Question: this type of question has been asked many times. So apologies; I have searched hard to get an answer - but have not found anything that is close enough to my needs (and I am not sufficiently advanced **(I am a total newbie)** to customize an existing answer). So thanks in advance for any help.  

Here's my query:  

* I have 30 or so csv files and each contains between 500 and 15,000 rows.
* Within each of them (in the 1st column) - are rows of alphabetical IDs (some contain underscores and some also have numbers).
* I don't care about the unique IDs - but I would like to identify the duplicate IDs and the number of times they appear in all the different csv files.
* Ideally I'd like the output for each duped ID to appear in a new csv file and be listed in 2 columns **("ID", "times\_seen")**

It may be that I need to compile just 1 csv with all the IDs for your code to run properly - so please let me know if I need to do that  

I am using python 2.7 (a crawling script that I run needs this version, apparently).  

Thanks again

Answer: 
That's rather easy to achieve. It would look something like:

```
import os

# Set to what kind of separator you have. '\t' for TAB
delimiter = ','

# Dictionary to keep count of ids
ids = {}

# Iterate over files in a dir
for in_file in os.listdir(os.curdir):
    # Check whether it is csv file (dummy way but it shall work for you)
    if in_file.endswith('.csv'):
        with open(in_file, 'r') as ifile:
            for line in ifile:
                my_id = line.strip().split(delimiter)[0]
                # If id does not exist in a dict = set count to 0
                if my_id not in ids:
                    ids[my_id] = 0
                # Increment the count
                ids[my_id] += 1

# saves ids and counts to a file
with open('ids_counts.csv', 'w') as ofile:
    for key, val in ids.iteritems():
        # write down counts to a file using same column delimiter
        ofile.write('{}{}{}\n'.format(key, delimiter, value))

```
Check out the pandas package. You can read an write csv files quite easily with it. 

<http://pandas.pydata.org/pandas-docs/stable/10min.html#csv>

Then, when having the csv-content as a dataframe you convert it with the `as_matrix` function. 
Use the answers to this question to get the duplicates as a list. 

[Find and list duplicates in a list?](https://stackoverflow.com/questions/9835762/find-and-list-duplicates-in-python-list)

I hope this helps