9

I know this is probably an easy answer but I can't figure it out. What is the best way in Python to keep the duplicates in a list:

x = [1,2,2,2,3,4,5,6,6,7]

The output should be:

[2,6]

I found this link: Find (and keep) duplicates of sublist in python, but I'm still relatively new to Python and I can't get it to work for a simple list.

6
  • Have you tried one of the one liners in that post? Commented Apr 4, 2013 at 13:25
  • Do you need to preserve order? Commented Apr 4, 2013 at 13:28
  • @DSM -- Looks like we were thinking the same thing ... Commented Apr 4, 2013 at 13:28
  • Post the code you are having trouble with, otherwise this is a complete retread of that other question. Commented Apr 4, 2013 at 13:28
  • 2
    @StevenRumbalski: Not precisely, the other question is also flattening nested lists at the same time. Commented Apr 4, 2013 at 13:31

5 Answers 5

16

I'd use a collections.Counter:

from collections import Counter
x = [1, 2, 2, 2, 3, 4, 5, 6, 6, 7]
counts = Counter(x)
output = [value for value, count in counts.items() if count > 1]

Here's another version which keeps the order of when the item was first duplicated that only assumes that the sequence passed in contains hashable items and it will work back to when set or yeild was introduced to the language (whenever that was).

def keep_dupes(iterable):
    seen = set()
    dupes = set()
    for x in iterable:
        if x in seen and x not in dupes:
            yield x
            dupes.add(x)
        else:
            seen.add(x)

print list(keep_dupes([1,2,2,2,3,4,5,6,6,7]))
Sign up to request clarification or add additional context in comments.

11 Comments

However you lose the order of the elements in the output.
Yep. There are a lot of situations where this isn't the best way to go. It also requires the input be hashable... But, it's O(n) even for un-sorted lists which is nice.
The shortest ordered variant I can think of offhand is [k for k in OrderedDict.fromkeys(x) if counts[k] > 1].
@DSM -- Why do you need an OrderedDict there? why not just [k for k in x if counts[k] > 1]? Actually, that's better than what I have. I'll update...
@mgilson: try it and see.. :^)
|
10

This is a short way to do it if the list is sorted already:

x = [1,2,2,2,3,4,5,6,6,7]

from itertools import groupby
print [key for key,group in groupby(x) if len(list(group)) > 1]

2 Comments

This will also work with python2.6 which is a problem with mine.
@luchosrock: No, groupby groups consecutive elements
3

List Comprehension in combination with set() will do exactly what you want.

>>> list(set([i for i in x if x.count(i) > 1]))

[2, 6]

Comments

0

keepin' it simple:

array2 = []
aux = 0
aux2=0
for i in x:
    aux2 = i
    if(aux2==aux):
        array2.append(i)
    aux= i
list(set(array2))

That should work

2 Comments

Won't that give [2,2,6]?
@DSM ahaha you're totally right, I edited my answer, Thanks :)
0

Not efficient but just to get the output, you could try:

import numpy as np

def check_for_repeat(check_list):
    repeated_list = []

    for idx in range(len(check_list)):
        elem = check_list[idx]
        check_list[idx] = None

        if elem in temp_list:
            repeated_list.append(elem)

    repeated_list = np.array(repeated_list)

    return list(np.unique(repeated_list))

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.