How do you remove duplicates from a list using Python?
To remove duplicate elements from a list in Python use a list comprehension to create a new list with no duplicates with this code:
[x for idx, x in enumerate(original_list) if x not in original_list[idx+1:]]
Here is an example demonstrating how this code works:
>>> original_list = [1, 2, 1, 3, 1, 2, 3]
>>> [x for idx, x in enumerate(original_list) if x not in original_list[idx+1:]]
[1, 2, 3]As you can see from the example above the result from the list comprehension produces a unique list from the original.
How does this code work?
A list comprehension is an operation in Python that allows for the creation of a new list using a single line of code.
The structure of a list comprehension, when read from left to right, starts with the element to be inserted into the new list. In this case, the element being inserted is
x
.
The element
x
is derived from the following
enumerated
for loop. The enumerated for loop allows for the capture of the index number as well as the element in the original list.
After the for loop is an
if
condition which checks if the current element can be found in the
original_list
using the
slice operator
which captures the next element to the end of the list. If the element cannot be found in the next element to the end then the condition is satisfied and the element is saved into the list comprehension.
Here’s how each element would have been processed individually from the example above:
First element is
1
. Is
1
not
found in the remaining elements in the list
[2,
1
, 3,
1
, 2, 3]
? It is found (multiple times), therefore it is excluded from being added.
Second element is
2
. Is
2
not
found in the remaining elements in the list
[1, 3, 1,
2
, 3]
? It is found, therefore it is excluded from being added.
Third element is
1
. Is
1
not
found in the remaining elements in the list
[3,
1
, 2, 3]
? It is found, therefore it is eliminated from being added to the new list.
Fourth element is
3
. Is
3
not
found in the remaining elements in the list
[1, 2,
3
]
? It is found, therefore it is not included into the new list.
Fifth element is
1
. Is
1
not
found in the remaining elements in the list
[2, 3]
? It is
not
found, therefore it is included into the new list. The list comprehension would now contain its first element
[1]
.
Sixth element is
2
. Is
2
not
found in the remaining elements in the list
[3]
? It is
not
found, therefore it is included into the new list. The list comprehension would now contain its second element
[1, 2]
.
Last element is
3
. As there are no other remaining elements in the list it would automatically be added to the working list comprehension which produces the output
[1, 2, 3]
.
The list comprehension is an easy way to create a unique list.
Summary
To create a unique list from an original list use the list comprehension to create a Python one-liner code:
[x for idx, x in enumerate(original_list) if x not in original_list[idx+1:]]
.
Next, you might want to check the article on how to get a list of duplicates from a list that uses the same code above with some minor modifications.
