How would I select rows in pandas that match a list of strings, not just one particular string?

Question

Let's say we have a dataframe- df and a column labelled 'A'. For selecting rows that match ONE string -'some_string', df['A'].str.contains('some_string') works great.

My question is, is there a corresponding method to pass to contains a list of strings, so that partial matches can be gotten? instead of 'some_string' can I give it a list of strings? I am trying to avoid using a for loop and slicing the data frame and concatenating into a new dataframe.

Lets say the dataframe is

pd.DataFrame(np.array([['cat', 2], ['rat', 5], ['ball', 8],['string', 8]]),columns=['A', 'B']))

and

list =['at','ll','ac']

So I want to select the rows with cat, rat, ball. Sorry for the artificially contrived example.

can you create a example df and an expected df? this would help us replicate your question. :) Thanks — anky
– anky, Commented Mar 7, 2019 at 17:07
You might be able to use df['A'].str.contains("|".join(list_of_strings)). — Graipher
– Graipher, Commented Mar 7, 2019 at 17:10
Possible duplicate of check if string in pandas dataframe column is in list — G. Anderson
– G. Anderson, Commented Mar 7, 2019 at 17:11
Check out this answer as well. We can write any logic in the lambda function. — Nagabhushan S N
– Nagabhushan S N, Commented Mar 21, 2020 at 11:24

Graipher · Accepted Answer · 2019-03-07 17:23:07Z

5

pandas.Series.str.contains takes either a string or a regex. So you could just build a regex from the list of strings:

import pandas as pd

strings = "fo", "ba"
x = pd.Series(["foo", "bar", "baz", "buzz"])
x.str.contains("|".join(strings))
# 0     True
# 1     True
# 2     True
 #3    False
# dtype: bool

This might be slow if your list of strings to match against is very long and you might need a na=False to ignore NaN values, as mentioned in the comments by @anky_91.

edited Mar 7, 2019 at 17:23

answered Mar 7, 2019 at 17:13

Graipher

7,24630 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

anky Over a year ago

also add` na=False just incase. :)

danz · Accepted Answer · 2023-07-06 18:54:43Z

2

The most simplistic and Pandas friendly is:

list_of_strings = ['string1', 'string2']

df[df['A'].isin(list_of_strings)]

From https://sparkbyexamples.com/pandas/pandas-use-a-list-of-values-to-select-rows-from-dataframe/

answered Jul 6, 2023 at 18:54

danz

834 bronze badges

Comments

Tacratis · Accepted Answer · 2019-03-07 17:10:44Z

0

If A always contains exactly the string you want to find in the list, you can do this:

df['A'].map(lambda x: 1 if x in list_of_strings else 0)

the lambda function will check, for each row, if the value in 'A' (temporarily stored in x exists as one of the elements in list_of_strings, and return 1 or 0 accordingly.

You can then filter when this new mapped column is 1

answered Mar 7, 2019 at 17:10

Tacratis

1,0551 gold badge7 silver badges16 bronze badges

3 Comments

user1775614 Over a year ago

will this work with substring matching? instead of if x in list_of_strings can I say contains?

Tacratis Over a year ago

Yes, but it is probably not very efficient. You can use a list comprehension: lambda x: True in [x in s for s in list_of_strings]

Graipher Over a year ago

@Tacratis You should use any(...) instead of True in [...].

Collectives™ on Stack Overflow

How would I select rows in pandas that match a list of strings, not just one particular string?

3 Answers 3

1 Comment

Comments

3 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

3 Comments

Linked

Related