How to obtain data from csv files using regex in python

Question

I am trying to filter a csv file from a column contains many minus numbers.

I found a source code on the website, which worked on small lists, but it does not work on data from a csv file.

here is an example of the data I have.

and here is the code I am using

import pandas as pd
from pandas import DataFrame
import numpy as np
import re

import csv
from re import findall

ful = pd.read_csv(r'/home/aziz/Desktop/testminplus.csv')
ful1 = ful[0:]

#full = ['1', '-3']
full = ful1

regex = re.compile(r'(-\d*)')
# use only one of the following lines, whichever you prefer
#filtered = filter(lambda i: not regex.search(i), full)
filtered = [i for i in full if not regex.search(i)]
print(filtered)

The results are as the following:

[' ', ' ', ' ', ' ', '8', '2', '3', '\n', '0', ' ', ' ', ' ', ' ', '6', '0', '9', '\n', '1', ' ', ' ', ' ', ' ', '6', '2', '7', '\n', '2', ' ', ' ', ' ', ' ', '2', '1', '1', '\n', '3', ' ', ' ', ' ', '2', '2', '6', '\n', '4', ' ', ' ', ' ', ' ', '9', '2', '1', '\n', '5', ' ', ' ', ' ', ' ', '8', '2', '9', '\n', '6', ' ', ' ', ' ', ' ', ' ', ' ', '1', '\n', '7', ' ', ' ', ' ', ' ', '9', '7', '2', '\n', '8', ' ', ' ', ' ', ' ', '1', '7', '3', '\n', '9', ' ', ' ', ' ', ' ', '1', '8', '1', '\n', '1', '0', ' ', ' ', ' ', '6', '6', '\n', '1', '1', ' ', ' ', ' ', '9', '6', '\n', '1', '2', ' ', ' ', ' ', '5', '7', '3', '\n', '1', '3', ' ', ' ', ' ', '8', '9', '5', '\n', '1', '4', ' ', ' ', ' ', '1', '1', '8', '\n', '1', '5', ' ', ' ', ' ', ' ', '7', '\n', '1', '6', ' ', ' ', '6', '9', '8', '\n', '1', '7', ' ', ' ', ' ', '3', '5', '1', '\n', '1', '8', ' ', ' ', ' ', '9', '3', '3', '\n', '1', '9', ' ', ' ', ' ', '9', '3', '2', '\n', '2', '0', ' ', ' ', ' ', '7', '3', '2', '\n', '2', '1', ' ', ' ', '6', '6', '0', '\n', '2', '2', ' ', ' ', '4', '6', '5', '\n', '2', '3', ' ', ' ', ' ', '3', '4', '5', '\n', '2', '4', ' ', ' ', ' ', ' ', '1', '8', '\n', '2', '5', ' ', ' ', ' ', '1', '2', '0', '\n', '2', '6', ' ', ' ', '2', '7', '0', '\n', '2', '7', ' ', ' ', '2', '3', '3', '\n', '2', '8', ' ', ' ', '1', '5', '2', '\n', '2', '9', ' ', ' ', ' ', '1', '8', '6', '\n', '3', '0', ' ', ' ', '3', '9', '6', '\n', '3', '1', ' ', ' ', '5', '3', '5', '\n', '3', '2', ' ', ' ', ' ', '3', '5', '9', '\n', '3', '3', ' ', ' ', ' ', ' ', '1', '\n', '3', '4', ' ', ' ', '5', '3', '3', '\n', '3', '5', ' ', ' ', ' ', '8', '1', '2', '\n', '3', '6', ' ', ' ', ' ', '5', '4', '6']

The desired output is something like the following:

Any idea how to solve this problem?

Is there a requirement to use pandas/regex here? Is your question ultimately - "How can I get a list of non-negative numbers from a file" or there abouts? — Jon Clements
– Jon Clements, Commented Oct 25, 2018 at 20:53
No requirements. I just need the results shown above...and yes, I need to exclude the negative numbers from the csv file. — Abdulaziz Al Jumaia
– Abdulaziz Al Jumaia, Commented Oct 25, 2018 at 21:05

Jon Clements · Accepted Answer · 2018-10-25 21:10:36Z

1

If you've just got a file with one number per line (and not an actual CSV file with multiple fields which doesn't appear to be your case) then you can do:

with open('/home/aziz/Desktop/testminplus.csv') as fin:
    # generator to yield each line as an integer
    data = (int(line) for line in fin)
    # list-comp to only include positive numbers...
    positive = [n for n in data if n >= 0]

answered Oct 25, 2018 at 21:10

Jon Clements

143k34 gold badges254 silver badges288 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Anton · Accepted Answer · 2018-10-25 21:58:07Z

1

Pandas solution is probably an overkill here, but also works quite well

import pandas as pd
# read file
df = pd.read_csv("/home/aziz/Desktop/testminplus.csv", 
                 header=None, 
                 converters={0: int}) # spits an error if non-numbers are present
# filter positives
df = df[df[0]>=0]
# write back 
df.to_csv("/home/aziz/Desktop/positives_only.csv", 
          header=False, 
          index=False)

answered Oct 25, 2018 at 21:58

Anton

666 bronze badges

Collectives™ on Stack Overflow

How to obtain data from csv files using regex in python

2 Answers 2

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Related