how to convert a list of list inside a csv file to numpy array

Question

I have a csv file that looks like below

[0.037621960043907166, 0.04622473940253258, 0.9161532521247864]
[0.030109738931059837, 0.03261643648147583, 0.9372738003730774]
[0.030109738931059837, 0.03261643648147583, 0.9372738003730774]

I need to convert this to numpy array. If I use below code

data = genfromtxt(file, delimiter=',', encoding="utf8")

I get nan in the output.

If I do this

np.genfromtxt (file, encoding=None, dtype = None)

It fails to remove the starting and ending brackets of the list and outputs like

array = ([['[0.037621960043907166,', '0.04622473940253258,',
        '0.9161532521247864]'],
       ['[0.030109738931059837,', '0.03261643648147583,',
        '0.9372738003730774]'],
       ['[0.030109738931059837,', '0.03261643648147583,',
        '0.9372738003730774]']], dtype='<U22')

the expected output is

array = ([['0.037621960043907166,', '0.04622473940253258,',
            '0.9161532521247864'],
           ['0.030109738931059837,', '0.03261643648147583,',
            '0.9372738003730774'],
           ['0.030109738931059837,', '0.03261643648147583,',
            '0.9372738003730774']], dtype='<U22')

How can I get the expected output? Seems I need to remove the brackets 1st before applying the numpy operations. Any suggestion?

meTchaikovsky · Accepted Answer · 2019-09-27 05:57:36Z

1

As long as you know the format of the content, I think a simple slicing will do

import numpy as np

tmp = open('tmp', 'r').readlines()
tmp = np.array([[float(num) for num in item[1:-2].split(',')] for item in tmp])

answered Sep 27, 2019 at 5:57

meTchaikovsky

7,6963 gold badges18 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

xiaotong xu · Accepted Answer · 2019-09-27 06:04:27Z

0

what you need is eval()

from numpy import array
with open('your file name', 'r') as f:
    str_lines = f.readLines()
    lines = [eval(x) for x in str_lines]
    ary = array(lines)
f.close()

answered Sep 27, 2019 at 6:04

xiaotong xu

3113 silver badges12 bronze badges

Comments

Sayandip Dutta · Accepted Answer · 2019-09-27 06:08:59Z

When you have text file like:

[0.037621960043907166, 0.04622473940253258, 0.9161532521247864]
[0.030109738931059837, 0.03261643648147583, 0.9372738003730774]
[0.030109738931059837, 0.03261643648147583, 0.9372738003730774]

You can try this:

np.genfromtxt(filename,dtype=str,encoding=None, converters ={0: lambda s: s.strip('['), 2:lambda s: s.strip(']')}, delimiter = ',')

Output:

array([['0.037621960043907166', ' 0.04622473940253258',
        ' 0.9161532521247864'],
       ['0.030109738931059837', ' 0.03261643648147583',
        ' 0.9372738003730774'],
       ['0.030109738931059837', ' 0.03261643648147583',
        ' 0.9372738003730774']], dtype='<U20')

Collectives™ on Stack Overflow

how to convert a list of list inside a csv file to numpy array

3 Answers 3

Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Related