python conversion of two dimensional list of float to two dimensional array of float

Question

I am trying to read a column of float into a list for each of a set of CSV files, then appending to a two dimensional list, later converting that to a 2D array, but the array doesn't convert into a two dimensional array of float (paraphrased below). Where am I going wrong?

import numpy
symbols = ['CVX', 'COP', 'MMM', 'BP', 'HMC', 'TM']
AA_lst = []
nSyms = len(symbols)
shortest = 99999
for sym in symbols
    fn = "PricesOf_" + sym + ".csv"
    col = getCSVcolumn( fn, "Close" )
    print( "type(col)="    + str(type(col)) )     # --> <class 'list'>
    print( "type(col[0])=" + str(type(col[0])) )  # --> <class 'float'>
    shortest = min(shortest,len(col))
    AA_lst.append(col)                            # appended as a row of AA_lst

AA = numpy.array( AA_lst )
print( "type=(AA)="     + str(type(AA)) )         # --> <class 'numpy.ndarray'>
print( "type=(AA[0]="   + str(type(AA[0])) )      # --> <class 'list'>
#print( "type=(AA[0,0]=" + str(type(AA[0,0])) )   # --> Error, too many indices

# fix up dimensions (so rows are all the same length)
AA = numpy.resize( AA, (nSyms, shortest) )
print( "type=(AA)="     + str(type(AA)) )         # --> <class 'numpy.ndarray'>
print( "type=(AA[0]="   + str(type(AA[0])) )      # --> <class 'numpy.ndarray'>
print( "type=(AA[0,0]=" + str(type(AA[0,0])) )    # --> <class 'list'>

# desire something of the form:  array([[1,2,3] [4,5,6] [7,8,9]])
# i.e. expecting type(AA[0,0] to be <class 'float'>, not <class 'list'>

This is probably happening because you don't have a 2D array of dtype float, you have a 1D array of dtype object, because numpy doesn't support arrays of heterogeneous size, and so it's doing the best it can. — DSM
– DSM, Commented Jan 4, 2014 at 4:03
I was paraphrasing the code... symbols is a list of tokens extracted from filenames of cross-correlations (two symbols per filename), with no duplicates. So, how do I set up to get a 2D array of float? — DarkSky
– DarkSky, Commented Jan 4, 2014 at 18:06
If I create a homogeneous rectangular list of lists (and skip the resize), the AA[i,j] have type float. Problem solved. — DarkSky
– DarkSky, Commented Jan 5, 2014 at 2:13

Yariv · Accepted Answer · 2014-01-04 22:05:04Z

1

Make sure the lists in AA_lst are of the same length.
Consider using pandas.

answered Jan 4, 2014 at 22:05

Yariv

13.4k21 gold badges57 silver badges76 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

DarkSky · Accepted Answer · 2014-01-05 02:18:51Z

1

Problem solved by replacing numpy.resize(...) with:

AAA_lst = []
for row in AA_lst:
    AAA_lst.append( row[:shortestUnshifted] )

and feeding AAA_lst to numpy.array()

answered Jan 5, 2014 at 2:18

DarkSky

452 bronze badges

Collectives™ on Stack Overflow

python conversion of two dimensional list of float to two dimensional array of float

2 Answers 2

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Related