How to perform operations with row arrays using numpy?

Question

I have a matrix of absorbance values that I've pulled from a whole wack of spectra. I call this matrix "specdt"

Each row represents the values over multiple samples at a specific wavelength. I want to find the r^2 values of the regression against a seperate array of concentration values called "Concentration."

Here's what I have so far:

regression = []
for row in specdt:
    x = Concentration
    y = specdt[row,:]
    slope, intercept, r_value, p_value, std_err = stats.linregress(x,y)
    regression.append(r_value**2)

regression_n = numpy.asarray(regression)
numpy.savetxt("r2_2.csv", regression_n, delimiter=",")

I get the error:

Traceback (most recent call last):
   file "blah blah", line 42, in <module>
   y = specdt[row,:]
InexError: arrays used as indices must be of integer (or boolean) type

I suspected this is because "row" isn't an integer, so I tried to iterate over a "t" variable instead; no luck.

I suspect it's the way I'm trying to pull the row into the y values for linregress, but I can't seem to find another way to do this.

Any advice is greatly appreciated!

edit: I should mention that

y = row

was the first thing i tried.

It gives me the following error:

Traceback (most recent call last):
  File "C:\Users\ME\Downloads\Personal\Spectrometer\test\Spectrum3.py", line 42, in <module>
    slope, intercept, r_value, p_value, std_err = stats.linregress(x,y)
  File "C:\Python27\lib\site-packages\scipy\stats\_stats_mstats_common.py", line 92, in linregress
    ssxm, ssxym, ssyxm, ssym = np.cov(x, y, bias=1).flat
  File "C:\Python27\lib\site-packages\numpy\lib\function_base.py", line 2432, in cov
    X = np.vstack((X, y))
  File "C:\Python27\lib\site-packages\numpy\core\shape_base.py", line 230, in vstack
    return _nx.concatenate([atleast_2d(_m) for _m in tup], 0)
ValueError: all the input array dimensions except for the concatenation axis must match exactly

The dimensions of the conncentration array and the row should be the same.

linregress works beautifully if I pull out a single column (I transposed specdt.) This is the working code, if that helps:

##take only column 26 or the values for 2268; print stuff
#Absorbance2268 = spectral_data[:, 25]

#print(Absorbance2268.shape)
#print(Absorbance2268)
#
##manual entry of concentration values + array info
#conc =[0,0,0,0,0,6,6,6,6,6,6,6,6,6,6,8,8,8,8,8,10,10,10,10,10,4,4,4,4,4]
#Concentration = numpy.asarray(conc)
#
#print(Concentration.shape)
#print(Concentration)
#
##performing linear regression.
#x = Concentration
#y = Absorbance2268
#
#slope, intercept, r_value, p_value, std_err = stats.linregress(x,y)
#
#print "r-squared:", r_value**2

Assuming specdt is a numpy array or matrix, then row is already the row you want (not the index of the row) so y=specdt[row,:] should simply be y=row. — Chris Mueller
– Chris Mueller, Commented Mar 31, 2017 at 19:51
Thanks Chris, I've edited to show what I've done up to that point. — FLAV10
– FLAV10, Commented Mar 31, 2017 at 20:13

kennytm · Accepted Answer · 2017-03-31 19:51:38Z

1

for y in specdt:    # <---
    x = Concentration
    slope, intercept, r_value, p_value, std_err = stats.linregress(x,y)

The for loop gives the content of the rows themselves already. If you want the row index, use

for row, y in enumerate(specdt):
    ...

answered Mar 31, 2017 at 19:51

kennytm

526k110 gold badges1.1k silver badges1k bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

FLAV10 Over a year ago

Thanks for taking the time Kenny. Unfortunately, this also gives the error: file "blah blah", line 230, in vstack return _nx.concatenate([atleast_2d(_m) for _m in tup], 0) ValueError: all the input array dimensions except for the concatenation axis must match exactly

kennytm Over a year ago

@FLAV10 please show the full stack trace in the question. Is "blah blah" the real file name?

kennytm Over a year ago

@FLAV10 Looks like your y should be the columns, not rows?

FLAV10 Over a year ago

I don't think so. Instead of pulling a specific column, I transposed the entire data set: specdt = spectral_data.T Am I missing something here?

FLAV10 Over a year ago

WELLLLLLLLLLL that's interesting. sure enough, 41 and 42; a mismatch. I TRIPLE checked by counting instead of just printing the shape. I see I made a typo now. Your solution works perfectly. Thank you so much!!!!

|

Collectives™ on Stack Overflow

How to perform operations with row arrays using numpy?

1 Answer 1

6 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Related