python reshape structure array to normal numpy array

Question

I have a structure array which looks like this

[(1, 2, 3, 4) (5, 6, 7, 8)]

and I removed the first column and make it looks like this

[(2, 3, 4) (6, 7, 8)]

but when I reshape it to array, it looks like this

[[1 2 3 4]
 [5 6 7 8]]

the '1' and '5' was not supposed to be there

This is my code

import numpy as np

array = np.array([(1,2,3,4), (5,6,7,8)],dtype=[('a', 'i4'), ('b', 'i4'), ('c', 'i4'),('d', 'i4')])
names = list(array.dtype.names)
new_names=names[1:]
data = array[new_names]
new_array = data.view('i4').reshape(len(data),-1)

can I know why and how to edit it?

Answer 1

In [128]: array = np.array([(1,2,3,4), (5,6,7,8)],dtype=[('a', 'i4'), ('b', 'i4'), ('c', '
     ...: i4'),('d', 'i4')]) 
     ...: names = list(array.dtype.names) 
     ...: new_names=names[1:] 
     ...: data = array[new_names]                                                         
In [129]: array.dtype                                                                     
Out[129]: dtype([('a', '<i4'), ('b', '<i4'), ('c', '<i4'), ('d', '<i4')])
In [130]: names                                                                           
Out[130]: ['a', 'b', 'c', 'd']
In [131]: data                                                                            
Out[131]: 
array([(2, 3, 4), (6, 7, 8)],
      dtype={'names':['b','c','d'], 'formats':['<i4','<i4','<i4'], 'offsets':[4,8,12], 'itemsize':16})

Note that the data.dtype has offsets. In the latest numpy versions, selecting a subset of the fields produces a view. array['a'] is still there, just 'hidden'.

Along with that change, they've added some some functions to the recfunctions:

In [133]: import numpy.lib.recfunctions as rf 

To make a copy without the 'a' data:

In [134]: data1 = rf.repack_fields(data)                                                  
In [135]: data1                                                                           
Out[135]: 
array([(2, 3, 4), (6, 7, 8)],
      dtype=[('b', '<i4'), ('c', '<i4'), ('d', '<i4')])

and to make an unstructured array:

In [136]: rf.structured_to_unstructured(array)                                            
Out[136]: 
array([[1, 2, 3, 4],
       [5, 6, 7, 8]], dtype=int32)
In [137]: rf.structured_to_unstructured(data)                                             
Out[137]: 
array([[2, 3, 4],
       [6, 7, 8]], dtype=int32)
In [138]: rf.structured_to_unstructured(data1)                                            
Out[138]: 
array([[2, 3, 4],
       [6, 7, 8]], dtype=int32)

These functions are documented at:

https://docs.scipy.org/doc/numpy/user/basics.rec.html#accessing-multiple-fields

Since all fields have the same dtype ('i4') view works - to a degree

In [142]: data.view('i4')                                                                 
Out[142]: array([1, 2, 3, 4, 5, 6, 7, 8], dtype=int32)
In [143]: data1.view('i4')                                                                
Out[143]: array([2, 3, 4, 6, 7, 8], dtype=int32)

But it's a view of the base data, and the shape is messed up. This shape issue existing in earlier versions. So it's best to read up on the changes, and use the recommended functions.

In previous SO questions I might have recommended using a list as intermediary:

In [144]: data.tolist()                                                                   
Out[144]: [(2, 3, 4), (6, 7, 8)]
In [145]: np.array(data.tolist())                                                         
Out[145]: 
array([[2, 3, 4],
       [6, 7, 8]])

Answer 2

Try slicing at the end:

new_array = data.view('i4').reshape(len(data),-1)[:,1:]

Result:

[[2 3 4]
 [6 7 8]]