The most efficient way to loop n-d np.array

Question

1. Introduction

Assuming there is a 3-d array in the shape of (365,100,100): Prec.

It represent the daily precipitation condition of an area for whole year.
Apparently, the first dimension represent the time series.
The last 2 dimensions represent the spatial distribution(for example, there are 10000 grids in size 1km x 1 km)

2. Attempt

Test each grid for whole area whether its precipitation above certain value Pd which separate dry and wet. I want to sum the dry day for the whole year.

3. My code

freq = np.zeros(100,100).reshape(100,100)
Pd = xxx
for i in range(0,prec.shape[0],1):
    for j in range(0,prec.shape[1],1):
        for k in range(0,prec.shape[2],1):   
            if prec[i,j,k] < Pd:
               freq[j,k] +=1

I think too many loop must waste time. Are there some cleanest way to achieve similar work?
Any advices would be appreciate!

Divakar's answer below is excellent. Personally, I think that for this type of stuff, numpy is too low level, and pandas is the way to go. — Ami Tavory
– Ami Tavory, Commented Mar 26, 2016 at 11:30
I'm only familiar with dataframe with pandas. I'll try n-d array in pandas some day! Thanks! — Han Zhengzu
– Han Zhengzu, Commented Mar 26, 2016 at 11:35

Divakar · Accepted Answer · 2016-03-26 11:39:05Z

2

You are comparing and summing along the first axis of prec. That comparison could be performed using NumPy broadcasting in a vectorized manner and then sum along the first axis with .sum(0), like so -

freq = (prec < Pd).sum(0)

edited Mar 26, 2016 at 11:39

answered Mar 26, 2016 at 11:27

Divakar

222k19 gold badges273 silver badges374 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Ami Tavory Over a year ago

No loops - very elegant.

Han Zhengzu Over a year ago

May I ask what the principle of these kind of code? I have met with some similar code like array[array > 0.5]. The efficiency are so greater than loop. Why they are so fast? Don't they need loop to traverse all the value ?

Divakar Over a year ago

@HanZhengzu This might shed some light. NumPy is built for performing the same operation on a huge number of elements in a vectorized fast way. I think the philosophy with it is SIMD.

Collectives™ on Stack Overflow

The most efficient way to loop n-d np.array

1. Introduction

2. Attempt

3. My code

1 Answer 1

3 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1. Introduction

2. Attempt

3. My code

1 Answer 1

3 Comments

Linked

Related