efficient algorithm for list edits

Question

I have a 3D-point cloud, saved in two lists. Example with 5 points (x,y,z): (3,3,3), (1,1,1), (4,4,4), (2,2,2), (5,5,5) -> My lists looks like this:

z = [3, 1, 4, 2, 5] # the z values
pts = [(3,3), (1,1), (4,4), (2,2), (5,5)] # the x and y values

Now I want to eliminate all values where the z-value is higher than 3:

# what I want to receive:
z = [3, 1, 2]
pts = [(3,3), (1,1), (2,2)]

My algorithm is here:

k = -1
for i in range(len(z)):
    k += 1
    if z[k] > h:
        z.pop(k)
        pts.pop(k)
        k -= 1

This returns me exactly what I want - but it's very slow (for >100,000 values). I thought about first sorting my list via z.sort() and then do z = z[:index] - but when I do this for my z-list then my pts-list is still unsorted. And even if I could get both lists sorted, doesn't I also have to go through a long loop to find the index where my condition is true? Does anyone knows a more efficient solution?

Answer 1

z, pts = zip(*[(z, pt) for z, pt in zip(z, pts) if z <= 3])
print z, pts

Output

(3, 1, 2) ((3, 3), (1, 1), (2, 2))

Answer 2

With itertools:

from itertools import izip, ifilter

zip together

zipped_coords = izip(z, pts)

filter (in post you mention higher, but expeted results are in fact lower, choosed first one)

filtered_coords = ifilter(lambda x: x[0]>=3, zipped_coords )

unzip

znew, ptsnew = map(list, izip(*filtered_coords))

or all-in-one oneliner

>>> znew, ptsnew = map(list, izip(*ifilter(lambda x: x[0]>=3, izip(z, pts))))
>>> print znew, ptsnew
[3, 4, 5] [(3, 3), (4, 4), (5, 5)]

Answer 3

Because you are removing elements from the list as you iterate it is a O(N^2) algorithm. You can use a simple list comprehension along with zip to do this in linear time.

z = [3, 1, 4, 2, 5] # the z values
pts = [(3,3), (1,1), (4,4), (2,2), (5,5)] # the x and y values

merged = zip(z, pts)    
filtered = [x for x in merged if x[0] <= 3]
z, pts = zip(*filtered)