RecursionError while iterating over list

Question

I have a list of geodesic points by the format: [lat, long, elevation, index, land/sea binary classifier] in a grid formation with regular spacing throughout the dataset. I'm trying to find all neighbouring points that are land (elv > 0) to the current point in the list.

I keep getting this error: RecursionError: maximum recursion depth exceeded while getting the repr of a list and although I understand the sort of thing that could be causing this, I have no idea how this applies to this situation as I'm not explicitly using recursion. How am I able to remedy this error? How can I understand the problem better as well?

(topLat, bottomLat, westLong, eastLong are the lats and longs for the first and last point in the grid/map to identify points at the edge of the map)

def buildNeighbours(point, dataset):
    neighbours = []
    ix = int(point[3])

    if point[0] != topLat and point[0] != bottomLat and point[1] != westLong and point[1] != eastLong:
        nw = dataset[ix - (rowLength + 1)]
        n = dataset[ix - rowLength]
        ne = dataset[ix - (rowLength - 1)]
        e = dataset[ix + 1]
        se = dataset[ix + (rowLength + 1)]
        s = dataset[ix + rowLength]
        sw = dataset[ix + (rowLength - 1)]
        w = dataset[ix - 1]
        neighbours = [nw, n, ne, e, se, s, sw, w]
        point.append(neighbours)

    else:
        point = []

    return point


for point in dataList:
    point = buildNeighbours(point, dataList)

print(dataList[2000])

You could get this if a list item is in fact a reference to the original list. repr would keep circling back to the top of the nested lists. Your code is really long so I haven't tried to figure out where the problem is. Do you have a shorter version that demonstrates the problem. — tdelaney
– tdelaney, Commented Oct 16, 2016 at 2:31
Whenever it has to excute the if statement, the error occurs — DanielSon
– DanielSon, Commented Oct 16, 2016 at 2:37
The error is very much self-explanatory - you've managed to implement infinite loop somehow. Short sample (3-7 entries) of your data list plus the way you run this script may help the community to assist you. None seems to be possible to evaluate from your current code though — agg3l
– agg3l, Commented Oct 16, 2016 at 2:56
I think that you get an error in one of the if calculations, python tries to raise an error, but hits a recursion error when generating the error message. Put a print(repr(point)) at the top of the function and it will likely hit. Then you can deal with that. — tdelaney
– tdelaney, Commented Oct 16, 2016 at 3:16
What is point? It seems like a list of ints or floats but then you append a list to it. Is that intentional? — tdelaney
– tdelaney, Commented Oct 16, 2016 at 3:17

Kevin J. Chase · Accepted Answer · 2016-10-16 06:50:53Z

To stringify a point (really a list), print must first get the string representation of every element. The last element of each point is a list of neighboring points, each of which is a list that contains yet another list of neighboring points... one of which is the original point. And so it continues...

list's __repr__ attempts to limit recursive cases, eventually giving up and returning '...'. Assuming it uses the same defaults as reprlib.Repr objects, the maxlevel (max recursion depth) is 6. With 8 neighbors each, that could mean thousands of visits to a relatively small number of unique points.

I was able to print a 3×3 grid, where the fan-out is limited because most of the points only have 3 or 5 neighbors (corners and sides). My simplified point lists, which didn't contain altitude or land/sea elements, required about 700kiB to represent the whole grid... about 40KiB for the upper-left corner alone. On a 4×4 grid, a single point ballooned up to about 16MiB.

That said, I'm guessing what your inputs look like, and I probably haven't reproduced what you're really doing. More importantly, I did not get a RecursionError like you did, perhaps because I gave up waiting for it.

With those caveats in mind, I suggest:

In each point's neighbors list, store the indices of the neighbors. Look them up later whenever you need them. (This is the simplest solution I could come up with.) Write a couple helper functions that calculate the northwest or south or whatever neighbor of a given index, since you'll be doing that a lot.
Alternatively, consider creating a neighbors dictionary, mapping each point's index to a list of indices. You'd have to keep that dictionary alongside dataList at all times, but it would let you remove the list of neighbors from all of your points.
If you really have to store the neighbors themselves, create a Point class with custom __str__ and __repr__ methods that don't try to print the neighbors. As a bonus, a class would let you refer to fields with names like lat, lng, and index instead of mysterious subscripts like [1] and [3].

Collectives™ on Stack Overflow

RecursionError while iterating over list

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related