How does python iterator for list actually works?

Question

Let's say we have the following list and we are creating an iterator for it:

lst = [1,2,3]
itr = iter(lst)

Next lets say we are changing our list with completely different values:

lst = ['a', 'b', 'c']

And if I we run the following loop:

for x in itr:
   print x

We will get '1,2,3'. But why? As far as I understand, iterator doesn't copy all values from iterating object. At least iterator for list from three elements has the same size as a list of 100000 elements. sys.getsizeof(i) returns 64. How can iterator be so small by size and keep 'old' values of list?

You haven't changed the list, you've created a new, completely unrelated one. — jonrsharpe
– jonrsharpe, Commented Oct 22, 2016 at 14:58

Ignacio Vazquez-Abrams · Accepted Answer · 2016-10-22 14:56:15Z

4

The iterator itself contains a reference to the list. Since lst is rebound instead of mutated, this reference does not change.

>>> lst = [1, 2, 3]
>>> itr = iter(lst)
>>> lst[:] = ['a', 'b', 'c']
>>> for x in itr:
...   print x
... 
a
b
c

answered Oct 22, 2016 at 14:56

Ignacio Vazquez-Abrams

803k160 gold badges1.4k silver badges1.4k bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Moses Koledoye · Accepted Answer · 2016-10-22 16:14:12Z

The iterator references a list object not a name. So reassigning the name lst to another object does not affect the iterator in anyway; names are bound to objects, and refer to objects, but the names are not the object themselves.

You can get a snoop of the object the iterator is referencing with gc.get_referents:

>>> import gc
>>> lst = [1,2,3]
>>> itr = iter(lst) # return an iterator for the list
>>> lst = ['a', 'b', 'c'] # Bind name lst to another object
>>> gc.get_referents(itr)[0]
[1, 2, 3]

As you'll notice, the iterator is still referring to the first list object.

The following reference will help you learn more about names and binding in Python:

Execution model - Naming and binding

jatinderjit · Accepted Answer · 2022-12-06 10:35:26Z

Welcome to Python's object reference system. The variable names do not really have a deep relationship with the actual object stored in memory.

lst = [1, 2, 3]
itr = iter(lst)  # iter object now points to the list pointed to by lst

print(next(itr))  # prints 1

# Both `lst` and `lst1` now refer to the same list
lst1 = lst

# `lst` now points to a new list, while `lst1` still points to the original list.
lst = ['a', 'b', 'c']

print(next(itr))  # prints 2

lst.append(4)
lst1.append(5)  # here the list pointed to by `itr` is updated

for i in itr:
    print(i)  # prints 3, 5

TL;DR: Python variable names are just tags, that refer to some object in space. When you call iter on the list named lst, the iterator object points to the actual object, and not the name lst.

If you can modify the original object, by calling append, extend, pop, remove, etc, the iterator's output will be affected. But when you assign a new value to lst, a new object is created (if it didn't previously exist), and lst simply starts pointing to that new object.

The garbage collector will delete the original object if no other object is pointing to it (itr is pointing to it in this case, so the original object won't be deleted yet).

http://foobarnbaz.com/2012/07/08/understanding-python-variables/

Extra:

lst1.extend([6, 7, 8])
next(itr)    # raises StopIteration

This doesn't have anything to do with object referencing, the iterator just stores internally that it has iterated the complete list.

Collectives™ on Stack Overflow

How does python iterator for list actually works?

3 Answers 3

Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Related