How do Python generator functions maintain local state?

Question

According to the docs at https://docs.python.org/2/reference/simple_stmts.html#yield,

all local state is retained, including the current bindings of local variables, the instruction pointer, and the internal evaluation stack: enough information is saved so that the next time next() is invoked, the function can proceed exactly as if the yield statement were just another external call.

Here's a simple case:

def generator():
    my_list = range(10)
    print "my_list got assigned"
    for i in my_list:
        print i
        yield
    return

In the shell, generator() behaves like this:

>>>>generator().next()
my_list got assigned
0
>>>>generator().next()
my_list got assigned
0

I would have thought that my_list would not get reassigned each time .next() is called. Can someone explain why this happens, and why it seems like the docs contradict this?

You're creating a new generator object each time. Call next twice on the same object and you'll get the output you expect. — dano
– dano, Commented Apr 20, 2015 at 22:03

Martijn Pieters · Accepted Answer · 2015-04-20 22:03:52Z

9

You are creating a new generator object each time. Create one instance:

g = generator()
g.next()
g.next()

Here g references the generator object that maintains the state:

>>> def generator():
...     my_list = range(10)
...     print "my_list got assigned"
...     for i in my_list:
...         print i
...         yield
...     return
... 
>>> g = generator()
>>> g
<generator object generator at 0x100633f50>
>>> g.next()
my_list got assigned
0
>>> g.next()
1

answered Apr 20, 2015 at 22:03

Martijn Pieters

1.1m325 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

Comments

chepner · Accepted Answer · 2015-04-20 23:25:22Z

5

Yes, the generator maintains state, as you correctly found in the documentation. The problem is that in your example, you're creating two generators. The first one is not assigned to any variable, so it's discarded immediately after .next() completes. Then you create the second generator, with its own local state, that starts at the beginning.

Try this:

>>> mygen = generator()
>>> mygen.next()
my_list got assigned
0
>>> mygen.next()
1

edited Apr 20, 2015 at 23:25

chepner

538k77 gold badges594 silver badges746 bronze badges

answered Apr 20, 2015 at 22:05

Philip

1,5521 gold badge12 silver badges23 bronze badges

Comments

mway · Accepted Answer · 2015-04-20 22:06:56Z

1

You're creating a new instance of generator when you call generator(). If you instead did

my_generator = generator()
my_generator.next() # my_list got assigned, 0
my_generator.next()  # 1

the list would only be assigned once.

answered Apr 20, 2015 at 22:06

mway

6435 silver badges15 bronze badges

Collectives™ on Stack Overflow

How do Python generator functions maintain local state?

3 Answers 3

Comments

Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Linked

Related