How to keep iterating after an exception?

Question

I am confused by interaction between Python-2 iterators and exceptions.

Specifically, given the following code:

def gen_int():
    if not hasattr(gen_int,"x"):
        gen_int.x = 0
    while True:
        gen_int.x += 1
        yield gen_int.x

def gen_odd():
    for x in gen_int():
        if x % 2:
            yield x
        else:
            raise ValueError("gen_odd",x)

(please assume the above are out of my control!), I write

def gen_all():
    it = gen_odd()
    while True:
        try:
            yield it.next()
        except ValueError as exn:
            print exn
            func_name, x = exn
            assert func_name == "gen_odd"
            yield x

in the hope of recovering the full stream generated by gen_int.

However, the iteration stops after the first exception:

def gen_test(top):
    for x in gen_all():
        print x
        if x > top:
            break

Here are 3 invocations:

>>> gen_test(20)
1
('gen_odd', 2)
2
>>> gen_test(20)
3
('gen_odd', 4)
4
>>> gen_test(20)
5
('gen_odd', 6)
6

The question is: How do I modify gen_all so that gen_test will print all ints below top?

PS. apparently, the exception in gen_odd works as return - it marks the iterator as exhausted. is this really the case? is there a workaround?

Why do you raise a ValueError in the first place in gen_odd? — Chris_Rands
– Chris_Rands, Commented Oct 26, 2016 at 16:23
gen_int is a strange generator. It saves global state that each new instance uses. The other 99.99% of generators in the wild don't work this way. — tdelaney
– tdelaney, Commented Oct 26, 2016 at 16:42

Moses Koledoye · Accepted Answer · 2016-10-26 16:45:32Z

4

Reassign gen_odd() in the except block:

def gen_all():
    it = gen_odd()
    while True:
        try:
            yield it.next()
        except ValueError as exn:
            print exn
            func_name, x = exn
            assert func_name == "gen_odd"
            yield x
            it = gen_odd() # here

The generator function gen_odd stops once that ValueError exception is raised. You have to recall the function to create another gen function object after the previous one stops. gen_odd picks up from where it stopped since the yielded values from gen_int are bound to the function object; states are saved, else this would not have worked.

>>> gen_test(5)
1
('gen_odd', 2)
2
3
('gen_odd', 4)
4
5
('gen_odd', 6)
6

edited Oct 26, 2016 at 16:45

answered Oct 26, 2016 at 16:21

Moses Koledoye

78.8k8 gold badges139 silver badges141 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

tdelaney Over a year ago

This is a great answer but it only works because gen_int works off of a singleton gen_int.x which means it is saving state outside of the generator. It would not work for the most generators that raise exceptions.

Moses Koledoye Over a year ago

@tdelaney Yes, I figured the yielded values are bound to the function object, else it would have been very different and probably more difficult

Collectives™ on Stack Overflow

How to keep iterating after an exception?

1 Answer 1

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Linked

Related