1

If I understand properly, we in Python we have:

  • Iterables = __iter__() is implemented
  • Iterators = __iter__() returns self & __next__() is implemented
  • Generators = an iterator created with a yield statement or a generator expression.

Question: Are there categories above that are always/never consumable?

By consumable I mean iterating through them "destroys" the iterable; like zip() (consumable) vs range() (not consumable).

2
  • Small correction: yield in a def statement creates a generator, but you can also create a generator with a generator expression, e.g. (x for x in [1,2,3]). Commented Nov 15, 2020 at 16:50
  • Coming back to this to mention that this page can be useful in finding the Python docs definitions of these concepts: docs.python.org/3.8/library/collections.abc.html Commented May 3, 2021 at 13:02

1 Answer 1

2

All iterators are consumed; the reason you might not think so is that when you use an iterable with something like

for x in [1,2,3]:

the for loop is creating a new iterator for you behind the scenes. In fact, a list is not an iterator; iter([1,2,3]) returns something of type list_iterator, not the list itself.


Regarding the example you linked to in a comment, instead of

class PowTwo:    
    def __init__(self, max=0):
        self.max = max

    def __iter__(self):
        self.n = 0
        return self

    def __next__(self):
        if self.n <= self.max:
            result = 2 ** self.n
            self.n += 1
            return result
        else:
            raise StopIteration

which has the side effect of modifying the iterator in the act of returning it, I would do something like

class PowTwoIterator:    
    def __init__(self, max=0):
        self.max = max
        self._restart()

    def _restart(self):
        self._n = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self._n <= self.max:
            result = 2 ** self._n
            self._n += 1
            return result
        else:
            raise StopIteration

Now, the only way you can modify the state of the object is to do so explicitly (and even that should not be done lightly, since both _n and _restart are marked as not being part of the public interface).

The change in the name reminds you that this is first and foremost an iterator, not an iterable that can provide independent iterators from.

Sign up to request clarification or add additional context in comments.

5 Comments

Thanks! So a range() object is not an iterator but is an iterable?
Correct; in particular, the CPython implementation of range.__iter__ returns something of type range_iterator.
Thanks so much! One last question: I have seen custom iterators that aren't consumable, hence my confusion (for ex the class "PowTwo" in this tutorial: programiz.com/python-programming/iterator). So is that considered bad practice, or it's just that all in-built Python iterators follow the non-consumable rule of thumb?
I would argue it's bad practice, if only because __iter__ has a side effect. It might look like you can create two independent iterators of this class (p = PowTwo(); i1 = iter(p); i2 = iter(p)), but since i1 is i2 is true, advancing one advances the other.
That makes sense. Thank you!

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.