Difference between Generators and Functions in Python

Question

I'm trying to understand the difference between functions and Generators and have used the code below for that purpose. However, I don't understand the Outputs.

class GeneratorsSample(object):

    def DoReturn(self):
        counter, maxCounter = 0, 5
        listResults = []
        while counter < maxCounter:
            print "tic"
            listResults.append(counter*2)
            counter += 1
        return listResults

    def DoYield(self):
        counter, maxCounter = 0, 5
        listResults = []
        while counter < maxCounter:
            print "tic"
            listResults.append(counter*2)
            yield listResults  #to return the entire list
            #yield listResults[counter] #you can only return one item at a time
            counter += 1
        return


generatorSample = GeneratorsSample()

I don't understand why, the Output for DoReturn() is different from that of DoYield(). For example,

returnResults = generatorSample.DoReturn()
for r in returnResults:
    print "toc", r

Outputs:

tic
tic
tic
tic
tic
toc 0
toc 2
toc 4
toc 6
toc 8

And,

yieldResults = generatorSample.DoYield()   
for r in yieldResults:      
    print "toc", r

Outputs:

tic
toc [0]
tic
toc [0, 2]
tic
toc [0, 2, 4]
tic
toc [0, 2, 4, 6]
tic
toc [0, 2, 4, 6, 8]

Answer 1

This is probably a better example:

class GeneratorsSample(object):
    def DoReturn(self):
        counter, maxCounter = 0, 5
        listResults = []
        while counter < maxCounter:
            print "tic"
            listResults.append(counter*2)
            counter += 1
        return listResults

    def DoYield(self):
        counter, maxCounter = 0, 5
        while counter < maxCounter:
            print "tic"
            yield counter*2
            counter += 1
        return


generatorSample = GeneratorsSample()

ret = generatorSample.DoReturn()
yld = generatorSample.DoYield()

for r in ret: print "toc", r
for r in yld: print "toc", r

print ret
print yld

First look at these lines:

for r in ret: print "toc", r
for r in yld: print "toc", r

The same values are generated, but in the "return" version, the tics all come first, then all of the tocs. In the "yield" version the tics and tocs are interspersed.

But the key difference between the two methods are illustrated by these lines:

print ret  # prints: [0, 2, 4, 6, 8]
print yld  # prints: <generator object DoYield at 0x0000000002202630>

Here, ret is the list of all the values generated. That is, when this assignment is made:

ret = generatorSample.DoReturn()

The entire list is generated then, and returned as the complete list.

With the generator approach, the entire list is not generated, in fact, no elements are computed. Just a reference to a generator that will produce elements "only the fly", as needed.

In other words, the "return" approach:

generates a number
generates a number
generates a number
...
uses that number
uses that number
uses that number
...

while the generator approach:

generates a number
uses that number
generates a number
uses that number
...

The efficiency of generators are in the fact that they only take the time generate a single element, as they are needed (if they are ever needed). If the maxCounter was, say, 1 million, and the computation was more complex than counter * 2, there would be a noticeable improvement in the amount of time it took for you to get your first output.

In fact, you can see that by adding artificial delays (here, with time.sleep(1):

import time

class GeneratorsSample(object):
    def DoReturn(self):
        counter, maxCounter = 0, 5
        listResults = []
        while counter < maxCounter:
            v = counter * 2
            time.sleep(1)
            print "[DoReturn] computed %d" % v
            listResults.append(v)
            counter += 1
        return listResults

    def DoYield(self):
        counter, maxCounter = 0, 5
        while counter < maxCounter:
            v = counter * 2
            time.sleep(1)
            print "[DoYield]  computed %d" % v
            yield counter*2
            counter += 1

        return


generatorSample = GeneratorsSample()

ret = generatorSample.DoReturn()
yld = generatorSample.DoYield()

for r in ret: print "[return loop] using", r
print("")
for r in yld: print "[yield loop]  using", r

The output being:

[DoReturn] computed 0
[DoReturn] computed 2
[DoReturn] computed 4
[DoReturn] computed 6
[DoReturn] computed 8
[return loop] using 0
[return loop] using 2
[return loop] using 4
[return loop] using 6
[return loop] using 8

[DoYield]  computed 0
[yield loop]  using 0
[DoYield]  computed 2
[yield loop]  using 2
[DoYield]  computed 4
[yield loop]  using 4
[DoYield]  computed 6
[yield loop]  using 6
[DoYield]  computed 8
[yield loop]  using 8