26

I have a very simple example:

#!/usr/bin/env python

#a()  # 1: NameError: name 'a' is not defined
#b()  # 1: NameError: name 'b' is not defined
#c()  # 1: NameError: name 'c' is not defined

def a():
    c()   # note the forward use here...

#a()  #2: NameError: global name 'c' is not defined 
#b()  #2: NameError: name 'b' is not defined
#c()  #2: NameError: name 'c' is not defined

def b():
    a()

#a()   #3: NameError: global name 'c' is not defined    
#b()   #3: NameError: global name 'c' is not defined
#c()   #3: NameError: name 'c' is not defined

def c():
    pass

a()    # these all work OK...   
b()
c()

I have 3 functions named a(), b() and c() defined in a Python source file in alphabetical order. The body of each function definition is a call to one of the other functions. You can see by my comments that I have to have the initial call to the first of these functions BELOW their definitions (in the text file), but you do not necessarily need definition of a function above another function that calls it.

Certainly it seems to be common practice to have the first executable code below all the function definitions (in Python and many other languages), and now I can see why. In C and C++, header files take care of this. In Pascal you must have name definitions prior to their use.

Assume, for example, that you have this in Python:

def a(a_arg):          c(a_arg)
def b(b_arg):          a()
def c(a_arg,b_arg):    b(b_arg)
a(1)

It will fail properly with TypeError: c() takes exactly 2 arguments (1 given) at runtime where the other errors are compile time. (in C, this would compile then fail mysteriously...)

In Perl, since subroutines names are USUALLY resolved at run time, you can have Perl definitions and code in any order:

#!/usr/bin/env perl

a();
b();
c();

sub a{ c(); }
sub b{ a(); }
sub c{ return; }

In C, it is either an error or a warning (implementation dependent) to use a function that has not been prototyped and shouldn't be ignored.

You can have this:

void a(void) { c(); }   /* implicitly assumed to be int c(...) unless prototyped */
void b(void) { a(); }
void c(void) { return; }

int main(void) {
    a();
    return EXIT_SUCCESS;
}

My assumptions and confusion is this: If Python does not resolve subroutines names until runtime, why does the source compile phase fail with the forward declaration of subroutine names that have not been defined yet? Is it documented somewhere (other than by observation of other code) that you cannot have code in a source file above definitions of subroutines?

It seems that Python has elements of dynamic name resolution (the use of c() in a() prior to its definition below in the source file) and elements of static name resolution (the failure of Python to run the call to a() if placed above its definition in the source file.)

Is there a Python version of THIS DOCUMENT that covers the lifecycle of a Perl executable and how names are resolved between source file interpretation and runtime?

Is there a definitive description somewhere on the order of definitions for a Python script that states functions can have forward definitions of other subroutine names but main code cannot?

Edit and conclusion

After some spirited comments, and some research on my part, I have concluded that my question is really more about how names are resolved, and how namespaces, scopes and modules are defined in Python.

From carot-top:

"a callable must be defined before it is called in the current namespace." and this link on scopes and names

From S.Lott:

"When a name is used in a code block, it is resolved using the nearest enclosing scope." and this link to the execution life of a Python script.

From the Python documents:

"A scope defines the visibility of a name within a block." From the Python Execution model

"A module can contain executable statements as well as function definitions." in more about modules

"In fact function definitions are also ‘statements’ that are ‘executed’; the execution of a module-level function enters the function name in the module’s global symbol table." in the footnote thereto.

And my own realization (Duh!) that:

  1. Every Python source file is treated as a "module" by Python: "A module is a file containing Python definitions and statements."

  2. Unlike Perl (which I have more experience with) Python executes modules as they are being read. Hence the failure of a immediately executable statement referring to a function not yet defined in the same module.

12
  • "forward definitions"? What does that mean? Are you asking about forward "references"? The answer is trivially yes. I'm sorry, but the question confuses me. Commented Feb 10, 2011 at 0:16
  • I used "forward definition" because of the original question being about when and where def c() is used in relationship to def a() and in relationship to code that immediately executes. In C and Pascal, it would be called a "forward" or "external" declaration by a function prototype in a header or otherwise prior to use. In Python it seems to matter sometimes but not others. Once again: my question is where is the documentation that covers this? What can be resolved at runtime and what cannot be? I have read the Python web site docs. Commented Feb 10, 2011 at 17:24
  • @S.Lott: I always thought of "forward reference" as the term used in a two pass compiler which Python is not. Commented Feb 10, 2011 at 17:36
  • @S.Lott: You say the answer is "trivially yes." As an additional example of my question: why can you have def a(): return Global_B with Global_B being a forward reference / declaration to a statement below or elsewhere but you cannot have a forward reference to the statement that defines a function outside of a function? That is not "trivial." It is either a random side effect or formally defined in the language specification in a document that I have not read. Commented Feb 10, 2011 at 17:39
  • @drewk: I'm not sure what more you could be looking for. Name resolution happens when the name is encountered. There cannot be a "forward" references because there are no "declarations" in any form. Your question is very confusing because it uses non-Python concepts like "declaration" and "forward". Commented Feb 10, 2011 at 18:22

5 Answers 5

27

The order of definitions is simply "everything has to be defined before you call it". That's pretty much it.

edit (to include answer in comments, elucidated):

The reason something like

def call_a():
    a()

def a():
    pass

call_a()

works when you've got a() in call_a() before a is even defined as a function is because Python only actually looks up values for symbols on an as-needed basis. When call_a is evaluated, the a() call is basically stored as bytecode instructions to "look up what a is and call it" when the time comes, which isn't until you get down to the actual invocation of call_a() at the bottom.

Here is what the disassembled bytecode of call_a looks like (via dis.dis):

Disassembly of call_a:
  2           0 LOAD_GLOBAL              0 (a)
              3 CALL_FUNCTION            0
              6 POP_TOP
              7 LOAD_CONST               0 (None)
             10 RETURN_VALUE

So basically, when you hit call_a, it loads whatever is stored as a onto the stack, calls it as a function, and then pops the return value off before returning None, which is what implicitly happens for anything that doesn't explicitly return (call_a() is None returns True)

Sign up to request clarification or add additional context in comments.

13 Comments

But the example shows otherwise. You can call functions below your code so long as the main body has not executed. The definition of a() with c() below would fail in C.
Yes, you are not actually invoking any of the functions until after they are defined. At the time it evaluates the function definition it just says "okay, I'll look up c in the symbol table when someone calls me" but at that moment it doesn't matter.
@drewk: I guess you're confused by the fact that the body of a function doesn't matter until it's executed. You should, perhaps, review the Python language reference. The "definitive description" is trivial -- nothing matters until it's actually evaluated.
@S.Lott: Believe me, I did RTFM! I cannot find in the TFM these things: 1) that there are no forward definitions or prototypes in Python; 2) Python is kinda sorta evaluated top to bottom but the Real Rule is "define it before you call it" as Daniel DiPaolo stated. In C, Pascal, Perl, C++, Objective C, etc. the concept of name resolution at compile time and run-time are very formal. I have not seen that same treatment for Python. Hence the question.
@Daniel DiPaolo: Thanks! Your edited question (and some other SO posts) cleared it up enough for me to now worry about it any more.
|
5

Following the various comments and struggling to understand some Perl concepts from a Python history, let me take a crack at this. Please reset your brain on some of things you learned in Perl. They do not apply in Python. (And vs vs...)

There are no forward declarations in Python. None. Technically, ALL functions are anonymous objects; they just happen to be bound to the name you used to define it. You can re-bind them at will.

The dictionary of these functions can be found using the locals() function like this:

>>> def a(): b()
... 
>>> locals()['a']
<function a at 0x100480e60>
>>> locals()['b']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'b'
>>> a()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in a
NameError: global name 'b' is not defined

If Python required b() to be defined BEFORE writing a(), this would be a problem in the Python interpreter. You would need to write all you functions in strict order.

Since all builtin function names are just bounded names, you can easily override builtin functions:

>>> abs(-1)
1
>>> def abs(num): print "HA Fooled you!!!"
... 
>>> abs(-1)
HA Fooled you!!!
>>> abs=__builtins__.abs
>>> abs(-1)
1

It is much more difficult (but possible) to override built-ins in Perl. (The disadvantage here is a mistyped def [builtin]: can unintentionally override the built-in function without warning)

The best description I can refer you to for names and scope in Python is actually the tutorial on Classes -- section 9.2

There is not actually chapter and verse why def need to come before executable code because this is not a true statement. Consider:

#!/usr/bin/env python

def fake_a(): print " a fake"
a=fake_a
a()  
def a():  print "this is a()"
a()

Or even:

def a(): print " first a()"
a()  
def a():  print "second a()"
a()

What is true is that a callable must be defined before it is called in the current namespace. Hence, usually, above the point of being called by executable code in a source text file or module. Each function has its own namespace; calls to other functions not yet defined only fail when that function is called in that local and executing namespace -- when the callable is called from the namespace of the function. This why you can have what looks like a "forward declaration" in your example. The module call of a "forward" callable outside a function fails because the function def has not been executed yet so it is not in the current namespace.

1 Comment

I think you are right. I was thinking that the ability to refer to a function not yet defined implied some sort of forward reference ability. It is more a "rolling namespace"
3

This is exactly the same as in C/C++ etc. You can't use something until it exists. In C/C++ you can't refer to something until it has been declared. Remember that a Python file is processed from top to bottom so if you attempt to call a function or reference a variable that does not exist, then it fails.

2 Comments

"This is exactly the same as in C/C++" Well no it is isn't at all really. You can have a forward definition of a function in C but you cannot in Python. You can have run-time checking of functions in Python but cannot in C. You can have implicit prototypes in C and that is similar.
@drewk In other words, C/C++ are different from Python. Yes, I can't deny that. You are taking me too literally. You can't name a symbol in C/C++ until it has been declared. That's all I'm saying.
1

http://docs.python.org/reference/executionmodel.html


When a name is used in a code block, it is resolved using the nearest enclosing scope. The set of all such scopes visible to a code block is called the block’s environment.

The following constructs bind names: formal parameters to functions, import statements, class and function definitions (these bind the class or function name in the defining block), ...

That's it. There's no concept of "forward" or "reverse" or "declaration" or anything like that.

4 Comments

@drewk: "Believe me, I did RTFM!". I'm puzzled how you could claim that and yet, somehow, not have found this.
While I said this was "helpful" it still does not answer the question I asked about why Python code can refer to an undefined subroutine inside a subroutine but not in the main code. Please point to where in that link it says that. You seem after me about this question. I have been respectfully trying to clarify. What's up? Seriously? I asked a question that compared what I saw in Python to what I have seen in other languages. You have been berating me about it. Why? Is SO just going to erode to a place where the knowledgeable berate those with less knowledge to go read the docs?
@drewk: "berating"? I'm puzzled because I'm not able to transfer concepts among languages. It puzzles me and I can't understand the question. I'd like to answer, but I can't understand it because it seems to involve concepts (like "declaration") that I can't apply to Python. I'm trying to help, but I just can't understand the question.
I am very thankful for the help you have given to me and others here. Comments like You should, perhaps, review the Python language reference. and trivially yes and Continuing to demand a Python document that magically matches a document for an utterly different language and I'm puzzled how you could claim that [you RTFM] and yet, somehow, not have found this can come across as condescending, berating and frustrating to someone on the steep part of the learning curve. It takes lots of time and work to to master all this. What is obvious to you might not be to others.
0

I know this question has been answered for quite a while now, but I also had trouble understanding this and would like to demonstrate how I found the answer with a simple example:

def a():
     print globals()
     b()
a()

In the example above, globals would display the location for the function that was defined with the name b if your main executable code defined a function with the name b. If not, the globals would be empty and call to a would result in

NameError: global name 'b' is not defined  

such as in the example of the wolf. What this means is that while function names need to be declared before they are called for them to exist in the namespace of the main executing code, the function that the main executing code calls already have the definitions found in main which is why function calls from functions need not follow a strict order.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.