Python Scope Declarations: Implicit, Global and Nonlocal

in python

Python provides 3 different scoping declarations for variables: implicit (default) scope, global and nonlocal. Let's explore their semantics and differences.

Assignment vs. Declaration

Python's assignment operator (=) is used in two different meanings: it performs both assignment to existing variables and initialization of new variables. There is no explicit new variable declaration keyword in the language.

Understanding the different semantics is necessary if we want to understand Python's scope rules. Here's the same operator with its two different meanings:

# Declaration and initialization of new variables to new data
x = 10
y = [1, 2, 3]
z = 'hello world'

# Assigning new values to existing variables
x = 50
y = { 'a': 10 }
z = 'bye bye'

Of course in the above example one could delete the first block and then the second block will be the one declaring the variables for the first time.

But the two are not always interchangable. Examine the following snippet:

x = [10, 20, 30]
x[0] = 'hello'

The second line could never declare a new array variable. Without the first, the second line will raise the error:

NameError: name 'x' is not defined

Assigning new values to variables will declare them. Modifying existing data structure requires the data to be declared beforehand.

Implicit (default) Scope Declaration

Declaring a variable in a function creates a new variable scoped to the function. Declaring a variable outside any function creates it in the global scope.

Global variable declarations:

x = 10
y = 20

# x = 10
print("x = ",x)

# y = 20
print("y = ",y)

Function variable declarations:

def foo():
    x = 10
    print(x)

# NameError: name 'x' is not defined
print(x)

Now this begs the question what happens if the same name is used for both a function variable and a global variable. The answer depends on the usage:

  1. Declaring a variable inside a function creates a new variable that hides the global name. Example:
x = 10

def foo():
    x = 20
    x += 1
    # prints 21
    print(x)

foo()
# prints 10 - variable is not affected by the function
print(x)

Note that it doesn't matter where in the function the variable is declared (even in unreachable code). This next program raises an error:

x = 10

def foo():
    print(x)
    if False:
        x = 10

foo()
print(x)

But if we delete the entire if False block it will begin working.

  1. Assigning to a global variable or reading its value from within a function will modify or show the value of a global variable, if no local is declared in the same name. Example:
x = [1,2,3]

def foo():
    x[0] = 10
    # prints [10, 2, 3]
    print(x)

foo()

# prints [10, 2, 3]
print(x)

Scope Quiz

Try to guess what the next program prints and why:

funcs = {
        idx: lambda: print(idx) for idx in range(4)
        }

funcs[0]()

The program creates a dictionary of indicses and functions, where each function's code prints the index. One might assume the last line prints 0, but that's incorrect.

The program is roughly equivalent to the longer code:

idx = 0
def f1():
    print(idx)

idx = 1
def f2():
    print(idx)

# prints: 1
f1()

Both functions refer to the same global variable idx, and as its value changes their behaviour is changed accordingly. The last line in the quiz will therefore print 3.

The keyword global

There may be a need to modify a global variable from within a function. Consider the next example:

_total = 0

def add(x):
    _total += x

# Error: UnboundLocalError: local variable '_total' referenced before assignment
add(1)
add(2)
add(4)

The program fails because the function includes a declaration, so _total inside the function is a new variable that is referenced before assignment.

One workaround that would get us the same functionality is move to object oriented style and write the following using classes:

class Accumulator:
    def __init__(self):
        self.total = 0

    def add(self, x):
        self.total += x

a = Accumulator()
a.add(1)
a.add(2)
a.add(4)

# This works. Prints 7
print(a.total)

Alternatively we could use an already existing data structure such as an array or dictionary to hold the value. In all cases you are required to change your code because of a limitation of the language.

The python solution is the keyword global. That keyword binds a local name to the global namespace. If no global variable in that name exists, it is created by the global declaration. Following a global statement, all assignments to the variable will affect the global one.

This is how we can use global to fix the previous program:

_total = 0

def add(x):
    global _total
    _total += x

add(1)
add(2)
add(4)

# OK - prints 7
print(_total)

The keyword nonlocal

The two scopes mentioned above can create confusion when we declare nested functions. Consider the following snippet:

def calc(x):
    def twice():
        return x * 2

    return twice() + 5

# Prints 25
print(calc(10))

A nested python function has access to another scope: That of its containing function. This scope is not its local scope but it's also not the global scope.

As the above example shows, reading or modifying values from that outer scope is allowed. Python will automatically search the name in the local scope, and if it's not found continue searching for it in all outer scopes until it reaches global scope.

However changing variables in outer scope will have the same limitations as we saw in global. This next example works:

def calc(x):
    z = [10]
    def twice():
        z[0] *= 20

    twice()
    twice()
    twice()
    return z[0] + 10

print(calc(10))

Because it changes an existing array data (assignment), but this one fails:

def calc(x):
    z = 10
    def twice():
        # UnboundLocalError: local variable 'z' referenced before assignment
        z *= 20

    twice()
    twice()
    twice()
    return z + 10

print(calc(10))

Because the internal function now includes a declaration.

The global keyword won't fix this problem as the variable is not declared in the global scope, so a new keyword is required. This is nonlocal. Declaring a variable nonlocal causes Python to search it up in outer scopes until it is found (or raise an exception if none is found).

Thus this code snippet also works:

def calc(x):
    z = 10
    def twice():
        nonlocal z
        z *= 20

    twice()
    twice()
    twice()
    return z + 10

# Prints 80010
print(calc(10))

Notes & Extra Reading

  1. The keyword nonlocal is a new addition made in Python3. Python2 developers still need to use the workarounds described above (i.e. using object oriented syntax or data structures).
  2. The document PEP 3104 describes nonlocal keyword spec.
  3. Toptal has made a list of
    10 most common python mistakes
    . The scope rules described here cover 2 of them, but you should really read their whole piece and make sure you know all 10.

Comments