How does Python referencing work?

Question

I am confused with Python referencing. Consider the following example:

My task : To edit each element in the list

d = { 'm': [1,2,3] }
m = d['m']
m = m[1:]   # m changes its reference to the new sliced list, edits m but not d (I wanted to change d)

Similarly:

d = { 'm': [1,2,3] }
m = d['m']
m = m[0]    # As per python referencing, m should be pointing to d['m'] and should have edited d

In python everything goes by reference, then when is a new object created? Do we always need copy and deepcopy from copy module to make object copies?

Please clarify.

Answer 1

In Python a variable is not a box that holds things, it is a name that points to an object. In your code:

• d = { 'm': [1,2,3] } --> binds the name d to a dictionary
• m = d['m'] --> binds the name m to a list
• m = m[1:] --> binds the name m to another list

Your third line is not changing m itself, but what m is pointing to.

To edit the elements in the list what you can do is:

m = d['m']
for i, item in enumerate(m):
    result = do_something_with(item)
    m[i] = result

Answer 2

Ethan Furman did an excellent job of explaining how Python internals work, I won't repeat it.

Since m really does represent the list inside the dictionary, you can modify it. You just can't reassign it to something new, which is what happens when you use = to equate it to a new slice.

To slice off the first element of the list for example:

>>> m[0:1] = []
>>> d
{'m': [2, 3]}

Answer 3

In python everything goes by reference

In Python, everything is a reference, and the references get passed around by value.

If you want to use those terms. But those terms make things harder to understand.

Much simpler: in Python, a variable is a name for an object. = is used to change what object a name refers to. The left-hand side can refer to part of an existing object, in which case the whole object is changed by replacing that part. This is because the object, in turn, doesn't really contain its parts, but instead contains more names, which can be caused to start referring to different things.

then when is a new object created ?

Objects are created when they are created (by using the class constructor, or in the case of built-in types that have a literal representation, by typing out a literal). I don't understand how this is relevant to the rest of your question.

m = m[1:]   # m changes its reference to the new sliced list

Yes, of course. Now m refers to the result of evaluating m[1:].

edits m but not d (I wanted to change d)

Yes, of course. Why would it change d? It wasn't some kind of magic, it was simply the result of evaluating d['m']. Exactly the same thing happens on both lines.

Let's look at a simpler example.

m = 1
m = 2

Does this cause 1 to become 2? No, of course not. Integers are immutable. But the same thing is happening: m is caused to name one thing, and then to name another thing.

Or, another way: if "references" were to work the way you expect, then the line m = m[1:] would be recursive. You're expecting it to mean "anywhere that you see m, treat it as if it meant m[1:]". Well, in that case, m[1:] would actually mean m[1:][1:], which would then mean m[1:][1:][1:], etc.