[ACCEPTED]-How do lexical closures work?-python-closures

Python is actually behaving as defined. Three separate functions are 11 created, but they each have the closure of the environment they're defined in - in this 10 case, the global environment (or the outer 9 function's environment if the loop is placed 8 inside another function). This is exactly 7 the problem, though - in this environment, i is mutated, and 6 the closures all refer to the same i.

Here is the best solution 5 I can come up with - create a function creater 4 and invoke that instead. This will force different environments for 3 each of the functions created, with a different i in 2 each one.

flist = []

for i in xrange(3):
    def funcC(j):
        def func(x): return x * j
        return func
    flist.append(funcC(i))

for f in flist:
    print f(2)

This is what happens when you mix 1 side effects and functional programming.

The functions defined in the loop keep accessing 11 the same variable i while its value changes. At 10 the end of the loop, all the functions point 9 to the same variable, which is holding the 8 last value in the loop: the effect is what 7 reported in the example.

In order to evaluate 6 i and use its value, a common pattern is 5 to set it as a parameter default: parameter 4 defaults are evaluated when the def statement 3 is executed, and thus the value of the loop 2 variable is frozen.

The following works as 1 expected:

flist = []

for i in xrange(3):
    def func(x, i=i): # the *value* of i is copied in func() environment
        return x * i
    flist.append(func)

for f in flist:
    print f(2)

Here's how you do it using the functools library 3 (which I'm not sure was available at the 2 time the question was posed).

from functools import partial

flist = []

def func(i, x): return x * i

for i in xrange(3):
    flist.append(partial(func, i))

for f in flist:
    print f(2)

Outputs 0 2 1 4, as expected.

look at this:

for f in flist:
    print f.func_closure


(<cell at 0x00C980B0: int object at 0x009864B4>,)
(<cell at 0x00C980B0: int object at 0x009864B4>,)
(<cell at 0x00C980B0: int object at 0x009864B4>,)

It means they all point to 3 the same i variable instance, which will 2 have a value of 2 once the loop is over.

A 1 readable solution:

for i in xrange(3):
        def ffunc(i):
            def func(x): return x * i
            return func
        flist.append(ffunc(i))

What is happening is that the variable i 26 is captured, and the functions are returning 25 the value it is bound to at the time it 24 is called. In functional languages this 23 kind of situation never arises, as i wouldn't 22 be rebound. However with python, and also 21 as you've seen with lisp, this is no longer 20 true.

The difference with your scheme example 19 is to do with the semantics of the do loop. Scheme 18 is effectively creating a new i variable 17 each time through the loop, rather than 16 reusing an existing i binding as with the 15 other languages. If you use a different 14 variable created external to the loop and 13 mutate it, you'll see the same behaviour 12 in scheme. Try replacing your loop with:

(let ((ii 1)) (
  (do ((i 1 (+ 1 i)))
      ((>= i 4))
    (set! flist 
      (cons (lambda (x) (* ii x)) flist))
    (set! ii i))
))

Take 11 a look here for some further discussion of this.

[Edit] Possibly 10 a better way to describe it is to think 9 of the do loop as a macro which performs 8 the following steps:

1. Define a lambda taking a single parameter (i), with a body defined by the body of the loop,
2. An immediate call of that lambda with appropriate values of i as its parameter.

ie. the equivalent to 7 the below python:

flist = []

def loop_body(i):      # extract body of the for loop to function
    def func(x): return x*i
    flist.append(func)

map(loop_body, xrange(3))  # for i in xrange(3): body

The i is no longer the 6 one from the parent scope but a brand new 5 variable in its own scope (ie. the parameter 4 to the lambda) and so you get the behaviour 3 you observe. Python doesn't have this implicit 2 new scope, so the body of the for loop just 1 shares the i variable.

The problem is that all of the local functions 4 bind to the same environment and thus to 3 the same i variable. The solution (workaround) is 2 to create separate environments (stack frames) for 1 each function (or lambda):

t = [ (lambda x: lambda y : x*y)(x) for x in range(5)]

>>> t[1](2)
2
>>> t[2](2)
4

I'm still not entirely convinced why in 11 some languages this works one way, and in 10 some another way. In Common Lisp it's like 9 Python:

(defvar *flist* '())

(dotimes (i 3 t)
  (setf *flist* 
    (cons (lambda (x) (* x i)) *flist*)))

(dolist (f *flist*)  
  (format t "~a~%" (funcall f 2)))

Prints "6 6 6" (note that here the 8 list is from 1 to 3, and built in reverse"). While 7 in Scheme it works like in Perl:

(define flist '())

(do ((i 1 (+ 1 i)))
    ((>= i 4))
  (set! flist 
    (cons (lambda (x) (* i x)) flist)))

(map 
  (lambda (f)
    (printf "~a~%" (f 2)))
  flist)

Prints "6 6 4 2"

And as I've mentioned already, Javascript 5 is in the Python/CL camp. It appears there 4 is an implementation decision here, which 3 different languages approach in distinct 2 ways. I would love to understand what is 1 the decision, exactly.

The variable i is a global, whose value is 10 2 at each time the function f is called.

I 9 would be inclined to implement the behavior 8 you're after as follows:

>>> class f:
...  def __init__(self, multiplier): self.multiplier = multiplier
...  def __call__(self, multiplicand): return self.multiplier*multiplicand
... 
>>> flist = [f(i) for i in range(3)]
>>> [g(2) for g in flist]
[0, 2, 4]

Response to your update: It's not the 7 globalness of i per se which is causing this behavior, it's 6 the fact that it's a variable from an enclosing 5 scope which has a fixed value over the times 4 when f is called. In your second example, the 3 value of i is taken from the scope of the 2 kkk function, and nothing is changing that 1 when you call the functions on flist.

The reasoning behind the behavior has already 9 been explained, and multiple solutions have 8 been posted, but I think this is the most 7 pythonic (remember, everything in Python 6 is an object!):

flist = []

for i in xrange(3):
    def func(x): return x * func.i
    func.i=i
    flist.append(func)

for f in flist:
    print f(2)

Claudiu's answer is pretty 5 good, using a function generator, but piro's 4 answer is a hack, to be honest, as it's 3 making i into a "hidden" argument with a 2 default value (it'll work fine, but it's 1 not "pythonic").

I didn't like how solutions above created 1 wrappers in the loop. Note: python 3.xx

flist = []

def func(i):
    return lambda x: x * i

for i in range(3):
    flist.append(func(i))

for f in flist:
    print f(2)