[ACCEPTED]-Dynamic and Lexical variables in Common Lisp-lexical-scope

What's going on?

You say: feel like there is nothing special is going on here. The outer foo in bar increments the global x, and foo surrounded by let in bar increments the shadowed x. What's the big deal?

The special that's going on here is that 42 LET can shadow the value of *x*. With lexical variables 41 that's not possible.

The code declares *x* to 40 be special via the DEFVAR.

In FOO now the value of *x* is looked 39 up dynamic. FOO will take the current dynamic binding of *x* or, if 38 there is none, the symbol value of the symbol 37 *x*. A new dynamic binding can, for example, be introduced 36 with LET.

A lexical variable on the other hand 35 has to be present in the lexical environment 34 somewhere. LET, LAMBDA, DEFUN and others can introduce 33 such lexical variables. See here the lexical 32 variable x introduced in three different 31 ways:

(let ((x 3))
  (* (sin x) (cos x)))

(lambda (x)
  (* (sin x) (cos x)))

(defun baz (x)
  (* (sin x) (cos x)))

If our code were:

(defvar x 0)

(let ((x 3))
  (* (sin x) (cos x)))

(lambda (x)
  (* (sin x) (cos x)))

(defun baz (x)
  (* (sin x) (cos x)))

Then X were special in all 30 three above cases, because of the DEFVAR declaration, which 29 declares X to be special - globally for all levels. Because 28 of this, there is the convention to declare 27 special variables as *X*. Thus only variables with 26 stars around them are special - by convention. That's a useful 25 convention.

In your code you have then:

(defun bar ()
  (foo)
  (let ((*x* 20))
    (foo))
  (foo))

Since 24 *x* has be declared special via the DEFVAR above in your 23 code, the LET construct introduces a new dynamic binding for 22 *x*. FOO is then called. Since inside FOO the *x* uses 21 dynamic binding, it looks up the current one and finds 20 that *x* is dynamically bound to 20.

The value 19 of a special variable is found in the current dynamic 18 binding.

Local SPECIAL declarations

There are also local special declarations:

(defun foo-s ()
  (declare (special *x*))
  (+ *x* 1))

If 17 the variable had been declared special by a DEFVAR or 16 DEFPARAMETER, then the local special declaration can be omitted.

A 15 lexical variable directly references the 14 variable binding:

(defun foo-l (x)
  (+ x 1))

Let's see it in practice:

(let ((f (let ((x 10))
           (lambda ()
             (setq x (+ x 1))))))
  (print (funcall f))    ; form 1
  (let ((x 20))          ; form 2
    (print (funcall f))))

Here all variables are 13 lexical. In form 2 the LET will not shadow the X in 12 our function f. It can't. The function uses 11 the lexical bound variable, introduced by 10 the LET ((X 10). Surrounding the call with another 9 lexically bound X in form 2 has no effect on our 8 function.

Let's try special variables:

(let ((f (let ((x 10))
           (declare (special x))
           (lambda ()
             (setq x (+ x 1))))))
  (print (funcall f))    ; form 1
  (let ((x 20))          ; form 2
    (declare (special x))
    (print (funcall f))))

What now? Does 7 that work?

It does not!

The first form calls the function 6 and it tries to look up the dynamic value 5 of X and there is none. We get an error in 4 form 1: X is unbound, because there is no dynamic 3 binding in effect.

Form 2 would work, since the 2 LET with the special declaration introduces a dynamic 1 binding for X.

When a variable is lexically scoped, the system looks to 16 where the function is defined to find the value 15 for a free variable. When a variable is 14 dynamically scoped, the system looks to where the function 13 is called to find the value for the free variable. Variables 12 in Common Lisp are all lexical by default; however, dynamically 11 scoped variables can be defined at the top 10 level using defvar or defparameter.

A simpler example

lexical scoping (with setq):

(setq x 3)

(defun foo () x)

(let ((x 4)) (foo)) ; returns 3

dynamic scoping (with defvar):

(defvar x 3)

(defun foo () x)

(let ((x 4)) (foo)) ; returns 4

How does the let know if 9 a variable is lexical or dynamic? It doesn't. On the 8 other hand, when foo goes to find the value 7 of X, it will initially find the lexical 6 value defined at the top level. It then 5 checks to see if the variable is supposed 4 to be dynamic. If it is, then foo looks 3 to the calling environment, which, in this 2 case, uses let to overshadow the value of 1 X to be 4.

(note: this is an oversimplification, but it will help to visualize the difference between the different scoping rules)

Maybe this example will help.

;; the lexical version

(let ((x 10)) 
  (defun lex-foo ()
    (format t "Before assignment~18tX: ~d~%" x)
    (setf x (+ 1 x))
    (format t "After assignment~18tX: ~d~%" x)))

(defun lex-bar ()
  (lex-foo)
  (let ((x 20)) ;; does not do anything
    (lex-foo))
  (lex-foo))

;; CL-USER> (lex-bar)
;; Before assignment X: 10
;; After assignment  X: 11
;; Before assignment X: 11
;; After assignment  X: 12
;; Before assignment X: 12
;; After assignment  X: 13

;; the dynamic version

(defvar *x* 10)
(defun dyn-foo ()
  (format t "Before assignment~18tX: ~d~%" *x*)
  (setf *x* (+ 1 *x*))
  (format t "After assignment~18tX: ~d~%" *x*))

(defun dyn-bar()
  (dyn-foo)
  (let ((*x* 20))
    (dyn-foo))
  (dyn-foo))

;; CL-USER> (dyn-bar)
;; Before assignment X: 10
;; After assignment  X: 11
;; Before assignment X: 20
;; After assignment  X: 21
;; Before assignment X: 11
;; After assignment  X: 12

;; the special version

(defun special-foo ()
  (declare (special *y*))
  (format t "Before assignment~18tX: ~d~%" *y*)
  (setf *y* (+ 1 *y*))
  (format t "After assignment~18tX: ~d~%" *y*))

(defun special-bar ()
  (let ((*y* 10))
    (declare (special *y*))
    (special-foo)
    (let ((*y* 20))
      (declare (special *y*))
      (special-foo))
    (special-foo)))

;; CL-USER> (special-bar)
;; Before assignment X: 10
;; After assignment  X: 11
;; Before assignment X: 20
;; After assignment  X: 21
;; Before assignment X: 11
;; After assignment  X: 12

0

You can tell your Lisp to bind local variables 1 dynamically, too:

(let ((dyn 5))
  (declare (special dyn))
  ... ;; DYN has dynamic scope for the duration of the body
  )

Rewrite example from PCL.

;;; Common Lisp is lexically scoped by default.

λ (setq x 10)
=> 10

λ (defun foo ()
    (setf x (1+ x)))
=> FOO

λ (foo)
=> 11

λ (let ((x 20))
    (foo))
=> 12

λ (proclaim '(special x))
=> NIL

λ (let ((x 20))
    (foo))
=> 21

Yet another great 54 explanation from On Lisp, chapter 2.5 Scope:

Common 53 Lisp is a lexically scoped Lisp. Scheme 52 is the oldest dialect with lexical scope; before 51 Scheme, dynamic scope was considered one 50 of the defining features of Lisp.

The difference 49 between lexical and dynamic scope comes 48 down to how an implementation deals with 47 free variables. A symbol is bound in an 46 expression if it has been established as 45 a variable, either by appearing as a parameter, or 44 by variable-binding operators like let and 43 do. Symbols which are not bound are said 42 to be free. In this example, scope comes 41 into play:

(let ((y 7)) 
  (defun scope-test (x)
  (list x y)))

Within the defun expression, x 40 is bound and y is free. Free variables are 39 interesting because it’s not obvious what 38 their values should be. There’s no uncertainty 37 about the value of a bound variable—when 36 scope-test is called, the value of x should 35 be whatever is passed as the argument. But 34 what should be the value of y? This is the 33 question answered by the dialect’s scope 32 rules.

In a dynamically scoped Lisp, to find 31 the value of a free variable when exe- cuting 30 scope-test, we look back through the chain 29 of functions that called it. When we find 28 an environment where y was bound, that binding 27 of y will be the one used in scope-test. If 26 we find none, we take the global value of 25 y. Thus, in a dynamically scoped Lisp, y 24 would have the value it had in the calling 23 expression:

> (let ((y 5)) (scope-test 3))
    (3 5)

With dynamic scope, it means 22 nothing that y was bound to 7 when scope-test 21 was defined. All that matters is that y 20 had a value of 5 when scope-test was called.

In 19 a lexically scoped Lisp, instead of looking 18 back through the chain of calling functions, we 17 look back through the containing environments 16 at the time the function was defined. In 15 a lexically scoped Lisp, our example would 14 catch the binding of y where scope-test 13 was defined. So this is what would happen 12 in Common Lisp:

> (let ((y 5)) (scope-test 3))
    (3 7)

Here the binding of y to 11 5 at the time of the call has no effect 10 on the returned value.

Though you can still 9 get dynamic scope by declaring a variable 8 to be special, lexical scope is the default 7 in Common Lisp. On the whole, the Lisp community 6 seems to view the passing of dynamic scope 5 with little regret. For one thing, it used 4 to lead to horribly elusive bugs. But lexical 3 scope is more than a way of avoiding bugs. As 2 the next section will show, it also makes 1 possible some new programming techniques.