Closure bug

Nesting block closures with block temporaries and accessing them may crash the VM or return unexpected objects.

Example #1 that crashes the VM:

[     | var1 |
 
      var1 := 'AB'.
 
      [     | var2 |
 
            var2 := 'CD'.
 
            [     | var3 |
 
                  var3 := 'EF'.
 
                  [ var1, var2, var3 ] value.
 
            ] value.
 
      ] value.
 
] value.

Example #2 – a block temp is no string any more:

[    | var1 |
 
      var1 := 'AB'.
 
      [     | var2 |
 
            var2 := 'CD'.
 
            [     | var3 |
 
                  var3 := 'EF'.
 
                  [ var1, var3, var2 ] value.
 
            ] value.
 
      ] value.
 
] value.

Explanation:

The bug is present in the nativization for bytecode 252, IndirectEscape. This bytecode has three sub-bytecodes we can say, for Pushing, Loading and Storing closure/environment variables. What makes sense, as environment variables are only used in blocks referring variables defined in an outer scope.

The bug is due to a desintelligence between the Smalltalk encoder (DefaultEncoder) and the JIT nativizer itself. Each of the 3 subbytecodes has two different encodings, one encoding for blocks deep up to 3 nesting levels, and a different encoding for deeper nesting levels. The problem shows up for deeper nesting levels, because the encoder assumes a 0-based count for nesting levels, and the JIT nativizer assumes a 1-based count for nesting levels.

Solution:

option 1: fix the encoder – the method Encoder>>#putCode:min:max:range: should be implemented like this

putCode: code min: min max: max range: idx
    (idx >= min and: [idx <= max])
        ifTrue: [1/>self putNext: code - max + idx - 1]
        ifFalse: [
            1/>self putNext: code.
            1/>self putIndex: idx + 1]

option 2: use the VMPatcher from Source code for VS-Smalltalk to fix this bug.

This bug was reported by Thomas Muhr on 2010.08.09. The explanation and solution was developed by Gerardo Richarte.