Virtual Machine stack overflow

When does a Virtual Machine stack overflow happen?

Usually you will get a stack overflow if you write recursive code that does not terminate correctly.

If the VM detects a stack overflow, an internal flag is being set.

The stack overflow triggers a walkback, which calls Process class>>#queueWalkback:makeUserIF:resumable:.

#queueWalkback:makeUserIF:resumable: will call Process>>#dropSenderChain during the error processing.

#dropSenderChain will clear the stack and will set up a new stack (process). It also clears the internal flag, that has been set during a stack overflow.

If an error occurs before the #dropSenderChain has been done, additional stack frames must be created and additional stack overflow checks will occur. If the internal “stack overflow” flag is still set, a virtual machine stack overflow will appear.

One possible reason that will trigger a virtual machine stack overflow are the senders of #osError in CursorManager. If your app does a lenghty computation and the Windows screensaver will be activated, many cursor related API calls will cause an OS error.

During the error handling in #queueWalkback:makeUserIF:resumable: “CursorManager normal change” will cause further walkbacks causing the stack to fill until the stack overflow will be triggered.