It turned out that SBCL compiler implements a lot of optimizations that allow us to enjoy quite fast generated code, but some of which require exponential time and memory resources.

I wonder what optimizations he's talking about. (declaim (optimize ...)) is pretty coarse, but I've never failed to convince the compiler not to do the thing I didn't want it to do before. And now there's sb-ext:restrict-compiler-policy and sb-ext:with-compilation-unit if you just don't want to litter your code.