Guile 3 is interesting because of the JIT. They've been working towards it for over 10 years. Guile is made for embedding in applications written in other languages, particularly C, the way emacs lisp is embedded in emacs. It wants to integrate more tightly with the application its embedded in than Python or Lua do, and that imposes constraints that have historically made it very slow.

But you're asking about lisp in general. I'll talk about Common Lisp instead of Scheme, because I know it better.

So every dynamic language has a thing called a repl, but it's not the same thing as the lisp repl. It's only a little facitious to say this is a complete Common Lisp implementation (that only works with sbcl):

    (sb-ext:save-lisp-and-die "dumblisp" :toplevel #'(lambda () (loop (print (eval (read)))) :executable t)

read with no arguments reads the next lisp object from standard input. Type (+ 2 2) and you get a list containing the symbol '+', the integer 2, and the integer 2.

eval takes a lisp object and, well, evaluates it. The language isn't defined in terms of text, it's defined in terms of how eval interprets its argument. (eval (list '+ 2 2)) is equivalent to typing (+ 2 2) at the repl, (eval "(+ 2 2)") is not.

A corollary of that is transforming programs is just manipulating lists, and you can teach eval new transformations. So, if you really miss while loops from some other language, you can write

    (defmacro while (test &body body) `(do () ((not ,test)) ,@body))

And now

    (let ((i 0)) (while (< i 10) (incf i) (format t "~a~%" i)))

works like while in C. You've added while loops to your lisp. That's how most control structures are implemented. If you expand that expression all the way (unfortunately there's no builtin function to do it in CL) you'll get something involving TAGBODY, which is equivalent to a set of labels and gotos in C and translates straightforwardly into jump instructions when passed to the compiler.

Common Lisp's support for object oriented programming, the Common Lisp Object System, was originally implemented as a library using the same mechanism, and in fact most implementations just incorporated the library with minor optimizations rather than modifying their compilers when it was incorporated into the standard. Here's an implementation of Prolog in around 400 lines.

There's other stuff of course. A lot of us just like the syntax. For greybeards it's synonymous with functional programming. Image-based persistence is really cool. A high-level language that can be about as efficient as C if you care enough to give it hints is handy for projects where that matters. Convenient metaprogramming is the feature that really distinguishes lisps from other languages now though, things like garbage collection and strong but dynamic typing started with lisp but have since become ubiquitous.

If you're interested, Practical Common Lisp and Paradigms of Artificial Intelligence Programming (in that order!) are both great books to start with.