R1 performing so well has me itching to finally try and run LLMs locally with Ollama (once I am over this flu I’ve got).

I do think we will see some remarkable nonincremental jump in performance in the next year or two, considering there are more and more vectors that can be used to increase performance. Model size and training data size were thought of as the only important ones, but just the last few months have shown larger context windows (Gemini 2), reasoning time (o1) and learning method (r1) to all amp up results. I’d be very surprised if we have hit a wall in all these dimensions and were unable to come up with new ones. I guess it depends on what one calls incremental progress.

Sidenote: unless I’m mistaken, the whole “they created a model on the cheap for 90% less training costs!” aspect of this is a bit misleading. What they did was to continue developing open source LLMs, which were trained at great expense by others, right? It’s still very impressive but it’s not like they started from scratch if I read the paper correctly.