I am very excited to be back and writing again, and boy do I have things to say.
So Jev, everyone is talking about it, and I am seeing a TON of comparisons to Laya (a small local model) and other classes of models now popping up. Most of them are interesting and cool that they are small and local and open, but I have opinions about this new class of "System One" models.
First off, what is Jev and what is System One. Well if LLMs think and make decisions based on those thoughts, they have to tell you about them, but we don't really get to see the decisions they made unless they say them out loud (out loud in their head?). What System One means is that you only get the decision, you don't get the words and the explanation that is accompanied by it. Meaning you cannot ask it open-ended questions. It's either true/false, or multiple choice. I believe that System One is a reference to "Thinking, Fast and Slow" by Daniel Kahneman where System One is your initial and fast thought to a question or a problem. System Two is when you actually use brain power and have to truly "think" to work through something to get the answer to the question. This is an over simplification but I think it conveys the point.
I think that yes it's cool and it's what anyone who is building systems with AI needs as a substitute for pure LLMs. The funny thing about this new class, is that it actually requires System Two thinking in order to use the bloody thing. In order to use it to its fullest extent, you must use well defined question/answer groupings. In TypeSafe's (maker of Jev) docs they say:
"System One models work best when each question asks one specific, well-scoped thing. Think of each question as a gut-check determination: the kind of judgment a highly knowledgeable person could make in a few seconds given the right context."
I like this because it actually brings cognitive thinking back into the equation. You have to know the possible answers ahead of time in most cases. Some issues that I see after playing with it for an hour or two, is that 1) I am realizing how much of my System Two I have offloaded to LLMs as I asked an LLM to write the questions and the answers and the harness for my testing 2) it has to be the right question with the best answers. Asking the right question is usually the hardest part. And my prompt to the LLM to help me evaluate the System One model was "could we ask a better question?".
TypeSafe has use cases on their app to show as examples. And really it's a decision maker at the end of the day. When we make a decision, we weigh things in our mind as to what the "best" decision is. We don't get to see the probability of why it's the best, but we do that, hundreds to thousands of times a day. You can't offload that here. And I like that.
Where I see arguments and new models popping up that I see people starting to make the WRONG assumption is that faster means better. Because it's System One, it's like well how fast is fast enough? At a certain point, fast enough happens. People argue that Jev is fast but local models can be faster. But if the difference between a decision at 180ms versus a decision at 60ms is make or break, I think now we are talking crazy talk. Also, I think that it's actually about BETTER answers rather than FASTER answers. If it's about getting the answer right, I'd rather have that than answer faster and get it wrong. I could finish an exam faster than anyone, but if I fail the exam who cares that I finished first? I think Jev is still the one to beat here and I think it does have an advantage over the major AI Labs. When you make better decisions, your forward momentum is greater than those that misstep.
Frontier labs are using their LLMs to make decisions. This is where I hate the thought of self recursion on LLMs because they often get decisions wrong. We have to tell them why or why not, and when we offload too much to them, we weaken that muscle ourselves. When an LLM gives you options and "recommends" an answer, don't just blindly accept that. It took your cognitive abilities away from you and your decision making part of your brain is quite good at discerning information. Read through the options and if none of them are good, argue with it. During a system architecture design, Claude thought it knew best, and I fought with it until it started asking the right question. We must keep this skill for the sake of humanity.
The most interesting use case along the LLM lines is tool calling. When you have an LLM agent (which is really System Two) it has access to several tools it can use to solve problems. But LLMs often fuck up the tool they need to use and start going down rabbit holes or making shit up. Now thinking about this, what if we gave an LLM one tool, called System One, and it asked System One what to use. Well that's interesting, BUT is the LLM going to ask the right question? My brother said today, this is getting confusing. And I agree, it's not getting easier to understand, it's getting more complex. But I think that complexity is what is going to save us as humans here, because it forces us to think again. I don't think racing to keep pace with the frontier labs is what we need. We need to start using our brains again and start asking the right questions instead of letting agents loop their way to death.
I believe that this is definitely unlocking things for me in my systems, because most of the time I put an LLM in the middle of a process to make a decision, where now that gets replaced with a System One model. BUT it's crucial that in order to get the right decision you have to ask the right question and have thought about the answers in advance. It's important to be asking, is this the right question to be asking? Will it make better decisions if I ask better questions? Will it make a better pizza if I use better ingredients? Jev says yes at a 0.82 probability.