LLMs for Mortals 6 months later

Part of the difficulty of writing a high quality book on LLM APIs is that the APIs change so often. You can read some of my initial post comparing my book to competitor books, which by the time they went to print were already out of date and had code that no longer worked.

I compile Large Language Models for Mortals using Quarto, so every code snippet in the book ran at the time I built it. But that is only a guarantee as of the day I compiled it. So around 6 months after release, I recompiled the whole book to make sure my code had not rotted the same way. For now it mostly still runs. Here are my notes on what broke, what I got right, and what I got wrong.

What broke in the recompile

The book has 264 python code cells that are executed, calling OpenAI, Anthropic, Google, and AWS. I did not update any of the python libraries (so still openai 2.14.0, anthropic 0.75.0, google-genai 1.55.0). Only three models had been retired:

- Claude Opus 4 (claude-opus-4-20250514), which I used for the extended thinking and web fetch examples. I swapped in Opus 4.5.

- Claude 3 Haiku, which I used in Chapter 4 to show testing an older, cheaper model against Haiku 4.5. There is no cheaper Anthropic model than Haiku 4.5 anymore, so that section now compares Haiku 4.5 to Sonnet 4.5 (3 times the price). The lesson is the same – on my small test set of 20 narratives the two models are near identical, so no reason to pay for the bigger model.

- Gemini 3 Pro preview. The API error helpfully says to use gemini-3.1-pro-previewinstead, so that is what I did.

Everything else, including the many examples using gpt-4o-mini, the AWS Bedrock examples, and the agent examples, ran without changes. So really you just need to change a few of the model names and the code snippets still run as is.

What I got right

Fine-tuning. In the final chapter I said I was not sure the foundation model providers would keep supporting fine-tuning, given how rarely I have seen it used. Custom trained GPT models have since been deprecated. (I did not cover fine-tuning in the book at all, so no code to fix.)

The major API components are all still relevant. Chat completions and the responses API, structured outputs, and the various RAG patterns (in memory, on disk, and the hosted vector stores) have not changed in any way that matters for the book.

For coding agents, you should still focus on test driven development (TDD). A good test suite is how you know the changes the agents make actually work.

What needs updating

AWS Inference Profiles. Bedrock can now use API keys, which is much simpler than the IAM role and inference profile setup I walk through in the book just to call a model. You likely still need much of the IAM setup for other services though, like S3 vectors.

Open source models. I mostly omitted these (I show DeepSeek on Bedrock and a small Qwen model locally). I did not expect how good the open weight models from GLM, Kimi, and DeepSeek would get. In particular, there are now US based endpoints that serve them, like Databricks and Baseten. For my work, sending data to servers outside the US is often a non-starter, so the US endpoints are what make these models usable. AWS Bedrock is clearly falling behind here.

There will be more things that need updating, likely before a year from now (older models being retired, the AWS approach). The main one though is temperature=0, which I use throughout the book. It is often not supported by the newer thinking models, which may be all that is available in the near future it seems. I may also need to deprecate the section on using logprobs for confidence in classification. (Or I may use open source models to show that, will need to wait and see.)

Let me know if there are other technical components not covered you would like to see (I may update the coding tools chapter to include Grok, and maybe web search with Perplexity APIs, but these would be tiny sections in the current book).

What I got wrong

These are not code examples, but parts of the narrative.

Agents. In the last chapter I guessed the python agent libraries may undergo major changes, and I would not make firm bets on how people will build agents. There has been no major change in agents in the prior six months. Codex now works much like Claude Code (the OpenAI agents API and codex and now one and the same). But otherwise there have been no major API changes from the foundation model providers. Most of the work has gone into isolation, so it may be whole environments that end up changing more drastically, and less query a foundation model endpoint.

Small code changes. In the coding tools chapter I pretty heavily emphasized having the coding agents make small code changes, and reviewing each one. It is easily possible now to hand the agents much larger tasks. Which goes back to the TDD point, the tests are what make that reasonable.

A new edition?

I am not releasing a new version of the book for now, but will likely in 2027. The basics – API calls, structured outputs, RAG, tool calling – are holding up. If I do update it, previous buyers will get the new epub version for free.

So there is no reason to wait. You can preview the first 60+ pages, and use promo code SIXMONTHSEPUB for $30 off the epub, or SIXPRINT for $20 off the paperback. And if you are newer to python, my first book, Data Science for Crime Analysis with Python, is the place to start.