The enthusiasm around AI has been focused on large language models for years now. For quite a few folks, this is their first real experience with AI at all. And I am sure you have noticed something:

LLMs are non-deterministic.

In other words, repeated invocations of the algorithm with the same inputs typically produces different outputs.

You can demonstrate this yourself. Open a chat AI tool in two incognito tabs, asking it a substantial prompt like "explain LLMs to me", and you will see the responses are different.

With all the noise about generative AI like this in the past 5 years, it's easy to forget there is a whole universe of AI that is completely deterministic.

The simplest example is linear regression.

Lots of great info about this online, so I will not explain it here. But when someone asked me recently how to start learning AI - not just how to use AI tools, but actually understand AI from first principles - my suggestion was that he start by learning linear regression. It is tremendously useful in the real world today, and is simple enough that you can actually grok it without a PhD.

The relevant fact about linear regression here:

It is deterministic.

The nuance is that some algorithms used to train linear regression - for example, gradient descent- deliberately inject randomness, because it is useful to do so.

But once trained, linear regression models are completely deterministic. Every time you stuff the same input into one, the exact same output comes out.

This has implications for the software systems we design.

When a program needs an AI component, and you are able to use a deterministic algorithm:

• You can write unit tests and have a good time doing it.
• QA will have a much easier time nailing down any issues and making them reproducible (and thus much easier to fix).
• You won't have bizarre surprising behavior pop up in production as often.
• That part of the software system will behave in more consistent and understandable ways.

Of course, if your application actually needs an LLM - or any other form of nondeterministic AI - then that's what you have to use. But if and when you have a choice, these are some things to think about.

If you liked this, you will enjoy the Powerful Python Newsletter.

P.S. When we say "LLMs are non-deterministic", that is technically a lie. It is more correct to say that LLMs are chaotic, in the mathematical sense of sensitive dependence on initial conditions; and on top of that, deliberate randomness is injected into how the LLM is applied, via the temperature parameter.

But nobody understands all that. "Non-deterministic" does describe how LLM-based tools feel to people at the UX level, so I'll go with the crowd and use that term.