A Primer: Large Language Models and Transformers

Much of the recent revolution in tools like ChatGPT was driven by large language models, themselves driven by an underlying technology called transformers. These AIs excel at generating content (starting with text) by analyzing textual patterns and, driven by a prompt, generating new content, one unit (often a word) at a time.

Even before AMI Labs, Yann LeCun has famously stated that he believes this structure is limited, in that the approach of generating content one unit at a time leads to loss of coherence as the output gets longer. There has been pushback, with others developing methods to counter the issue (such as chain of thought prompting or search-based decoding to improve large language models) and using various methods like integration with reinforcement learning, to add more logical understanding to LLM outputs.

What Is A World Model?

World models are an approach, where rather than trying to understand an input as a series of small sub-components (such as a sentence being a set of words, or an image being a series of pixels), the AI sees the input more conceptually, reflecting its internal training on how things work in the world. This can be the physical world, where scientific laws apply, or the business world, where unspoken and spoken rules exist that should be followed, in other domains. This conceptual modeling (referred to in technical terms as latent space - where the AI works with a conceptual map rather than raw data) is where world models construct outputs.

In a world model, an AI learns in a simulated environment where it can explore inputs and outputs and learn patterns that reflect the rules of the environment as stated in conceptual terms. It can then understand not just the input but also what-if scenarios and strategize the impact of various outcome choices.

But Isn’t This Reinforcement Learning?

There is a strong synergy between world models and reinforcement learning. Reinforcement learning is the body of AI where the AI learns the outcomes of actions. You can think of a world model as being the teacher to a reinforcement learning AI. RL today is notoriously data-intensive, often requiring simulators, such as in the case of self-driving cars. RL also uses armies of human trainers, such as in reinforcement learning with human feedback used in LLMs.

What Are The Benefits?

It is too early to definitively tell, but some areas where benefits can exist are;

• Areas where AI models must obey underlying environmental rules. For example, in physics, the law of gravity does not need to be learned by an AI from patterns in data. It can be specified at a conceptual level and then applied. Similarly, AI models generating recommendations for new drug molecules should be able to avoid recommending molecules that nature will never allow to physically exist. These kinds of issues are known to exist, where AI predictions in drug discovery, for example, have struggled when faced with real biological complexity in validating trials.
• Areas where a massive amount of training data is required to train an LLM (either as raw input or scenario assessments). When preexisting rules exist, less data is needed to learn them, possibly making it possible to create AI for more complex scenarios than is possible today.

What Are The Risks?

Again, it is too early to tell definitively, but the fundamental changes that these models present suggest that new progress may need to be made in related areas to manage risk, particularly in production. For example:

• Security issues now appearing for generative AI are likely to be quite different from whatever security issues will appear in world models.
• Techniques being developed to make existing AIs explainable (where they are able to articulate the reason for predictions, outputs, and decisions) are not likely to apply to these new models.

Must It Be One Or The Other?

No. For example, recent research at PAN demonstrates that there can be a benefit to integrating the world model concepts of abstraction with existing techniques used in large language models. PAN is a way to simulate world realities by using large language model reasoning as a backbone that is anchored in physical correctness.

It is also worth noting that other techniques have attempted to add constraints to AI modeling to reflect physical realities, such as work in drug discovery, to force AI models to consider physical constraints when evaluating the merit of outputs.

Implications For Business

World models are early, but they could, over time, dramatically expand the functionality available from AI, expanding beyond well-defined tasks to broader strategy and design problems. It may also become possible to teach an AI on concepts (much as humans are trained today) and have the AI execute on tasks that it has not seen patterns of before. If this occurs, it does not remove the need for human-in-the-loop, but it can further change the role of the human into that of a high-level overseer of increasingly capable AIs. The human role may also increase in areas of testing and security as the AI complexity and autonomy increase.

At the moment, the only action I believe is required is to stay aware and track the progress of these technologies. The fundraise is likely to increase excitement and activity in exploring this branch of AI, further accelerating the rate of progress.