Popular transformer-based AI models, such as GPT, Gemini, Claude, and Llama, have become the standard for natural language processing, thanks to their powerful classification and language generation capabilities. Models like these rely heavily on multi-head attention mechanisms and feed-forward networks, which consist of a large number of parameters that consume substantial computational resources. Together, these requirements contribute to these models needing notoriously high energy consumption and long training times.

But what if there were a way to cut down the resources it took to run useful models? Rather than relying only on incremental improvements to transformers, researchers could be experimenting with completely new architectural ideas.

Enter CoFrGeNets, which stands for Continued Fraction Generative Networks. They’re a new class of model architecture designed by a group within IBM Research. CoFrGeNets replace conventional transformer-based model components with structures derived from continued fractions, a mathematical representation that can express complex functions compactly. Relying on continued fractions allows a model to act more expressively with fewer parameters, since multiple components can combine in a way that increases representational complexity without proportional increases in size. The team outlined the evidence for CoFrGeNets in a new paper and presented the results this week at the International Conference on Machine Learning (ICML) in Seoul, South Korea.

The primary motivation behind the work is twofold, according to Amit Dhurandhar, principal research scientist at IBM Research. First, the field of LLM architecture design currently relies on a relatively small set of dominant ideas, particularly the transformer model. By introducing an alternative, Dhurandhar and his fellow researchers hope to expand the range of tools available for building models that are more efficient, compact, and scalable without sacrificing performance. Second, the team wanted to systematically explore the limits and capabilities of this function class.

CoFrGeNet builds on prior work where the team applied continued fractions to classification tasks. That research showed advantages over conventional architectures in a much more restricted setting, said Dhurandhar. The current project extends those ideas to generative models, a significant conceptual and engineering jump.

At the heart of CoFrGeNet is the use of continued fractions to represent functions. A continued fraction expresses a value as a nested sequence of divisions, which can capture complex relationships in a compact way. When adapted to neural networks, these structures form “ladders” of computations where each layer depends on the reciprocal of the previous one.

Ensembles of these ladders can approximate very complex functions while using fewer parameters than traditional neural networks. This makes them a potentially powerful alternative to dense layers such as feed-forward networks. In practice, an input is passed to each “step” of the ladder, where it gets multiplied by model parameters and added to the reciprocal of the previous step. In this way, the causal chain of token generation is preserved, with each subsequent step on the ladder being defined by what came before it.

Dhurandhar and his colleagues sought to determine whether architectures based on these ladders can directly replace the two core parts of transformer blocks in the following ways:

Attention replacement: Performing token-to-token interactions while preserving the causal structure required for auto-regressive language generation.

Feed-forward mechanism replacement: Replacing standard feed-forward networks to enable feature mixing with fewer parameters.

The development process began with small-scale experiments using a lightweight, open-source toolkit called nanoGPT, where they tested feasibility of different architectures that used these two approaches before scaling up. “If things don't work there, they're definitely not going to work for something bigger,” said Dhurandhar. “If things do work there, it doesn't necessarily mean it'll work for something else. But it was a good starting point.”

To test how CoFrGeNet components stack up, the team swapped out multi-head attention and the feed-forward network with their own ladders, then compared them against strong baselines like GPT2-xl (1.5B) and Llama-3.2B. The setups were broad, pre-training models on large datasets like OpenWebText, GneissWeb, and a mixed Docling data mix. Each combination of continued fraction ladder components was evaluated according to downstream benchmarks like GLUE (for classification), plus perplexity tests on multiple datasets.

The results were impressive for such a different architecture. Across downstream classification tasks, the CoFrGeNet variants (especially one called CoFrGeNet-F) often matched or slightly beat GPT2-xl, even though they used far fewer parameters, often hundreds of millions fewer.

Perplexity scores, which quantify how well a model can predict samples, told a similar story. CoFrGeNet models consistently performed better than GPT2-xl and beat efficient attention baselines like Synthesizer-D and Sparse Attention in many cases. CoFrGeNet models also demonstrated efficiency gains in the form of quicker training time and faster inference.

Even on the larger Llama-3.2B experiments, the CoFrGeNet models stayed competitive on tasks like open-domain Q&A and reasoning, while still being smaller and faster to train. Taken all together, these preliminary findings indicate that CoFrGeNet holds up in practice, giving similar or better performance while being leaner and more efficient.

Optimizing the new method

A major challenge with continued fractions is that they are computationally expensive and potentially unstable on modern hardware, Dhurandhar said. For example, if you get very small numbers in the denominator, a value could go essentially to infinity. Those kinds of figures can be dicey for today’s GPUs.

“The first thing we did to reduce divisions is we adopted this alternative formulation of representing continued fractions using something called continuants,” said Dhurandhar. Continuants are another way of representing a continued fraction that reduces it to a ratio of two polynomials. Irrespective of how deep the ladder goes, you only need to do one division. This particular element is what makes CoFrGeNet efficient to train and infer on modern digital hardware.

To improve training stability, the team also introduced an “incremental training” method, where model layers are gradually introduced during training rather than optimized all at once.

One of the defining advantages of CoFrGeNet is its flexibility. The approach is largely “plug and play,” meaning it can be integrated into existing model pipelines with minimal changes. Developers can selectively replace either attention layers, feed-forward networks, or both, and can mix the new components with traditional ones.

“This approach is very complementary to existing things,” said study co-author Vijil Chenthamarakshan, a senior technical staff member at IBM Research. “If there is a new development in some other aspect of LLMs, you can use that along with our model. It's not conflicting in any sense.”

Compared to existing efficiency techniques like pruning or sparsity methods that reduce parameters within an unchanged architecture, CoFrGeNet offers a novel solution, said Dhurandhar. It introduces a fundamentally different way of representing functions, opening new directions for research while still being compatible with other optimization strategies.

What’s next

In terms of practical impact, the results suggest strong potential for replacing feed-forward network components, which account for a large proportion of parameters in many models. Early collaborations with teams working on IBM Granite family models suggest that CoFrGeNet could reduce parameter counts or improve performance at similar scales. The approach from preliminary tests appears to be even more effective for mixture-of-experts (MoE) architectures, which power most frontier models. This is because each expert typically caters to only a subset of tokens, and hence there is more potential for compression.

Looking ahead, the team is exploring further hardware acceleration strategies. In an ongoing partnership with researchers at Rensselaer Polytechnic Institute (RPI), the team is now experimenting with performing the division operations on non-GPU hardware, like analog processors or field-programmable gate array (FPGA) boards. These have the advantage of faster division performance than GPUs can achieve, and then the GPUs take over to do the multiplication operations they excel at.

“That speeds up things a lot, and it also leads to a lot of savings in power,” said Dhurandhar.

Overall, CoFrGeNet has the potential to be both a practical tool for improving model efficiency and a conceptual shift that broadens how we think about neural architectures.