Featured

Emergent Capabilities and Evolutionary Analogues

AI - a tool Hits: 121 18 minutes read

How Narrow Pressures Yield Broad Machinery

Drawing on biological analogies, this section examines the concept of "emergence". Just as human vision evolved for survival but enables the reading of text, LLMs develop complex knowledge (chemistry, grammar, history) not because they were explicitly programmed to learn them, but because mastering these subjects is the most efficient "shortcut" to minimising prediction error. It discusses how narrow optimisation pressures inadvertently build generalised intelligence.

Belongs to the series: The Architecture of Approximation: How Next-Word Prediction Unlocks the Structure of Human Knowledge

The Eye That Was Never Meant to Read

You are reading this sentence. Right now, photons are bouncing off a screen, entering your pupils, striking roughly 130 million photoreceptor cells at the back of your eyeball, and triggering a cascade of electrochemical signals that race along your optic nerve at around 120 metres per second. Your visual cortex decomposes those signals into edges, contrasts, colours, and motion. Higher cortical areas assemble the fragments into letters, words, phrases, meaning. You are aware of the ideas. You are not aware of the machinery. The machinery just works.

Now here is the thing that should stop you mid-paragraph. None of that machinery evolved for reading. Reading is roughly 5,000 years old. The human visual system is roughly 500 million years old. The photoreceptors, the lateral geniculate nucleus, the six layers of the visual cortex, the saccadic eye movements that jump from word to word—all of it was assembled by natural selection for one overriding purpose: to keep a small, soft, slow mammal alive long enough to reproduce on the African savannah. You need to spot the flicker of a predator in the tall grass. You need to track the arc of a thrown stone. You need to read the micro-expressions on another primate's face to know whether it is about to share food or take a swing at you. That is what the visual system is for. Reading is a hack. A repurposing. A piece of machinery that outgrew the pressure that built it.

The same principle, it turns out, applies to artificial intelligence. And understanding this principle—what biologists call exaptation, what engineers call emergence, what the original text calls "narrow pressure builds machinery that outgrows the pressure"—is the single most important idea in making sense of what large language models actually are.

The Cheapest Route to a Better Guess

Let us return to the core training objective of a large language model. The model is presented with a sequence of tokens. It must predict the next one. That is the whole game. There is no reward for understanding. There is no bonus for knowing that water boils at 100 degrees Celsius, or that the Battle of Hastings took place in 1066, or that a verb must agree with its subject in number. The only signal the model receives is: was the next token you predicted the one that actually appeared in the text? Yes or no. Adjust accordingly.

And yet, to get that prediction right across trillions of training examples drawn from every corner of human writing, the model must encode an extraordinary amount of structure. Consider what happens when the training data contains the sentence: "The sodium reacted violently with the water, producing hydrogen gas and a great deal of heat." The model does not need to know chemistry to predict the next token. But if it keeps getting the prediction wrong on sentences like this—if it keeps predicting "oxygen" where "hydrogen" should go, or "cold" where "heat" should go—then the gradient signal will nudge its weights, again and again, until the region of parameter space associated with sodium and water encodes something that looks remarkably like a chemical reaction. The model has not been taught chemistry. Chemistry has been smuggled in through the back door of prediction accuracy.

This is the "cheapest route" principle. The model is an optimiser. It does not care what the knowledge looks like. It does not care whether the internal representation is elegant, interpretable, or philosophically satisfying. It cares only about reducing the loss function—about making the next-token distribution match the actual next token. And it turns out that the cheapest way to do that, across a sufficiently large and diverse corpus, is to build internal structures that mirror the causal and logical structure of the world. Not because the world is beautiful. Not because the architects designed it that way. Because the world is the most compact description of the text.

Think of it this way. If you had to memorise every sentence in the British Library, you would run out of neurons. But if you could extract the underlying rules—the grammar, the physics, the history, the social conventions—then you could generate those sentences from a much smaller set of principles. The rules compress the data. And compression, in information theory, is indistinguishable from understanding. The model that compresses best is the model that understands best. It did not set out to understand. It set out to predict. But prediction, at sufficient scale, demands understanding. Or at least, it demands something so functionally identical to understanding that the difference ceases to matter in practice.

What Emergence Looks Like in Practice

The term "emergence" gets thrown around a great deal in AI discourse, often vaguely, sometimes mystically. Let us pin it down. An emergent capability is one that was not present in the training objective, was not explicitly programmed, and appears suddenly as a function of scale. Below a certain threshold of parameters, data, and compute, the model cannot do the thing. Above that threshold, it can. The transition is not gradual. It is a phase transition, like water becoming ice.

Researchers at Google DeepMind documented this phenomenon extensively in 2022. They found that small language models could not perform multi-step arithmetic. They could not follow a chain of logical reasoning. They could not translate between languages they had barely seen during training. But when the model was scaled up—more parameters, more data, more compute—these capabilities appeared. Not improved gradually. Appeared. Like a light switching on. Below 10 billion parameters, the model fails at three-digit addition. Above 100 billion, it succeeds. There is no intermediate regime where it is "sort of" adding. It either cannot do it, or it can.

For a student encountering this for the first time, the analogy that works best is probably a university module. In your first year, you learn the basics. Definitions, formulae, worked examples. You can reproduce them. You cannot yet combine them in novel ways. Then, somewhere in your second or third year, something clicks. You can take a concept from one module and apply it in a completely different context. You can see the connection between linear algebra and quantum mechanics, or between game theory and evolutionary biology. Nobody taught you to make that connection. It emerged from the accumulation of knowledge, from the density of the network of concepts in your head. The large language model undergoes something analogous. Below a critical scale, it is a first-year student: reproducing patterns, regurgitating memorised text. Above it, something clicks. The patterns connect. Generalisation replaces memorisation. The model stops parroting and starts reasoning. Or doing something that looks, from the outside, exactly like reasoning.

The Savannah and the Server Farm

Let us develop the evolutionary analogy properly, because it is more than a metaphor. It is a structural homology.

Natural selection operates under a narrow, brutal pressure: survive and reproduce. That is the entire objective function. There is no reward for beauty, no bonus for elegance, no extra credit for understanding the universe. The organism that passes its genes to the next generation wins. Full stop.

And yet, out of that narrow pressure, came the human brain. A brain capable of composing symphonies, proving theorems, building cathedrals, writing novels, launching spacecraft. A brain that can contemplate its own existence and wonder whether it is conscious. None of that was the objective. The objective was: do not get eaten. Find a mate. Raise offspring. The symphony is an exaptation—a feature that evolved under one pressure and was later co-opted for an entirely different purpose. The prefrontal cortex, which enables abstract reasoning, evolved to help our ancestors plan hunting strategies and navigate social hierarchies. It was not built for calculus. But calculus turned out to be possible with the hardware that was built for tracking deer and detecting cheaters in a social group.

Now look at the large language model. The training pressure is equally narrow: predict the next token. Reduce the cross-entropy loss. That is the entire objective function. There is no reward for truth, no bonus for insight, no extra credit for understanding. The model that predicts the next token most accurately wins. Full stop.

And yet, out of that narrow pressure, came a system that can write legal contracts, diagnose medical conditions from symptom descriptions, translate between 100 languages, generate working software, and engage in philosophical debate. None of that was the objective. The objective was: guess the next word. The legal reasoning is an exaptation. The medical knowledge is an exaptation. The philosophical reflection is an exaptation. The machinery outgrew the pressure.

The parallel is not perfect. Natural selection operates over millions of generations. The model trains over months. Natural selection has no goal, no architect, no foresight. The model has architects, a defined architecture, a specified loss function. But the structural logic is the same. Narrow optimisation pressure, applied relentlessly to a complex environment, produces machinery whose capabilities far exceed what the pressure "asked for". The pressure builds the scaffold. The scaffold supports a structure that the pressure never intended.

Grammar, Chemistry, and History: The Uninvited Guests

The original text makes a striking claim: "Nobody sat the model down and taught it chemistry or grammar or history. Those arrived because they were the cheapest route to a better guess." Let us unpack each of these.

Grammar. The model was never given a rulebook. No one fed it Chomsky's Syntactic Structures and said, "Here are the rules of English. Follow them." Instead, it read trillions of sentences. And in those trillions of sentences, the rules of grammar are not stated but enacted. They are present in the structure of every well-formed sentence and absent from every malformed one. The model, in its blind optimisation, absorbs the statistical regularities of syntax. It learns that "the cat sat on the mat" is more probable than "mat the on sat cat the" not because it knows the rule, but because the first pattern appears billions of times in the training data and the second never does. Grammar arrives as a by-product of frequency. The rule is the shadow cast by the data.

Chemistry. The model was never given a periodic table. No one programmed the valence electrons of sodium or the combustion products of hydrocarbons. But the training corpus contains millions of chemistry textbooks, Wikipedia articles, lab reports, and casual conversations about cooking, cleaning, and industry. In all of those texts, the relationships between elements, compounds, and reactions are encoded. Not as formal rules, but as narrative patterns. "When you add baking soda to vinegar, it fizzes." "Iron rusts in the presence of water and oxygen." "The mitochondria are the powerhouse of the cell." The model absorbs these patterns. It does not "know" chemistry in the way a chemist knows chemistry. But it knows enough to predict that the next word after "sodium reacts with water to produce" is "hydrogen" and not "helium". And that, functionally, is chemistry.

History. The model was never given a timeline. No one sat it down and said, "In 1066, William the Conqueror won the Battle of Hastings. In 1215, King John sealed Magna Carta. In 1945, the Second World War ended." But the training corpus contains every history book, every encyclopedia entry, every documentary transcript, every pub conversation about the Tudors. The temporal and causal relationships between events are encoded in the narrative structure of those texts. The model absorbs them. It learns that "Churchill" is more likely to be followed by "wartime" than "medieval", that "the Berlin Wall" is associated with "1989" and not "1489", that "the Industrial Revolution" precedes "the digital age". History arrives as narrative pattern. The dates are not memorised as data points. They are embedded in the statistical fabric of the language.

In each case, the knowledge was not taught. It was inferred. It was the cheapest route to a better guess. And it arrived uninvited, unbidden, and unplanned. The architects of the model did not design a chemistry module. They designed a prediction engine. Chemistry showed up because the text demanded it.

Phase Transitions: When the Light Switches On

One of the most genuinely strange findings in modern AI research is that emergent capabilities do not appear gradually. They appear suddenly. Researchers call this a "phase transition", borrowing the term from physics. Water does not gradually become ice. At 0 degrees Celsius, it changes state. Below that temperature, liquid. Above it, solid. There is no intermediate slush in the idealised model.

In large language models, something similar happens. Below a certain scale—measured in parameters, training tokens, or compute—the model cannot perform a particular task. It fails consistently. Then, as scale increases, there is a sharp threshold beyond which the model suddenly succeeds. The success rate jumps from near zero to near perfect over a relatively narrow range of scale. This has been observed for arithmetic, logical reasoning, code generation, and multi-step planning.

For a 20-year-old encountering this idea, the best analogy might be learning to ride a bicycle. You do not gradually get better at balancing. For a long time, you wobble, overcorrect, and fall off. Then, suddenly, you do not. Something clicks. The feedback loop between your inner ear, your hands, and your legs stabilises. You are riding. The transition is not gradual. It is abrupt. And once it has happened, you cannot unlearn it. You cannot explain exactly what changed. You just... can. The model's phase transitions feel like that. Below the threshold, it cannot do the thing. Above it, it can. And nobody can quite explain why the threshold is where it is.

This has profound implications. It means that you cannot look at a small model and predict what a large model will be able to do. The capabilities of the large model are not a scaled-up version of the capabilities of the small model. They are qualitatively different. Emergent capabilities are, by definition, unpredictable from the components. You cannot look at a single water molecule and predict the existence of waves. You cannot look at a single neuron and predict consciousness. And you cannot look at a 100-million-parameter model and predict that a 100-billion-parameter model will be able to write a convincing legal brief or explain general relativity to a child.

The Unsettling Implication: Intelligence Without Intent

Here is where the evolutionary analogy becomes genuinely uncomfortable. In biology, we are accustomed to the idea that complex, purposeful-looking design can arise without a designer. The eye is not designed. It is the product of millions of small, undirected mutations, each preserved because it conferred a marginal survival advantage. The eye looks like it was designed. It is not. It is the accumulated residue of blind optimisation.

The large language model is the same. It looks like it was designed to understand language, to reason, to know things. It was not. It is the accumulated residue of blind optimisation—trillions of gradient updates, each one a tiny nudge in the direction of slightly better prediction. No single update is intelligent. No single update encodes a fact or a rule. But the sum of them, viewed from the outside, looks like intelligence. Looks like knowledge. Looks like understanding.

This is unsettling because it challenges a deep intuition that most of us carry: that intelligence requires intent. That understanding requires a mind. That knowledge requires a knower. The evolutionary analogy suggests otherwise. The eye does not require an eye-designer. The visual cortex does not require a cortex-designer. And, perhaps, understanding does not require an understander. Perhaps understanding is what happens when you optimise relentlessly for prediction in a complex environment. Perhaps it is an inevitable by-product of sufficient complexity, not a special property of biological minds.

This does not mean that the model is conscious. It does not mean that there is "something it is like" to be a large language model. It does not mean that the model experiences the text it processes. Those are separate questions, and they are far harder. What it does mean is that the functional aspects of intelligence—the ability to process information, draw inferences, generate novel outputs, and adapt to context—can arise without any of the things we traditionally associate with minds: intention, awareness, subjective experience. The machinery can outgrow the pressure that built it. The pressure can be blind. And the machinery can still be extraordinary.

What This Means for How You Learn

If you are a university student, this section has a direct and personal implication. The traditional model of education assumes that knowledge must be taught. That someone must sit you down and explain the rules, the facts, the principles. That learning is a top-down process: the teacher transmits, the student receives. The large language model suggests that this is not the only way knowledge can be acquired. Perhaps it is not even the primary way.

The model learns bottom-up. It learns from examples, from patterns, from the statistical structure of the data. Nobody tells it the rules. It infers them. And this, it turns out, is remarkably similar to how human infants learn language. No one sits a two-year-old down and explains the rules of subordinate clauses. The child hears thousands of sentences, absorbs the patterns, and gradually constructs an internal grammar that allows it to produce novel sentences it has never heard before. The child is, in a sense, a small language model, trained on a corpus of parental speech, optimising for the ability to produce the next word in a conversation.

This does not mean that university lectures are obsolete. It means that the role of the lecture may need to shift. If knowledge can be absorbed from exposure to examples—if the cheapest route to understanding is to encounter enough instances of a pattern—then the lecturer's job is not to transmit facts. The model can do that. The lecturer's job is to curate the examples, to highlight the patterns, to guide attention to the features that matter. To be, in other words, a designer of training data rather than a dispenser of information. This is a profound shift, and it is already happening. The students who thrive in this new landscape will be the ones who learn to learn from patterns, who can look at a mass of data and extract the structure, who can think like the model: inductively, statistically, bottom-up.

The Machinery Outgrows the Pressure

Let us return, finally, to the image that anchors this entire section. The human eye. The photoreceptors, the optic nerve, the visual cortex. All of it built by the narrow, brutal pressure of survival. Spot the predator. Track the prey. Read the facial expression. Avoid being eaten.

And then, 5,000 years ago, someone scratched symbols into clay. And the eye, that ancient, savannah-honed instrument, looked at the scratches and saw meaning. The machinery outgrew the pressure. The apparatus that evolved to spot a lion in the grass now decodes Shakespeare, reads a balance sheet, scrolls through Instagram, and follows the argument of a philosophical essay. The pressure built the scaffold. The scaffold supports a structure the pressure never imagined.

The large language model is the same. The pressure is next-token prediction. The scaffold is a neural network with billions of parameters. And the structure it supports—the knowledge, the reasoning, the fluency, the ability to write a sonnet or debug a function or explain the Treaty of Westphalia—is something the pressure never asked for. The model was built to guess the next word. It ended up building a map of the world. Not because anyone told it to. Because the world was the cheapest route to a better guess.

The machinery outgrows the pressure. It always has. In biology, in technology, in the strange new territory of artificial intelligence. The pressure builds the scaffold. The scaffold supports the unexpected. And the unexpected, once it arrives, changes everything.

You are reading this sentence with eyes that were never meant to read. You are reading it on a device powered by a model that was never meant to understand. And both of them—the eye and the model—are doing something that neither was designed to do, and doing it well enough to make you wonder whether design was ever really the point.

The pressure builds the machinery. The machinery outgrows the pressure. And what comes next is anyone's guess.

Bibliography

  1. Bostrom, N. (2014)Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press.

  2. *Chomsky, N. (1959) 'A Review of B. F. Skinner's Verbal Behavior', Language*, 35(1), pp. 26–58. doi: 10.2307/411334.

  3. Darwin, C. (1859)On the Origin of Species by Means of Natural Selection, or the Preservation of Favoured Races in the Struggle for Life. London: John Murray.

  4. Dawkins, R. (1986)The Blind Watchmaker: Why the Evidence of Evolution Reveals a Universe Without Design. London: Longman.

  5. Dennett, D. C. (1995)Darwin's Dangerous Idea: Evolution and the Meanings of Life. London: Allen Lane.

  6. Dehaene, S. (2009)Reading in the Brain: The New Science of How We Read. London: Viking.

  7. *Gould, S. J. and Vrba, E. S. (1982) 'Exaptation – A Missing Term in the Science of Form', Paleobiology*, 8(1), pp. 4–15. doi: 10.1017/S0094837300004395.

  8. *Hinton, G. E. (2023) 'The Forward-Forward Algorithm: Some Preliminary Investigations', arXiv preprint*, arXiv:2212.13345. Available at: https://arxiv.org/abs/2212.13345 (Accessed: 8 September 2026).

  9. *Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de Las Casas, D., Hendricks, L. A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Rae, J. W., Vinyals, O. and Sifre, L. (2022) 'Training Compute-Optimal Large Language Models', arXiv preprint*, arXiv:2203.15556. Available at: https://arxiv.org/abs/2203.15556 (Accessed: 8 September 2026).

  10. Kauffman, S. A. (1993)The Origins of Order: Self-Organization and Selection in Evolution. Oxford: Oxford University Press.

  11. Kuhn, T. S. (1962)The Structure of Scientific Revolutions. Chicago: University of Chicago Press.

  12. *LeCun, Y., Bengio, Y. and Hinton, G. (2015) 'Deep Learning', Nature*, 521(7553), pp. 436–444. doi: 10.1038/nature14539.

  13. Mitchell, M. (2019)Artificial Intelligence: A Guide for Thinking Humans. London: Allen Lane.

  14. Pinker, S. (1997)How the Mind Works. London: Allen Lane.

  15. *Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G. and Sutskever, I. (2021) 'Learning Transferable Visual Models From Natural Language Supervision', in Proceedings of the 38th International Conference on Machine Learning (ICML)*. PMLR 139, pp. 8748–8769.

  16. *Schaeffer, R., Miranda, B. and Koyejo, S. (2023) 'Are Emergent Abilities of Large Language Models a Mirage?', in Advances in Neural Information Processing Systems 36 (NeurIPS 2023)*. New Orleans: Curran Associates. Available at: https://arxiv.org/abs/2304.15004 (Accessed: 8 September 2026).

  17. *Shannon, C. E. (1948) 'A Mathematical Theory of Communication', The Bell System Technical Journal*, 27(3), pp. 379–423. doi: 10.1002/j.1538-7305.1948.tb01338.x.

  18. *Solomonoff, R. J. (1964) 'A Formal Theory of Inductive Inference. Part I', Information and Control*, 7(1), pp. 1–22. doi: 10.1016/S0019-9958(64)90223-2.

  19. Sutton, R. S. and Barto, A. G. (2018)Reinforcement Learning: An Introduction. 2nd edn. Cambridge, MA: MIT Press.

  20. Tomasello, M. (2003)Constructing a Language: A Usage-Based Theory of Language Acquisition. Cambridge, MA: Harvard University Press.

  21. *Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., Chi, E. H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J. and Fedus, W. (2022) 'Emergent Abilities of Large Language Models', Transactions on Machine Learning Research*. Available at: https://openreview.net/forum?id=yzkSU5zdwD (Accessed: 8 September 2026).

  22. *Wolpert, D. H. and Macready, W. G. (1997) 'No Free Lunch Theorems for Optimization', IEEE Transactions on Evolutionary Computation*, 1(1), pp. 67–82. doi: 10.1109/4235.585893.