Ideas Are Rivalrous: The Thermodynamic Cost of Innovation

The 2025 Nobel Prize in Economics was awarded to Mokyr, Aghion and Howitt for “for having identified the prerequisites for sustained growth through technological progress”i. Mokyr’s historical research attempted to explain why innovation flourishes. Each new technology competes to replace the old, creating both incentives for innovators and disruption for incumbents.

In traditional economic models, physical capital is rivalrous: only one factory or person can use a machine at a time. In contrast, it is claimed that ideas can be reused infinitely; once an innovation exists, it can boost productivity everywhere without being depleted. This is misleading. New ideas need to be learned and implemented via new algorithms and machines, and this depletes capital. These resources are rivalrous and their allocation to new technology can lead to obsolescence and scrapping of old capital—a key theme in the Aghion–Howitt model of creative destruction.

There is a hidden assumption in the claim that ideas can be reused without limit: all humans are equipped with roughly the same biological machinery for learning. This obvious fact is the result of the very wasteful process of evolution. The cost of ideas has already been paid in countless rounds of variation and selection, trial and error. The rise of AI has dramatically highlighted the fundamental cost of ideas (or intelligence): the number of tokens you can generate per unit of energy. What is the link between the ‘token accounting’ of AI and the cost of ideas?

Life on earth is possible because the planet is near a hot star and surrounded by cold space. Heat energy emitted from the sun is absorbed by the earth and remitted into empty space at the same rate. It is in a steady-state, but far from a thermal equilibrium. Between absorption and emission of heat by the earth, some energy is stored as Free Energy ¹. This is life. All living things are evolutionary, metastable states of a complex network of electro-chemical processes driven far from thermal equilibrium by access to low-entropy energy, a.k.a food. We can think of a store of free energy as a battery. We can extract work by using it up as the store is returned to thermal equilibrium.

Natural selection drove the development of the central nervous system (CNS) in locomotive animals, dramatically increasing free energy by lowering entropy and allowing a learning machine to arise. Fossil fuels also store free energy, but the free energy stored by evolution in the CNS of animals is vastly greater. How is it stored?

The relentless pursuit of sustenance demands energy expenditure. Every organism is locked in the struggle to find effective strategies to acquire energy, ensuring survival and the perpetuation of life itself. This delicate balance is critical; expending excessive time and energy in the quest for nourishment jeopardizes the future, risking the ability to reproduce. The evolution of animals with a central nervous system allowed this process to become more efficient through the development of biological learning machines. A learning machine stores free energy in the way it compresses the information extracted from its interactions with the world.

This may sound difficult to believe. A simple story may help. Imagine you are lost and starving in the jungle. The you see a fisherman by a river. You could eat the Fisherman, but a better long-term strategy is to ask the Fisherman to teach you how to fish. This only works because evolution has equipped you with the same kind of learning machine used by Fishermen. That makes it very inexpensive to learn by talking with this Fisherman.

Humans learn novel and effective strategies to intervene in the world. They have ideas, make tools, and teach each other how to use them. Ideas and learning are not free. They are objective features of a special kind of complex machine and, like all machines, constrained by thermodynamic principles.

Learning machines are dissipative systems driven from thermal equilibrium by an external energy source and necessarily generate waste heat. The current focus on AI algorithms masks this feature, although the huge cost of electricity to power the machines running the algorithm gives the game away. This is fundamental and not an accident. There is a minimum thermodynamic cost to learning and modern AI systems are extremely far from it. The biological learning machines built by evolution are an enormous store of free energy in the form of a complex system of vastly reduced entropy.

Evolution paid an enormous thermodynamic cost to produce biological learning machines. This cost is largely invisible as the waste products of the evolutionary learning process are dead. We take the incredible thermodynamic advantage provided by biological learning machines for granted.

The reason that learning consumes energy, and requires work, is easy to see. The irreversible dynamics of learning necessarily lowers the entropy of any physical system that supports the process. This low entropy state becomes the thermodynamic resource that enables learning machines to lower the entropy of raw materials in their environment and create wealth.

A homonid randomly chipping stones may accidentally produce a stone axe, but the amount of waste stone generated will be enormous. Most of the agent’s internal energy will be used up producing the waste and not reducing the entropy of the raw materials at all but increasing it. The solution to this problem was solved by evolution; build a learning machine. It is the learning machine inside the agent that enables it to change the distribution in feature space to enable optimal control.

The fundamental law of learning machines says that a properly configured learning machine will minimize the probability of making an error when it expends the least amount of energy in each trial. The training process is a feedback from output to the internal configuration of the machine that minimizes the error. This process is irreversible as the change in the internal configuration is made in the presence of friction.

A learning machine once trained will produce goods with the least consumption of energy per unit. So long as the efficiency in production so gained pays for the energy lost to train the machine, you win. The cost of building and training the learning machine is a capital expenditure that contributes directly to production. In the case of axe production, the physical labour is spent in learning to minimise the amount of energy required to produce each axe. It is essential that we not only include the cost of labour and the capital cost, but the cost of learning when we seek a formula to estimate productivity.

The AI industry focus is rapidly moving from chip price, capital expenditure, and traditional KPIs to “tokens per watt”. In AI economics, it’s not just about more powerful processors—it’s about optimizing token throughput relative to energy consumed, aligning computation with business outcomes.

A machine learning algorithm requires a machine to run it on, an Nvidia GPU for example, and it consumes energy as it switches transistors on and off while the calculation proceeds. Suppose we submit a query to a large language model (LLM) like ChatGOT for example. The first step is to encode all the input as words, sub-words, or punctuation marks are split into tokens. Each token is assigned a unique numerical ID, allowing it to be represented in a way the model can understand and manipulate.

Each token ID is linked to a high-dimensional vector called an embedding. This vector captures the meaning and correlations between parts of the input. These embeddings are the actual inputs to neural network layers in modern AI models. Models like transformers use sequences of these embeddings to learn patterns and connections by adjusting the numbers in embedding space. During training, the model processes millions of token sequences and gradually updates its internal settings to understand language, images, or sound. For language models, the AI learns to predict the next token based on previous ones and context. For other types of data, like images or audio, tokens represent parts of the input, and the model learns to interpret or rebuild them step by step.

At inference, the model generates new tokens, turning internal representations back into data (such as text) for the user. By analyzing, learning relationships between, and generating tokens, learning algorithms develop the ability to understand, reason, and communicate in natural or structured domains.

Processing more tokens demands proportionally more RAM, VRAM, and GPU cycles. Transformer architectures—core to LLMs—scale computationally with the number of tokens, sometimes quadratically (especially in attention mechanisms).

In the last twelve months, the approach to machine learning has been fundamentally changed by the implementation of reasoning models. OpenAI introduced its first reasoning model—known as o1—on September 12, 2024, moving beyond the Chatbot phase of early consumer models. In reasoning models, tokens are generated from inside the inference phase itself. Reasoning models often produce chains of thought or detailed solution traces before generating a final answer. These reasoning traces can be several times longer than standard outputs—up to 20x higher in some advanced reasoning tasks and models. While a standard question/answer model might generate a concise answer, a reasoning LLM will output extended explanations, step-by-step logic, or code to justify its conclusion, each segment adding tokens. Models produce these extra tokens sequentially—each new token depends on all previous ones—so memory requirements and compute time scale with output length. This impacts inference slowdown and resource consumption as token sequences get longer.

While this increases the performance of the models to the point where they can ace international mathematics competitions, it dramatically increases the cost of inference by generating tokens for reinforcement learning. The change in token count is highly task-dependent, model-dependent, and can be controlled for efficiency in some architectures. About 60% of businesses using LLM APIs have exceeded their planned budgets due to unanticipated token use, underscoring the importance of robust monitoring and control tools.

Errors in prediction or classification are dependent on the mathematical form of token embedding. High-quality token embeddings enable learning algorithms to better capture semantic meaning, relationships, and context among data units such as words or code, which leads to more accurate and reliable predictions. Poorly constructed embeddings may miss complex correlations in the token sequences. Models with robust embeddings are less likely to make these errors because they have a more faithful representation of language or other underlying data.

In summary, it is tokens in and tokens out. Tokens determine the model’s context window—how much information it can process at once—and directly impact accuracy, efficiency, and inference cost. In between input and output there are trillions of transistors switching on and off each second, transforming embedded tokens.

Switching a transistor on and off costs energy and this is where the fundamental cost is paid. Larger and more expressive embeddings often lead to better prediction quality but require more energy, memory, and computational power at inference. The most efficient token embedding systems are co-designed to match hardware capabilities—using hardware-aware algorithms for dynamic token pruning, data compression for sparse input, and pipelines that execute key computations near memory arrays (processing-in-memory/PIM). These innovations cut data movement overhead and maximize tokens-per-watt metrics.

Nvidia’s Jensen Huang has proposed the concept of an AI factory where hardware co-design strategies for token reduction are fundamental to maximizing efficiency measured as tokens generated per unit of energy and capital. The strategy is to achieve exponential performance growth while driving down the cost per token. By focusing on algorithm-hardware integration, each incremental improvement translates directly into lower operational costs and higher return on investment.

In a traditional factory raw materials are processed to produce lower entropy products by doing thermodynamic work, but a key role is played by learning. A homonid learns a procedure for making stone axes. It is not error-free. Even if it is followed exactly, it can fail to produce a useful axe, but most of the time it produces a good axe. A recipe used by a baker to make a cake is another kind of procedure. A cake has less entropy than the raw ingredients that went in it. Henry Ford was one of the first innovators to realizethat the algorithm organizing the factory can make a huge difference to entropy reduction per unit cost.

Jensen Huang’s concept of an AI factory makes a direct link between learning and entropy reduction. Making this process more thermodynamically efficient is the key to wealth growth. In the AI factory, raw data (text, images, experimental data) enters, is processed with massive computation, and intelligence, in the form tokens, exits—ready to be used by businesses, software, robots, and other industries. They key is to maximise the entropy reduction per unit of free energy used. Better hardware co-designed with better algorithms is the new drive of GDP growth.

AI factories run continuously, much like power plants, operating at scale to produce high-value intelligence through enormous compute infrastructure. Tiny improvements in quality or efficiency produce outsized gains in engagement and profitability for clients. Industrial scale intelligence will transform our world in a matter of years. The real bottle neck is the need to talk to humans. If we require that, the machine needs to transform its internal embeddings into the algorithm of human language which evolved over millions of years of trial and error. There is no good reason for this if the real objective is to transform matter. Exponential growth in GDP will only occur when humans are out of the loop.

The Origins of Self-Awareness: Why Any Learning Agent Needs a Self-Model

A conscious AI simulates all AI’s that do not simulate themselves.

In the previous post I introduced a simple learning agent. Confined to the circumference of a circle, it can send and receive optical pulses. All pulses it transmits are ultimately received back due to a reflecting boundary that coincides with the circle on which it is confined. It can change the angle at which it emits a pulse and it can count the ticks of a clock until the pulse is received back. I showed how such an agent could learn the geometry interior to the circle without ever moving off the circumference. Suppose now there were other, similar agents on the circle. How does this change the learning protocol?

The first problem faced by each agent is that they can receive pulses of light that did not originate with them. Can they learn to distinguish anomalous pulses from those they sent themselves? Could they learn that there are other agents, like them, on the circle? The ability to distinguish sensations that are correlated with an agent’s own actions from those that are not is the second step in self awareness.

When an agent has learned the relation between the angle at which pulses are emitted and the time taken for their return, it can use this to ‘signal itself’. It can encode a message for its future self by modulating head angle. The ability of an agent to signal itself is the true origin of self awareness and the concept of an enduring, unitary identity. Our obsession with artificial self-images is evidence enough of this. But the signalling could be more immediate. Standing in front of a mirror and watching the results of actions in your motor cortex is an example.

Signalling yourself is a kind of self reference. Douglas Hofstadter wrote very elegantly on this in “I am a strange Loop”. Recurrent neural networks embody loops like this to achieve some surprising things, including memory. Self reference is central to some of the deepest theorems in mathematics, Goedel’s theorem for example. The theory of recursive functions lies at the core of Turing’s idea of computation.

A learning machine encodes the relations between actions and sensations in the physical states of its components. In a machine that can signal its future, these sensations are caused by prior states of the machine itself. But it treats these sensations like all others and learns the correlations that result from this causal link. In other words, the machine learns to simulate its states as if they are states of the external world. It learns to model itself as a part of the external world.

This is not a new idea. Merlau-Ponty, the French philosopher of phenomenology, argued that self-awareness is fundamentally a consequence of our lived bodily experience and our direct engagement with the world. For Merleau-Ponty, self-awareness is inseparable from our perception and interaction with our environment. It is learned in the same way we learn everything else about our world and for the same reason: to enable us to implement effective strategies that will enhance our survival.

A similar idea was recently elaborated by Gemma De les Coves and her collaborators and draws attention to a defining characteristic of self-reference sometimes called the ‘Russell’s paradox’. Let me state this in an AI-centric way:

A conscious AI simulates all AI’s that do not simulate themselves.

Russell’s paradox arises from self-referential definitions—most famously, the “set of all sets that do not contain themselves.” In that classic form, we ask: Is the set itself a member of itself or not?—if it is, then by definition it shouldn’t be; if it isn’t, then by definition it should be.

We need to be more precise about simulation. I will take it to be synonymous with prediction. In my model of a learning machine, when an agent acts on the external world it also acts on its internal learning machine. The leaning machine then tries to generate, or predict, the actual sensation returned from the world to the agent at a later time. Learning consists in making the probability of an erroneous prediction as small as possible.

Let action a result in a predicted sensation ,

$\tilde{s}=f(a)$

The world however produces the actual sensation

$s=\Phi(a)$

Assuming both f and $\Phi$ are invertible functions we have that

$s=\Phi(f^{-1}(\tilde{s})$ ,

and

$\tilde{s}=f(\Phi^{-1}(s))$ ,

It follows that,

$\tilde{s}=s$ iff $f=\Phi$ .

This would imply that the agent is identical to the world, which cannot be right — the agent is a very small subset of the world and all learning machines necessarily make mistakes. The most we can say is that the prediction error is small. Prediction is uncertain.

The uncertainty arises in many ways. Firstly, the world function depends on many more variables than we can control, or even know. Secondly, it may not belong to the same class of mathematical functions the learning machine can learn. Thirdly, the actual sensation the machine records, $s$ , labels a state of the agent, not of the world, as do the actions, $a$ . The states of the world are fundamentally unknown and unknowable. They transcend the agent.

Suppose that the states of the agent are restricted in such a way that they are in one to one correspondence with the positive integers, while the world states are real numbers. There are infinitely many states of the world that are in one to one correspondence with possible states of the agent, but infinitely many more that are not. The world function must be like

$y=\Phi(x)$

where $x,y$ are real numbers. When the input is restricted to integers, $n=1,2,3\ldots$ the output is also an integer. One example is any polynomial function with integer coefficients. There are also polynomials with rational coefficients (not necessarily integers!) that always take integer values at every integer. These are called integer-valued polynomials. The binomial coefficients are an example. Then there are piecewise functions such as any function that is defined to equal an integer for each positive integer, but is arbitrary elsewhere, and periodic functions such as

$\Phi(x)= x \mbox{ mod } n$

More generally, for any sequence of integers, $(a_1,a_2,\ldots a_n)$ there exists a function on the reals such that $a_n=f(n)$ defined by interpolation (such as Lagrange interpolation), but takes arbitrary values outside the positive integers.

If world functions are in this class, there is going to be a problem with inversion. For example, the only polynomials with integer values at all integers and are invertible as real functions are the linear polynomials with integer slope. This is an example of a strictly monotonic (increasing or decreasing) real functions whose restriction to positive integers is injective and which assign only distinct integer values to positive integers. If you want the function to be both elementary and invertible, essentially only these strictly linear forms fit the criteria. Certainly periodic functions do not fit this category.

Restricting the class of world functions accessible to us has the consequence that we can never ensure that $\tilde{s}=s$ for all possible actions. Predictions will necessarily be imperfect. While some actions may result in completely predictable sensations, there are infinitely many actions that do not.

Can an agent learn an approximation to $\Phi(x)$ even if it is not invertible? Yes! Machine learning algorithms—especially neural networks—can approximate arbitrary integer-valued functions, including those that are not invertible. For a function that is constant or stepwise—clearly not invertible—neural networks can still learn to approximate the mapping from inputs to integer outputs. The price that must be paid is a small but non zero probability of making an error. That is to say the error probability to predict $\tilde{s}=s$ cannot be made to go to zero.

In the course of a long sequence of interactions with the world, an agent learns many world functions, including causal functions that enable it to control the world around it. But the agent itself is embedded in the world, so some of the functions it learns refer to its own physical operations.

This is what it means for a learning agent to ‘signal itself’. It learns how to predict its own behaviour, but not perfectly. It will always make mistakes, especially if it has access to restricted kinds of world functions. Such an agent is necessarily surprised by itself. Its ability to predict its future is limited. It never, truly, knows itself. A long life of interactions with the world, including self signalling, endow the agent with an internally stored prediction engine for its own behaviour in a changing world, but it is necessarily imperfect.

Different agents may differ in there ability to signal themselves and learn how to predict there future behaviour. Perhaps fruit flies can’t do as well as humans in this respect. It suggests that levels of consciousness are linked to the ability to learn, and to predict your own future behaviour. This has implications for how we understand free will. It also has implications for building engineered embedded learning machines.

A physical learning agent must always surprise itself because it is part of a world that necessarily transcends its finite and limited internal learning capability. The actual physical embedded thing transcends its own learned self image.

The Origins of Consciousness — Know Thyself: A Physical Theory of Self-Modelling

Consciousness is overrated. We spend large fraction of our lives in willing forfeit of it: too much consciousness will kill you. It is not involved in every action that we take. Each time we act in the world, new motor programs are generated unconsciously depending on the setting. If we had to be consciousness of every action and sensation, every minute of our waking lives, we would also likely die. There is just too much at stake in surviving a single day to hand it all over to conscious control.

Hyper vigilance is the name given to a state of “too much consciousness”. Hyper vigilance can cause increased anxiety, elevated heart rate, and challenges in social interactions. These symptoms can impact your life, making it difficult to relax or focus on daily tasks. Thinking becomes difficult, and perception is altered, leading to physical and mental fatigue, decreased fine motor skills, and impaired decision-making.

Consciousness is costly. The need to sleep is the ‘smoking gun’ for consciousness. All animals must spend a good part of every day recovering from it. That cost could only be born if it conferred a considerable advantage in survival rates. It seems unlikely that this advantage would accrue to humans alone. Animals need to sleep too, indicating that they bear the cost of conscious. It cannot be exclusive to humans but a widespread evolutionary adaptation.

What is the necessary hardware required for a living thing to be conscious? It seems likely that a central nervous system is essential. Simple organisms (e.g., bacteria, plants) lack the necessary neural complexity to be conscious. Insects and small animals may have minimal forms of awareness but not full consciousness. Humans, primates, cetaceans, and some birds exhibit strong evidence of consciousness. But what precisely does a central nervous system do, and are there other hardware enablers for consciousness?

All animals are learning machines. They consume energy to continuously reconfigure brains to predict the ever changing sensations that follow actions. In so doing they necessarily lower their internal entropy paying a price by dissipating heat into the environment. Contemporary engineered learning machines do the same but much less efficiently. They consume far more energy, and dissipate far more heat, than is necessary for learning.

All animals are learning machines, but are all animals conscious? Humans claim they are, but struggle to explain what this means, and are unsure if other animals are conscious. If animals are not consciousness, learning is not sufficient for consciousness.

A minimal model for a learning machine requires actuators, that change the local environment, and sensors that respond to changes in the environment. A small subset of those changes are due to internally generated actions. A learning machine must predict what sensations follow a given action. To do so they must distinguish internally generated changes in the local environment from those that are not. Evolution will drive this process to become ever more efficient.

Self-awareness is a necessary precondition for learning in embedded learning machines. If self-awareness is necessary for consciousness, then learning is necessary for consciousness, but is self-awareness sufficient for consciousness? Are all embedded learning machines conscious?

The simplest actuator for a learning machine is the ability to move. Llinas has made a strong case for motility as a necessary precondition for the evolution of a central nervous system in animals. Motility is not essential for robots, but it would appear to be a highly desirable feature. It certainly makes it far more difficult to engineer a robot if it is required to move rather than remain stationary.

The minimal requirement for an autonomous motile learning machine is an internal gyroscope and internal clock. Of course, a motile learning machine need not be autonomous. A drone equipped with sensors and actuators can learn a lot about the world, and effect changes in it, even if its motion is controlled remotely by a controller. Modern warfare has made this very apparent.

Here is a very simple example. Imagine an autonomous learning machine — an agent — confined to move on the circumference of a disc. In addition to an internal clock and a gyroscope, the agent can send and receive pulses of light. An internal actuator enables it to rotate the direction of emitted pulse —the pointer. I will assume that the boundary of the disk is perfectly reflecting so that a pulse of light sent into the interior will eventually be received back by the agent, if sent at rational multiples of pi. The gyroscope estimates the direction of its pointer as a function of the ticks of the internal clock. An example of possible configurations is shown below.

The gyroscope is a simple sensor that measures angular accelerations of the agent’s head direction. When combined with a clock, it enables the agent to keep a record of head direction. The agent can only `know’ its internal states indexed by ticks of the internal clock.

I will assume that the agent emits a pulse immediately after receiving a pulse: if a pulse is received at a clock count of m, a pulse is emitted at the same clock count m. The head direction at each step determines the clock count of the next emission. We can label the head direction `cause’ and the clock count for the next emission as ‘effect’.

Each emission event corresponds to a distinct internal state labelled by a setting of its internal gyroscope, labelled a, and a reading of the clock at pulse emission, n. Our protocol means that a pulse received at clock count of n is a pulse returning from the previous emission when the head direction was a. The only thing the agent has access to are these two things. An internal state is an ordered pair of numbers S=(a, n) where n is the clock count for the next emission, and a is the record of head direction at the previous emission. The first entry of the pair is a cause while the second entry is the effect, see figure below.

We now equip the agent with an internal learning machine. It works like this. For the first emission the agent elects a head setting, a, at random and sets the clock counter to n=0. Simultaneously, the learning machine is sent the head direction, a. It then quickly tries to predict when the next light pulse will be received by generating an integer n. For simplicity, I will assume there are only four settings for a, designated (1,2,3,4). These are labelled with angles given by

, but the agent does not know this. It only knows that there are four different headings, as recorded by its internal gyroscope. At each emission step one of these four headings is chosen at random.

Given our external god-like view of the agent’s world, we can easily give a relation between head angle and time taken for a pulse to return to the agent. If we assume that space is Euclidean inside the disk, the time taken is then given by

where I have used an arbitrary scale for time units. The shortest period occurs for a pulse sent along a diagonal and the longest for a pulse sent at 45 degrees to the diagonal.

The agent does not know this function. It simply generates a list of ordered pairs made up of a label for the head angle and number of ticks of the clock until the pulse returns. The data can be displayed like the table below.

Heading	Count
1	41
2	50
4	56
3	54
3	54
3	54
3	54
4	56
3	54
4	56

I will assume that the agent has machine that enables it to learn this function. This can be almost anything, for example a physical neural network or a decision tree. I will assume that is a machine that is described by a learning algorithm (nearest neighbour) and that it has a very large number of examples to train on. In the figure below I plot an example of the predictions made by this machine once it has been trained using 1000 training pairs.

The machine can use its sensors and actuators and learn to predict the return time of a pulse for a given head direction, working only with its internal states. If such agents were inclined to speculation, they might propose that the world they inhabit is a boundary to a two dimensional Minkowski disk. Or they might not. In any case, once the agent has leaned this relationship it can control when it will receive a pulse emitted at any head direction. This is the point of learning: once learned, the relationship can be used to intervene effectively on the world.

Once the agent has learned a good approximation to the ground truth function, it can begin to classify received pulses that it did not originate as spurious background pulses. This is the origin of self awareness. It is an entirely physical process and intrinsic to a physical learning machines. Even very simple learning machines would easily develop this ability.

Without self awareness, a robot equiped with a learning machine would be hardly viable and possibly dangerous. I conclude that embedded learning machines would be driven by evolution to become self aware.

How do we get from self awareness to consciousness? Before I can answer that, I need to consider how stable communities of learning agents arise. Agents must not only learn to distinguish self generated sensations from extrinsic sensations, they must be able to distinguish sensations that arise from the actions of other agents of the same kind. I will address this in the next post.

What Is a Photonic Qubit? How Single Photons Compute

In April 2024, PsiQuantum announced it will build the first utility-scale, fault-tolerant quantum computer in Brisbane, Australia, as part of an investment and partnership with the Australian Commonwealth and Queensland governments. This outcome builds on three decades of quantum computing research and development in Australia. The plan is to build a machine using millions of photonic qubits. What is a photonic qubit?

A computer is a machine. It consumes energy and generates heat in order to do work. What precisely is that work? I do work and dissipate energy when I walk up the stairs. The wind does work when it drives a turbine to produce electric power and also dissipates heat, as little as engineers can manage. Clearly a computer is dissipating heat, but what work is being done?

We all have an intuitive idea of what a computer does. They do calculations. That sounds like a rather abstract thing to be constrained by the laws of thermodynamics. But computation is no more abstract than a hand axe made by a neolithic hunter. Computers, like any other tool, bring order to the physical world so that it may be controlled. A very simple calculation might simply decide an answer to a question, TRUE or FALSE. Suppose we want to know if the number of people who have entered a room is greater than the maximum capacity of the room as stated by the occupational health and safety regulation. Let the maximum number be an integer, M, and the number counted into the room be an arbitrary number, N. The function we need to compute is f(N,M) =Max(M,N) . If f(N,M)=M, we comply with the rule. If f(N,M) ≠ M, we do not comply. The output has only two values , 0 or , (NOT 0). It is binary function with two inputs.

In this example, the kind of function is known and the input data are the numbers M and N. A simple analogue machine to compute this function is shown below. Each time someone enters the room they place a red marble in a bucket on the right. When the scale tips, the door to the room closes before the next person can enter.

A simple analogue binary comparator for two integers N and M . We run the device by loading M marbles into one bucket and N marbles into the other. If the device switches, N>M.

If the fulcrum has a lot of friction, the device has two stable states — OFF/ON — depending on which bucket has the most marbles. It is an over-damped binary “switch”. In a conventional computer we don’t use binary switches of this kind, we use transistors, but the principle is the same, a heavily damped system with two stable states — OFF/ON — and a control (a bias) to make it switch. The mechanical analogy can be made a little more realistic by adding a procedure to empty the bucket on the right enabling the switch to return to its initial state.

The basic elements of any switch are:

two stable output states, labelled S= (1,0) for (on or off).
a control input with states C that enables the state of the switch (S) to be changed
A lot of friction (damping) to ensure the state S does not randomly change independent of input (no bouncing, no leaking marbles, no leaky buckets).

This idea can be captured in the diagram below.

We can account for low friction by treating the switch as probabilistic, a bit more like a coin toss, where the control can continuously bias the coin. This is depicted below.

We would like this to be as sharp a transition as possible but a fundamental law of physics (thermodynamics) says it cannot be arbitrarily sharp.

Another kind of switch is the optical switch. This is a conventional device that controls the propagation of light in optical fibres or integrated optical circuits. The switching is all-optical in the sense that there is no need to convert pulse of light to pulses of current or voltage. Optical switches are essential in telecommunications for managing data traffic, network configuration, and switching between different signal paths. A number of physical phenomenon can be used for light-light switches, including mechanical, electro-optic, acousto-optic, and thermo-optic mechanisms.

To build a computer we need to build a cascaded array of irreversible switches, synchronised by a clock signal, so that the change in state of one switch can act as a control for another. In a modern silicon computer the basic controllable switches are transistors and signals are voltages. It is a very large array (trillions) of cascaded switches. We do not use optical switches for computer chips as the resulting device would be too large, however they are used extensively for optical communication systems. Optical switches use pulses of classical light generated by a laser. A quantum optical switch uses quantum states of light … single photon pulses.

An example of an classical optical switch is shown below.

An all-optical switch. Pulses of light travelling in optical fibres interact via a special material to enable switching. In this case two input are labelled as On (1) and OFF (0) and a control pulse can switch them from inout to output. In the non-linear material the refractive index changes with the intensity of the light and ideally no light is absorbed. A clock needs to synchronise pulses so that they have maximum overlap in the non linear material.

A quantum computer is a cascaded array of reversible quantum switches, synchronised by a clock signal. The overall device is reversible (in a perfect device) right up until the output is readout. That step is necessarily irreversible. Once a measurement is made a quantum switch cannot be reversed. This is a fundamental feature of measurement in quantum theory (sometimes called ‘collapse’). Thermodynamics again ensures that no measurement can be perfect so some measurements make mistakes and accidental measurements are also irreversible. This is how errors enter a quantum computer.

In a quantum optical switch, special quantum sources enable the creation of optical pulses that contain at most one photon per pulse. This is a relatively recent technology. The switch is now probabilistic but the theory of quantum optics enables us to calculate how the probability to detect the photon at either output changes as we vary the strength of the non linear refractive index change. This is shown below

An example of an quantum optical switch is shown below.

A quantum optical switch. Pulses containing one or no photons enter into optical fibres and interact via an optical non linear material. This changes the probability for detecting a photon at photon counters in such a way that the probability can be controlled by changing the strength of the nonlinear material. In all case the intensity of the control pulse is unchanged. The output is labelled by a single binary digit, 1 or 0, as only one of the two detectors can count a photon.

What is the difference between this and the classical optical switch? The surprising answer can be seen when we cascade two of these switches together with the non linearity set so that the there is an equal chance of getting a photon in the previous experiment at either photon counter .. a coin toss.

Two cascaded quantum optical switches. If w do not know which way the photon went in the middle section we can completely undo the first switch making the output certain.

The answer is a surprise. Now the photon is counted for certain on the same path in which it was injected. Putting the optical switches back to back in such a way that we have no knowledge of the path taken by the photons has taken complete uncertainty and turned it into certainty. This is the strange way probabilities can be controlled in the quantum world. The output of the first optical switch has no value at all as there is no way we can know it. The output state of the light at the first switch is not a single binary digit … a bit. We call it a qubit to make the difference explicit. It is not right to say it is both one and zero as it is ‘unknowable’. There is still light there, and it is in a definite quantum state, but it is not describable in natural language using the abstract noun ‘bit’.

The basic switch I have described is a Fredkin gate. I proposed this design way back in 1989. It suffers from a serious problem: no such non linear material exists .. yet. But in 2001 Manny Knill, Ray Laflamme, and I found another way to do it, generally known as “KLM”.

PsiQuantum are building vast cascaded arrays of photonic switches. The objective is to be able to inject data encoded into strings of single photons and arrange all the paths so that the probability of getting the value of the required function is as close to one as possible.

There are many engineering challenges. Here are some of them:

finding a way to make single photon pulses behave this way without using a non linear optical material.
making single photons on demand
detecting single photons ( that is to say, being able to discriminate between a count of n-1, n, n+1 …
not losing any photons inside a vast cascaded array of switches.

All these problems have been solved at some level of reliability. The solution to the first problem was a breakthrough in 2005 when Dan Browne and Terry Rudolph figured out a out a far more efficient way to use the KLM scheme. This is what made PsiQuantum’s approach technically viable. Fortunately much of the optical engineering is a refinement of classical time multiplexed optical networks fabricated in monolithic materials not fibres.

The PsiQuantum scheme is in reality a fully quantum implementation of integrated optical circuits using highly quantum single photon states rather than classical laser pulses. Seen this way the engineering challenge comes into focus — and a pretty good focus at that. One day I hope I will be able to send Cloud queries to an optical quantum AI made in Australia.

Unredeemable Time: Why Every Clock Is an Irreversible Machine

What is time? If clocks measure time, can we answer the question by a careful study of clocks?

A clock is a machine and, like all machines, subject to the laws of thermodynamics.
In popular discussions, a clock is often presented as the epitome of a reversible and
predictable dynamical system. Nothing could be further from the truth. A careful examination of the requirements for a clock show that a it cannot be reversible, and in fact requires friction, entailing heat loss, to operate. Irreversible systems generate entropy. A clock is a flow meter for entropy.

Clocks come in many forms, from bio-chemical to astronomical, from
mechanical to statistical. A dictionary definition states, “a clock is a device to
measure time”, which begs the question. Other definitions emphasise the repetitive, or periodic, nature of clock states. Yet radio carbon dating is a clock that is anything but periodic, as is the water clock used by Galileo at the birth of modern physics.

I will take the view that a clock is a physical device used to coordinate local co-
incidences of physical events. If clocks measure time, this definition captures the relational character of time: for three local events, if event C coincides with event A and also coincides with a distinct event B, then A happens at the same time as B, up to an error determined by the precision of the clock. There is no absolute simultaneity, even putting relativity aside. If clocks measure time, then now is fuzzy.

As an example, suppose you are in a kitchen with a dripping tap. A digital clock/timer on the wall is counting up in unit steps. When a drip hits the basin, a sound pulse is generated. At each repetition of the sound, take a note of the count on the clock and tabulate the results. Place a 0 if no sound occurs at a particular count and a 1 if a sound does.

Event (x)	Count (n)
1	1
0	2
1	3
0	4
1	5
1	6
0	7
1	8
0	9
1	10
0	11
1	12

A clock table of count versus event.

It is clear that the event is almost periodic according to this clock but this is entirely relational. There appears to be an error at n=6, but we can equally well claim that the event is truely periodic but that the clock is imprecise. I have not said what the units of the clock are … they are irrelevant. The phenomenon is already captured by the table.

The table represents an integer valued function of a single binary variable. Where is the continuous time of physics, the little t ? A little known theorem of integer valued functions tell us that they can always be parameterised in terms of a real variable t. As far as clocks are concerned this is a mathematical trick, but it makes the job of theoretical physicists much easier. But keep in mind that little t is not time. It is simply a mathematical device. All real parameterisations are equivalent. We chose the one that makes the calculations easier. This was the key insight that Einstein injected into general relativity.

The relational nature of time as measured by clocks is critical to understanding the nature of time. Ernest Mach, the famous 19th century physicist that inspired Einstein, saw this very clearly.

Physics sets out to represent every phenomenon as a function of time. The motion of a pendulum serves as the measure of time. Thus, physics really expresses every phenomenon as a function of the length of the pendulum. . . If one were to succeed in expressing every phenomenon. . . as a function of the phenomenon of pendulum motion, this would only prove that all phenomena are so connected that any one of them can be represented as a function of any other. Physically, then, time is the representability of any phenomenon as a function of any other one

Returning to clocks as machines. The pendulum clock invented by Huygens in the 17th century is typical. Friction causes it to run down unless we give it some kicks. Huygens figured out a way to do this using a falling mass and an escapement wheel coupled to the pendulum. The escapement gives the pendulum two kicks every cycle. That is the sound you hear as tik-tok.

The escapement must be heavily damped to prevent it from bouncing. Most of the energy per cycle is lost here. The combined system is no longer a simple oscillator but a special kind of non liner oscillator, a relaxation oscillation. It remains to add a counter…the dial.

There is a price to pay for stable, continuous operation. The clock is no longer reversible and necessarily dissipates heat. The laws of thermodynamics mean that the period of such a system is not fixed but suffers small fluctuations from one cycle to the next. The average period can be fixed but the uncertainty cannot be reduced to zero. Engineering a good clock means finding a way to make the uncertainty in the period as small as possible.

It turns out that the greater the rate of heat and entropy produced by the clock the better it is. Good clocks are pumping disorder into the world and the faster the better. A clock is a flow meter for entropy. If it measures anything it measures the rate at which disorder is increasing in the world.

You may object that astronomical time, based on the motion of the earth and planets, looks pretty regular, quite ‘clockwork’. Where is the heat being generated? The answer is surprising: heat is generated in the planets due to non uniform gravitational fields stretching and squeezing the matter and generating heat. This is called tidal heating. It leads to a loss of precision over vast numbers of orbits. The phase of the orbits becomes scrambled on the order of 100 million years. This is much longer than our quotidian concerns, and so astronomical time is good enough for us.

National governments spend billions of dollars annually to support laboratories to make better clocks. Our sophisticated technologies for computing and communication require better and better clocks. Our global navigation system is limited by the precision of clocks in satellites. Currently the best clocks under development are based on a special kind of relaxation oscillation, the laser, locked to a quantum oscillation of the charge distribution in an atom that the light illuminates. Engineering better clocks requires new quantum technologies and this is big business.

Clocks are irreversible, entropy producing devices. What does this tell us about the nature of time? Firstly, time is not the reversible, perfect little-t of physics. The equations of physics are reversible in little-t. The world is not, and neither are clocks. The equations of physics provide an idealisation of the changing world. Secondly, we are only too aware that time is irreversible for that is what aging is. Time is personal for us. We change and decay in our little patch of the universe. All time is unredeemable and local. Here and now is all there is.

Why Learn? Learning as Entropy Reduction

The purpose of a learning machine : first discover patterns in the world, and second, exploit those patterns to intervene more effectively in the world. The flip side of optimal learning is optimal control.

Suppose you are walking across a stony plane, perhaps an ancient riverbed. Most of the rocks you encounter are smooth and rounded, of various sizes, a dull collection produced by the power of running water. Then you encounter something surprising: a rock with sharp edges, concave ablations, and an easy fit to your hand. Your surprise reflects the fact that you need to update your expectations about the stony plane. Intuitively, you update your statistical expectations. The distribution of stones has changed. The randomness of stones has been lowered by the unexpected find.

Entropy has two meanings in science. It has a purely mathematical meaning related to statistical uncertainty as the previous paragraph suggests, but that is not how it first entered science. In the early days of the Industrial Revolution, entropy had a very different meaning. It was used as an index to order the kinds of physical changes, natural or engineered, that can occur; heat flows from hot to cold, a drop of ink diffuses uniformly in a glass of water. In both cases the final state has higher entropy than the initial. These processes are spontaneous. The real question is how did the initial state of lower entropy come about in the first place? Why is the distribution of stones not as random as possible.

We can transform a localised system from a high entropy to low entropy by physically intervening on that system. This costs energy and it must be supplied by a controller, for example, a battery or the sun. The energy can be in the form of heat or due to applied forces, for example an electric field in a transistor. However it is done, the price to pay for lowering the entropy of a system is to increase the entropy in the environment by at least as much as the entropy is reduced in the controlled system. Entropy, in this physical sense, imposes constraints on the kinds of interventions that are possible.

The connection between the statistical/information sense of entropy and the physical (thermodynamic) sense can be made in many ways. The most important of these connects physics and learning, and returns us to the stony plane. The sharp-edged stone axe heads, scattered among the water tumbled smooth stones, is evidence of a special kind of controller. Considerable energy was expended in their making for sure, but also a very special kind of controller, a learning machine.

A machine could make stone tools by randomly chipping away at smooth stones, but almost every stone would be useless as a tool and a great deal of energy would be wasted. Almost every attempt would be an ‘error’ , a non-tool. A machine that monitors its own performance and changes its internals settings to reduce the error probability in each trial will learn to make good stone tools efficiently.

This does not come for free as the learning process itself generates waste heat, dissipated energy. Learning itself reduces the thermodynamic entropy of the learning machine and this enables it to more efficiently create good tools with little error. Learning enables a controller that is a pool of low entropy that can be used to efficiently lower the entropy of its surroundings.

Here then is the link between informational entropy and thermodynamic entropy. Learning necessarily lowers the error of control and in so doing enables a powerful thermodynamic resource for reconfiguring the world. Biology discovered this trick a long time ago. We are now learning how to emulate it, thereby creating new paths to wealth and security. After all, a hominid with a hand tool is wealthy relative to hominids without one, and a threat.

Strange Things, Straight Talk: What Quantum Superposition Really Means

When forced to explain quantum superposition in plain English, physicists have an unfortunate tendency to equivocate. They will utter variations on ” a quantum superposition means a particle is in two different STATES at the same time”, or even worse “a quantum superposition means a particle is in two PLACES at the same time”. No experiment in any lab has ever seen this. A particle is always detected at the same location as the particle detector.

Why is this equivocation? It is trading on the ambiguity of the word STATE. In physics, the state of a physical system is a list of numbers that enables you to predict, with little error, the results of any measurement you wish to make upon it. No measurement is ever perfect so this list does not determine the result of measurement (see my earlier post).

In classical mechanics (CM) this list is the position and velocity of every particle in the system. Given this, Newton’s equations predict the position and velocity of every particle into the future (and the past). The position and velocity are real numbers and we cannot give them arbitrary precision. In many case finite precision will suffice for predictions a few time steps at a time without too great an effect on error. Anything you can measure is a real valued function of the numbers in this list. Classical physicists tend to think the state is ‘out there’ waiting to be discovered in greater and greater accuracy. They think that a physical state has an objective status that is discovered by measurement.

In quantum mechanics (QM) the physical state is a list of complex numbers. (Each complex number is actually comprised of a sub-list of two real numbers.) Together with Schroedinger’s equation, this list of complex numbers (or list of pairsof real numbers) determine the probability to obtain a result for any kind of measurement you care to make upon the system. This is a very different kind of state to the classical state. We use the same word ‘state’ to refer to both lists as they determine the statistics of outcomes for anything you care to measure.

Physicists, if forced to explain quantum superposition, equivocate over the precise meaning of `state’ they are using. In both classical and quantum physics a carefully prepared physical system is only ever in a one state at a time. It is the meaning of the word state that is different in the two theories.

Now here is a curious fact. In CM, if you know the state with perfect accuracy, you can predict with certainty the results of every measurement you may care to make. In QM however if you know with perfect accuracy the list of numbers for the quantum state, there is always one measurement for which the results are completely random. This is the most general statement of the Heisenberg uncertainty principle. In CM, if you know the state perfectly, all measurement results are determinate. Knowledge of the whole is knowledge of the parts. This is never the case in QM. Knowledge of the whole can coexist with total ignorance of the parts.

Is the quantum state out there? Is it an ontological fact about the world? Or is it something else, perhaps simply a way to encapsulate all we have learned about our interventions in the quantum world. Physicists are still arguing about this, but the argument is moribund. We have been arguing about it for almost a century yet this has not slowed the discovery of new physics.

The concept of learning machines gives an alternative response: neither CM or QM make ontological claims. They make claims about the kinds of functions physical learning machines like us can learn in order to intervene effectively in the world. Kant said sometime similar long before the quantum world was discovered. It is time to heed the message. The problem is not the interpretation of quantum mechanics, the problem is the interpretation of classical mechanics.

Learning Is Not Memorisation: The Energy Cost of Remembering Everything

There is but one goal for a biological learning machine: survive long enough to reproduce. Learning changes the odds in your favour only if you don’t spend so much time and energy learning you die young. Ultimately, the goals of evolution and thermodynamics are aligned.

A simple learning task is classification. For example, suppose some red berries you encounter are packed with energy, but some kill you. To distinguish the two kinds of berries requires taking careful note of a lot more than simply the colour of the berry. Perhaps the good red berries have clustered skins (raspberries), but the ones with smooth skins will kill you. Perhaps some smooth-skinned red berries are OK but not if they are overripe. To find the safe red berries you need to take account of many more features.

You could simply not eat any red berries, and die of hunger in the winter. Another way to approach this problem; try to remember all the local factors that were present the last time you saw a member of your tribe eat a berry and die. The ‘memory first’ approach requires a lot of experience and a very good memory. There might be a lot to learn and a lot to recall and that may take so much time that … well, you are deselected. The objective in learning is to avoid memorisation. You don’t need to recall every detail in every situation in which you encountered a red berry. You only need to recall the patterns associated with good berries, and the patterns may be relatively easy to recall … if you can find them efficiently.

The kind of memorisation I am talking about is not how learning agents do memory. We tend to think our memories are like records in the old, analogue world, simply a photo or movie reel, that ‘we’ view in the Cartesian theatre. Our memories are not like that. If they were, it is doubtful we could function at all, like Funes the Memorious in the Borges story. The memories of a learning machine are reruns of learned predictive simulations, and every re-run is a new, different experience.

It takes a lot of resources to memorise every instance of a problem and a lot of time to search the data to know what to do in a novel encounter with a similar problem. In general the effort required, in both time and space, grows exponentially with every instance of the problem, even if the output is simply yes or no. For example, suppose you need to answer all possible 20 yes/no questions before you can decide if the outcome is good or bad. There are 2 raised to the power of 20 such inputs and each one could give two possible outcomes. That means to memorise this function you need to store

sets of yes/no values in order to have a complete look-up table. That is a very big number. Yet the key feature you need to learn might simply be that the good output occurs, almost every time, when all the twenty questions have an equal number of yes answers as no questions (even parity). You don’t need to memorise every instance of the problem, you only need to learn a particular feature of the problem that gives a low chance of making a mistake. And that may take a lot less effort to acquire … if you are a learning machine and are faced only with binary valued functions of many binary inputs.

The thermodynamics of this problem are important. If you simply memorise all instances of the problem you need to set aside locations in memory for 2²⁰ bits. Yet if you learn the key feature of the problem , say all you need to know is if the input has even parity , then you can store much less data.

If all inputs are equally, likely then this means the average entropy of what you need to store has decreased dramatically. In a physical machine (like you) entropy reduction like this requires that you pay a thermodynamics price by emitting enough heat into the environment to raise its entropy by as least as much as your learning machine has reduced your internal entropy. This is why all learning necessarily requires heat generation. This is simply a restatement of the Landauer erasure cost, if that means anything to you. Learning offers a huge evolutionary advantage.

Current AI (based on conventional silicon chips) learns well at a huge thermodynamic cost. Is it the minimum cost required? Absolutely not. If it were, no biological learning machine would ever have evolved on Earth. What is the minimum cost? AI engineers are trying hard to figure this out before they go broke paying the power bill. (That is what natural selection means in silicon valley. ) I think the answer is quantum.

Quantum — Noise and Error: Why Some Uncertainty Can’t Be Removed

Every physical experiment is subject to some uncertainty. Repeated measurements on physical systems prepared in the same way necessarily do not give identical outcomes. The results fluctuate.. This is noise. If the measurements are designed to verify that the preparations are identical, fluctuations will ensure that some give YES, but a few will give NO. These are errors. If the systems in questions are quantum systems, fluctuations in the measurement results is called quantum noise.

If all noise refers to fluctuations of measurement results why should we distinguish quantum noise from classical noise? The difference is subtle. In the classical case ( the physics of Newton, Maxwell, Boltzmann…), we assume that, with enough effort, we can make noise and error arbitrarily small, for example by lowering the temperature. In the quantum case this is impossible. The quantum world is a source of irreducible uncertainty. Fortunately quantum theory shows us how to manipulate the odds to our advantage. We have many more levers available than simply lowering the temperature. Quantum technology is the business of engineering those levers to control noise by directly intervening in the quantum world. The discovery of quantum mechanics is the discovery of new ways to intervene in the physical world.

Learning is impossible without noise and error. It should come as no surprise that quantum noise enables new kinds of learning machines. I will return to this in the next post.

Machine Learning or Learning Machine? The Difference Physics Makes

Biology offers abundant evidence that physical systems can learn, that is to say, physical systems can exhibit stable behaviour, conditioned on prior interactions with an external environment, in order to achieve a goal. We are entering an era in which learning machines can be engineered. In which case, what are the physical principles in play?

A learning machine can be instantiated in any physical system and not necessarily digital. Biological learning in brains is not based on algorithms running on digital computers, even if it can be simulated that way. What are the physical principles required for a machine to learn?

A learning machine, like any machine, is an open, dissipative physical system driven far from thermal equilibrium by access to a low entropy source of energy, for example, a battery. I will focus on simple classification in supervised learning. Here the objective is to learn a binary valued function, f(x), of the input data, x, by giving the machine a list of examples (x, f(x)) and adjusting the parameters of the machine through feedback so that the actual outputs are correct almost all the time. Error cannot be removed in a learning machine: it is an inherent feature of all learning. If you never make a mistake, then you never learn anything. if you only make mistakes , then you never learn anything either.

In a learning machine, reducing the error to zero in a finite machine would violate the laws of thermodynamics. The goal is to reduce the error probability, while making efficient use of the available thermodynamic resources.

A machine learning algorithm however is a mathematical procedure for approximating functions running (usually) on a conventional CMOS based von Neumann computer. There are very many machine learning algorithms and the discovery of new ones proceeds at an incredible pace. I want to contrast algorithms run on computers to actual machines that learn by thermodynamic constraints. In many ways this reduces to the question of who or what sets the goal? Who or what sets the error function? In a learning machine the goals are ultimately set by thermodynamics (in an evolutionary setting). In contrast, in ML algorithms, the algorithm designer sets the goal.

I am interested in quantum machines operating at very low temperature (they are cheaper to run), in which case the goal is to learn by exploiting quantum noise. How can quantum noise be harnessed for efficient learning? I will pursue this approach in future posts.