Where machine learning comes in

Start with Part 1 if it is still unread. This page assumes force is the slope of the energy, and that a pair potential adds up every pair of atoms.

1. The best answer is quantum mechanics, and it is painfully slow

Every model in Part 1 was a guess at the shape of the energy. There is one way to stop guessing: work out what the electrons are actually doing. That is quantum mechanics, and it gives real answers, bonds that break, metals that behave like metals, chemistry that happens.

The catch is the cost. Here are the real numbers from a simulation of one small molecule of five atoms, measured on a laptop:

2.6 seconds does not sound slow. But remember the loop: we need this every single step, and a step is one femtosecond. To simulate one nanosecond, still a very short time in biology, we need a million steps.

And that is for five atoms. A protein has tens of thousands. This is the wall the whole field ran into.

2. The idea: learn the answer from examples

Here is the trick, and it is simpler than it sounds.

We cannot afford to run quantum mechanics a million times. But we can afford to run it a few thousand times. So do that: pick a few thousand arrangements of the atoms, and compute the exact energy for each one. That is our set of examples.

Then fit a flexible mathematical function to those examples, one with enough knobs to bend into almost any shape. Afterwards, throw away the quantum calculation and just use the fit. The fit runs in microseconds.

The widget below shows this happening. The grey curve is the truth, which in real life we never get to see. The dots are the expensive calculations we chose to run. The blue curve is what a machine learning model builds from only those dots.

3. The catch: we cannot just feed in the coordinates

So we need a function that takes atom positions and returns energy. The obvious thing is to feed the raw numbers, x, y, z for every atom, straight into a neural network. This does not work, and the reason is important.

A molecule sitting in empty space has the same energy no matter where it is, no matter which way it is turned, and no matter what order we happen to list identical atoms in. Physics does not care about any of those. But the raw numbers change completely.

If we trained a network on raw coordinates, it would waste all its effort learning that turning a molecule does nothing, and it would still get it wrong for angles it had never seen. Worse, it would give two different answers for the same molecule described two different ways.

4. The fix: describe each atom by what it can see

Instead of feeding in positions, we give the network a description of each atom's surroundings, built only out of distances and angles to its neighbours.

Think of it as a fingerprint of the neighbourhood. "There are two atoms about 1.1 Å away, one at 1.8 Å, nothing else nearby, and the angle between the first two is 109°." That description is unchanged if we move the molecule, turn it, or relabel identical atoms, because it never mentions where anything is in absolute terms.

5. What exactly does the model predict?

This is the part people get most confused about, so here it is spelled out.

What comes out of the model is one number for the whole system: the total energy of all the atoms together. Plus the forces, which are just the slope of that one number, three numbers per atom. That is all. There is no separate prediction for each atom.

But inside, the model works atom by atom. Each atom's fingerprint goes through a small network which returns a number, and those numbers get added up:

So does that mean the model learns "the energy of one atom"? No, and this is the important bit.

The per-atom numbers are the network's own invention. It is free to divide the total between the atoms however it likes, as long as the total comes out right. Two differently trained networks will split it differently and both be equally correct. So we should not read a single atom's number as "how much energy this atom has": that quantity does not exist, because an atom's energy only means anything in relation to its neighbours.

Why bother splitting it up at all, then? Two reasons, both practical:

• It works for any number of atoms. Train on molecules with 5 atoms, then run on 500. There is no fixed-size input, just one network applied once per atom.
• It builds in the physics we want for free. Adding up per-atom terms makes the energy automatically unchanged when we relabel identical atoms, and it makes a system of two far-apart molecules come out to exactly the sum of their separate energies.

The widget below uses a real trained model. Drag the slider to stretch a bond in a methane molecule and watch what happens.

6. Two more pieces, and we have a real model

The sum is many-body, even though it looks like the pair sum

Adding up per-atom terms looks like the pair sum from Part 1, but there is a crucial difference. A pair term only ever sees two atoms. An atom's fingerprint sees all of its neighbours at once, so the network can learn exactly the kind of crowding effect that EAM had to hard-code, and anything else besides. It is many-body without anybody deciding in advance what the many-body rule should be.

It also means the model does not care how big the system is. Every hydrogen atom shares the same small network. Train on small molecules, run on large ones.

Learn from forces, not just energies

Every quantum calculation gives us the energy, one number. But it also gives us the force on every atom, almost for free. For a molecule of N atoms, that is 3N more numbers, all from the same expensive calculation.

7. Putting it together

Here is the whole recipe, in order:

1. Choose a few thousand arrangements of the atoms.
2. Run the slow, accurate quantum calculation on each one. Keep the energy and the forces.
3. Turn each atom's surroundings into a fingerprint of distances and angles.
4. Train a small network: fingerprint in, that atom's energy out.
5. Add up the atoms to get the total energy. Take its slope to get forces.
6. Put those forces into the loop from Part 1 and run the simulation.

That is a machine-learned interatomic potential. It is not magic and it is not a black box replacing physics; it is a very good curve fit to a physics calculation we could not afford to keep repeating.

8. What it does not fix

Worth being straight about, because these are real limits.

• It only knows what it was shown. Ask it about a situation unlike anything in its training set and it will answer confidently and possibly wrongly. This is the central difficulty in the field.
• It still needs the expensive calculations. We have not escaped quantum mechanics; we have spread its cost out.
• It is slower than a pair potential. Roughly a thousand times slower, just a hundred thousand times faster than quantum. It sits in between, which is exactly why it is useful.

None of that stops it working. Modern general-purpose versions of this idea are trained on millions of structures across most of the periodic table, and they have genuinely changed what simulations people can run.

Aayush Neupane
CS Researcher & Engineer · MS CS, University of Kansas

More physics →