One of the main challenges of distributing a physics simulation across many independent physics workers is how to make them collaborate. Off-the-shelf physics engines are fully authoritative over the entities they know about, and aren’t designed to be aware that other physics engines exist, let alone collaborate with them.
Consider the case of a line of dominoes. When the whole line is toppling, each piece is pushed by the one before it, and in turn pushes the one after it, so there is no clean way to split the entities in two disjoint subsets that will be simulated independently.
But suppose the line of dominoes you want to simulate is so long that two physics workers are required to deal with the workload. SpatialOS delegates authority over each domino entity to only one of them, but for the reasons outlined above, some regions of space need to be simulated simultaneously by both.
This is a really hard problem that hasn’t been solved before in a satisfactory way. Distributed physics is not an embarrassingly parallel problem.
To convince ourselves that our solution works, we designed the harshest, most difficult test case we could think of: not just co-simulation at region boundaries, but fully overlapping authority regions — and randomly switching the authority for each entity between the two physics workers. If we got it wrong, we would see blocks passing through other blocks, the tower tumbling, and massive forces being applied by one block to another causing catastrophic structural instability.
In this video, two physics workers are collaboratively simulating a stack of cubes. While each physics worker can “see” all the cubes, it only has authority over half of them, coloured green, and no control whatsoever over the other half, coloured red; and these subsets are constantly changing. Despite all this, the tower is as structurally sound and as realistically simulated as if it were on a single physics engine.
SpatialOS’s ability to transparently combine the output of each of the workers into the canonical state of the entity in the simulation, and synchronise this state to the other worker, makes it possible to add co-simulation support to any engine. Worker allocation is extremely dynamic and fluid — in this video, their regions are fully overlapping, and authority switches randomly several times per second.
Of course, this is a specifically designed, pathological edge case that will rarely occur in a real-life application, but it’s an extreme example of SpatialOS’s ability to distribute physics, and indeed any other kind of computation such as traffic simulation or flocking behaviour, without the notion of server boundaries.