Networking & Collaboration

The module

Networking in the Engine

All of the engine's networking lives in one module, HorizonNet. It is a layered stack: sockets at the bottom, encryption over them, a message layer over that, and — at the top — two independent consumers that share everything below and nothing above.

Why the two top layers are separate

Gameplay and collaboration share the transport and nothing above it, because their requirements are opposites. Collaboration replicates authored edits: rare, reliable, and they must never be lost — losing one silently discards somebody's work. Gameplay replicates simulation state: about thirty times a second, and loss-tolerant, because a dropped snapshot is corrected by the next one a few milliseconds later.

Forcing gameplay down the collaboration path would make every position update a reliable, ordered message. Forcing collaboration down the gameplay path would quietly drop an edit. So they are two consumers, not one generalised system.

What else uses the network

The engine reaches the network in only a few other places, all through the same module's HTTP clients — which are platform-native (Apple, Windows and libcurl back-ends) rather than a bundled TLS stack:

Architecture

The Transport Stack

Networking lives in HorizonNet, a layered stack where each layer wraps the interface below it rather than the concrete type. Encryption is a decorator, so it can be dropped in or tested around without either side knowing.

Why the split matters

HorizonNet does not know what a scene is. It sits below the scene layer, so reaching up into it would invert the dependency and drag the entity system into every networking test. Snapshot capture and application are abstracted behind an interface the editor implements; the tests supply a trivial in-memory stand-in instead.

The same discipline runs through the protocol. Lock subjects, presence selections, document scopes and the project key are all opaque values to HorizonNet — it relays and compares them without ever interpreting them. What they mean is decided one layer up.

Encryption

The handshake derives a session key from an ephemeral X25519 exchange, so sessions have forward secrecy: the private halves are wiped once the handshake completes, and recording the traffic and later learning the join code does not let an observer decrypt it. Data frames are AES-256-GCM with a counter nonce and an authentication tag, so tampering is detectable and a captured handshake cannot be replayed into a session.

Getting through

Discovery & NAT Traversal

Two machines on the open internet cannot simply find each other. One sits behind a router that drops unsolicited inbound connections, and neither knows the other's address. HorizonNet addresses both, and is honest about the cases where nothing can.

The session directory

A host publishes where it can be reached under a short, high-entropy session id; a peer looks that id up. This is address discovery only — no session traffic ever passes through it.

The division of responsibility is deliberate. The directory learns an address, a port and a display name. It never learns the join secret: authentication is engine-to-engine, so a compromised directory still cannot join or read a session. The server records the address it sees the request arrive from rather than one supplied in the body, so a client cannot point other peers somewhere else. Registering returns a management token, so merely knowing a session id is not enough to drop or hijack an entry.

Asking the router for a port

Before publishing, the engine tries to open the way in by itself: it discovers the router with an SSDP multicast search, then asks for a port forward over UPnP, falling back to NAT-PMP. Routers generally speak one or the other — an Apple base station answers NAT-PMP and stays silent to SSDP entirely — so a UPnP failure says nothing about the second attempt. For IPv6, where there is no address translation to undo, it asks for a firewall pinhole instead.

Then the directory tries to connect back. That verification matters: a mapping can succeed while the host is still unreachable, and publishing an endpoint nobody can dial produces a session that simply never works, with no clue why.

When it cannot work

A failed mapping is a normal outcome, not a bug. Many routers ship with UPnP disabled — it is a known attack surface — and corporate and campus networks block it deliberately. Behind carrier-grade NAT, common on mobile connections and some ISPs, there is no forwardable port at any level: not automatically, and not by hand either.

The editor reports the router result and the reachability result separately, because they can disagree, and then says what you can actually do — forward the port yourself if the router is yours, otherwise put everyone on a mesh VPN such as Tailscale or ZeroTier, whose free tiers relay the traffic. Peers on the same local network always get through without any of it. Under CGNAT the advice deliberately omits manual forwarding: suggesting it would send someone off to spend an afternoon on something that cannot work.

Layer 3a

Gameplay Replication

The networking layer for a running game, as opposed to the editor. It is server-authoritative: the server simulates, samples the world at a fixed tick, and sends each client the entities relevant to it. Clients apply what arrives.

What is replicated

Entities carrying a Network component, and only those. An entity without one is purely local — which is how muzzle flashes, debris and other cosmetic effects stay off the wire entirely. The component carries the entity's network id, which participant is allowed to drive it, and two bandwidth levers:

Two mechanisms, two different problems

Other entities are interpolated between the last two snapshots, so they do not visibly step at the tick rate on a higher-refresh display.

The player's own entity is predicted: input is applied locally the instant it happens, then replayed on top of whatever the server later confirms. Without prediction, moving would only respond after a full round trip — which feels wrong above roughly 50 ms of ping, and no amount of interpolation hides it, because interpolation is about smoothness, not latency.

Positions are quantised against a configured world extent rather than sent as raw floats, which is where most of a snapshot's size saving comes from.

Layer 3b

Collaboration Sessions

A collaboration session lets several people edit one project at the same time. One editor hosts; the others join. Everyone sees the same scene, watches each other's cameras move, and edits graphs and widgets side by side — the changes appear as they are made, not when someone saves.

The design is host-authoritative. The host owns the participant registry, assigns ids, and holds the only copy of the lock table. Every change a client makes is checked by the host before it is relayed. That is what makes a lock mean something: a request is answered by one authority, in order, with no window in which two people can both believe they hold it.

Getting in

Hosting & Joining

Open the Collaboration panel and press Host. The editor opens a port, generates a join code, and publishes the endpoint to the session directory under a short session id. A guest needs only those two values — never an IP address, and never a port number.

The join code is machine-generated, not chosen by the user. Anyone who captures a handshake could brute-force a weak passphrase offline at their leisure, so the engine does not offer the option to pick one.

Sessions on this network

A host also announces itself on the local network, and the panel lists what it hears: who is hosting, which project, how many people are in it, and — when the session carries them — that it also transfers meshes, textures and audio. Picking a row fills the session id in for you. The join code is still typed, because it is the only thing keeping strangers on the same network out.

A row that cannot be joined says why on the row itself: a different engine version, or a different project. A session that is merely missing from the list sends people off to ask each other what is wrong.

When peers cannot reach you

Publishing runs the port-forward attempt and the reachability probe described under Discovery & NAT Traversal. Both results appear in the panel, and when the host is not reachable it says what you can actually do about it. Guests on the same local network always get through regardless.

Everyone needs the same project

A session addresses everything — scene entities, asset references, lock subjects — by UUIDs that only mean something inside one project. Joining a host who has a different project open is therefore refused, and the refusal names the project you need to open.

The check uses a stable id in the .heproj manifest, not the project name: two people routinely have differently named copies of the same project, and identically named copies of different ones. Projects created before this existed get an id the first time they are opened.

Protocol

The Join Handshake

Once the encrypted link is up, the collaboration protocol takes over.

# client → host
JoinRequest        protocol version, display name, project key,
                   client key, avatar, colour preference,
                   "I accept large assets"

# host → client — checked in this order, first failure wins
JoinRejected       VersionMismatch       # different engine build
                   Banned                # the host ejected you from this session
                   ProjectMismatch       # different project open (+ the host's project name)
                   LargeAssetsRequired   # this session carries them and you have not agreed
                   SessionFull           # participant limit reached
                   SnapshotFailed        # the host could not capture its own scene

# host → client, on success
JoinAccepted       assigned id, state sequence, assigned colour,
                   "this session carries large assets", current roster
LockTable          every lock that already exists
SnapshotBegin
SnapshotChunk…     the scene, chunked (progress bar, bounded allocation)
SnapshotEnd

# host → everyone else
ParticipantJoined
# …and on disconnect
ParticipantLeft

The order of the checks is deliberate. The version check runs first, so a peer on an older build is told it has the wrong build rather than the wrong project — an older build does not send a project key at all, and reading a missing field would produce exactly that wrong answer.

The snapshot is captured before the peer is admitted. If the host cannot produce one, the join is refused instead of completing with a peer that sits on an empty scene believing it is in sync. It is sent in chunks rather than as one blob so the joiner gets a progress bar and never has to accept an arbitrary allocation announced by the other side.

Reference

What Is Synchronised

Three layers, each at the granularity that suits what it carries.

Per item, immediately

Node graphs and the UI designer sync one item at a time. The receiving editor patches the document it is already showing — its canvas, selection and undo history all survive.

Ordering travels too, where it carries meaning: sibling order in the UI designer is draw order, so a pure reorder — where no element's own data changed — is its own kind of change.

Per entity, immediately

Entities the engine generates per machine — terrain chunks, the environment's sun and moon — are deliberately excluded. Every peer makes its own, so sending them would duplicate them on arrival.

Whole file, about once a second

Underneath the live layer, an edited asset is written to disk and pushed to peers on a short debounce. This is the baseline: it persists everyone's edits into their own copy of the project, serves a peer who opens the same tab later, and covers the asset types that have no item structure to diff.

The heavy files are a decision, not a rule

Meshes, textures, audio, fonts, shaders and baked animation clips stay out of a session by default. Not because they matter less — they are simply the largest files a project holds, and pushing them through a live link turns it into a poor file-sync tool. Source control carries those, which is what the engine's Git integration is for.

The host can switch that off. A session started with large asset sync enabled carries everything, imports included, and everyone in it has agreed to that before joining — see Large Assets.

Shared files

Assets in a Session

The content browser is part of the session too. Assets and folders that anyone creates appear for everybody; anything that destroys or moves work is a request the host answers.

Creating is open to everyone

Make a material, a widget, a HorizonCode class, a struct or enum type, a folder, a C++ class, and it appears for everyone. The host settles the name, which matters more than it sounds: if two people create a material at the same moment, one of them is quietly renamed rather than one of them being lost — and the person it happened to is told.

Destructive changes are requests

Deleting, renaming and moving an asset all go through the host, whether you are hosting or joining. Dragging a file into another folder is the same operation as renaming it — the new name simply sits in a different folder — so it asks in exactly the same way. Any of these that happened on one machine alone would leave every other peer holding a path that no longer exists, and the next scene save would hand them a reference to a file they do not have.

That is a decision rather than a check, so a person makes it. The host is never interrupted by it: requests collect in a list in the Collaboration window with a count in the footer. With several people in a project, a dialog per request would mean the host spends the session clicking things away rather than working, and would eventually approve something by reflex.

One gesture is one row

Selecting twenty assets and pressing Delete is one decision. It arrives as one row — “Delete 20 assets” — that opens into the full file list, with Approve all and Deny all as well as a button per file, so the host can wave through nineteen and keep the twentieth. Dragging a multi-selection into a folder batches the same way.

Nothing happens to an asset until the host says so, there is no timeout, and nothing is ever approved automatically. Requests still waiting when the session ends simply lapse — nothing was applied, so there is nothing to undo. While they wait, everyone in the session can see that they are waiting.

Re-importing

Re-importing an asset from its source file rewrites bytes that other people are working against, so in a session it passes the same lock gate an edit does — and, when the session carries large assets, the new file is sent in full afterwards rather than leaving peers on the old one. The lock is claimed for the duration of the import and released again when it finishes, so nobody can take the asset out from under a rewrite that is already running.

A rename fixes references everywhere

Renaming or moving an asset rewrites every reference to it on every machine, not just the one that asked. References live in two forms — a path inside an authored asset, a UUID inside a scene — and both are followed. That work runs on a background worker: on a large project, walking the whole content tree is far too slow to do between two frames.

When the far side refuses a file

A peer can refuse an incoming file — most often because it is over that machine's size ceiling. The refusal travels back to whoever sent it. Before that, the lower of two ceilings won in silence: the sender's own limit let the bytes go, so nothing on its side failed and nothing on its side ever said otherwise, while the only editor that knew the file had not been written was the one that refused it.

Bandwidth

Large Assets

Meshes, textures, audio and fonts can travel a session, but only when the host has decided the session works that way and everyone in it has agreed to it. It is off by default, because on a metered or slow connection it is the difference between a few kilobytes an hour and a few hundred megabytes.

The host decides, you consent

The switch lives in Preferences ▸ Collaboration and it means two different things depending on which side of the session you are on. As the host, it decides what this session carries. As a guest, it is your answer to being asked.

Joining a session that carries large assets without having agreed is refused, with that as the stated reason — and the editor turns the refusal into a question rather than an error, because the whole point of refusing was to get you asked. Say yes and it retries the same host with the setting on. Say no and you stay out, which is a legitimate answer: someone on a mobile connection should not have a project's textures pushed at them because they clicked a session id.

How big one file may be

Separately from all of that, each machine sets a ceiling on a single transfer — 1 to 512 MiB, 64 MiB by default. It is per machine, not per session: each side refuses what it is not willing to hold, so two peers may disagree and the lower of the two is what actually gets through. That is also why this one can be changed during a session while the switch above cannot — it commits nobody but you.

Raising it is not free, and the setting says so where you change it. A file that travels is held whole in memory on both machines — read into one buffer here, queued a second time in the outgoing connection buffer, assembled into a third at the far end before it reaches disk. It also goes out ahead of everything the session still has to say, so a large file on a slow connection makes editing together feel frozen until it is through. And this number is the only thing standing between another peer and an allocation of that size on this machine: the receiving side reserves the announced size before a single byte arrives.

That is where 512 MiB comes from — already about a gigabyte and a half of memory on somebody else's say-so, and as far as this is willing to go. The lower bound is 1 MiB because below it ordinary authored assets — a scene, a widget with embedded art — stop travelling, and a session where saves silently fail to arrive is worse than having no control at all.

Knowing before you join

A session announced on your local network says whether it carries large assets, so the browser can show what it will cost you before you click it. A session you join by session id shows nothing beforehand by design — the directory knows an address, not a policy — so for those the question is asked at the join instead.

How it works

Live Document Editing

The per-item layer is what makes a shared graph feel live rather than reloaded. It is built on one generic message, and one idea.

One message for every document type

A document delta names a single item inside a single document: which asset, which document within it, what kind of item, which id, and either the item's JSON or a removal. HorizonNet interprets none of that. Because every graph and the element tree all decompose into identified items with a JSON form, one message type covers all eight editors — and a ninth costs no protocol work.

Deltas are batched. Everything one edit produced travels as one frame, so pasting thirty nodes arrives atomically instead of interleaving with another peer's work.

Two documents, one asset

A UI widget file holds both an element tree and a HorizonCode graph. They share a single lock — deleting an element breaks the nodes that reference it, so splitting the lock would only move the conflict from the lock table into the data — but they are addressed as separate scopes, so a delta can never land in the wrong one. Element ids and node ids are both small integers; the scope is what keeps them apart.

Deltas are found by diffing

The editors are not instrumented to report what they changed. Each graph editor already reports only that something changed, and threading a per-action delta through every editor would have to be redone for every action added afterwards — the add menu, paste, box-select drags, every context menu, undo and redo.

Instead the engine keeps a shadow copy of each open document and diffs it. That covers every one of those paths at once, cannot drift from what the document actually holds, and only runs on the frames an editor reported a change. The shadow copy doubles as the echo guard: applying a peer's delta updates it in the same step, so their edit is never diffed back out as yours.

When a delta is refused

A single item's payload has a size bound. Past it the batch is refused rather than truncated, and the whole-file path takes over for that change. Half a node's JSON is not a smaller edit — it is an unparseable one that would drop the item on the far side while looking like a success on yours.

Arbitration

Locks & Handover

A lock says who currently owns the right to edit something. The host holds the only copy of the table and answers every request itself, so there is no window in which two people both think they hold one.

Locks are taken on entities (the one you have selected) and on assets (the tab you are editing). Both use the same table — the subject is just an opaque 64-bit id — so an entity and an asset are arbitrated by exactly the same mechanism.

Assets lock lazily

Opening a tab is reading, and reading together is the point of a session, so opening does not take a lock. The first edit does.

But opening does ask the host who holds the asset. The replicated lock table every peer keeps is right almost always, and up to one round trip stale — which is exactly the moment a tab opens and has to decide whether it is editable. For the frame or two the answer takes, the tab draws but takes no input, behind a short notice. Then:

Closing the tab releases the lock — holding an asset nobody is even looking at any more would block everyone else for no reason.

An asset somebody is editing carries a small padlock in the content browser, in that person's colour — the same colour as their cursor and their name everywhere else. You see it before you invest in opening the file.

Asking for it

The banner on a read-only tab has an Ask to edit button, and so does the right-click menu on the tile. That request does not go to the host. Deleting and renaming destroy work, and the host decides those; asking to edit interrupts work, so it goes to the person being interrupted. They see it in the same quiet list the host's requests live in, and they answer it — hand it over, or keep it. Saying yes moves the lock across in a single step, so nobody else can take it in between. If nobody holds the asset at all there is nobody to ask, and it is simply yours.

Being read-only means you cannot edit it, not that you cannot look at it: a graph you are not holding still pans and zooms, and the holder's changes appear in it live.

Background work holds the lock too

Anything that rewrites a shared file without a tab open — a re-import is the case that exists today — claims the lock for the length of the operation and gives it back afterwards. Merely asking whether the asset is free and then writing leaves a gap in which somebody else can take it, which is the exact race the lock is there to close. If you already held the lock beforehand, finishing the import does not take it away from you.

On a guest the claim is a round trip away and the write does not wait for it, so there the lock is optimistic rather than settled: if the host turns out to have given it to someone else in that window, the race is reported — as a problem in the notification list — rather than prevented. Silence would be the one unacceptable outcome.

The host re-checks everything

A client that skips its own gate is still stopped: the host verifies the sender holds the lock when a transform, a component update, an asset or a document delta arrives, not merely when it is sent. Without that the lock would be a suggestion.

Awareness

Presence & Undo

Every participant broadcasts their camera position, orientation and current selection. Peers draw each other as camera gizmos and highlight what the others have selected, in a colour derived from the participant id — so both sides pick the same colour for the same person without having to agree on one.

Presence is deliberately disposable: it is rate limited to roughly ten updates a second, a lost update is simply corrected by the next one, and unchanged presence is still resent occasionally so late joiners converge without anyone having to move. A stationary editor sends nothing at all.

Undo is per user

Undo in a session is not a rewind of shared state — that would undo other people's work. Each participant has their own history of their own changes, and undoing one publishes the inverse as an ordinary change. Everyone sees it the same way they saw the original edit.

Awareness

Notifications

A session is full of things that happen to you rather than because of you: a delete a peer could not apply, an asset the host never answered about, a file the far side refused. The bell in the editor's footer is where those go, with a count of what you have not read yet and a flyout that expands into the list.

It is deliberately not a log. A log answers “what happened”; this answers “what still needs you”. Entries are written in sentences, the ones that concern a particular file carry its path, and there are three levels: something worked and you should know, something did not fully work but nothing is lost, or something is out of step and will stay that way until someone acts.

The list is bounded, and when it has to drop something it drops the oldest entry you have already read — so a run of chatter can never push an unread problem off the end. Identical messages arriving in a row collapse into one row with a count, which is what keeps a worker that failed on four hundred files from producing four hundred entries.

Reference

Message Reference

The complete collaboration protocol. Client-to-host and host-to-client forms are separate messages on purpose: the client's form carries no participant id at all, so a client cannot claim to be someone else — the host stamps the id from the connection the message arrived on.

Identity across peers

Entity handles are allocator indices — the same number names different entities on different machines. Everything on the wire therefore uses an entity's stable UUID, which each peer computes from its own world, so no side has to translate the other's numbering.

Compatibility

Protocol Versions

The version is exchanged in the very first message and a mismatch is refused. Peers that disagree are never allowed to talk: the failure mode of "close enough" is one side silently misreading the other, which is far worse than a clear refusal.

Honest scope

Limits & Known Gaps