The Idea

Tabletop roleplaying rarely fails because of the players. It fails because of the game master: someone has to prepare for weeks, keep the rules in their head and hold the evening together. The idea behind ai-dm is a virtual tabletop in the browser where that seat can be taken by a human or by an AI.

I started building it at the beginning of September. After a little over three weeks the whole chain works end to end: a map with tokens and real-time sync, a scene editor with walls, vision and lighting, the D&D 5e (2024) ruleset, a 3D dice tray, a character builder, and behind all that an AI that runs the table, remembers previous sessions, writes a newspaper about the world between game nights and listens in over a voice channel.

This is what I learned along the way.

1. The most important safety rule is a missing permission, not a prompt

The core risk of any AI game master is that it can be talked into things. "My spell hits, right?" – and the model gives in.

My first instinct was a stricter prompt. The better answer is in the architecture: every action at the table goes through the same pipeline of authorization → validation → event → state → per-recipient view. A human game master may override the rules engine, as a visible event in the log. The AI simply doesn't have that permission. If the engine says no, all the AI can do is describe the no.

So the LLM never decides hit points or dice. It sends requests to deterministic code and narrates the result. There is no instruction for a persistent player to talk their way around.

2. The event log as the single source of truth

Instead of changing state directly, the server writes events and folds the state from them. This has paid off several times:

• ▸Undo means marking a batch as reverted and folding again. No applying changes in reverse, no special cases.
• ▸Per-role visibility is a projection of the same state. The game master and the players see different maps because the server builds a separate view for each recipient, not because the frontend hides something.
• ▸Dragging is presence, dropping is an event. Sixty positions per second don't belong in the log, but everyone else should still see the movement.

Kept deliberately simple: every change sends the full view state, with no patch system. With five players that's a few kilobytes, and a whole class of bugs goes away.

3. Everything a human can do, the AI must be able to call

For a long time I built the table without AI and polished each surface before starting the next. Then I noticed the AI had never been connected, even though it's the whole reason for the project.

That led to two rules:

• ▸Get it working end to end first, then polish. The whole chain runs once, even if it's rough, before anything gets pretty.
• ▸Every capability as a tool. At the table, the game commands are the AI's tools. For preparation (creating characters, building maps, setting up scenes) there is a catalog from which an MCP server generates its tools automatically. A new capability is one catalog entry plus a route, and the tool follows from that.

At the table the AI works with any model that has an OpenAI-compatible API, so I'm not locked into one provider.

4. Building with AI agents means building guardrails

Large parts of the code were written together with AI coding agents (Claude Code), often several in parallel. That's what made the pace possible, but it also created its own problems:

• ▸A shared working directory goes wrong. On one day, each of the rules I wrote down afterwards got broken once: one agent switches the branch and another commits to it without noticing; git add -A picks up someone else's half-finished work. The fix: every piece of parallel work gets its own git worktree with its own port pair and a copy of the dev database, set up with one command.
• ▸Report errors where they happen. A hook runs the TypeScript typecheck after every file change the agent makes, so type errors show up right away instead of three steps later.
• ▸The docs are the memory. An agent doesn't remember the last session. That's why every decision and its reasoning go into the architecture and planning docs, and commit messages explain the why.
• ▸Look instead of imagining. The agents check UI changes in a real headless browser, with screenshots and measured spacing. Otherwise a silent console error reads like a design question in the source code.

5. Small lessons along the way

• ▸Color management in three.js: The landing page background got brighter as soon as an effect switched off. The cause wasn't the lighting: three.js applies tone mapping and sRGB conversion only on the final draw to the screen. Whenever the post-processing chain changed, the stage that got that conversion changed too. A fixed OutputPass at the end fixed it.
• ▸Controlling isn't owning. A character only recorded who controls it. Nobody controls non-player characters, so they belonged to no one and disappeared from every list after being moved. Ownership and authority are two different things and need two fields.
• ▸Test your permissions. Every API route is tested from three perspectives: game master, player, stranger. Identity comes only from the token, never from a field in the request.

Where it stands

213 tests, a clean typecheck, and the table is fully playable without AI. The biggest open question, honestly: no real language model has seen the table yet. Everything involving AI is tested against mocked model servers. Next up are real model runs, then the first game night with friends. Whatever breaks there decides what comes next.

To be continued.