AI direction and quality

From source to scene: why a generated visual novel needs a director

Generating one or two attractive images is relatively easy. The real difficulty begins after the tenth image: a character must remain the same person, a location must remain the same place, a prop must remain the same object, and every frame must continue both the source and the previous scene. Sending one paragraph after another to an image model produces an album of illustrations, not a working visual novel.

I ran into this while adapting a literary work. Individual images looked convincing, yet together they exposed failures that no isolated asset check could find: the hero changed size, people floated above the ground, the same coast rebuilt itself between adjacent shots, music restarted, and an important ring moved to another hand, another finger, or multiplied.

Generation therefore became a production pipeline with separate roles: text analyst, artist, compositor, MCP agent, technical validator, and AI director. Its main output is not an image. It is a verifiable connection between the source, the scene, its assets, and what a player actually sees in Player.

![Romergo pipeline from source through story structure, continuity bibles, assets and MCP assembly to director review, with regeneration and recomposition loops](direction-flow)

1. Fix the source before drawing anything

The input may be plain text, HTML, Markdown, or a text-based PDF. The agent first records language, edition, chapter order, and the available text layer. Rights to the original, the particular translation, and embedded illustrations are confirmed separately; an old work does not automatically make every translation publishable.

romergo_inspect_story_source and romergo_plan_story_source can build an inventory and preview without writing a project. This stage must not “improve” the work or invent branches. A linear source should still produce a linear graph.

2. Split the text into continuous scenes

A scene boundary is defined by a change in place, time, participants, or action, not by paragraph length. Every segment keeps its exact sourceExcerpt and receives a visual contract:

place, time, and physical participants;
speaker, listeners, and action;
gaze targets and visible supports;
details that must appear;
faces that must survive mobile cropping;
anchors that connect adjacent shots.

This is already a director's breakdown. “They sat with their backs to the window” becomes a testable body-orientation requirement instead of decorative prose that a generator may ignore.

3. Let an independent auditor extract props and states

Because a planner can miss a small detail, a separate text auditor reads the exact excerpt again and extracts every object a character touches: a book, fishing rod, bottle, watch, or ring.

Each object enters a continuity ledger with a canonical description, immutable traits, holder, state, visibility, instance count, and event chain. A “thin gold ring,” for example, remains on one specific hand and finger of Gray before the transfer; it is visible between his fingers and Assol's little finger during the action; afterwards it is absent from Gray and remains on Assol's little finger.

That is stronger than a generic prompt to “keep continuity.” The review knows not only that the ring exists, but exactly where it belongs at every beat.

4. Temporary documents form a production graph

A long chapter cannot live in one prompt. The pipeline creates temporary documents linked by stable IDs and hashes of the source excerpt:

the scene map stores text boundaries and visual contracts;
the character bible fixes face, age, stature, clothing, and pose vocabulary;
the location bible keeps geometry, light, and recurring landmarks;
the prop ledger records identity, holder, and state changes;
the asset manifest maps scenes to backgrounds, poses, and portraits;
generation jobs repeat the required traits for their exact representation;
audit reports are tied to the exact render by SHA-256.

These files do not replace the literary source and are not published with the quest. They are scaffolding: they explain every decision, let the system repeat only a failed step, and prevent details from disappearing between agents.

5. A scene has two main representation modes

Not every scene should be assembled the same way.

baked_narrative is for narration without direct speech. Characters are generated into the background, so lighting, contact with the ground, sitting, lying, and prop interaction can look natural. Several sequential illustrations can replace one universal background.

layered_dialogue is for conversation. The location stays stable while speaking and listening poses remain separate sprites. A reaction can change with the line, and the dialogue card can keep the correct name and face.

A hybrid is possible but explicit: a sleeping heroine may be baked into the background while an approaching sailor remains a sprite. Accidental layer mixing is one of the fastest ways to create physically impossible staging.

6. Generate from the exact scene

Character identity references and the world's visual style come first. Scenes are then processed one at a time. A background prompt is built from the exact excerpt, location bible, and current prop state. Only then does the pipeline select an existing pose or generate a missing speaking, listening, or action pose.

Asset QA catches extra limbs, broken hands, text in the picture, repetitive foliage, melted branches, transparency halos, and cropped silhouettes. Yet an isolated PNG cannot tell whether a person stands correctly in the final scene. That requires the next layer.

7. MCP assembles a normal editable project

Approved assets enter a draft and MCP tools create chapters, scenes, text, graph transitions, timeline, music, and character placement. This is not a flattened image export: it remains a regular Romergo project that can be opened and changed in Builder.

Every write is followed by persisted readback. The agent uses romergo_get_chapter, runs romergo_validate_project, and finishes through romergo_verify_quest_change. The check covers saved relationships, media references, and the runtime representation, not merely a successful tool response.

8. The AI director reads the text and final frame together

Final review happens in Player, not against source PNGs. The whole chapter is played on desktop and at 390 × 844. Frames at 0, 100, and 500 ms after scene entry expose black transitions, white mattes, and stale sprites that appear only briefly.

For each frame the director receives the exact excerpt, visual contract, adjacent-scene context, and screenshot. It checks source fidelity, physical support, relative scale, occlusion, gaze logic, location and prop continuity, mobile-safe faces, dialogue identity, text sequence, and uninterrupted music.

The verdict must prescribe a repair. If the background is correct but a sprite floats, MCP can recompose position, scale, layer, or crop. If a wrong pose is baked into the background, the asset must be regenerated. A fresh render is then reviewed again.

Three failures that created new rules

Failure 1. The hero literally sat on the heroine

The wide composition seemed acceptable, but mobile cropping merged the layers: Gray's knee landed across sleeping Assol. “Both characters are visible” was technically true and completely insufficient. Review must reason about support, depth, and body intersection in every viewport. A layered scene needs recompose; a fused image needs regenerate.

![Mobile frame where Gray's layer lands over sleeping Assol after cropping](direction-failure-overlap)

Failure 2. One ring became several and changed hands

Checking only for a gold object was too weak. Across assets the ring moved to another hand or finger and sometimes multiplied. The contract now requires exactly one instance, a specific anatomical side and finger, plus before, during, and after states.

![Nine failed ring-placement crops where the ring misses the finger, floats beside it or lands on the wrong finger](direction-failure-ring)

Failure 3. Characters looked through the window although the text said the opposite

The tavern shot was attractive, but the source explicitly said the men sat with their backs to the window so they would not turn around by accident. This is not a layout problem: the wrong orientation is baked into the image. It must be regenerated with the source's spatial constraint stated verbatim.

![Tavern scene where seated characters face the window against the source instruction](direction-failure-window)

Why earlier checks missed them

Early checks were local: the file existed, alpha was present, the character fit the frame, and graph edges connected. Each answered a narrow question, but nobody compared the source verb, final pose, mobile crop, and prop history in one pass.

Reliability improved only when requirements came from exact text, Player became the controlling surface, review received adjacent scenes and prop history, and every failure had to choose a repair followed by an independent rerun.

The result is repeatable direction, not a pile of pictures

All of this is still far from fully automatic generation. Human visual literacy, experience, and feeling remain the main measure of success. I do not expect to say with confidence that generation is fully autonomous any time soon.

The current pipeline already saves me many hours, and I hope it can save time for other creators too. Keep your time for creative work; let scripts and AI carry the difficult technical routine.

That is how text, HTML, or PDF becomes a working visual novel: the source establishes facts, contracts make them testable, generation creates the required representations, MCP assembles an editable project, and director review compares the finished experience with the text. Only then do separate attractive images begin to behave like one story.

Back to dev blog