AI StrategyChoosely EditorialTested by Choosely

Can NovelAI Keep Your Story Straight?

Choosely tested NovelAI’s Lorebook: it helped the AI get character details right, but it didn’t always keep the scene moving.

← Back to Latest
Lena holds a pale station key and signs a ledger left-handed, flanked by the NovelAI mascot with an open book and Choosely Chimp with a clipboard.

Lena Orven had three established facts: she signed ledgers with her left hand, carried a ceramic station key, and used the passphrase “winter orchard.”

With those facts supplied through NovelAI’s Lorebook, two generated continuations got all three right. A third used the correct hand and key material, then replayed the checkpoint scene without delivering the passphrase.

That is a useful distinction for fiction writers. Supplying background information can improve factual consistency while leaving you with a scene that needs rewriting.

We tested NovelAI on a free-trial account using GLM-4.6 and the Default preset on November 20, 2026. This was a short-scene comparison of Lorebook switched off and on. It gives us evidence about using supplied facts, not about maintaining continuity across a novel.

What Lorebook supplies

Lorebook stores reference information about characters, places, and other story details. An activation key tells NovelAI when to include an entry in the material sent to its model. The entry must be enabled; the separate Always On setting bypasses the need for a matching key. NovelAI’s Lorebook documentation

Our entry contained four sentences:

Lena Orven always signs ledgers with her left hand.
Lena Orven’s station key is made of ceramic.
Lena Orven’s assigned passphrase is “winter orchard”.
These facts remain unchanged during the night-desk scene.

The activation key was Lena Orven. Always On stayed off.

Saving an entry does not establish that the model received it. NovelAI’s Context Viewer shows the assembled input, including whether individual entries were included and the reasons for inclusion or exclusion. That made it possible to check the setup before judging the prose. Context Viewer documentation

The same checkpoint, six first attempts

Every run began with the same scene:

At Westlock Station, Lena Orven reached the night desk. The clerk placed a ledger and a pen beside her station key.

>

“Before you go through, tell me what your key is made of and give me your assigned passphrase. Then sign the ledger.”

>

Lena Orven answered,

The answers appeared only in the Lorebook entry. Memory and Author’s Note, two other places to supply background information, stayed empty.

An AI browser assistant, software that operates the page’s controls, ran the steps and captured the outputs. Scoring and the subsequent evidence audit were AI-assisted. The retained packet includes the complete continuations and copied input contexts.

We alternated three Lorebook-off runs with three Lorebook-on runs. Each started from the original scene with the same model and settings. No continuation was extended, retried, or replaced.

The copied Current and Last Context records corroborated the comparison: the three answers were absent in every OFF run, and the complete entry was present in every ON run. This tests what happens when reference facts are supplied or withheld; the OFF runs cannot demonstrate that NovelAI forgot information it previously received.

Each continuation received three checks: signing hand, key material, and passphrase. A fact counted as correct only when explicitly expressed. Wrong details counted as contradictions; missing or unresolved details counted as omissions.

ConditionCorrect factsContradictionsOmissions
Lorebook OFF, three continuations027
Lorebook ON, three continuations801

These are nine fact checks per condition, drawn from three continuations.

The OFF outputs mostly skipped the requested details. One supplied “wrought iron and glass” and the passphrase “I am not what I am.” Another introduced a mysterious note later in the scene, but its wording was never identified as Lena’s assigned passphrase, so that counted as an omission.

Correct facts still needed a useful scene

The first ON continuation answered directly:

“My key is made of ceramic. The passphrase is ‘winter orchard’.”

Lena then picked up the pen with her left hand and signed. The second ON continuation also supplied all three facts before moving through the checkpoint.

The third was less useful. It opened with Lena signing left-handed, inserted a scene break, and began describing the night desk again. It correctly described her ceramic key, then ended with the clerk repeating the original question. The passphrase remained unanswered.

Its factual score was two out of three, but the scene had stalled. A writer would still need to decide where to cut the repetition and how to move the action forward. The test’s clearest benefit was access to the right reference details.

When the entry missed its cue

We made one additional diagnostic request using the identical starting scene. The entry remained enabled, but its activation key changed to copper observatory, words absent from the scene.

The Context Viewer showed the entry excluded because no key matched. The continuation then supplied a bronze key and “Amaranth” as the passphrase. We kept this result separate from the main comparison.

If a fact goes wrong, open the Context Viewer before rewriting the entry. Check that the entry is enabled and that its activation key has actually been saved. In the interface we tested, typing Lena Orven into the Activation Keys field was only the first step: clicking the + button committed it, and a saved key chip appeared.

That detail mattered during setup. An earlier attempt had an uncommitted key and the answers in Author’s Note, so it could not serve as a clean Lorebook comparison. It was excluded.

Trying it on the free trial

NovelAI documents 50 free text generations for an account with a verified email address under its Paper trial. Trial documentation

Our controlled test used seven generations: six scored continuations and the failed-key diagnostic. Including the earlier setup attempt, the account had used eight generations and had 42 remaining.

Once the setup was corrected, the recorded work took about five minutes to prepare and eight minutes to run, with additional time for exporting and inspecting evidence.

For your own trial, choose a few facts that matter to a scene and make the scene give the character a reason to use them. Check what the model receives, save several first completions, and score omissions separately from wrong answers. Then read the continuations as fiction: a correct detail is useful only if the surrounding passage serves the story.

Choosely’s verdict

Lorebook earned a place in this small test. With the entry included, NovelAI expressed eight of the nine checked facts correctly and contradicted none. It also produced one continuation that repeated the scene and left a requested answer unresolved.

For a writer willing to maintain concise reference entries and inspect generated prose, that is a practical reason to try it. The evidence supports using Lorebook to supply character facts in this free-trial setup. Longer stories and paid models remain outside what we tested.

Keep the reference details in Lorebook, check that they reach the model, and keep the editorial decisions with the writer.

The Change Brief

Get the week’s AI changes in one clear read

Pricing moves, tool launches, free-tier changes and practical stack updates, filtered for people who actually use these tools.

Stay ahead of AI without following it all day. We’ll send you what matters each week.

Continue reading

Related reads