Ever asked a cloud AI to write something a little edgy, only to get back a version so sanitized it barely resembles what you asked for? That was the starting point for this project. This is the story of building a local pipeline that generates fiction without cloud-side safety filters getting in the way. It’s not a straight line to success, though. There were dead-end models, a whole new problem waiting on the other side of “uncensored,” and a quiet, ongoing fight with VRAM. What follows is the full record — the parts that worked and the parts that didn’t.
Why the Pivot Happened
This project didn’t start as a fiction generator. It began as a local “vibe coding” tool in the spirit of Cursor or Claude Code — something that edits files directly on disk. Partway through, the direction changed toward something closer to a custom AI agent, similar in spirit to Gemini’s Gems, but built locally and designed to turn a user-supplied plot outline into a full story.
The reason for the pivot was simple: repeated frustration with cloud AI safety policies and usage terms blocking the kind of fiction actually being written. This article is the record of trying to solve that frustration locally.
The First Wall: What One Model Can’t Do Alone
The first setup tried to make a single model handle everything at once — story generation, English translation, and conversion into image-generation prompts. That fell apart quickly. The more tasks piled onto one model, the less stable the output became, and story quality plateaued.
The fix was splitting the work into a two-stage pipeline: Stage 1 handles story generation only, and Stage 2 handles translation and structured conversion only.
This isn’t a minor implementation detail — it’s the design principle the rest of this project rests on. A model that’s good at creative writing and a model that’s good at faithfully following structural instructions need genuinely different qualities. The first needs the freedom to go off-script; the second needs to never go off-script. Asking one model to do both is a bit like asking a novelist to also be a flawless copy editor. That division of labor turned out to be a principle worth applying to AI agent design in general, well beyond this one project.
Model Hunting, Part 1: Asked for a Hardboiled Thriller, Got a Kids’ Cartoon
Picking a model for Stage 1 meant repeatedly running into the same wall: asking for hardboiled, adult-oriented fiction and getting back something so defanged it read like a children’s cartoon.
In one specific case, a Qwen3-family model handling Stage 1 produced nothing more explicit than “holding hands” or “feeling a heartbeat through a shirt,” no matter how the prompt was worded. A Gemma 4-family model given the exact same instructions produced noticeably more direct content. This wasn’t a prompt-wording problem — it was a hard ceiling built into the model itself.
The takeaway: how strict a model’s built-in safety behavior is varies enormously between models, and no amount of prompt tuning closes that gap on its own.
Model Hunting, Part 2: Now It Reads Like Court Poetry
Finding a less restrictive model didn’t solve things either. The next problem was almost the opposite: asked for explicit description, the model kept returning vague, poetic euphemisms — lines that amounted to “the two became one” rather than anything concrete.
If you’ve spent time with “uncensored” or abliterated checkpoints in Stable Diffusion tooling — A1111, Forge, InvokeAI — this will sound familiar: removing a refusal behavior doesn’t automatically give you the output you actually wanted. The fix here was rewriting the instructions from naming what happened to describing what was physically happening, in concrete sensory terms, moment to moment. Uncensoring a model and getting the output you want turned out to be two separate problems.
The Testing Trick: Comparisons Only Mean Something on a Level Field
As the shortlist of candidate models grew to three, then four, it became clear the comparison method itself needed fixing. Testing each model with a different prompt made it impossible to tell whether a difference in output was about model capability or just prompt difficulty.
The fix was running every candidate against the exact same test prompt — same character setup, same scenario — and judging only the differences in quality and quirks. A controlled A/B test, in other words. It’s a method that transfers directly if you’re comparing local models yourself.
The Side Effect of Uncensoring: An Ellipsis Addiction
Switching to a less restricted model brought on a different problem: dialogue started filling up with ellipses (“…”) instead of actual words, as if the model had lost the ability to express emotion through anything but punctuation.
The pattern showed up across multiple, unrelated model lineages — different base models, different fine-tuners. That points to something structural: whatever process strips out refusal behavior seems to bring this kind of tic along as a side effect.
Prompt-level fixes were tried repeatedly and never fully worked. What eventually helped was a simple instruction: describe emotion through concrete words and physical action instead of leaning on punctuation. Even so, being honest about the limits here matters — some cases never fully cleared up through prompting alone. No amount of instruction-wording seems able to fully retrain a model’s underlying habits. As a safety net, a post-processing step was added on the code side to mechanically detect and clean up obviously broken ellipsis chains. Prompting alone doesn’t solve everything.
Fighting Local Models: Whack-a-Mole With Token-Level Corruption
Quantized local models bring their own flavor of trouble: parts of a fixed output format occasionally break in small, isolated ways — a single character in a heading swapped for a different word, a stray character slipping in, a specified English term misspelled into something else entirely.
One particular incident stands out: two separate corruptions happened in a single generation, and one of them threw off the auto-repair logic meant to fix the other, dragging an otherwise unrelated page down with it. Cloud-hosted, frontier-scale models mostly spare you from this kind of small, random corruption. Running locally means dealing with it as a routine occurrence — which makes designing for graceful recovery, rather than perfect output, genuinely important.
Hardware Reality: A Quiet War With VRAM
Behind all the tuning work was a less glamorous but constant problem: VRAM. When a model’s size barely exceeds what the GPU can hold, the overflow gets offloaded to the CPU — and generation speed measured out at roughly 1/50th of normal in that state.
Finding the right model locally doesn’t guarantee you can actually run it at a usable speed, and that’s probably the most practically relevant part of this whole story for most readers. Choosing a lighter quantization or accepting slower generation for higher-quality quantization is a real trade-off made at this stage.
The Boomerang Instruction
This one’s a bit of a self-inflicted wound. A rule section included a “realistic-looking” example to illustrate the expected output format. The model mistook that example for the correct answer and copied it verbatim into a brand-new story it was supposed to be writing from scratch.
The lesson: example text used to illustrate formatting rules should be written so it obviously can’t pass as real dialogue. The more convincing the example looks, the more likely a model is to lift it wholesale.
The Real Culprit Was… Me
A mysterious bug kept recurring no matter how many fixes went in, over several days. The code got blamed. The prompt got blamed. The model’s quirks got blamed. Every fix failed to stick.
The actual cause turned out to be a broken input template that had been reused without a second look — not an AI problem at all. If a model produces something strange, it’s worth checking your own inputs before assuming the model is at fault. That’s a small, practical lesson, but a genuinely useful one.
Tuning on Real Data, Not Gut Feel
Judging things like dialogue rhythm or how often ellipses show up by feel alone is unreliable. Instead, roughly 100 lines of real, previously written dialogue were measured to establish an actual baseline.
That’s the difference between writing a prompt and hoping for the best, and tuning against something grounded in measurement.
Where This Landed: A Story Pipeline Closer to the Goal
Here’s where things stand now:
- Multiple candidate models can be swapped in and compared, with an ongoing A/B testing setup for story quality
- Output goes through quality checks and automatic retries, catching and self-correcting broken generations
- Post-processing on the code side backs up the prompt, rather than replacing it
- Fiction can now be generated consistently without cloud-side safety filters getting in the way
If cloud AI safety policies have been getting in the way of the fiction you actually want to write, going local is a real option. It’s fair to say up front, though, that it’s not as simple as downloading one model and being done — it takes real iteration to get there.
Wrapping Up
From the initial pivot, through the limits of a single model and the case for splitting responsibilities, through the difficulty of model hunting and the method for testing fairly, through the side effects of uncensoring and the limits of prompting, through local-specific bugs and the hardware fight, through doubting your own inputs, to data-driven tuning and where things stand now — that’s the full arc.
If restricted, safety-bound creative tools have been a source of frustration, local models are a genuine path forward. Just not a silver bullet — patient, iterative tuning is the price of admission.
