Chapter 32

The Cost of Changing a Model and a Storage Structure

Chapter 31 ended at the gate it could not price: migration economics. Every number there compared two models. None of them costed the act of moving from one to the other, and that act turns out to be the expensive part.

It was rehearsed rather than estimated. A throwaway store, twice, with the second pass building the path the first pass discovered did not exist. No production store was rebuilt during it, and the numbers below are from that rehearsal. The hub store was later rebuilt along this path for real, on 11 September 2026, and one defect the rehearsal had not seen appeared then; the end of section 8 has it.

1. A vector's dimension is a storage contract

The width of an embedding vector is not only a property of the model. Once storage is built to hold vectors of that width, the width is a contract the storage holds the rest of the system to.

The store here uses a fixed-width array column. So adopting a model of a different width is not a configuration change, and this is where the estimate and the measurement first parted company.

2. The migration call that raises nothing and changes nothing

The plan had said that 1024 dimensions were "mechanically supported, but a schema migration on every store". Measured, that was wrong in both halves.

Asking the store to provision at the new width on an existing store returns all thirty one of its statements, raises nothing, and leaves the column exactly where it was. The migration path runs its statements through a helper that swallows any error containing "already", and there is no statement that widens a fixed-width array in place. So the call reports success and does nothing.

The mismatch surfaces at the next write, as a conversion error naming both widths. By that point the model switch has already deleted every vector and marked the whole corpus pending. Adopting a wider model without a purpose-built path therefore does not degrade search. It empties the store's vectors and then wedges, with no error at all until the first refill attempt.

The other half was wrong too: it is not a migration on every store. The shipped desktop bundle excludes the encoder stack by name, vectors are written in exactly one place reached only by the worker, and account synchronisation does not carry vectors. So a desktop store's vector table is empty from the start, and the dimension change there is a table swap that finishes in under two seconds on the full corpus and loses nothing. The fleet-wide rebuild is one hub store.

3. Changing the schema and regenerating the vectors are different things

Changing the schema does not move existing documents into the new semantic space. Every chunk still has to go through the new model: re-embed the existing chunks with it to get new vectors, rebuild the index, and verify retrieval.

Each of those steps was, at the start of the rehearsal, either missing or quietly lossy.

4. What the rebuild path has to do, in order

The order is the contract, and each step is there for a failure that was observed rather than imagined.

1. Drop the old vectors from the dump. 768-wide vectors cannot be cast into a 1024-wide column.
2. Import including the metadata tables. The export deliberately excluded the underscore-prefixed tables, so a naive export and import loses the document metadata entirely. Then the requeue step sees zero documents and requeues zero: re-embedding never happens and nothing errors. Measured, ten documents became zero and zero were requeued.
3. Build the indexes, before re-embedding starts. The import builds none and says so. Without this step the new store has no full-text index either, so search is not degraded but dead.
4. Requeue every document and stamp the new model identifier, returning the document count and the requeued count together so a caller can see them disagree.

Step 2 turned out to be larger than a rebuild concern. The exclusion lived in a shared function, so backup restore and device migration had been losing the same metadata all along, including the content hashes that drive change detection and the cursors that drive account synchronisation. The fix went to the shared function, not to the rebuild.

5. What correctness cost, measured in the same session

Both variants were run back to back on one machine against one source store, which is the only comparison Chapter 29 allows.

Correctness cost 2.57 seconds, against a refill of between 410 and 1345 seconds on the same corpus. Five regression tests hold it, and they exist because every value in that table disappears without raising: a store missing its metadata looks perfectly healthy.

6. The bill is re-embedding, and it is three orders of magnitude larger

Store work and model work are not the same size.

The candidate costs 3.5 times the wall clock for the same corpus and about 305 MB more peak memory, single-threaded. Both scale linearly in chunk count, so the bill is the chunk count divided by the rate: roughly 1.8 hours for fifteen hundred chunks, roughly twelve hours for ten thousand. Read those as a shape, not a constant, because the container's processor is not the one measured.

Running several stores at once scales at about 83 percent on eight logical cores, and the binding limit is memory rather than processor: each worker peaks near 1.8 GB, so a 16 GB machine tops out near eight concurrent. What degraded under concurrency was the store work, not the encoding, which is disk contention on the import rather than model work.

One number is worth keeping for its own sake. Making the path correct did not make it slower: the candidate re-embedded at 0.225 chunks per second through the finished path against 0.23 through the minimal one, which is the same rate inside noise.

7. Rolling back is another rebuild

Going back moves the column the other way, takes the identical path in the opposite direction, and pays the identical store cost plus a full re-embed at the destination model's rate. Measured through the finished path: 4.23 seconds of store work plus 410.1 seconds of re-embedding.

The asymmetry favours going back, because the older model re-embeds about 3.5 times faster. It is still not cheap and it is not instant. There is no switch that flips the model back, and a bad deployment is recovered by another multi-hour rebuild during which vector search is off.

What rolling back does not cost is the stored information. The full cycle was run forward and then back, and the document metadata was compared field by field against the original: all 38 documents matched on content hash, synchronisation cursor and creation time, zero mismatches after two rebuilds. Retrieval came back to exactly its pre-rebuild values.

8. An interrupted rebuild has to be distinguishable from a slow one

A rebuild was stopped deliberately part way through. The store opens, documents and chunks are intact, the completed vectors are kept, and the refill resumes where it stopped because progress is recorded per document.

It also alarms, and that is the design. The model state reports that a migration is in progress with a pending count, and once a threshold passes it reports itself stalled, so a worker that died mid-rebuild is distinguishable from a backlog that is merely long.

The first attempt failed the one promise that mattered here. The claim that keyword search keeps answering through a model change is true of an in-place switch and not of a rebuild, because a rebuild makes a new store and the import moves rows only. After the index step was put in the right place, an interruption at 104 of 303 chunks still answered both an English query and a Korean one with five hits each. Index construction is a required step of any real rebuild path, not an optimisation. Without it, the window in which the hub answers nothing is not the seconds of the swap but the hours of the refill.

The defect the rehearsal could not see appeared when the hub was actually rebuilt on 11 September 2026. The dump read the logical ids with the owner stripped off rather than the physical keys that carry it, so in the new store every exact id lookup died and the Nodes of two owners sharing a logical id folded into one row. A rebuild now reads physical keys, and if the source already holds logical keys it stops loudly instead of copying the fault into a new file. The throwaway store had a single owner, so the rehearsal could not have shown it.

9. Choosing a model is choosing a path

Choosing a model means choosing, together, whether the schema and the worker fit, whether the model is registered, and whether the path through re-embedding, index rebuild and retrieval check exists. Read that list against Chapter 31 and the shape of Part 4 appears: a model comparison is the cheap half of a model decision.

One thing about this chapter is worth noticing before the next one. Every defect in it was silent. The provision call that raised nothing. The export that lost metadata without an error. The import that built no index and left search dead rather than degraded. The requeue that counted zero and reported success. None of them produced a failure; all of them produced a healthy-looking store.

That is a category of its own, and it is the next chapter.

Continue with Chapter 33, Failures That Look Like Success.