Chapter 15

How a Document Becomes Nodes

Chapter 12 said where documents and the graph each stand, and Chapter 14 said what is allowed. This chapter follows one document, in order, from the moment it arrives until Nodes come out of it.

1. The path a document travels

Write the document, extract candidates from its headings, and a person applies them explicitly Write the document store, hash written last link mentions Extract candidates from headings only Apply a person, explicitly proposed, evidence filled

A document write needs an owner. Source text without one is refused. If the content hash and the title are both unchanged, nothing is written and the call answers with success. A changed title alone triggers a rewrite.

The order of a rewrite matters. The old Chunks are deleted first, the new ones stored, and the document's hash is recorded last. If the process dies partway, the hash is not updated, so the next write does not skip; it rewrites everything. That is how the store heals itself without a transaction.

Extraction reads only second and third level headings. The top-level heading is the document's title. A heading has to be shaped like a name to become a candidate. The structure words every document repeats (method, provenance and the like), headings that open like a clause (What, How, Why), and headings longer than six words are all rejected. A candidate with no explanatory sentence under its heading is dropped too, because without one there is nothing to judge it by. Six words is the linker's own ceiling for finding a title, so a candidate that could never be linked is never born. At most twenty-four candidates. A document with no headings yields nothing. That is a limit, not a defect.

Evidence filling has two rules. When a Chunk mentions two Concepts and their titles appear verbatim together in the same sentence, that sentence becomes the relation's evidence, with a confidence that falls from 0.9 to 0.5 with the distance between the titles. When the titles do not appear verbatim but the key words of both are all in one sentence, that sentence is used too, at a fixed confidence of 0.4, so a confidence below 0.5 marks the looser rule. Either way, appearing in the same document is not evidence, and the number never exceeds 0.9 so that it says no person confirmed it. A relation that already has evidence is left alone.

2. Automatic extraction is not the same as a meaning a person refined

Extraction is a pure function. No model, no network, no store. The same document yields the same candidates anywhere. And extraction returns candidates only. Writing to the graph happens when a person applies explicitly. Drop a file into the app and it is stored, the extraction is previewed, and one more click applies. If arbitrary headings became Concepts on their own the graph would be polluted, and the cleanup would be a person's work.

Three rules hold when applying.

An existing Concept is never overwritten. If the identifier already exists, nothing is created and it is counted as matched. A title or summary a person refined is kept. Only an empty time or an empty topic is filled in, and even that fill is stamped. Someone else's Concept is not touched at all.

A document with no topic is refused. If the document neither declares a topic nor has one derived from its mentions, the answer is to give it a topic first. The earlier rule created untopiced Concepts and left the fix for later. Later never came, and one thing with no topic became eleven.

Applying again creates nothing. Zero created, n matched. It is safe to press as often as you like.

Two more things. Full-text search works the moment a document is written; vector search follows later. Embeddings are filled in the background per document, and a document counts as done only when every Chunk is filled. And documents synchronize on a channel of their own, apart from Nodes. The document is the unit, only the restored text travels, and the receiving machine re-chunks and re-embeds on its own. A conflict is settled per document by the later writer, because a document is atomic.

3. What going from document to Node means

Going from document to Node does not mean compressing a document into a Node. The document preserves context, the Graph structures meaning, and the two stay connected. You can go from a Concept back to the source text, or from a document, along the Concepts it mentions, into the graph.

This closes Part 1. It has shown in what form memory is stored. Now the stored memory has to be found when a task needs it. Search narrows where memory might be; settling which one it is belongs to a different step. Part 2 begins with that problem.

Continue with Chapter 16, How Memory Gets Found.