Chapter 12

How Documents and the Graph Connect

Chapter 11 structured memory into Nodes and Edges. That does not make the original document unnecessary. The sturdier the graph gets, the more the question “where did this Node's claim come from” matters.

1. The same sentence is not the same memory

Suppose the sentence “we are adopting a new vector database” appears in three documents. That does not mean three memories should exist. All three may describe the same decision in the same project from three points in time. On the other hand, merging decisions from different projects because the wording is alike erases the real difference between what each project chose.

So the test for “is this the same memory” is not what the sentences look like. The test lives on the graph side. The document is where wording and context are preserved. The graph is where an identifier settles whether those wordings point at the same thing.

2. The source text is the canonical record

In Vinc a document's source text lives inside the graph. There is no separate document blob and no reference to an original file. A file is a convenience at load time; once loaded, the text survives the file being deleted. This is the canonical record Chapter 2 spoke of.

The text is stored in pieces called Chunks. A Chunk is not a new unit of meaning. It is a unit of storage and search.

One document cut at the boundaries 0, 1840, 3610 and 4102 into three Chunks, which join back into the source text Document Chunk 1Chunk 2Chunk 3 0184036104102

What matters is how the pieces are made. A Chunk is not a cut but a slice taken from a list of boundaries. The first boundary is zero and the last is the length of the text, so joining the pieces gives back the original by construction, not by checking. Overlap and dropout cannot be expressed. A bad boundary produces an ugly split, never a broken document. Within a window of two thousand characters, a paragraph end is chosen first, then a line end, then the edge of the window.

On read, two things are checked together: that the offsets start at zero and run without gaps, and that the hash of the joined text matches the recorded hash. When both hold the document is verified. When they do not, that is a report, not a failure. It means legacy data written before offsets were recorded, or an interrupted write, and writing the document again heals it.

A document's identifier comes from its content. Load the same document twice and it stays one; load it again on another machine and it converges on the same identity. Editing passes the existing identifier so the document stays itself. A Chunk's identifier comes from its position, so after an edit the piece at the same position keeps the same identifier.

3. A Chunk mentions a Concept

When a document arrives, the system looks inside it for the names of Concepts already in the graph. When it finds one, it leaves a MENTIONS relation between that Chunk and that Concept.

Chunk ──MENTIONS──> Concept

This link is made deterministically. If the origin matches, or the Concept's title and stem tokens appear in the Chunk, the link is drawn, and the rule that drew it is recorded on the relation. No person judges it and no model is involved. The same document against the same graph yields the same links.

Two things are worth stating up front. A wikilink that addresses a document id is not a mention, because the linker matches Concept titles. And unless a topic is set explicitly, a document's topic is derived from the topics of the Concepts it mentions; a document with no mentions has an empty topic.

Because of this link, a Concept exists independently in the graph and can still be followed back to the sentences that support it, and search can return not only the matching Chunk but the neighbors that Chunk mentions.

4. Linking is automatic; identity is a person's call

Two jobs have to be told apart here.

Linking a Chunk to an existing Concept is automatic. If the title appears, the link is drawn. That is a mention, not a judgment.

Deciding that two Nodes are the same thing is never automatic. In Vinc two Nodes are the same only when their identifiers are the same. A similar title is not grounds for merging, and neither is a shared type name.

So there is a covenant on the writing side: search first, and reuse any identifier already there before making a new Node. The server does not enforce it. Enforcing it would take a similarity judgment, which the server has undertaken not to make. Instead, when the covenant goes unkept and near-duplicates pile up, the system proposes candidates by title similarity and a person folds them. A pair once judged distinct is remembered and not proposed again.

How that identity gets settled in retrieval is Chapters 16 and 18.

5. What dividing the responsibility buys

The reason for dividing responsibility between document and graph is simple. When a document changes, the document is corrected. When the graph's meaning changes, the Nodes and relations are corrected. Bundle the two and every edit to a document forces a revisit of every concept derived from it, or every refinement of a concept reaches into the source text.

Kept apart, each moves on its own. Rewrite a document and its Chunks are replaced and its mentions relinked, while the Concepts and their relations stay as they were. If the document is the original of a memory, the graph is the set of handles built to pull pieces of that original out again and again.

The next chapter asks how far these handles should go, and why the system of concepts is managed apart from the structure of the graph.

Continue with Chapter 13, How the Concept System Is Managed.