KMesh: Local updates. Continually improving knowledge networks

The Basic Idea

KMesh is a research project I have been working on recently. Its central question is: Can AI continuously acquire new capabilities through local knowledge updates, without retraining the entire system each time?

KMesh combines a small, shared computational core with an external knowledge network that can keep growing. Knowledge is organized into knowledge patches: a patch can express a fact, rule, condition, exception, or procedure, has a stable identity and version, and connects to other patches.

The inspiration comes from how the human brain works—or, more precisely, my own understanding of it. Knowledge forms a network, not a strict hierarchy. Learning often updates local knowledge without disrupting the overall structure. Intuitively, each learning event is like applying a patch to the knowledge network. Just as many open-source projects have evolved through countless patches over the years, knowledge can develop through continual updates.

All patches use a unified representation and read/write interface, without an upfront distinction between “summary knowledge” and “ordinary knowledge.” The network as a whole is not a strict hierarchical tree, either. Through the network structure and training methods, we hope to encourage different reading preferences at different stages of computation: sometimes retrieving specific details, sometimes drawing on rules reusable across tasks, and sometimes reaching beyond the current neighborhood to find knowledge not yet considered.

In this design, the computational core selects, reads, and combines knowledge; the knowledge network stores and accumulates it. New knowledge can be added as new patches, and incorrect knowledge can be corrected through versioned updates. Further research will explore whether training only the representations of a small number of patches can produce transferable capability improvements while preserving other knowledge that remains valid.

Local updates do not mean that every change will improve every task. What KMesh aims to test is whether such updates can drive continual improvement in overall capability with controllable interference and cost.

What Will Be Tested First?

The project begins with small rule-based worlds that can be solved exactly, rather than directly training a large language model.

The first stage will compare standard retrieval, knowledge-graph access, and learnable staged reading strategies. It will test whether a model can actually use external patches, combine previously unseen knowledge, and adjust its answers correctly when key patches change. Subsequent tests will examine continual addition, modification, and retraction of knowledge, followed by local parameter learning with the shared core frozen.

These experiments will explicitly distinguish “answering correctly after the reference material is replaced,” “acquiring new capabilities through local training,” and “automatically forming abstract knowledge.” They will not treat these as the same achievement.

Long-Term Direction

If the core hypothesis holds, KMesh will further explore the formation of reusable abstractions and tiered knowledge storage across GPU memory, main memory, and disk. Keeping only the currently needed working set on the GPU would allow knowledge capacity to grow without requiring GPU memory to grow at the same rate.

KMesh serves a longer-term AI for Everyone vision: intelligent systems that can run inexpensively on ordinary devices and continually accumulate and correct knowledge through use.

For now, it remains a research proposal whose hypotheses have yet to be validated. Through reproducible experiments, we hope to gradually answer: Can a small computational core keep expanding its capabilities by drawing on a continually evolving knowledge network?

Research Progress

There are currently no experimental results that answer whether most LLM training requires only local updates, or how much money this would save. We are still preparing the foundational tools needed to test this question.

Confirmed Progress

Three things have been confirmed so far:

Knowledge and rules can be represented clearly and checked for basic errors.
Two independent reasoning programs produce the same answers in the 64 small rule-based worlds tested so far.
A reasoning trace can be checked independently for validity. The associated correct, incorrect, and edge cases have passed acceptance checks.

These developments are recorded in the implementation status.

What Remains Untested

Current reasoning is performed by programs written to follow explicit rules. Experiments on neural networks reading knowledge, combining knowledge, and learning locally have not yet begun. There are therefore no conclusions about learning performance, forgetting, or training costs.

Next Steps

Following the research plan, the next steps will address these questions in order:

Can a small model truly understand and combine external knowledge?
Can modifying only a small amount of knowledge or a small number of parameters enable it to learn something new while retaining existing capabilities?
After repeated local updates, can these pieces of knowledge still be used together correctly?
When applied to LLM training, does this actually reduce costs at the same level of performance?

Two questions must be tested separately: whether changing fewer parameters is sufficient, and whether changing fewer parameters actually saves computation and time.

Project Link

KMesh