LoreTokens
Compressed Meaning For AI
LoreTokens are an AI-native serialization format that lets models work on compact meaning instead of huge blobs of JSON. They compress schemas, behaviors, and typical values into short, LLM-readable lines that can be expanded into full knowledge on demand.
Engineered alongside SAIQL and used inside Nova, an enterprise crypto trading AI.
Why LoreTokens?
Semantic Compression
Instead of squeezing bytes, LoreTokens compress meaning. A single line can encode the domain, concept, subject, fields, and output shape - which modern LLMs can expand into many pages of detail.
LLM-Native
LoreTokens are designed the way language models already think: regular, symbolic, and explicit. That regularity is what lets a model expand a single line back into full detail.
Field-Proven With SAIQL
LoreTokens power SAIQL's compressed storage paths, GPU hooks, and Nova's long-term trading memory. Together they have hit order-of-magnitude speedups over Postgres-only stacks in internal tests.
Formats At A Glance
Symbolic (.sym)
Machine-centric glyph stream used by SAIQL nodes and GPU hooks. Highest compression, least human-friendly - ideal for hot paths and logs.
LoreToken Standard (.lt)
Balanced format for CLI tools, diffs, and repos. Short but still readable once you know the glyphs.
LoreToken Ultra (.ltu)
Human-friendly variant for docs, audits, and exported memories where clarity matters more than every last byte.
Tooling and GPU Hook
CLI and Converters
Translate between symbolic, standard, and ultra formats with converter and C-based search tools built into the stack.
GPU Hook
LoreToken-GPU compresses buffers before they hit the GPU, saving memory and bandwidth with minimal overhead.
The Layer Under Our Stack
LoreTokens are the foundation SAIQL, the GPU hook, and Nova are built on. This page is informational - to see the technology in action, explore SAIQL.