Long-running agents usually pay the model to run their own memory — an LLM call to summarize on every write and another to rank and rewrite on every recall. Move both operations onto deterministic graph traversal plus encoder embeddings so no memory step spends LLM tokens, and reserve the model for the final answer alone.
Run the Agent's Memory Layer on Encoders and Graphs, Not on the LLM
🔒 Pro tip · Advanced
Unlock this tip — and 114 more
This is one of 115 advanced, fact-checked tactics reserved for Pro. Get the full 137-tip library, a searchable archive, and a new tip every morning. Free for 7 days, then $9/mo.
Prefer to browse? The 22 Beginner tips are free forever.
More in Retrieval & RAG
🔎Retrieval & RAG
Eliminates the embedding API call and vector search on cache hits. The saved cost tracks your hit rate; it cuts embedding/retrieval spend, not the tokens sent to the LLM.
Cache the Context, Not Just the Answer
Cache the retrieved chunk set keyed by a normalized query, so popular or repeated questions skip the embedding call and vector search and reuse the same context block instead of rebuilding it every time.
🔎Retrieval & RAG
70-95% on document-heavy prompts
Stop Pasting Whole Documents: Retrieve the 3 Chunks That Actually Answer the Question
Dumping a full PDF or knowledge base into every prompt bills you for thousands of tokens the model never needed. Retrieve only the passages relevant to the question instead.
🔎Retrieval & RAG
20-40% fewer retrieved tokens at equal accuracy
Chunk on Structure, Not Character Count, So You Retrieve Fewer (and Smaller) Chunks
Naive fixed-length chunking splits ideas mid-sentence, forcing you to retrieve more chunks (and more overlap) to capture one answer. Chunk on semantic boundaries to send fewer tokens per query.