An agent without memory is a very expensive goldfish. It can call tools, reason about what comes back, and produce something useful — and then the session ends and all of it is gone. The next run starts from zero and makes the same mistakes in the same order.
In the third session of building Whale, our agent harness, I didn’t write a line of engine code. I spent the whole session deciding what to put behind the memory hook we had already wired into the main loop. This is the write-up of that comparison: four candidates, the criteria that actually mattered, and the decision at the end — which, as it turned out, did not survive the week.
Memory is not chat history
The first thing to get out of the way: keeping the conversation around is not memory. Chat history is a transcript that grows until it blows the context window. Memory is the opposite move — deciding what is worth keeping, in what shape, so it can be retrieved by relevance rather than by recency.
That reframing is what turns the question from “which database” into something harder. The engine has to answer four things:
- Short-term against long-term. Does it distinguish what happened in this session from what the agent knows in general, or is everything one undifferentiated pile?
- Vector store against knowledge graph. Similarity search finds things that read alike. A graph finds things that are connected. Most real questions an agent asks are about connections.
- Token efficiency. Whatever comes back out of memory gets pasted into a prompt. An engine that returns generously is an engine that burns your budget on every single step.
- How it attaches to the loop. We already had the hook. The engine either plugs into it cleanly or it wants to own the control flow — and an engine that wants to own the loop is not a component, it’s a rewrite.
How we did the research
Not by reading documentation for four hours. I spun up a separate research agent per candidate — same prompt each time: read the repository, read the docs, read the comments, explain how it works, then give your opinion on whether it can serve as a memory agent for a harness like ours.
Four agents working in parallel while I read their output as it arrived. It is a small thing, but it is the honest answer to “what do you actually use agents for”: this, exactly this. Reading four codebases at once is the kind of work where an agent earns its cost, because a wrong summary is cheap to catch and a right one saves an afternoon.
What follows is my reading of what came back.
The four candidates
Semantica — “the open source Palantir for AI agents”, in its own words — turned out not to be a memory library at all. It is a platform with a memory-shaped API — the centre of gravity is a data pipeline with ETL, ingesting documents and personalising them, with a vector store underneath. If you need a data lake that an agent can also talk to, it is genuinely powerful. But we weren’t looking for a lake. We were looking for something small enough to sit inside a loop, and the dependency weight pointed the other way.
The Neo4j agent memory layer is the opinionated one, and I mean that as a compliment. It arrives with a strong view of what agent memory is: three tiers — conversation, session, message — plus entity extraction and facts stored as subject-predicate-object triples, which is RDF thinking under a friendlier surface. The API surface is wide, the entity recognition is good, and it is well traced. The price is the backend: you are in Neo4j’s world, self-hosted or managed, and you pay for that in operational weight.
OpenWolf is the minimalist, and the only one I had already run in a different project. It keeps one project memory across Claude Code, Codex and OpenCode, it works through hooks, and it is strikingly efficient with tokens — that last part is not a small virtue. But it is solving a slightly different problem: it keeps memory structured and present, it is not a general-purpose engine you build on. It stayed in the comparison because its simplicity is a standing argument against everything heavier.
cognee is the closest competitor to what we want Whale’s memory to be, and it was the only moment in the session where I thought “someone already built this”. A pluggable, persistent, multi-layer memory for agents that combines a knowledge graph with a vector store and a cache, with pluggable providers at each level, a rule-based classifier that picks the search type for a given query, and self-improvement loops on top. It is an interface library in the best sense: it does not tell you which graph database to use, it gives you a factory and gets out of the way.
The comparison in one view
| Shape | Strength | What it costs you | |
|---|---|---|---|
| Semantica (MIT) | Platform with a memory-shaped API | ETL pipeline, vector storage, strong structure | Heavy; its centre is a pipeline, not a loop |
| Neo4j agent memory | Opinionated memory layer | Three tiers, entity extraction, triples, tracing | Backend binding, operational weight |
| OpenWolf (AGPL-3.0) | Minimal hook-based project memory | Token-efficient, simple, proven in-house | Narrow scope; not a general engine |
| cognee (Apache-2.0) | Pluggable multi-layer memory | Graph + vectors + cache, provider-agnostic, self-improving | Complexity; more machinery than a first cut needs |
What I decided, and what happened next
At the end of the session the answer was cognee, with one caveat spoken out loud on the recording: until next time, maybe better ideas will surface.
They did. Within the week a paper reframed the question for me — not “where do I store what the agent knows” but “what kind of thing is an agent’s memory in the first place”. The answer we ended up implementing stores procedures rather than facts, and rewrites itself when they fail. That is the next post, and the next session.
Which is the part of this I would keep if you take nothing else: the comparison was still worth doing. Not because it picked the winner — it didn’t — but because it produced the four criteria. When the better idea arrived, I could tell it was better in those terms, in about ten minutes, instead of arguing with myself for another week.
Watch the session
- The series so far: https://www.youtube.com/playlist?list=PLPbgiIdJb7gE
- Where the series started: https://nowtec.solutions/blog/creating-an-agent-harness/
- Code: https://github.com/nowtec/whale
This article was written with the support of AI tools and reviewed by our team before publication (EU AI Act, Art. 50).
