Architecting a Modern Data Stack for AI Agents

USE REFLECTION AND A SHARED MEMORY LOCATION

Reflection is a strategy in which one agent monitors and critiques the output of another. This can provide a powerful way to improve agent output in a loop, with one agent producing output, the other pointing out flaws, and the original agent fixing the flaws repeatedly until the second agent can’t find any more of them. This can be a more powerful strategy if the two agents are from different models, ChatGPT criticizing Claude, for instance. The reflective agent serves in many ways like a quality tester. Finding bugs in agent-created code is an excellent use of reflection, but to reiterate what I already said: A human should still test the code before it goes to production.

Each step in an agentic workflow should be aware of what the previous step did. Also, there is often the need for one step’s output to become another’s input. There needs to be a shared memory space to store context and state so the entire agentic workflow functions as one unit. This is also a good place to store past context, previous prompts, and templates. LLMs usually have a built-in cache, but it is often inadequate. Hopefully, over time, this will improve, but the cache for that model will still not transfer to another model’s agents, so you need one for the whole workflow.

USE API STANDARDS WHEN POSSIBLE

Model Control Protocol (MCP) is a nascent standard that makes it easier for agents to connect to various tools to do work. It remains to be seen if it will become the standard across the industry, but I have hope. Not confidence, just hope. SQL is supposed to be a standard, but Oracle SQL and SQL Server SQL are eminently not the same thing. And there’s always the issue of someone coming up with another competing “standard” and another and another. As long as there’s only one, use it and support it. A standard way for

AI agents to work with other applications makes development and modular interchangeability far easier. Continuously monitor and adapt. Observability is key to long-term production use and improvement over time, whether we’re discussing AI, ML, or any other type of data flow or application. Build telemetry flow into every part of the model, observe how it does over time, and adapt and improve based on that information. Some technology that can help with this is Arize Phoenix or LangSmith. Observability software like these are super helpful for iterating the agent’s design as well, so use them from the beginning and don’t ever stop.

GOING TO PRODUCTION IS THE BEGINNING, NOT THE END

For the TL;DR crowd, start with a solid data architecture as a foundation. Build for fast data, and scale from the beginning. Provide for vector and semantic search. Use solid agentic workflow best practices including reflection, parallelism, robust guidelines, MCP standard, and a single shared memory location for the whole workflow. Be kind to your future self by making everything as modular as possible and building observability in from Day 1.