Enabling Real-Time Data for AI Systems
SCHEMA AND DATA QUALITY
It’s not uncommon for the data infrastructure layer’s value to be based in part on adaptable schema mechanisms. Granted, options such as JSON, Avro, and Parquet have endowed organizations with schema flexibility prior to today’s advanced ML. Nonetheless, real-time data infrastructure is replete with numerous constructs to reinforce the schema flexibility helpful for AI deployments. With schema-on-write capabilities, “You don’t have to have everything mapped out perfectly beforehand and then manage the database that way,” Kaplan explained. “You don’t have to have this many columns and rows. You can say these are the things I want to collect, and I want to write it this way.”
Data quality is imperative for accurate model responses and tenable actions from agents. Numerous options, particularly those from streaming data platforms, include mechanisms so that “duplicates, late data, and malformed records get validated and filtered in flight, before they reach the model,” Falconer mentioned. Rectifying discrepancies in late data gives AI systems the proper sequence of time-sensitive data. High cardinality, which is said of columns in a database with many unique values, is another important notion that top time-sensitive data infrastructure natively supports. Kaplan described an optimal situation in which databases provide an “unlimited number of combinations of data, tags, and values that let you get a good picture of a system’s current state.”
LONG- AND SHORT-TERM MEMORY
The tenet of memory is ascending in importance across applications of language models, particularly when involving dynamic agents endowed with them. Real-time data infrastructure’s rapid ingestion rates are complementary to a storage architecture that facilitates data access for low latency interactions with models. Top platforms that avail themselves of SSD storage and DRAM, in addition to forms of parallelism to optimize transactional interactions, can quickly make data available “for short-term memory,” Srinivasan remarked. “You can easily create new data items on-the-fly and also share it across multiple agents as you are creating these items.” The capacity to combine these techniques for short term storage—or memory for agents—with long-term storage within the same engine also assists with low-latency AI deployments. Support for ACID transactions ensures data isn’t lost once it’s stored. By pairing long-term and short-term storage in the same repository, real-time data infrastructure underpins numerous agent-based use cases. “Agents can audit the workflows of other agents,” Srinivasan observed. “You can do audits after the fact or during the fact.”
The enlarging number of agent applications is emblematic of the need for real-time data for AI systems, while heightening other concerns such as controls for them. “We’re going through a shift from a world that’s hyper-static, from a security and control perspective, to one that has to be incredibly dynamic, because we have this new actor type that is probability distribution,” reflected Keycard CEO Ian Livingstone. “It has a different security profile, and what we want is dynamic access controls based on what the agent is assigned at that point in time.”
STREAMING DATA
The capacity for language model-powered agents to perform actions in real time typifies the evolutionary leap forward of modern AI applications. Although other forms of data infrastructure, including operational databases, are applicable to these model-infused bots, streaming data platforms have several characteristics that are ideal for such applications. In fact, the pendulum may be shifting so that “agents are the primary adopters of streaming systems,” Livingstone said.
The uninterrupted processing of streaming data platforms is utilitarian for low-latency agent deployments. Even popular open source tooling contains many time-sensitive calculations devised to transform data into a state consumable for downstream applications involving agents or foundation models. According to Falconer, “Tumbling and sliding windows give you rolling metrics such as the last hour’s error rate or today’s completed tasks. Session windows group a string of events, a user interaction, or a trip into one unit the agent can reason about.”
REAL-TIME AGENTS
Streaming data systems are particularly helpful when there are multiple data sources channeling continuous data flows. Such applications might involve reading sensors from a factory or supporting facets of robotics. The queuing constructs of popular streaming data resources such as open source Apache Kafka are ideal for these workloads. “When you’re having multiple streams, meaning multiple writes from different systems at different times, you can put it in one queue and pull it from that one queue,” Kaplan commented.
Such queue-based tooling typically adheres to a publisher subscriber paradigm, in which some sources write data to a queue while consuming applications, typically called sinks, receive it. Coupling this functionality with some of the temporal capabilities of streaming data platforms accelerates deployments of agents. According to Falconer, “Pattern detection recognizes a sequence of events that should inform or trigger the agent. And latest-state materialization collapses a changelog to the current value per key, so the agent reads the current status of an order and not every update it ever had.”
The complexity of the workloads supported by real-time agents is nothing short of admirable. They enable AI deployments at tremendous scale. Subsequently, “One person can now deploy tens or even hundreds of agents to do a bunch of tasks, and they’re autonomous,” Srinivasan revealed. “They go around generating more data and sharing more data and looking at things and making decisions, and all of this is machine-driven.”