Enabling Real-Time Data for AI Systems

The most immediate consequence of the widespread availability of modern statistical AI applications is perhaps their inordinate emphasis on enterprise data. Regardless of how capable they are, deployments of language models, intelligent agents, retrieval-augmented generation, and digital twins hinge on the data fueling them.

Moreover, these implementations introduce other data requisites which, if not necessarily novel, greatly transcend the scale at which they were previously necessary. Latency, even that involving a matter of seconds, has become all but intolerable. There is a renewed prioritization of data currency, for which traditional transactional or analytical engines simply aren’t designed.

According to Aerospike CTO Srini Srinivasan, the pervasive use of contemporary advanced machine learning (ML) systems has placed an onus on enterprise data infrastructure for “storing more data in real time, accessing more data in real time, and ensuring the data you access is accurate. This means you can’t use a cache as much, because then you get a wrong answer, maybe an answer based on data that was 2 hours ago, and [it] isn’t the right answer the user expects.”

Numerous technologies have emerged as viable options for accounting for the real-time data interactions driving today’s AI. Operational databases—including those specializing in time-series calculations—are gaining significant traction. The same can be said of streaming data tooling.

Regardless of which approach is invoked, the worth of real-time data access is predicated on considerations such as memory for AI resources, schema flexibility, high cardinality, data quality, and timely access controls. These concerns effectively ground lofty aspirations for statistical AI in the practical realities of the data driving them.

Or, as Sean Falconer, head of AI at Confluent, put it, “The moment AI has to do something useful, it needs data that changed seconds ago, sitting in operational systems the model has never seen.”

THE DATA LAYER

The immediacy of the data required to capitalize on current investments in advanced ML may be its most prominent attribute. What time series databases, other types of operational databases, and streaming data stacks actually furnish is the foundational data layer on which intelligence is applied. There’s a bifurcation to their utility for doing so, which Evan Kaplan, InfluxData CEO, described with a simple analogy. “We’re the sensors: the eyes, the ears, the nose, the skin, the touch, the taste, and the smell. We’re also the arms and legs that take action. But, we’re not necessarily the brain, which is built elsewhere.”

The sensors to which Kaplan referred are the respective data sources that real-time data infrastructure processes. They might include audio or textual inputs for model training data, network telemetry, or the latest signals from satellites in space. Moreover, once that data is processed and sent to applications of language models, it is often transmitted back for a defined business purpose.

What real-time data infrastructure actually does is purvey the “ingest, storage, and taking of action on the thing in the real world,” Kaplan summarized. It’s vital to realize the delineation between what the data layer and the intelligence layer achieve. Although several data platforms have mechanisms for analytics, “The LLM [large language model] has the intelligence to take data, which is real time, and then come up with the answer,” Srinivasan pointed out. “It has all the knowledge built into its models.” Consequently, there may be less of a need for analytical databases within these low-latency constraints.

INGESTION AND DATA AVAILABILITY

Real-time data ingestion is pivotal for making data available for AI deployments at the current pace of business. Time-series databases that serve operational purposes specialize in high ingestion rates. Kaplan described an Internet of Things use case at scale in which there’s a sensor network of batteries—each generating approximately 40 metrics—in a million homes; organizations need to determine the state of the network in the past 10 seconds.

“Normal databases can’t ingest like this because you have to index a bunch of stuff, and it tends to take off,” Kaplan revealed. “A time-series database ingests this stuff and makes it easily available in almost zero time to be read, so it’s available to act.” Interestingly, these and other facets of real-time data processing run on CPUs, as opposed to GPUs, which are more specialized and commonly found for running the actual AI systems. There are numerous ways to expedite and scale ingestion operations, including employing what Kaplan characterized as a “Write Ahead Log, which makes it so that before the data is resonant in the database, you can act on it.”