Building Trustworthy Data Foundations for AI
A plethora of reports highlight the prevalence and dangers of AI hallucinations and other forms of inaccuracies resulting from AI applications. These errors can be traced to many different root causes, but the solutions boil down to a few methods: restrict the content to a carefully defined and structured set of validated information, protect the application through governance and monitoring, and evaluate the output to provide continuous improvement.
AN AI PARTNER FOR MEDICAL WORKERS
After years working in a variety of healthcare disciplines observing the challenges faced by researchers and practitioners as they searched and digested large quantities of information, Ome Ogbru sought to remove some of the laborious aspects of their work by using AI to assist them in collecting, analyzing, and presenting medical data. A medical doctor, he founded a company called AI Narrative Generation and Engagement Solutions (AINGENS) and developed a software product called Medical Affairs Content Generator (MACg) that uses PubMed (pubmed.gov) and user-uploaded documents as its preferred sources for developing scientific content. PubMed consists of over 40 million citations for biomedical literature from MEDLINE, life science journals, and online books.
The collection is held in the National Library of Medicine and is considered a highly reliable source of data. In existence since 1996, it is extensively used by medical information professionals, including physicians, nurses, and librarians. MACg is designed to provide an AI-based workspace for medical writers, healthcare professionals, and researchers. “We cover the entire workflow,” said Ogbru, “from literature searches to summarizing, editing, and producing slide presentations.”
Having all these functions in one system gives a more seamless user experience, improves efficiency, and reduces cost for individual users and organizations. “The system cites its sources,“ he emphasized, “providing traceability and verifiability, which increases trust. The evidence-first approach ensures that the system is using curated sources, reducing the risk of hallucinations.”
“There are a few steps to take in order to maintain accuracy,” Ogbru continued. “One is to understand the capabilities of the AI models, which have context limits. Users should not overwhelm AI models with many large files or very long instructions. Using the AI model’s training knowledge as a source of truth also increases the risk of inaccuracy. MACg reminds users to provide a knowledge source, says when it does not have the right information to fulfill a request, and acknowledges when it does not know instead of fabricating information.”
BUILDING A SOLID AND PRECISE DATA FOUNDATION
Not every situation affords the opportunity to work with a focused set of information, which is particularly the case for large enterprises whose performance depends on numerous processes and volumes of disparate data.
“Many organizations have multiple data lakes, warehouses, and application data,” said Gaurav Pathak, SVP and GM for data governance and privacy and CLAIRE AI for Informatica, a Salesforce company. “Our message is that a strong data foundation is essential to any AI application.” Informatica has been managing enterprise data for over 3 decades and offers expertise in data integration, quality, governance, cataloging, and master data management through Informatica’s cloud.
“Our core differentiator is our unified metadata foundation that allows us to provide productivity and efficiency benefits across data management tasks with CLAIRE.” CLAIRE is the metadata-driven intelligence layer powering the Intelligent Data Management Cloud. This detailed metadata can be used to drive AI agents toward trusted data. For example, the agent’s goal may be to discover data that is managed by a certain owner, has been used repeatedly in reports, or has been published within the past month. “We can establish certified datasets that have been validated,” commented Pathak, “so that out of a huge body of information, the most trusted and relevant datasets are chosen. By using an index of high-quality data, we are saving time, reducing token costs, and increasing accuracy,” he pointed out. AI provides the benefit of scaling otherwise manual processes, but that scaling also produces risks.
“Even the smallest mistakes will be magnified when at scale,” Pathak noted. “Each of these mistakes compounds. In addition, with agentic AI, there is usually not just a single workflow, but multiple ones chained together. Even if each one is 95% accurate, having multiple workflows can quickly reduce the overall accuracy.” Other than having a solid data foundation, the most important action an organization can take in AI development is to evaluate system performance.
“Right now, evaluation building for AI is in the early stages, more art than science,” explained Pathak. “Getting performance metrics such as how many customer calls are deflected versus being resolved automatically is key to continuous improvement. Without exploring the root causes of any issues, you don’t know if the answers are incorrect or the system is just lacking the information to provide an answer.”
A SYSTEMIC APPROACH TO ATTAIN TRUST
Organizations should keep in mind that AI applications are systems, and all the components must work in concert for AI to be successful.
“Every piece of the system influences reliability,” said Ben Prescott, head of AI solutions at Trace3, an IT consulting firm. “The biggest misunderstanding organizations have about creating AI applications is that a new model will solve their problems. AI is reliant on data. If there are version control issues, data quality issues, or structural issues in the data, AI will not function properly, regardless of the model.” Design elements can help mitigate AI fabrications.
“Training data existed at a point in time and may have been accurate at that time, but once in production, flaws may become evident—a link from the training data may be dead, for example,” explained Prescott. “In that case, it is not technically wrong, but it is no longer producing reliable results.” By anchoring the AI to updated information from the organization or by implementing a validation loop to open and verify sources before presenting them to the user, these effects can be reduced. A key element for reliability is to provide governance constructs and ownership clarity.
“Who is responsible for reviewing and updating the data? This needs to be built into the AI lifecycle and measures taken to transfer ownership if the owner leaves that job,” advised Prescott. “In addition, there should be a way to identify data that is ready for AI, and then marry it up with business use cases, rather than trying to get all the data ready for any use case that might come up.” The systems approach should extend to the development process itself.
“It is better to not treat the AI development team as separate from other teams, but to have developers embedded within the business units and working closely with them,” commented Prescott. “Otherwise, the AI team can become its own silo, and that is a recipe for stalled or failed projects.”
Consistent interactions with the users and subject matter experts will help all the stakeholders develop the knowledge and skill sets they need for success. Metrics for success for AI applications are not as straightforward as they are for many other enterprise applications.
“Defining what ‘good’ looks like is a challenge,” Prescott acknowledged. “Success for a business user is not the same as success for an engineer.” How accurate does an AI answer have to be for it to be accurate enough? As good as or better than that of a human? “Every step has things we can tweak to improve reliability, and humans should always be in the loop to provide feedback,” he concluded. “It is not a static scenario, but an ongoing process.”
PROACTIVELY MANAGING COMPLIANCE
Building Trustworthy Data Foundations for AI Another approach to ensuring valid output from AI systems is to deploy a platform specially designed to manage data, risk, compliance, and privacy.
Daimler Truck North America (DTNA) wanted to use AI agents to provide early warning of supply chain problems, given the severe impact that such issues can create.
“Daimler has a million trucks on the road,” said Satyen Sangani, CEO and co-founder of Alation, “and if a truck breaks down, they guarantee delivery of parts within 24 hours.” The company developed AI agents in two categories: one group to serve as planners and another to carry out tasks. The agents monitor inventory, detect risks, and alert human planners about impending supply chain problems before small ones become big ones.
DTNA uses Alation to manage and govern the underlying data that informs the AI agents about the status of company resources and activities with respect to its manufacturing processes. Prior to launching this initiative, DTNA had also been using Alation to manage General Data Protection Regulation and other compliance requirements, finance-related matters, and assess data quality throughout the enterprise.
As a result of its reliance on Alation, DTNA is able to make evidence-based decisions across numerous enterprise functions, and Daimler has been forthcoming with its expressions of satisfaction about the role Alation is playing. Alation was founded to connect people with the high- quality data they need in order to do their jobs. The company first introduced a data catalog that combined machine learning with human curators. In the following years, Alation developed AI agents to manage and govern data and AI applications and, most recently, introduced the Alation Intelligence Operating System.
“Ultimately, our platform is designed to help organizations get reliable answers from AI,” explained Sangani. “The best way to launch an AI project is to start with a specific business problem, such as a compliance issue that has to be implemented. In such cases, it is usually clear what data is needed, and the outcomes can be tested.” One of the most difficult obstacles is that although customers may like the idea of AI, they don’t fully trust it and do not want to change the way they work. “People have legitimate reasons to be cautious,” agreed Sangani, “but when they work with a reliable system that proves its value over time, there can be real benefits to efficiency and ROI.”
GOVERNANCE AND ACCURACY FOR KNOWLEDGE MANAGEMENT
Knowledge management (KM) has been one of the major beneficiaries of AI. “Many have woken up to the fact that knowledge management as a discipline is fundamental and a good use of AI,” said Stephen Bedford, product director at ClearPeople. “However, provenance is increasingly important, and the output needs to be correctly sourced.” ClearPeople’s AtlasFuse is a KM platform designed for Microsoft 365 users. It was developed based on the company’s professional services experience in KM to address the lack of an existing KM platform for Microsoft 365.
“All of the security of Microsoft 365 is present, but we add several layers of governance,” continued Bedford. “The user can configure AI settings such as the model’s ‘temperature,’ which controls how deterministic or creative the responses are, as well as how restrictive the search should be.”
Typically, the user would limit the search to content in a particular collection, or only retrieve content within a certain timeframe or just the final approved version of a document. Users can also identify the owners of the information to allow for direct verification.
“Copilot alone can come up with an authoritative-sounding answer that is wrong,” he noted. “AtlasFuse can restrict the AI search, instructing it not to provide an answer if the specified content does not contain it.”
At the Royal College of Psychiatrists in England, one of the tests students must take requires an interview with an individual taking the role of the patient to ensure that the students are interacting appropriately.
In order to improve the performance of those who did not pass, feedback was provided by two evaluators who analyzed the conversation and actions of the student. AtlasFuse was trained to do these reviews, with issues of concern defined by subject matter experts, and now the time required for the feedback process has been reduced by 80%.
The Royal College of Psychiatrists methodically validated the system before deploying it by comparing its assessments against expert-reviewed, trusted reference answers across a representative set of examinations, refining the prompts and evaluation criteria until the AI consistently aligned with the expected outcomes.
In addition, the college can now provide useful feedback to all test takers, not just those who failed, which was not previously possible from a time and resources perspective. Building reliable, governed, and high-quality data foundations that reduce hallucinations and improve trust in AI-driven actions and outputs is an achievable goal. However, it requires a combination of planning, vision, and technical expertise. We are still on the learning curve, but the potential is significant and worth the effort.