Fast, Accurate, Relevant, Intuitive: The Future of Search

BUILDING A RAG SYSTEM

Let me give you an example from my personal experience. I worked with an AI team whose first major project was to build a retrieval-augmented generation (RAG) system for financial data. As a member of the search team, I was brought in to make sure that the “R” in RAG worked well. I dug into the solution and discovered that the AI had built a custom search index with 20 lines of Python. I was shocked that so little effort had been put toward having a strong search architecture, but as I worked through the app, what I discovered is that in this RAG application, the retrieval logic could not just send a query to a search engine, but also had to send a structured query to a SQL database as well as several custom REST APIs.

From the search team members’ perspective, they saw that retrieval doesn’t equal search. Retrieval means getting data from whatever system, using whatever access pattern is needed.

They focused on making the next steps, the “augmented generation” parts of their RAG solution, really robust, with amazing synthesis of all the various source data into a powerful answer that had embedded the next steps for users in their journey to accomplish a goal. They viewed success as, “Did the user take the next step in the journey based on the action?”

WHAT SEARCH TEAMS DO WELL

I was taken aback by this experience. Were all my years of building search applications for nought? Well, no! Because search teams bring a lot to the table:

• We are really good at collecting and dealing with complex, confusing, even contradictory, datasets. We can distinguish that some rakes are not suitable at all for cleaning up the yard, such as those used in working with concrete or for cleaning out a sludgy pond, from those that can work well enough! We are even comfortable with the idea that sometimes the best results are that you have no good results. Don’t propose to the user that they try to rake the leaves with a shovel, Mr. AI Smart System.
• Ongoing evaluation of quality has long been key to search, and that is a powerful capability that we bring to these new problems. We are accustomed to collecting fuzzy results, such as, “Did the leaf rake function much better than the lawn rake for the task of collecting the leaves?”
• We are also good at dealing with competing objectives. Search teams think a lot about business objectives, how they are measured, and how that translates down to the implementation of an algorithm. Looking for a primary care physician? A user probably wants the best one closest to them. But a hospital might be interested in boosting on a doctor who has a lighter patient load than one who has a heavier one. We build algorithms all the time that deal with these challenges. We work with our business stakeholders to evolve them as objectives change.

As you might have guessed, we also found that a retrieval engine built in 20 lines of Python meant that the classic “garbage in, garbage out” situation arose. Implementing a true search engine was a win for the field of IR and a powerful opportunity to educate the AI team on what search brings to the table.