Nuances of Build-or-Buy Decisions
The intricacies of the time-honored question of whether to build or buy take on new dimensions when it comes to enterprise AI. Organizations must not only account for traditional considerations like cost, time-to-market, and proof of concepts but also for those that are unique to advanced machine learning and language model deployments.
Concerns in our modern, AI-driven environment often center on notions of control of the data fueling the models, as well as control of models’ individual weights, parameters, and hyper-parameters. Users must consider how domain-specific or use-case specific a model is, which directly affects its performance.
Moreover, the build-or-buy decision is far from binary. Organizations must choose among options for building models and supporting systems from scratch, availing themselves of smaller open source models and possibly adopting much larger open source models. Alternatively, they can select between models for which the parameters are completely closed to customers and those that are amenable to fine-tuning.
Even open source options are far from straightforward as, depending on the particular open source model and its license, they might easily involve cost. As Perry Krug, head of field engineering at Pinecone, summarized it, “There’s no free lunch.” Other points of consideration include architectural, infrastructural, and networking requirements. Depending on which route a particular organization takes, it will be responsible for facilitating forms of prompt augmentation, context engineering, fine-tuning, and maybe even formal model training.
Many of these tasks are resource-intensive, time-consuming, and deterministic for building or buying AI systems. But they also help dichotomize the decision-making press into something closer to simple. “Build or buy is not so much about building everything from scratch or buying everything from scratch,” Krug reflected. “But, it’s a managed service versus putting the pieces together yourself and really managing and operating it by yourself.”
ORGANIZATIONAL COMPETENCIES
Even a cursory assessment of an organization’s strengths and weaknesses reveals which of the two options for adopting AI systems is best. Such an evaluation, which includes scrutinizing facets of funding, talent, and existing IT resources, is the starting point for gleaning whether “you and your engineering team are focused on managing database infrastructure, or on building your applications out, and your IP, and your customer requirements,” Krug said.
Paying for respective models from one of the larger model providers (such as OpenAI, Anthropic, or Google), which often entails per-token costs for transmitting data back and forth via API calls, provides an ease of adoption that’s difficult to match with open source or in-house options. According to Deepgram VP of product Natalie Rutgers, it may help to “get a proof of concept with your use case being solved, even with a closed source model first, just to get your feet wet and make sure it’s doing the job you need it to.” If so, users can later replace that model with one that’s less cost-intensive, including open source options.
MODEL COMPARISONS
Once organizations have determined a use case and specified whether a language model (or some other type of model) can solve their business problem, they’re tasked with choosing what Shawn Bradford, Laserfiche senior engineering manager, termed “the best operationally efficient model.” A systematic process for comparing models via predetermined criteria aligned with the business case is necessary to understand their respective utilities. That process includes what Bradford called a “testing framework” that involves several applications relevant to the foresaid business case.
For example, for those testing a model for summarization to aid with intelligent document processing, “We have certain criteria that we’re looking for that determines if the summary is good,” Bradford revealed. “The number of words in the summary, the number of concepts that were covered in the summary. We have this very systematic approach to measuring the quality of this particular use case, and we have several of them, up to 100 different use cases.”
GENERAL-PURPOSE LLMS
Although large language models (LLMs) available via the buy paradigm are often the quickest way to get started working with advanced models, there are several dimensions to employing this technology. These models are designed to complete a broad number of tasks with data available from public sources—which is why they’re a credible option for many statistical AI initiatives. According to Rutgers, “When bringing on a closed source language model that’s off-the-shelf, you’re hiring a generalist. The question is: How well can a generalist do in this role? How quickly can you train them?”
Vendors including Anthropic, Google, and OpenAI routinely compete with each other to provide the most capable model, and they do so with an onslaught of new model versions introduced with surprising rapidity. Within the enterprise, this involves a fair amount of retooling. Customers who purchase access to models through such providers aren’t tasked with this endless responsibility, which is why “the advantage of a company buying access to one of those models is companies will always continue updating them to get new capabilities,” Rutgers observed.
There are several cost considerations for deploying LLMs that aren’t open source. The most important of these factors pertain to inherent trade-offs (which are particularly acute at scale) between cost and performance. Krug mentioned that, because such models are general purpose, “They tend to be slower because they take a lot more into account. They’re more expensive, even if running on your own hardware, and certainly for a service you’re accessing too.”