Nuances of Build-or-Buy Decisions

FINE-TUNING

Tailoring models so they’re specific to an organization’s use case and business needs is a primary consideration when deciding whether to build or buy. Doing so often involves fine-tuning models, a process in which users can retrain the final or outer layers of a model with their own data.

Traditionally, the ability to fine-tune a model was one of the advantages of utilizing open source options, in which the weights and layers of models were exposed to organizations.

However, even the larger model providers are releasing models that users can fine-tune. “OpenAI has fine-tuning models,” Bradford remarked. Which models these larger providers enable fine-tuning for and when they’re released are two points of concern when relying on commercial model providers. According to Rutgers, “They don’t open up fine-tuning for every single model they release. There’s a lag; they’ll release a model early and then add on fine-tuning abilities later.”

PROMPT AUGMENTATION

Of the many forms of prompt augmentation, retrieval- augmented generation (RAG) and GraphRAG remain the most widely used mechanisms for making models more domain-specific. These constructs involve vector databases, graph databases, and semantic data technologies, respectively, as a means of providing additional context to models when answering questions. This is particularly useful, for example, with information expressly provided by an organization for its business case. It’s important to realize, however, that RAG needn’t necessarily be juxtaposed with fine-tuning when making models more domain-specific. “RAG and fine-tuning are not replacements for one another,” Krug said. “They’re not mutually exclusive. They don’t even provide the same set of capabilities.”

What both of these approaches have in common, however, is that they expand the amount of context for models to deliver more accurate outputs. They also reinforce the need for data quality, both for the data used to fine-tune models and the data that’s added to prompts via RAG. “A lot of companies think their data is easy to use and you can just feed it into the training process,” Bradford added. “But, in reality, it's not, and that requires some expertise to figure out what the quality is.”

THE RIGHT CHOICE

There are several nuances of the build-or-buy decision for advanced machine learning and language models along with their supporting systems. Organizations must determine how to make them specific for their particular business case while navigating facets of open source and commercial models, their own organization’s core competencies, obvious and less obvious costs, and how to properly evaluate models. The right choice hinges on each of these factors and will likely be different for each organization.