Understanding Foundation Models: The Technology Behind Today's AI Tools
Before foundation models, building an AI system for a specific task meant training a model specifically for that task. A spam filter was trained on spam. A translation system was trained on translated text pairs. A medical image classifier was trained on medical images. Each system was narrow, purpose-built, and expensive to create from scratch.
Foundation models changed that logic entirely.
A foundation model is a large AI model trained on a broad, general body of data at massive scale, designed to serve as a base that can be adapted to a wide range of downstream tasks. The term was coined by researchers at Stanford in 2021, though the models it describes had been emerging for several years prior. The defining characteristics are scale, generality, and adaptability. A foundation model isn't built for one thing. It's built to be the starting point for many things.
The training process is what makes this possible. A large language model like GPT-4 or Claude is trained on a vast corpus of text, absorbing patterns of language, factual associations, reasoning structures, and domain knowledge across an enormous range of subjects. That training is expensive, requiring significant compute resources and large quantities of data, but it only has to happen once. The resulting model can then be applied to tasks it was never explicitly trained on, simply by providing the right instructions or a small number of examples. This is called zero-shot or few-shot capability, and it's one of the things that makes foundation models qualitatively different from earlier task-specific models.
The same underlying model that can write marketing copy can also summarize legal documents, answer customer service questions, generate code, and translate between languages. Not because it was trained on each of those tasks separately, but because broad training at sufficient scale produces a model capable of generalizing across them. That generalization is the core capability that foundation models introduced, and it's what made the current wave of AI applications possible.
Foundation models aren't limited to language. The same principle applies to images, where models like DALL-E and Stable Diffusion serve as foundations for image generation tasks. It applies to multimodal models that handle both text and images. It applies to models trained on protein sequences for biological research, on code for software development assistance, and increasingly on combinations of modalities that allow a single model to work across very different types of input and output.
The "foundation" metaphor is intentional and fairly precise. Just as a building's foundation supports many different structures built on top of it, a foundation model supports many different applications built on top of it. Those applications might use the model directly through an API, fine-tune it on domain-specific data to improve performance on a particular task, or combine it with retrieval systems and other tools to extend its capabilities. The foundation is shared. The application layer is where differentiation happens.
This architecture has significant practical implications. Organizations building AI products today rarely train models from scratch. They start from a foundation model and adapt it, which dramatically reduces the cost and expertise required to build capable AI systems. It also means that the capabilities of the foundation model set a ceiling on what the application can do, which is why model selection, choosing which foundation model to build on, has become a meaningful architectural and strategic decision rather than a purely technical one.
The concentration of capability in a small number of large foundation models also raises questions worth understanding. When much of the world's AI capability runs on a handful of models developed by a handful of organizations, the training choices, biases, and limitations of those models propagate into a very large number of downstream applications. Understanding that the AI tools you use are built on foundation models, and that those foundation models have their own characteristics, limitations, and failure modes, is part of understanding what you're actually working with.