Every enterprise leader is asking the same question: how do we use large language models? The answer isn't a model. It's a strategy. And the most expensive mistake is choosing the wrong approach for your use case.
The Three Paths
When it comes to deploying LLMs in the enterprise, you have three fundamental approaches. Each comes with different cost profiles, capability ceilings, and operational requirements.
Path 1: API (Buy)
Use a foundation model through an API. OpenAI GPT-4, Anthropic Claude, Google Gemini, or similar. You send prompts, you get responses. No infrastructure, no training, no GPU management.
Best When
- • General-purpose text tasks (summarization, Q&A, drafting)
- • Speed to market is critical
- • Task doesn't require deep domain expertise
- • Data sensitivity is low or manageable via enterprise agreements
- • Volume is moderate (cost scales linearly with usage)
Watch Out For
- • Cost at scale: high-volume use cases get expensive fast
- • Data privacy: your prompts travel to a third party
- • Model changes: provider updates can break your application
- • Latency: API round-trips add 200–500ms minimum
- • Vendor dependency: your capability lives on someone else's roadmap
Path 2: Fine-Tune
Take an existing foundation model, typically an open model like Llama, Mistral, or Phi, and fine-tune it on your domain-specific data. The model learns your terminology, your style, your knowledge.
Best When
- • Task requires deep domain-specific knowledge
- • Output must follow a specific format or style consistently
- • Data is sensitive and must stay on-premises
- • Volume is high enough that API costs exceed infrastructure costs
- • Latency requirements are strict (< 100ms)
Watch Out For
- • Data quality requirements: garbage in, garbage out at scale
- • GPU infrastructure costs and expertise needed
- • Catastrophic forgetting: fine-tuning can degrade general capabilities
- • Evaluation is harder: you need domain-specific benchmarks
- • Ongoing cost of retraining as data and requirements evolve
Path 3: Build From Scratch (Pre-Train)
Train a model from scratch on your own data. This is the most expensive and complex path. Very few organizations should consider this.
Best When
- • Your domain is radically different from general text (genomics, chip design)
- • You have massive proprietary datasets (billions of tokens)
- • Competitive advantage depends on model IP
- • Regulatory requirements prohibit using external models entirely
Watch Out For
- • Cost: millions in compute for a competitive model
- • Talent: requires deep ML research expertise
- • Time: 6–18 months to a usable model
- • Risk: high probability of underperforming off-the-shelf alternatives
The Decision Framework
Here's the framework we use to guide enterprise LLM strategy decisions:
Start with the use case, not the model. What specific business problem are you solving? What does the input look like? What does the ideal output look like? How good does “good enough” need to be? These answers determine the path.
Default to API, graduate to fine-tuning. For 80% of enterprise use cases, starting with a frontier API model (GPT-4, Claude) plus good prompt engineering and RAG is the right first move. It gets you to value fastest. Only invest in fine-tuning when you've validated the use case and identified specific quality gaps that prompt engineering can't solve.
Never build from scratch unless you're sure. If you're considering pre-training a model, make sure you've exhausted fine-tuning first. In our experience, fine-tuned open models meet enterprise requirements 95% of the time at a fraction of the cost.
The Fine-Tuning Playbook
Since fine-tuning is where most enterprises land, here's a tactical playbook:
Choose the Right Base Model
Not all open models are equal. For enterprise workloads, we recommend:
- General tasks (7B–13B): Mistral 7B or Llama 3 8B. Excellent quality-to-cost ratio. Can run on a single GPU.
- Complex reasoning (30B–70B): Llama 3 70B or Mixtral 8x7B. Near-frontier quality. Requires multi-GPU or quantization.
- Code generation: CodeLlama or DeepSeek Coder. Purpose-built for code tasks and outperform general models.
Prepare Training Data Ruthlessly
Fine-tuning data quality is everything. We follow a strict pipeline:
- Collect 500–5,000 high-quality examples of the task (input/output pairs)
- Have domain experts validate every example. Remove anything ambiguous
- Format consistently (conversation structure, response length, style)
- Include edge cases and failure modes (teach the model what NOT to do)
- Split into train/validation/test with no data leakage
1,000 excellent examples consistently outperform 10,000 noisy ones. Invest in curation, not volume.
Use Parameter-Efficient Methods
Full fine-tuning of a 70B model requires enormous compute. Parameter-efficient methods like LoRA and QLoRA give you 90% of the quality at 10% of the cost. LoRA trains small adapter layers while keeping the base model frozen. This dramatically reduces GPU memory requirements and training time.
With QLoRA, you can fine-tune a 70B model on a single A100 GPU. This puts enterprise-grade fine-tuning within reach of any team with cloud GPU access.
Cost Comparison: Real Numbers
Here's a rough cost comparison for processing 1 million requests per month with a moderate-complexity task:
- GPT-4 API~$30,000–50,000/month (depending on token volume)
- Fine-tuned Llama 70B~$3,000–5,000/month (2x A100 on-demand) + one-time $2,000 training cost
- Fine-tuned Mistral 7B~$800–1,500/month (1x A10G) + one-time $500 training cost
At scale, fine-tuned open models are 10–30x cheaper than frontier APIs. But only pursue this path when the quality gap is closed. A 10x cheaper model that gives 20% worse answers is not a savings.
The Bottom Line
Enterprise LLM strategy is not a technology decision. It's a business decision. Start with the use case, validate with APIs, and graduate to fine- tuning when you've proven value and identified quality gaps.
The organizations winning with LLMs aren't the ones with the biggest models. They're the ones with the clearest use cases, the best data, and the discipline to match the right approach to the right problem. Build the strategy first. The models are commodity.