Interesting to see Mistral positioning Forge as the "all-in-one" platform. The trend is clear: every model provider is building their own platform layer.
The question for developers is whether they want lock-in to a single provider's platform or a middleware approach that abstracts across providers. If Mistral changes pricing or deprecates a model, you're migrating your entire workflow — not just swapping an API call.
The most resilient architecture: use provider-specific platforms for experimentation, but run production through an abstraction layer that lets you swap providers without code changes.
I am rooting for Mistral with their different approach: not really competing on the largest and advanced models, instead doing custom engineering for customers and generally serving the needs of EU customers.
Mistral has been releasing some cool stuff. Definitively behind on frontier models but they are working a different angle. Was just talking at work about how hard model training is for a small company so we’d probably never do it. But with tools like this, and the new unsloth release, training feels more in reach.
This is definitely the smart path for making $$ in AI. I noticed MongoDB is also going into this market with https://www.voyageai.com/ targeting business RAG applications and offering consulting for company-specific models.
Huh. I initially thought this is just another finetuning end point. But apparently they are partnering up with customers on the pretraining side as well. But RL as well? Jeez RL env are really hard to get right. Best wishes I guess.
How many proprietary use cases truly need pre-training or even fine-tuning as opposed to RAG approach? And at what point does it make sense to pre-train/fine tune? Curious.
And yet your blog says you think NFTs are alive. Curious.
But seriously, RAG/retrieval is thriving. It'll be part of the mix alongside long context, reranking, and tool-based context assembly for the forseeable future.
Note that any supervised fine-tuning following the Pretraining stage is just swapping the dataset and maybe tweaking some of the optimiser settings. Presumably they're talking about this kind of pre-RL fine-tuning instead of post-RL fine-tuning, and not about swapping out the Pretraining stage entirely.
The question for developers is whether they want lock-in to a single provider's platform or a middleware approach that abstracts across providers. If Mistral changes pricing or deprecates a model, you're migrating your entire workflow — not just swapping an API call.
The most resilient architecture: use provider-specific platforms for experimentation, but run production through an abstraction layer that lets you swap providers without code changes.
But seriously, RAG/retrieval is thriving. It'll be part of the mix alongside long context, reranking, and tool-based context assembly for the forseeable future.
https://docs.mistral.ai/api/endpoint/deprecated/fine-tuning
It's feasible for small models but, I thought small models were not reliable for factual information?
Foundational:
- Pretraining - Mid/post-training (SFT) - RLHF or alignment post-training (RL)
And sometimes...
- Some more customer-specific fine-tuning.
Note that any supervised fine-tuning following the Pretraining stage is just swapping the dataset and maybe tweaking some of the optimiser settings. Presumably they're talking about this kind of pre-RL fine-tuning instead of post-RL fine-tuning, and not about swapping out the Pretraining stage entirely.
Is it possible to retrain daily or hourly as info changes?