I eventually settled on Gemini 2.5 Flash Lite as my workhorse, using it against structured data/vectors as a chatbot. It was cheap, good enough, and most importantly, fast - but it's now being sunset and I'm not sure where to go to find similar performance.
Groq seems promising but developer access hasn't been available for months. I'd love to use open source, but I never found anything comparable (cost/performance/speed).
Would love to hear your suggestions.
If you just wanted to test a model or have super light work you can get some free access to alot of open source models through nvidia (https://build.nvidia.com).
There is also Runpod and a few more.
Is there something in particular you need? if its just cost/performance/speed, check hugging face for there inference providers, it compares them against these three.
Although it's closed, openai's luna model is a great balance between cost/performance/speed.
lastly if you wanted to help shape a new inference provider, I am working on a more pro/consumer focused platform. It has a nice iOS app, platform etc. Models are limited currently and it's in beta. if you want to take a look it's called Ando and can be found here - https://andoai.xyz. (Its very early in its build, it will break, any and all feedback is appreciated)