> A language server is a specialized program that processes language-related tasks. This includes activities like text generation, grammar correction, rephrasing, summarization, data extraction, and more.
This confused me initially. I've never heard language server in this context. To me, it has always been in the context of the LSP. But maybe this is a common usage of the term and I've just missed it?
Even if so, it seems like an odd term, since this is a self-contained desktop application and not really a server, right?
Now I'm imagining a natural-language language server -- a plugin for text editors which can provide services like text completion and transformations in plain text documents based on an underlying LLM. Why let programming languages have all the fun? :)
No, you are right, it's a new term I made up. The idea is that I see this as a RAD tool (for non-programmers). So you would have this app open all the time and it would serve different tasks you have prepared before. But that part is not finished yet and it will be limited in the free version.
As to Why --> this one actually has a decent GUI and one that's not (visibly) based on a locally running webserver -- so it seems like it's got a shot at getting 100x popular with just a YouTube video (or gasp Tiktok). With the popularity of open-source today I find it a little hard to trust that something is "ensuring maximum privacy" as you say without having the open-source badge (yes, I know I shouldn't be so naive, but no, I am not going to look at network level logs to find out if the app is truly contained to local).
I resent having to step in and say something here, but the way you describe things, I effectively don't exist. I read the app code. I write stdlib code. I grokk every line of every library I use. I share alike.
I'm not saying change your ways, just consider being a bit less flippant when you say things like "IMHO, at some point, it's just about trust". As someone who seeks to profit from this, that's about the worst example you can give a prospective user like me.
Also, FOSS has a lot more chances to have eyes on it. I definitely don't read every line, and i definitely don't expect anyone else to - but i do expect more eyes to be on it, to raise questions, spot interesting behavior, etc. If you see a questionable network IO you can look in the source to investigate, etc.
It has value even if every person doesn't individually read every line.
Yes, and a big part of establishing trust is sending the right signals. The risk vs reward differential is way higher for open-source compared to closed source, so all other things being equal I would trust open-source more. Note that all other things are actually not equal in this case, e.g. your closest competition is open-source: Ollama, GPT4All, Llamero.
> Fair point, but if this was open-source, would you really go and read all of the source code in several programming languages and also audit all of the dependencies?
This probably isn't the best justification to use on tech forums. Odds are higher here and in similar forums that some of audience do indeed read the source code, do static analysis on them, check the libraries for CVEs, etc.
Sidenote, i value FOSS but i'm supportive of your desire to keep it closed and make money.
With that said, i'd never pay $5/m these days for something unless is had very good value to me and obvious ongoing costs i'm using (servers/etc) for the dev/company. Those costs would need to be clear though, not just "we chose to run some servers so that we'd have features which justified subscriptions"
I would however, pay for upgrades - ie the licensing model employed by JetBrains/etc. Which can come out to $5/m (or yearly purchases, etc), or w/e, but most importantly the software keeps working if you decide to cancel.
edit: Also i'm on Linux. Looks like i can't buy it anyway haha
Exactly! I don't want to subscribe to anything, I want to pay once and have the license forever (for that specific version + bugfixes).
But that makes OSS more complicated - people usually don't pay for something they can clone & build themselves.
BTW: Linux and Windows will definitely happen, I just didn't have time yet. Sorry, it's a lot of work and time so I had to pick macos first, and even that alone is a lot of testing on different configurations.
I will say, as someone who values FOSS - if you see areas in your app that don't feel super proprietary and that you can library-out - that puts forward good will to people like me.
Ideally your app would be all FOSS components and i pay for the glue to tie it all together, but i've seen that become too easy to knock-off. Depends on the app of course. Still, FOSSing parts can do a fair amount of good will to the FOSS crowd, imo.
disclaimer: I'm one of the maintainers working on Ollama.
I would love to hear how you are using Ollama. One of the upcoming releases will involve an official release of Ollama on Linux with CUDA support of the box. From there, we will publish Ollama Docker images to enable GPU support as well.
Thank you for your amazing work! I more or less use it as a llama cpp replacement cause I honestly can’t figure out a good prompt structure so I get bad results. On my M1 MBP, I use it for stuff that can’t go to GPT but can only use upto 13B. I also run a 34B model with CUBLAS enabled on my Linux server by modifying the go files mentioned in one of the issue tickets.
- I was missing file sizes on the models list, and even in the installer window there's only a percentage indicator. Blurring the window while installing also stops me from visually exploring the app while waiting for the install to finish (maybe just a lil' bit of blur is enough to show that the rest of the window is inactive?).
- Scrolling up should stop auto-scroll in the chat, I couldn't even reach the Stop generation button while it was generating. Maybe put that button in a static location.
- Maybe an uninstall button for models is appropriate.
Thanks for the feedback, this is what I was hoping for.
> Scrolling up should stop auto-scroll in the chat
This is at the top-of the list for today, check the website/discord/twitter for updates. I want to do both - pin the button at the bottom, and also make sure that if you scroll, it will stay there. Currently, it's one component which contains the button and also scrolls and it was a bad idea.
> Blurring the window while installing
Agreed 100%, I definitely want to improve downloader a lot more but this was good enough for first release.
> Maybe an uninstall button for models is appropriate.
It should be there (in the upper area), I hope the 24h cache in cloudfront is not too much but the link should be pointing to 09-18 version which has Delete button.