Regarding 2 and 3: depending on the type of computations you need to do there are ways to verify the integrity of a delegated computation with very high confidence for very low cost. Typically if you're computations happens in a modular structure it's quite easy. See for example my THC (trustable homomorphic computation) project and the accompanying paper: https://pablo.rauzy.name/software.html#thc
There is also a video presentation because the conference where the paper was published took place during a COVID induced lockdown: https://www.youtube.com/watch?v=6DByVlqpH0s
Technically something like this is relevant, but most people would assume that in practice it is going to be way too much work to get it implemented on GPUs and in such a way that it doesn't add too much overhead.
Two and three are really the same problem and are solved by periodically running the same workload in duplicate on other systems and comparing results to detect unreliable sellers. Same as all the captcha systems. It's an overhead required to have trust built into a system. In the event of a faulty GPU, the owner would likely want to know anyways.
The first issue could be addressed by just never giving a seller enough data from a buyer to abuse, but that does require a critical mass of both buyers and sellers before you could distribute and dilute a single customer's workload enough. The company could bootstrap this with renting their own GPUs and subsidizing sellers at first though.
There's a privacy issue in both directions too: the tenant will be afraid their workload will be stolen, and the host will be afraid other data in their system will be stolen (if the sandboxing is imperfect).
Wait a year and see how kids get on blockchain to sell and buy GPU resources for rendering ‘trans furries’, or better analyse classmates’ stolen chats.
I can vividly remember we were by no means disclosing private info back in ‘94 only to see a world of influencers in 2024 sharing their very personal guts for profits.
...but also at no point in known history has it ever happened to find millions of people doing so in very mean and aggressive way, and worldwide scene to exploit.
agreed it IS a Big Brother of a sorts, where everyone participates to certain degree. but then written forums have been a thing before BigBrother and perhaps don't have this voyeurism added in the equation.
I know Azure has a confidential computing offering for GPUs, which I'm hoping will get broader uptake soon. It seems like the best way to address these concerns to me.
I'm not an expert on the area, but I've attended some conference talks on the subject at fosdem so I'll give it a go.
Essentially you're trying to provide a way to prove that the code running on the machine is what you instructed. This is achieved by a series of hardware attestations that measure and check the code to make sure it's what you requested. Generally this means encrypted ram at a minimum, and checks/balances that give you confidence this is the case (you have to trust someone, eg: Intel)
Is it perfect, probably not, but it's a lot better than just running VMs with unencrypted memory that any operator can jump into.
To my understanding most GPU workloads are not run in this way currently, and the operator can see/manipulate everything executed
I've used vast.ai (similar "Airbnb for GPUs" pitch) for years to spin up cheap test machines with GPUs you can't really find in the cloud (and especially consumer-grade GPUs like 4090s). Any insight into how this is different/better?
Main difference is that we are more opinionated (in terms of configurations) and sort of do the scrolling and sorting out for you – hopefully a bit smoother as a user experience. We sort out bad machines immediately. We're also directly working on making compute from unknown high-end data centers available, there's a lot of unused compute out there! See gpulist.ai
Also, don't know if vast.ai does this, but with us you can have 6 user sessions on your machine if you have six GPUs, so granular utilization is possible.
We own and operate 40+ data center GPUs (v100s, a100s, and ax000s) in a private cluster and use vast to rent unused capacity.
What would make you better than vast is extremely easy spot leasing and job prioritization.
I want to be able to have one of our training jobs finish, and then have the capacity immediately transition to a lease. With vast, we are renting in week long blocks.
You should be able to do that right now on vast. You just need to rent the gpus yourself with your own on demand instance(s) for your training job. As soon as it finished you then stop or destroy those instance(s) and the GPUs are available immediately (and if there are any other instances queued up in scheduling they will start up). Your actual job doesn't necessarily need to run in the container (if you know what you are doing).
(I'm the founder of vast btw - contact us for help on setting this up and/or any feedback on making it an easier/better process)
You can do that on llm.sxwl.ai
Shoot me an email at z@sxwl.ai for instructions, the web site is pretty outdated, the main UI is through restful API (which we don't have time to write doc yet)
Don’t build something if you don’t have a use case. All you have is a wishlist, until someone says “yes I want this here is $$$” which I assume the email will facilitate.
Im also interested in what the differing factor is. Would also like to see more documentation for onboarding rather than just "Ubuntu and root available".
vast you have to choose specific machines. gpudeploy routes to whatever resources are available.
vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way. Someone on HN wrote a script to run a test docker image on every machine and auto-tagged the machines' quality using their API, which is what I'd do if I was going to use vast seriously for compute.
I think it's more of a business strategy issue than a technical one.
I suspect it would be trivial for Vast or GPUDeploy to spin up a benchmarking job before allowing sales on that machine. I'm not an expert on PCIe lanes, but I would think the performance issues would be visible via bandwidth or latency on the lanes.
It kind of makes sense to me, though. If I were looking for absolute reliability and was willing to pay for it, I'd just go to one of the many GPU cloud vendors. Likewise, I suspect anyone willing to really work on getting good performance would rather be a real provider or sub-provider than being part of this nebulous C2C GPU cloud.
I feel they could onboard so many more users by having a fiat frontend that doesn't refer to blockchain. But maybe they're targeting crypto natives so users don't get shocked by price variability etc.
These links should work now, at least they do for me. We use a third party service called Termly for these, I think there was an issue with the url (missing the www).
> One good trick for describing a project concisely is to explain it as a variant of something the audience already knows. It’s like Wikipedia, but within an organization. It’s like an answering service, but for email. It’s eBay for jobs. This form of description is wonderfully efficient.
Seems the risk of this is a loose simile.
Edit: Though, thinking about it, Airbnb for GPUs is fairly accurate for this model of a marketplace where people let out their GPUs for others to rent.
Yeah I think in general it's a decent strategy. I'm this case it falls short for me because a key element of Airbnb is that the assets (homes/rooms) are provided by the user base, whereas this product doesn't seem to do that at all. So the strategy of using familiar companies is good, but I think it only works when the functional correlation is strong. In this case I don't think it is.
More so than the electricity of the GPU, is the electricity in cooling the GPUs heat. (In most cases cooling, in some cases, I’m guessing the heat might be a welcome bonus but that’s more rare)
Interesting, these companies are springing up left and right. Software solutions to a hardware problem. It looks like most of the available compute is allocated, so they are going to have to get a bunch more providers onboarded.
I wonder why y-combinator is stuffing their investments with multiples of these similar companies... https://www.shadeform.ai/ is another one.
As the post points out, this is a pivot from a company that was a consumer of this kind of service in its original incarnation; YC had presumably nothing to do with it.
That an interesting presumption. If a startup is "Backed by Y Combinator", and it pivots, you don't think that Y Combinator would have some sort of opinion on that?
I would expect some sort of conversation like: "Oh hey, we have another company, in our portfolio, that is far ahead of you, doing exactly the same thing. Maybe you should do something else?"
But of course, this is AI... plenty of space for gpu marketplaces.
YC partners might have an opinion and might offer advice but they're always going to support what the founders want to do—that's kind of the core principle of YC.
As for "other company in portfolio", that's unavoidable when funding thousands of startups and almost always turns out to be a non-issue.
Cool, thanks for the info Thomas. I honestly have never looked into YC at all, so I'm definitely not aware of how these things work. I was posting from an honest curiosity.
Nope. That allows them to control the market. Find new ideas that could threaten the big 5 status quo. Fund them. Get them bought by one of the big five (making a good amount of money off that). Let the big five shutdown the competition.
Conflict of interest only applies if you think they’re concerned with the public good. There is no assumption of that here. Every man for himself.
I'm a provider, I'd like to list my service. How do I do that? Since there is nothing obvious, then how do you know the providers they list are a valid representation of what is out there? Maybe there are other providers who don't want to pay to be listed? But how do you even pay?
I have read many of your comments on GPU-related posts. Occasionally, when I see your comments, I visit your website to see if I can learn anything about your business beyond what you are posting on HN. To this day, hotaisle.xyz contains nothing but a logo.
I don’t get it.
How can you expect a website to list your service when your own service’s website contains zero information? Why would you pay to list your service, when there is no information available about the service you provide? Am I looking in the wrong place? So confused…
To be fair, there is also a contact email at the bottom of the page.
> How can you expect a website to list your service when your own service’s website contains zero information?
We are the first and only (for now) verified MI300x provider on gpulist. In order to get verified, I contacted them directly, they asked for a few bits of information about my business, including my EIN. What I'm offering there, is exactly what I have today.
I know, it is ok. Let me explain a bit. We are starting small, so the website is the last focus right now. I know the general expected culture is to have some splashy page with a typeform on it, but hey... aren't we also a bit tired of that?
In order to even get access to buy these GPUs, you have to go through quite a lot of effort. You can't just buy them off the shelf from BestBuy. They are export controlled and I've agreed to not use them to build bombs. You have to have a valid business and a great story, or they won't even talk to you. Heck, I even had to prove my business was in good standing in Delaware. I'm pointing this out because I will need to know all my customers too. My business isn't something someone just signs up for on a website.
These GPUs are also extremely expensive. Imagine a 350lbs Ferrari. We started with 8 of them (one chassis) because they are super new and it was a proof of concept. Last year, we didn't even know for certain if AMD would double down on AI. This is all we initially raised funding for. As soon as we deployed the compute, we immediately had a customer on them, all without a website. Just word of mouth. By the way, the success of the PoC unlocked our next round of funding, and we are working on a much larger order of MI300x right now.
Don't worry, you'll get a website at some point. That said, these things sell themselves, you either have them or you don't. I've been very transparent and public about what we are up to. Would a website really help here? Maybe. But I've also started other extremely successful businesses originally without websites too. At the end of the day, I'd rather spend investors money on buying more compute, than a pretty website. Once I have some more revenue, I'll funnel that right back into the business and work on marketing/sales more.
If you're curious about anything, feel free to just reach out and ask. I'm not some corporate overlord suit wearing sales guy. I'm an open source tech nerd who's been in the business a long time. 20+ year ASF member, who co-founded Java @ Apache. Happy to answer any questions.
Who says they're even looking? Being able to contact them would tell you right away whether they simply overlooked you or whether you don't yet meet some criteria (which they might actually tell you).
I was wondering, are there any security guarantees for the providers? Assume I have a small GPU cluster at home, if I rent out my GPUs, what sort of access should I accept the renters to have? Only GPU kernels would be sent to machine? Or will they have a limited permission user access on my cluster?
Also instead of having the operatos openning ports in their routers, were there any considers of adding them to a private network for a more seamless experience? (Nebula/headscale and the likes)
I got it from the line about how they trained their robot models - think of Amazon pivoting to aws, the substrate becomes the commodity. Unless you're asking if it makes sense, or what went wrong? Then I don't know. I imagine startups that survive usually pivot from something else?
I've never run a start-up (successful or otherwise :) ) but I've also heard that being able to pivot can be really useful - Flickr is often cited as an example.
Also, "GPU on demand" sounds _a_lot_ easier than "drone-based delivery". Between Seti@Home/Folding@Home/etc, various grid-computing/clustering/orchestration stacks that already exist, etc it seems reasonably doable to implement in a year or so. "drone-based delivery" sounds capital-intensive, sounds like you'll need to spend a lot of time building a professional network of business people who might use the service (so there's a 'cultural friction' between techie founders and business folks, potentially), plus the ever-looming threat of Amazon/etc figuring this out first.
tl;dr: I agree it's weird pivot, and good on the founders for being able to make the change! :)
While most people think it's cool and some think it's scary, drone delivery is ultimately something that people (that is: companies with money) don't need.
Turns out, using drones to deliver is also not that competitive either. Delivery vans are very cost-efficient, and for food delivery / on-demand delivery, drones are not able to carry most orders. So it's not even the regulatory pains that make this difficult, which are unbearable in their own right.
It was a lot of fun to work on and we would have definitely stuck with it if there was any interest. There was none, so we had to admit that to ourselves.
This is a hard pivot, but it's been very stimulating to work on.
Btw thanks for this honest insight. When I read the bit about your pivot, I sort of rolled my eyes. But this is a really nice, sober reflection that actually builds credibility with people like me who might make snap judgements at first.
To add to "capital-intensive": the regulatory framework right now (in the US) for drone delivery is uncertain at best. While most of the rules & regs to enable drone delivery is already finalized & published, a few very important ones (such as BVLOS - beyond visual line-of-sight) have yet been finalized, and thus need one-off waivers from the FAA to allow. And getting those one-off waivers is what eats a lot of time & money.
What's preventing GPU providers from sending wrong results instead of actually running the computation? For example, send the last computed result? Is this something that the renter has to handle by adding their own checks?
In addition to the problem of the renter crashing your machine or reading your password through DMA, of course.
The incentive is huge, if I spend 2 milliseconds sending you your previous results instead of 2 hours running your new computation, I can (pretend to) run way more computations on the same hardware and collect hundreds of time more money.
NO. That's the worst way to do almost anything on the Internet, and should be considered a last-line defense, if nothing else can be done. Here, it can be. See my comment above.
That's my whole question, do they do random audits, or is it the job of customers to double-check their results for possible attack or compute-theft and report.
It seems wrong to call it a "job of customers". It's like you wrote a Bitcoin client which didn't verified hashes of transactions, "trusting" everything. Or like serving a website with login feature supporting only HTTP, not HTTPS. It is a very basic feature of whatever software would connect to such services.
I don't know how developed is it now, I'm not associated with the startup shown in any way. It's mainly a question to them. However, in terms of wider industry, in general distributed high-performance GPU(-like) computing "for everyone" is in its infancy. 99% of what was already done up to this point was targeted to people who would both buy and supply power "in bulk", not "in retail". Perhaps with a little exception of several excellent projects like Folding@Home and other @home's.
Run 1/10,000 - 1/100,000 of computations locally, and also send them as tasks to be send remotely. If compare yields difference, repeat both. After, say, 10 tries, blacklist the provider. Of course it will take a lot more nuances to implement that, but that's the general idea. It's a no-brainer.
At the moment, we manually verify operators and are currently onboarding some tier-4 operators. Down the line, we'll have a 2-tier system where you can choose whether you want a verified machine or not. From the operator's perspective, everything runs inside Docker, configured with security best-practices.
I've always understood that containers are not proper sandboxes and shouldn't be used for containing untrusted code, no matter the best practices used. Has this changed in recent years? Do you have documentation for what sorts of best practices you're using and why they are sufficient for executing untrusted code?
You are correct from my knowledge. I would expect that if the container is set to not run as root you might be able to enforce fine meaningful security but I’d still run it in a VM if feasible.
Having done a little bit of work in the area[1], I think you should publicly document exactly what those best-practices are. Are the workloads running in a networkless container? Do you limit IO? Do you limit disk usage? Answering these in detail would help you gain customer trust on both sides.
probably very basic... so don't run it on anything that has your own data on it (if you're an AI startup, definitely don't run it on your research cluster).
I think they mean don't lease out your research team's GPUs and allow random people to run untrusted code on your cluster, lest they figure out a way to break out of any sandboxing the software has in place and get loose in your network. The company's current answer to that concern is "everything runs inside Docker, configured with security best-practices", which is less than inspiring.
About half of that will go to electricity here in the UK (23.72p per kWh). After costs that's roughly $145 a month profit. Might be more worth it if you have rooftop solar and have free electricity during the day.
This is an interesting idea but it would be cool if it were more granular, like I pay while my payload is executing only, and an API abstracts away which GPUs I'm running on, the execution environment, etc and just let's me push in code and get out data when it's done. Maybe that's what this is, not sure. I hit an account sign up before I could figure it out.
Tried deploying an instance, didn't work, no option to let me delete my billing info/ account. Identical services with better features and more availability exist
I have a cluster on RunPod and it's great, but there's definitely some opportunity in the space, for example if you can focus on transparency (utilization rates, more info about end user and their purpose, reporting) that would be helpful.
I think RunPod and Vast have most the market share but t's still early to the game
> "if you’re an AI company with idle compute (hopefully not in a Stability AI way)"
What are they referring to? Stability AI over provisioned hardware or something? Sorry not up to date with what's happening here, have developed a blindspot for AI hype.
It was a fairly unique architecture that had some pros and cons. In reality what made it a winner was the steep subsidy by Sony (units were sold at a loss, making the assumption you would buy a few games) and the availability (later removed) of an official Linux distribution.
It was basically an all-purpose vector computing monster focused on SIMD. You could use it for physics simulations, animations, tesselation, etc. Basically everything you'd use a compute shader for nowadays.
That's why emulators need AVX512 support to match the PS3. It was incredibly powerful.
Obviously, in that era's single-threaded world no engine could make use of that functionality and few knew how to program for it. It was ahead of its time, by quite a while.
Remember around that time when the industry said that OpenCL would allow write-once run-everywhere compute code for a booming industry of diverse and competitive compute devices? I fell for that scam for a couple of years, before very fortunately getting a different job for long enough to watch the collapse from a safe distance.
It'd be nice if there was any documentation on how the sharing side of things worked pre-account creation. Is it a VM image you distribute? Docker container?
Seems like you have to rent for specific dates… that doesn’t feel very on-demand to me. It also means you have to know how much time you need. I’d love to be able to run a job with a remote gpu as if it were local, spinning up instances as needed. Then if things crash and I need some hours to figure it out, let me downsize it all.
I don’t get “final rate is usually lower”. Why is it lower, and by how much? And since it’s “usually”, what about the unusual case? Equal to the sticker price, or higher? But since you use “<” neither should happen? I would expect an exact price tag for a “no-nonsense” pricing scheme. Right now it’s just confusing.
Thanks for the input, you're right, maybe a fixed price is better. The idea currently is that the quoted price is an upper limit on the rate you pay and the final rate depends on what the GPU provider inputs, which will normally result in a lower rate than the upper bound. At the moment we have upper bounds if you want to be included in on-demand input, and also depending on the service you provide (security, network etc.). We determine pricing with partners 1-on-1 a the moment.
I like runpod the most. They offer attached network drives so you can start a GPU instance, stop it, and restart while retaining the data. This way you don't have to sync data to and from external data sources.
If I want to run stable diffusion on other provides I have to install docker image, get data from somewhere, run models, sync data to somewhere so I don't lose it. The process can be tedious and time consuming.
How much do these kind of services pay the GPU owners? Before I sign up and add my nodes I need to know if it will be worth it considering I'm paying the power to run it, depreciation, etc.
Hmm, interesting. At $0.49/hr I get $0.30/hr profit out of my 4090s. It's more like Uber for GPUs. When I have spare capacity on my 4090s maybe I will look into the service.
Pretty neat, but is there any Windows or MacOS support planned? I wouldn't mind renting out my GPU when it's idle, but I don't really want to go through the process of dual booting etc.
Have you looked at Nvidia GeForce NOW? It's like $10/mo for a pretty decent streaming gaming rig. I'm very happy with it - I don't have to deal with Windows and can play AAA games on my Macbook Pro at 60Hz (1080p).
I think you just answered yourself. Some of us like to play games at 4K at 80Hz+, with no subscription fees, no internet bandwidth requirements, no added latency, and ability to mod.
Yeah, but that has nothing to do with the context of the question. Someone is specifically asking about accessing a GPU over the internet for video games.
Nowadays it's rare that a Windows-only game that I want to run doesn't run flawlessly on my Linux machine (through Proton/Wine). I wouldn't recommend going outside of Steam though unless you're willing to do some troubleshooting.
Ah yes, apologies that's an important caveat I forgot to include.
Games that use anti-cheat are a mess on Linux. I don't play any of those games, and if you do then you're likely to run into some trouble with Linux-only.
I remember gamedevs wanting a company like this to test video game drivers and shader code on a slew of different cards and vendors but this seems clearly set up for LLM work.
Make sure your users can run profiling tools! Specifically Nsight Compute needs containers with some elevated privileges. That would get you an easy win over vast.ai .
This is really great! I hope you succeed. At Phind, we’re customers of Voltage Park and SF Compute for something similar. Let me know if you’d like to chat!
> French startup Qarnot (...) manufactures heaters and boilers with a special trick — they pack computers as computers tend to generate a lot of heat. Qarnot then lets companies leverage that computing power by running tasks on those unusual servers.
I am interested in the business logic of paying to hosts. Say someone had netted sub 10 dollars for hosting their GPU on the platform. How do you make that payment? Business have to pay fixed plus variable transaction fees. So the small payments to a large pool of users should be quite involved specially with international payments.
Since the dawn of torrenting (and to some extent the the Silicon Valley TV show) many folks tried to launch a P2P resource pool platform. The challenge is that small amount of money being distributed to a large pool is challenging. Then there is the issue of stability. Do you penalize hoster who fail to provide a stable service?
Hosters must have "trust" in the system and platform can only afford to pay out only when the payable amount to hosters reach a certain threshold. Like you can't withdraw unless you have about $100 charged invoices.
And that is where I see the concept of crypto tokens appear because trust as a short term mechanism can be consoled via crypto tokens.
Engineering an service to take advantage of idle resources is a non-trivial task specially if the service is hooked up to doing something dedicated. For the hosters energy is not cheap, maintenance is a hassle and guranteeing stability is a pain.
I would love to learn how you would address these issues.
1. Privacy. An attacker can set up a GPU honey pot and and sell the data they got.
2. Fake GPU computation. An attacker can fake GPU and send back dummy data sometimes to reduce the computation.
3. Corrupt GPU. Practically same with 2. But not malicious intent. It's just the faulty GPU.
The first issue could be addressed by just never giving a seller enough data from a buyer to abuse, but that does require a critical mass of both buyers and sellers before you could distribute and dilute a single customer's workload enough. The company could bootstrap this with renting their own GPUs and subsidizing sellers at first though.
I can vividly remember we were by no means disclosing private info back in ‘94 only to see a world of influencers in 2024 sharing their very personal guts for profits.
At the end of the days, everyone wants and hopes to be an idol or famous.
agreed it IS a Big Brother of a sorts, where everyone participates to certain degree. but then written forums have been a thing before BigBrother and perhaps don't have this voyeurism added in the equation.
https://azure.microsoft.com/en-us/blog/azure-confidential-co...
Essentially you're trying to provide a way to prove that the code running on the machine is what you instructed. This is achieved by a series of hardware attestations that measure and check the code to make sure it's what you requested. Generally this means encrypted ram at a minimum, and checks/balances that give you confidence this is the case (you have to trust someone, eg: Intel)
Is it perfect, probably not, but it's a lot better than just running VMs with unencrypted memory that any operator can jump into.
To my understanding most GPU workloads are not run in this way currently, and the operator can see/manipulate everything executed
I've used vast.ai (similar "Airbnb for GPUs" pitch) for years to spin up cheap test machines with GPUs you can't really find in the cloud (and especially consumer-grade GPUs like 4090s). Any insight into how this is different/better?
Also, don't know if vast.ai does this, but with us you can have 6 user sessions on your machine if you have six GPUs, so granular utilization is possible.
What would make you better than vast is extremely easy spot leasing and job prioritization.
I want to be able to have one of our training jobs finish, and then have the capacity immediately transition to a lease. With vast, we are renting in week long blocks.
(I'm the founder of vast btw - contact us for help on setting this up and/or any feedback on making it an easier/better process)
vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way. Someone on HN wrote a script to run a test docker image on every machine and auto-tagged the machines' quality using their API, which is what I'd do if I was going to use vast seriously for compute.
> vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way.
Wouldn't gpudeploy have exactly the same problem? How is it mitigated with gpudeploy?
I suspect it would be trivial for Vast or GPUDeploy to spin up a benchmarking job before allowing sales on that machine. I'm not an expert on PCIe lanes, but I would think the performance issues would be visible via bandwidth or latency on the lanes.
It kind of makes sense to me, though. If I were looking for absolute reliability and was willing to pay for it, I'd just go to one of the many GPU cloud vendors. Likewise, I suspect anyone willing to really work on getting good performance would rather be a real provider or sub-provider than being part of this nebulous C2C GPU cloud.
Dropping a note that I've found https://akash.network/ ~ https://akash.network/gpus/ to be impressive, as typically with crypto projects it's all scams, however in this instance there's demand and legit usage. https://stats.akash.network/
Something to consider!
> One good trick for describing a project concisely is to explain it as a variant of something the audience already knows. It’s like Wikipedia, but within an organization. It’s like an answering service, but for email. It’s eBay for jobs. This form of description is wonderfully efficient.
Seems the risk of this is a loose simile.
Edit: Though, thinking about it, Airbnb for GPUs is fairly accurate for this model of a marketplace where people let out their GPUs for others to rent.
Minimum rent 1 day? $100 cleaning fee? No quality control? Spy cameras in the bedrooms?
Airbnb implies a two way market place. Much more interesting prospect for those having idle GPUs .
Not sure if electricity costs justify renting out and what pricing looks like, those details apart.
I wonder why y-combinator is stuffing their investments with multiples of these similar companies... https://www.shadeform.ai/ is another one.
A few quick comments:
Reading the source of their install script:
https://gpudeploy-public.s3.us-west-1.amazonaws.com/join_clu...
It doesn't start off with set -e, which could result in an incomplete install, yet appear to finish.
It also installs some binary "instance-server"... who knows what it does... would you trust this on your server on your network?
It is nvidia specific... sadly, don't expect AMD gpus anytime soon.
Feels like a MVP, let's see how this grows over time.
Looks like you can get a MI210 on ebay for $6500, with 30 day warranty. heh.
Oh, there are a bunch of H100's there...
I would expect some sort of conversation like: "Oh hey, we have another company, in our portfolio, that is far ahead of you, doing exactly the same thing. Maybe you should do something else?"
But of course, this is AI... plenty of space for gpu marketplaces.
As for "other company in portfolio", that's unavoidable when funding thousands of startups and almost always turns out to be a non-issue.
I'm curious now, if you can say. Was advice offered in this case? If so, what was it?
What's YC going to do? They have a tiny stake in your firm. That's the point; that's what "founder-friendliness" means. They're not your board.
Conflict of interest only applies if you think they’re concerned with the public good. There is no assumption of that here. Every man for himself.
This seems much more in depth, and a true service, but for those that just want to compare prices check out gpumonger.
I don’t get it.
How can you expect a website to list your service when your own service’s website contains zero information? Why would you pay to list your service, when there is no information available about the service you provide? Am I looking in the wrong place? So confused…
To be fair, there is also a contact email at the bottom of the page.
> How can you expect a website to list your service when your own service’s website contains zero information?
We are the first and only (for now) verified MI300x provider on gpulist. In order to get verified, I contacted them directly, they asked for a few bits of information about my business, including my EIN. What I'm offering there, is exactly what I have today.
https://gpulist.ai/detail/3c18f8a
> So confused…
I know, it is ok. Let me explain a bit. We are starting small, so the website is the last focus right now. I know the general expected culture is to have some splashy page with a typeform on it, but hey... aren't we also a bit tired of that?
In order to even get access to buy these GPUs, you have to go through quite a lot of effort. You can't just buy them off the shelf from BestBuy. They are export controlled and I've agreed to not use them to build bombs. You have to have a valid business and a great story, or they won't even talk to you. Heck, I even had to prove my business was in good standing in Delaware. I'm pointing this out because I will need to know all my customers too. My business isn't something someone just signs up for on a website.
These GPUs are also extremely expensive. Imagine a 350lbs Ferrari. We started with 8 of them (one chassis) because they are super new and it was a proof of concept. Last year, we didn't even know for certain if AMD would double down on AI. This is all we initially raised funding for. As soon as we deployed the compute, we immediately had a customer on them, all without a website. Just word of mouth. By the way, the success of the PoC unlocked our next round of funding, and we are working on a much larger order of MI300x right now.
Don't worry, you'll get a website at some point. That said, these things sell themselves, you either have them or you don't. I've been very transparent and public about what we are up to. Would a website really help here? Maybe. But I've also started other extremely successful businesses originally without websites too. At the end of the day, I'd rather spend investors money on buying more compute, than a pretty website. Once I have some more revenue, I'll funnel that right back into the business and work on marketing/sales more.
If you're curious about anything, feel free to just reach out and ask. I'm not some corporate overlord suit wearing sales guy. I'm an open source tech nerd who's been in the business a long time. 20+ year ASF member, who co-founded Java @ Apache. Happy to answer any questions.
The lack of "contact us" was pure laziness at the time.
Sure, assuming the site will continue... my email is in my profile.
I was wondering, are there any security guarantees for the providers? Assume I have a small GPU cluster at home, if I rent out my GPUs, what sort of access should I accept the renters to have? Only GPU kernels would be sent to machine? Or will they have a limited permission user access on my cluster?
Also instead of having the operatos openning ports in their routers, were there any considers of adding them to a private network for a more seamless experience? (Nebula/headscale and the likes)
Also, "GPU on demand" sounds _a_lot_ easier than "drone-based delivery". Between Seti@Home/Folding@Home/etc, various grid-computing/clustering/orchestration stacks that already exist, etc it seems reasonably doable to implement in a year or so. "drone-based delivery" sounds capital-intensive, sounds like you'll need to spend a lot of time building a professional network of business people who might use the service (so there's a 'cultural friction' between techie founders and business folks, potentially), plus the ever-looming threat of Amazon/etc figuring this out first.
tl;dr: I agree it's weird pivot, and good on the founders for being able to make the change! :)
Turns out, using drones to deliver is also not that competitive either. Delivery vans are very cost-efficient, and for food delivery / on-demand delivery, drones are not able to carry most orders. So it's not even the regulatory pains that make this difficult, which are unbearable in their own right.
It was a lot of fun to work on and we would have definitely stuck with it if there was any interest. There was none, so we had to admit that to ourselves.
This is a hard pivot, but it's been very stimulating to work on.
I hope you guys find traction.
In addition to the problem of the renter crashing your machine or reading your password through DMA, of course.
[1]: https://containerssh.io/v0.5/reference/docker/#securing-dock...
Containers are not, and will never be, a secure isolation boundary.
...what‘s the threat, actually? GPU time sellers stealing your secret sauce?
https://news.ycombinator.com/item?id=40261591
I think RunPod and Vast have most the market share but t's still early to the game
What are they referring to? Stability AI over provisioned hardware or something? Sorry not up to date with what's happening here, have developed a blindspot for AI hype.
https://en.m.wikipedia.org/wiki/Folding@home
Was the cell processor in the PS3 really that efficient for this purpose?
That's why emulators need AVX512 support to match the PS3. It was incredibly powerful.
Obviously, in that era's single-threaded world no engine could make use of that functionality and few knew how to program for it. It was ahead of its time, by quite a while.
How times have and haven't changed.
If I pause a GPU instance will I be able to resume it on the same host so I can use the data generated before?
I've found that usually this involves something extremely simple and easy to understand, with clear up front numbers.
If I want to run stable diffusion on other provides I have to install docker image, get data from somewhere, run models, sync data to somewhere so I don't lose it. The process can be tedious and time consuming.
What does that mean? I’m likely missing some context, could anyone explain?
[0]: https://www.theregister.com/2024/04/03/stability_ai_bills/
The difference is that ours sell GPU compute hours, not machines.
I think you just answered yourself. Some of us like to play games at 4K at 80Hz+, with no subscription fees, no internet bandwidth requirements, no added latency, and ability to mod.
Games that use anti-cheat are a mess on Linux. I don't play any of those games, and if you do then you're likely to run into some trouble with Linux-only.
Short of dedicated hardware (Xbox/ps) I’m not sure what else could be done.
> French startup Qarnot (...) manufactures heaters and boilers with a special trick — they pack computers as computers tend to generate a lot of heat. Qarnot then lets companies leverage that computing power by running tasks on those unusual servers.
Since the dawn of torrenting (and to some extent the the Silicon Valley TV show) many folks tried to launch a P2P resource pool platform. The challenge is that small amount of money being distributed to a large pool is challenging. Then there is the issue of stability. Do you penalize hoster who fail to provide a stable service?
Hosters must have "trust" in the system and platform can only afford to pay out only when the payable amount to hosters reach a certain threshold. Like you can't withdraw unless you have about $100 charged invoices.
And that is where I see the concept of crypto tokens appear because trust as a short term mechanism can be consoled via crypto tokens.
Engineering an service to take advantage of idle resources is a non-trivial task specially if the service is hooked up to doing something dedicated. For the hosters energy is not cheap, maintenance is a hassle and guranteeing stability is a pain.
I would love to learn how you would address these issues.
I've been a happy user of vast.ai for some time now.