Maybe I'm being dumb, but I don't understand what the innovation is here.
I get that they're using liquid coolant at higher than usual temperatures, but why couldn't they do that before? Most of the comparison in the article is for air cooled datacenters but what about other liquid cooled ones?
Surely in all the previous datacenters that have been designed there has been someone doing the math and determining what temperature things need to run at, how much energy it will use, how much heat it all will produce, etc.
edit: just saw this:
>Previous liquid-cooled servers were hybrid: GPUs and CPUs got cold plates, but the rest of the system stayed air-cooled, with finned heat sinks designed to shed heat into moving air. In a fully liquid-cooled server, the cooling for these components needed to be completely redesigned to use liquid.
When my grandpa retired from Monsanto chemical back in the 90s, I helped him clean out his office and got a tour of a bunch of stuff.
He showed me their Cray, which had its own dedicated computer room, and they set it up with the coolant pump and fountain unit right in the middle in front of a glass wall facing the hallway so everyone could gawk at it.
Haven't AMD CPUs been targeting a 95°C limit for 5+ years already? I'd have guessed servers could do 60°C without degrading a whole lot before switching to more power efficient hardware is available.
Bad quality of water clogging the pipes integrated onto the PCBs (thus requiring to replace the PCBs) was said to be what were killing those few USSR Elbrus supercomputer installations.
> Surely in all the previous datacenters that have been designed there has been someone doing the math and determining what temperature things need to run at, how much energy it will use, how much heat it all will produce, etc.
It seemed like a pretty big deal ~ 2011 when big companies were running their (air cooled) datacenters closer to 95F (35C) vs the traditional 72F (22C). So jumping up a little more is maybe not super exciting, but it's still innovation.
And I think the answer to the "doing the math" question is, until you've actually collected the data, "what math?" Until someone actually puts a bunch of six-figure value hardware through its paces, pushes the previous limits, and sees what that does to its lifespan, there's nothing to meaningfully calculate.
And the fact that their system doesn't dump water. I think that is actually perhaps the bigger deal. Datacenters have been getting a lot of heat (pun intended) for using significant fresh water at the expense of local municipalities.
Closed-loop water cooling chips is nothing new. There are two separate water systems that often get conflated*. The loop warms up the water, which is recycled but first needs to be cooled externally somehow. Normally they use evaporative cooling towers that do use water, or chillers that don't use water but use more energy. But they're claiming they can get that water loop so much hotter than the outdoor environment that active cooling isn't needed. They attribute this to improving the chip-to-water interaction.
Even air-cooled datacenters work somewhat the same way, but instead of water to chips, it's air. The air goes into hot aisles then exchanges heat with water, after which, see above.
* Other datacenter marketing materials talk about how they have a "closed loop system that uses no water" and they do still use water in the evap towers. I was half expecting this article to be that again, glad it wasn't.
It was available, there are plenty of water-cooled datacenters already, or water-cooled racks fitted into existing sites. Nvidia improved the cooling efficiency though.
You have to design your hardware to tolerate being run in consistently hotter conditions. There's a tradeoff between cooling cost and failure rate / capex.
Doesn't look like they made the hardware more tolerant of temperature, rather they made it remove waste heat more quickly.
"NVIDIA’s thermal engineering team reworked how those components handle heat, designing cooling loops that simplify how liquid is routed to multiple high-power chips on the board using a single inlet and outlet, resulting in a cleaner tray-level cooling architecture"
Core temp though. Ambient temp is a different story, and also depends on air vs water. In fact the article suggests the difference is getting the water more directly onto the chips, no mention of running at a higher core temp.
Temperature ratings are the allowed ambient temperature. The actual silicon will inevitably operate somewhat higher, because coolers are just moving heat down a temperature gradient.
AMD CPUs basically all boost up to 90°C as a relatively normal operating temperature as long as the power (and some other factors) allow it to. I assume AMDs and NVs GPUs do to, but I play mostly CPU bound games so I see mine just sitting at ~60°C under load.
Speculating here - “effectively” cooling the CPU and GPU materially using this technique at datacenter scale may have never been done. Those things than run hot, easily crossing 100C. So the loop is doing a lot of work to keep them stable at 55C.
The innovation may be in the speed or volume flow of the coolant through different parts of the data centre to regulate the temperature. And of course, redesigning every component to be compatible with this fan-less design.
I think it’s only possibly because NVIDIA is much more vertically integrated than ever before.
This opens up an interesting synergy: district heating. 45C is low but not unworkable for a district heating loop, and a data center might be able to make a nice pitch to a community if the data center offers to provide heat to a district heating system for free. This brings the value to the local community of a nearby datacenter up from near zero to potentially a few million dollars per year.
Summer is still an issue, but fun solutions are possible. With the right geology, I think it’s possible to heat an underground volume in the summer and recapture (some of) that heat in the winter. In many, many climates, annual heating costs are far higher than cooling costs, at least if people aren’t stupid with skylights. [0]
[0] As a back-of-the-envelope heuristic, heating or cooling load due to conduction and air exchange is proportional to the difference between indoor and outdoor temperature. Outdoor temperatures of -10F to 30F are not unusual in the winter and are 40-80F away from an indoor temp of 70F. But outdoor temperatures in these climates rarely exceed 95F and are mostly lower in the summer, so that’s 15-25F of cooling. And heat pumps are more efficient at smaller temperature differences.
Microsoft's already building data centers hooked up to district heating (Espoo and Kirkkonummi, Finland). Heatpumps are amazing.
(Seasonal heat storage is also a thing, Espoo's neighbours have tens of GWh of storage, with a new 90 GWh cavern in the works. Not sure if the systems are interlinked.)
Yes, but the heat will still likely need boosting by about a further 10 degrees either at the source or end user.
DC inlet is 45°C, outlet is 55°C assuming a 10°C ΔT. By the time that's travelled 500m–1km through pipework you've lost a few degrees, so you're arriving at the HIU at maybe 50–52°C. The home radiator circuit then takes that down by around say 12°C, returning ~38°C. Factor in pipe losses on the return leg and you're back at the data centre with maybe 35°C inlet rather than 45°C — meaning the DC output is now only 45°C rather than 55°C, and the whole system gradually degrades each cycle. You could address this by mixing some hot output back into the return to keep the DC inlet stable at 45°C, but eh.
Do you live near a datacenter? Property value goes down, constant humming.. the way we heat up the earth right now, i don't think you have to worry about heating
I grew up in Northern Virginia, (of AWS US-East-1, MAE-East, and Equinix fame), where there are more data centers than anywhere else in the world, and I never heard organized opposition to them until the last couple of years. They were mainly viewed as a way for Loudoun County to build their industrial tax base without the downsides of having industrial workers, and allowed them to consistently lower property taxes while having excellent schools. Data centers are unsightly and use electricity and water, but so does literally any kind of industrial facility. They are also pretty quiet, if you exclude the ones using on site gas turbines for electricity.
Property values have consistently gone up in that region for decades, and are up to $6 million an acre if there's enough contiguous land to put another data center on.
Many of the people complaining about datacenters would also complain about literally any kind of development.
I’ve been to datacenters, but not the huuuge ones people seem to talk about in the context of AI. They are noisy inside (due to air cooling, which is largely avoided by the tech in the OP), but they’re entirely unremarkable outside compared to any other commercial or industrial building. Computers are not inherently loud, nor is power conversion.
Power plants are all over, even in populated areas. They’re not so bad either (except perhaps coal).
There is no fundamental reason that datacenters need to be especially unpleasant to their neighbors.
On-site natural gas turbines at a handful of DCs are genuinely loud. In general I agree that DCs are mostly fine neighbors, but maybe louder power plants aren't.
Yeah this is it. You can make really nice datacenters that are basically quiet and environmentally perfect.
This was never in dispute.
But that is not how corporations roll. They want the cheapest shit that they can get away with. No regulations only corruption.
Which is middle of nowhere America.
they just want data centers now. most companies would rather use solar, but they can't on short timelines due to land use regulations (and import tariffs)
It depends a lot on things like geology and some people are a lot more sensitive. It is really an issue.
I don't have any datacenters near me but I can hear some heavy hums from the washing machine 3 floors up when it put my head on my pillow, for some reason it just propagates through the building physically. When I walk around I don't hear it. Datacenter noise can be the same.
IMO they should be put away from habitation, there's no reason for them to be near there anyway
> I don't have any datacenters near me but I can hear some heavy hums from the washing machine 3 floors up when it put my head on my pillow, for some reason it just propagates through the building physically. When I walk around I don't hear it. Datacenter noise can be the same.
Right. Vibration propagates through solid (and liquid) materials.
But this can all be measured and controlled, and there's nothing special about datacenters. A building that is hundreds of feet away will couple to your pillow much less strongly than a washing machine in your building. And the washing machine often has a wildly unbalanced load and minimal decoupling between itself and the floor, whereas a big fan in a datacenter or other industrial building ought to be balanced and also ought to be installed on decoupling mounts.
If datacenter operators (cough xAI) are being lazy about properly selecting, installing and maintaining equipment, then you can have a problem. Otherwise you have a much smaller problem.
> IMO they should be put away from habitation, there's no reason for them to be near there anyway
I agree, but that's a hard problem (in the US anyway). Unless you're plopping data centers in the middle of national parks, or in the middle of the desert where water is going to be a problem, you are nearly always going to be within some small mile radius of civilization. Plus the cost of trenching new fiber out in the middle of nowhere.
The same reasons humans want to concentrate in a particular area (access to jobs, infrastructure) are the same things that data centers need.
Once water-less cooling tech like this improves then yeah, just plopping them in the middle of the unpopulated desert becomes viable (assuming you can get the fiber out there and latency is tolerable), so long as they generate their own power.
The climate requirements to run at this hotter temperature still probably means it'll require more active cooling in the desert during daytime /summers. Assuming we're talking about hotter desert environments like US southwest. That might make your proposal not as economical.
Imo we should just solve the problems with data centers being near cities. Manage/regulate the noise and any waste (heat included, it shouldn't drastically impact the neighbors) and make them pay for any utility capacity/reliability upgrades needed. If this article is right and water usage can be nearly eliminated then it seems like the rest should be solvable? Especially if we can take the extra heat and use it for local power or heating needs.
You might (but probably not) be able to do district heating with this, but electricity generation is not going to be efficient. Heat is a very low grade form of energy, and you need a large differential to drive a turbine efficiently.
If you cycle between 45 C and 55 C water temperature (as mentioned by the press release), you are only getting a 10 C delta. That isn't even enough for district heating, probably not even with heat pumps.
Now if you have something like a steel foundry, that have much hotter cooling water, you can absolutely use the heat for district heating, but even then it usually isn't enough for cost effective electricity generation. Even when it is waste heat, as the equipment to handle it still costs money and requires maintenance.
> If you cycle between 45 C and 55 C water temperature (as mentioned by the press release), you are only getting a 10 C delta.
You are calculating the wrong delta T. To heat a space, you need your working fluid to be warmer that the space you’re heating by an appropriate amount.
55°C is certainly on the cool side to heat a building, but it’s entirely workable with a high-area, highish-thermal-conductivity system. Here’s an actual chart:
This study doesn't factor in droughts, floods, crop death (and starvation), and other non-direct effects. It also doesn't consider wet-bulb events, because it's looking at average ambient temperatures.
I don't think this is climate change propaganda, but your application of this study by evoking it in a discussion about climate change feels like it.
Noise is a design choice and could likely be legislated away. Reject heat is different than heating from greenhouse gas effects that are “heating the planet”.
> No one bats an eye when an air conditioner runs.
I find their noise pretty obnoxious. Out of respect for my neighbours, when I get around to installing one, I'll be getting the absolute quietest model available.
Do you? I live at 4th and Brannan and there was one just off 3rd and Brannan in San Francisco. It was shut down when hosting.com sold it off but I didn't notice it while walking by then and I don't notice it while walking by now.
My GPUs at Hurricane Electric in Fremont are also completely unnoticeable outside the building. Inside, when I'm working at the cabinets it's obviously deafening. Outside you wouldn't even know. Realistically, the predominant sounds at my home are from the traffic on the Bay Bridge so it's nice when there's congestion because it's quiet.
Honestly, I wish there were more urban datacenters. It's getting quite annoying having to make a 1 hr trek to Fremont every time I want to rack a new server.
Imagine if one of the amenities in a high-end residential building could be a cabinet haha! That would be amazing. Doesn't make economic sense, but I'd love it. Would love to explain that to my wife. "I know that one has the pool, but we can rack my servers in this one if we live there".
They almost certainly need fans on the outside of the building to cool the 55C water back down to 45C. But correct, no fans on the servers themselves or even in the building. Except perhaps for the humans, so they can stand to work inside the building, when needed.
Some systems use liquid cooling for the GPU and CPU, but air cooling for the PSU, RAM and SSDs.
With that said, by the standards of industrial sites data centres are quiet, low traffic and smell free. An industrial area that can’t build a data centre certainly can’t build a steelworks or oil refinery or leather tannery.
If the outdoor temperature is cool enough (maybe 30C?), you just pipe the liquid outside through a large enough loop or heat exchanger to get it back down to under 45C. Even better if you can put the loop in a lake and dump the heat there (maybe not better from an ecological POV though). The pumps moving all that liquid becomes the noisiest component.
Coldest month average temperature where I live is around -7C, with peaks of -35C. Climate change is not going to increase that average, more like decrease. Typically, of course, electricity price is the highest during that month too.
>Do you live near a datacenter? Property value goes down, constant humming
I don't live next to one but I'd take constant humming over the constant stop/go traffic noise, honking, squeaky brakes, slamming doors and revving engines I now have on my western side of the apartment, thanks to the unemployment office the city opened on my street not too long ago.
So how come constant humming is somehow an illegal nuisance, but we've been expected to put up with the much more annoying urban traffic noise for decades just fine?
My parents apartment have constant humming anyway thanks to the HVAC system on the roof of the nearby supermarket and white/brown noise is far more tolerable and easy to tune out than traffic noises.
> we've been expected to put up with the much more annoying urban traffic noise for decades just fine?
For one, there tends to be little traffic at night when most people want quiet in order to sleep. Driving is also something (nearly) everyone does and benefits directly from, so negative externalities are easier to accept. It is much harder to accept a new source of noise near your home you haven't asked for and don't directly benefit from.
> Driving is also something (nearly) everyone does and benefits directly from, so negative externalities are easier to accept.
This reads a little too close to driving being an inherently good thing or some sort of objective requirement, but it's only that way in certain urban places because the built environment makes it as arduous as possible to do those things without.
Something that pisses me off about many urban places that don't even otherwise require people to drive, is that many who do use their cars the most often have their neighborhoods protected from the noise they contribute to everywhere else. This whole thing of putting apartments only where there's already the most disgusting car-infested thoroughfares; "sorry, can't have an apartment one street in off the main drag, that's only for bungalows! Don't like it? Get richer. Excuse me while I drive through your bedroom and park for free in front."
How dare those nasty, dirty, unemployed live their lives under likely desperate circumstance. They are so much worse than corrupt oligarchs pumping and dumping their way into the greed hall of fame.
> In favorable climates, NVIDIA’s 45-degree liquid-cooling architecture ....
What's a favorable climate, apart from, obviously, Greenland? The piece is a little light on details on the correlation between outside temperatures and efficiency & cost. It'd be nice to see even a broad-strokes discussion of that.
The university where i studied uses high temperature cooling since a few years. The weather on Germany ranges to quite high temperatures, but according to the tech stuff they only need active (as in AC) cooling for the higher end of the 30 degrees. The technology is quite fascinating.
Speaking from a UK perspective global warming is now noticeable, hot days are hotter and there's more likely to be a heat wave, and that's changed in the last decade.
I assume Germany is the same, many years ago really is different to today.
Yeah, this is part of the issue to be honest. You'd need outdoor air to be below ~37°C to guarantee 45°C water outlet temperature. In most locations you still need cooling towers or compressors some of the time, so you still have to build all the infrastructure that comes with them; though reducing their use is still great, saving serious amounts of water or energy.
For e.g you might think of the outskirts of London as fairly moderate, but this week it's been hot enough that supplemental cooling would likely have been needed at points. For a data centre here you'd typically design the cooling system to cope with outdoor temps in excess of 40°C, which is not a conservative number anymore.
Also, while Nvidia might be happy with you supplying water at 45°C I suspect you will get better longevity of the hardware at lower temps like say 35°C. GPUs are expensive, and extending longevity may well be 'worth' a bit more water or energy to you. In practice you are also likely to have air cooled systems that sit 'beside' the AI compute like storage severs, any extra CPU compute and network switches. So you are likely to need a separate room and cooling system for that. Great progress though.
The NASA Ames Research Center Modular Supercomputing Facility is highly efficient, both in terms of electricity and water use. The facility isn't air conditioned. The chips are water cooled, and the inlet water temperature is pretty high I believe - I think it's 90 degrees Fahrenheit.
I am pretty much doing the same but running the coolant at 40 deg C instead of 45 as my pumps are rated for 45 C max temp. Here is bit more about my setup https://sabareesh.com/posts/blackwell-waterblock/
“That unlocks something beyond energy savings: the possibility of eliminating water consumption entirely.”
They’ve made this claim numerous times in the article and I really don’t understand it. The building has tons of water being recirculated through it. That water came from somewhere in the surrounding natural world. How is that 0 water consumption?
As someone else noted - it's effectively zero compared to the water evaporated in evaporative chillers.
But it's still misleading. The major source of water use in datacenters, by _far_, is the water used in power generation. This improves PUE, which reduces power draw, but the savings are almost certainly under about 20% given that many modern datacenters already operate at a PUE of under 1.2. So if you're running on coal or gas, you're still consuming quite a bit of water indirectly.
Now that said again - the water consumption part of this equation is generally overhyped. The power draw is the problem, as are the really bad temporary hacks to the power problem (e.g., what x.ai is doing with "temporary" gas turbines).
After filling the tanks once, subsequent draw would be zero. Most data centers currently use evaporative cooling - where they pull tons of water in, and then... Let it evaporate.
Imagine a residential building that reclaims and reprocesses and purifies 100% of all the water it uses. This would be dramatically more difficult and better than the status quo, and would be called 'net zero' by any sensible accounting method.
---
Obviously, evaporative cooling is net-zero water use when accounted across the entire globe (the water falls as rain, somewhere, eventually), but it is net-negative for a local community.
>Previous liquid-cooled servers were hybrid: GPUs and CPUs got cold plates, but the rest of the system stayed air-cooled, with finned heat sinks designed to shed heat into moving air. In a fully liquid-cooled server, the cooling for these components needed to be completely redesigned to use liquid.
Does that mean the whole server board is running in liquid?
If true, how do they do maintenance? Replacing parts must be extremely difficult.
The article implied (but did not seem to ever state) that they simply added liquid cooled cold plates to all the other components that were previously air cooled.
But it did not sound like they were describing a Cray 2 style liquid immersion cooling system.
I never looked into this, but why would a datacenter consume water for cooling in the first place? Sure, they use some. But just like you fill up the cooling loop in a car, once it is there it just circulates between the heat source and radiators and/or heat exchangers, with perhaps some minimal top off needed (since flexible tubing isn't 100% water proof).
Or are they for some unfathomable reason using evaporative cooling in data centers?
Evaporative cooling consumes less expensive electricity than air conditioning. Electricity is much more expensive than water (for the same cooling load) in most places DCs are located.
It’s usually open loop - closed loop, so closed loop goes through CRACs or liquid cooled equipment manifolds. That heated water circulates through an heat exchanger on the roof that uses open loop cooling to shed the heat to the surrounding environment.
It is more efficient, in terms of cost the datacenter owner pays per unit of heat extracted. Water is cheap in places this is being done, relative to amount of heat vaporization can carry off.
You can certainly argue DCs should pay more for water than other uses, but who gets to decide what is a good vs bad use of water? Pricing in externalities is tricky, and water usage rights are especially complicated. I don't know what a good&fair solution is.
(ab)using fresh water in vast quantities is cheaper.
currently.
and also more energy efficient, because evaporating water away takes a lot of energy with it. you have to raise radiators to a higher temperature to keep up with that, or have much more surface area.
It’s very efficient. The net electrical energy saved using the latent heat of water is 30 to 100+ times greater than the energy required to desalinate or wastewater recycle the same volume of water.
You cannot have evaporative cooling without evaporating some water into the atmosphere.
A closed circuit cooling tower still has water spraying onto the closed loop process water heat exchanger coil and mixing with atmospheric air to evaporate and cool the process water indirectly instead of evaporating and recirculating the process water that doesn’t evaporate directly like in an open-loop cooling tower.
I suppose you could condense the evaporated water somehow by using a chilled umbrella or some other ridiculous contraption above the cooling tower, but why would you do that?
FWIW I sell and run commercial electrical work, primarily to mechanical contractors who are installing boilers, chillers, cooling towers, and pumps. I spend my professional life immersed in this type of equipment.
Not more efficient at heat dissipation, but radiative cooling panels can achieve sub-ambient dissipation in conditions (like high humidity) that evaporative cooling performance declines in, and with no water consumption.
Weird I was daydreaming about why isn't this done the other day (in the context of desert datacentres running on solar anf battery). Glad to see it is a thing.
Heat exchange is used instead of refrigerating the coolant. Makes sense. How do they manage the indoor climate for the humans working there though? Eventually everything will be at 45C in the building, will it not?
The heat exchange between that fluid and the ambient air isn't infinitely fast, if it's low enough they can just run "normal" A/C at low power for the humans. They just need to keep the heat in the fluid until it reaches… well… whatever heat dump there is. (cf. top-level post)
> Heat exchange is used instead of refrigerating the coolant.
There are some systems that pipe refrigerant around the building, but they’re relatively uncommon (VRF or variable refrigerant flow if you want more details).
Glycol and water is cheaper than refrigerant so there’s usually a chilled water loop that passes thru a heat exchanger that interfaces with a chiller (vapor compression refrigeration) to reject the heat from the chilled water loop.
This eliminates the need for evaporative cooling towers.
My understanding is that datacenter water consumption is not really that big of a deal compared to other industries, and it's mostly inflated by people who are looking for a reason to criticize AI/datacenters. If Nvidia is hoping to placate those people, I predict it will not work.
On the other hand: the heat has to go somewhere. So… where? Datacenters already create a warm microclimate in their vicinity, is that getting even worse?
This approach appears to directly reduce energy use (that's what the articles says). The heat would still be going into the local environment, but if there is a reduction in energy use, there should be less of it.
Actual heating due to human energy use is not really a big deal except perhaps locally. Climate change is caused by changing how much heat the earth retains from the sun. Maybe if we stopped using fossil fuels and used immense amounts of nuclear power, we would care about the waste heat. But solar and wind power largely redirect energy flows.
It’s kind of like how brine from desalination is not a global problem for the oceans at all — all that matters is diluting it enough that it doesn’t poison the local ecosystem.
This is kinda debunked / obviously false. It's almost entirely a land use issue: a building will create a heat island. Data centers aren't using enough energy to make a significant difference.
More on it at [0], but it doesn't take anything beyond a basic energy calculation to know that 1GW of energy is not going to have a significant effect "6.2 miles away".
Indeed. If the datacenter uses less total power, it produces less waste heat.
If you manage to use the waste heat to avoid generating heat somewhere else (that the article calls heat recovery) then there’s a further reduction in total heat output.
The temperature is independent of the actual heat flux. Also - a quick search suggests that at best the data center coolers run at COP of little more than 10. The inverse of that is the amount of heat wasted just on cooling. Having a system not relying on heat pumps would only make it better. A back of the envelope calculation based on PC AIOs suggests they would achieve a COP of 20 or more. A scaled up system would be more efficient than that, if not just for wider tubes.
It's a response to the FUD that is water usage in data centers. Data centers don't use a meaningful amount of water when compared to just about any other industrial application, or many recreational ones. Data centers used about 66 billion liters 2023. source: https://escholarship.org/uc/item/32d6m0d1
Golf courses used 2 trillion liters in 2020. We won't even bring up almond production.
I feel that the sad reality is that most blogs in the future will be addressed to AI and not humans, it's gonna be quite rare to read directly something as we will have built-in tools within browser and phone and OS and so-on that always rewrite on-demand based on current expertise, wanted tone and so-on. There is a recent study I believe that demonstrated that AIs digest better articles made by AI, which means that it might be just better to let AI write the articles so others AI have a better accuracy in digesting it (and incorporating it in their training data as well).
The same as technical docs for any codebase, humans will not read them anymore, only AIs which then translate it to human on-demand, it's already happening, I've worked recently with many new frameworks/codebases without even opening the doc (not even the Github page) and solely asking the agent to gather info for me about it.
PS: The reason I feel it will be this way is that it will allow to legitimatize mass data collection indirectly, instead of doing telemetry on page and software level, we will just send all the content automatically to some inference providers (probably provided for free by Google, MS and so-on)
We're there. The recent HN article about the Fender Stratocaster had some content from a Fender press release, which was regurgitating text from a legal aggregation site. It was, overall, bad coverage of the area of copyright on decorative but useful objects.[1]
Watch for cases where content has been through two layers of LLMs. It's not good.
This is also the type of thing that makes space based data centers more viable. I was previously more skeptical on the concept but have come around.
I do think ground based centers will have better economics when they can be built though, and this addresses noise and water complaints which are the big 2 regional complaints.
It seems like lots of bottlenecks are getting solved quickly, except for maybe memory.
How does that change the calculus for space datacenters? There is still no reasons or benefits to having them in space. You still have to rely solely on radiative cooling. That doesn’t solve any of the maintenance problems.
Space datacenters is a really dumb and unrealistic idea Musk is talking about to hype his companies, it’s not meant to actually be done. Anything in space is more expensive and way harder to do, for a datacenter there is no benefit. We aren’t lacking places where to have them on earth
Plans for space data centers should be seen with skepticism. However when they are backed by different parties who have stakes in the game, that's more credible. More than HN crowd for sure.
OK, here's my problems with space data centers. How many of them has China solved?
1. Space is terrible for heat regulation. It's a perfect insulator for everything except radiative cooling, which is the least efficient. Hot things stay hot.
2. Space is full of radiation. Everything has to be radiation hardened, which makes it heavier, more expensive, and, yes, more difficult to cool.
3. Space is far away. Well, farther than a data center on Earth can be. I know China hasn't solved the speed of light.
We put up with it with satellites because it still has some advantages over trying to run cables literally everywhere, but we do, in fact, still use cables laid on the bottoms of the oceans.
So, is physics wrong, or is a country known for making dumb decisions some times making a dumb decision?
I would comfortably bet at least $1000 that this is bs, and it would be more money than China has put in so far. They've got a "vow."
To answer your first question, yes. Even if they were US companies. Remember when each Big Tech player was claiming to build their own blockchain, except centralized which was a contradiction.
I honestly think musk wants them there because they are hard to reach.
I do really think that if large numbers of jobs are indeed going to be displaced by AI, movements will pop up of people attacking datacenters (and honestly I wouldn't blame them even though it won't really accomplish anything). Having them in space keeps them out of reach of anyone but state actors.
To be precise, heat rejection via radiative cooling scales with the fourth power of the temperature (in K) the radiator operates at, all else constant.
By definition it is compute nodes in space. That is what a router is, a computer. Just a matter of scale. They could be improved to more compute and more storage per node. The framework is already there: treat these as disposable vs having to think about supporting them through maintenance.
If you look at how small a Starlink sat is, and how much of that space is taken up by power generation and storage, antennas, signal conditioning, RF electronics and more, I'm sure that whatever resources are running the computing in the entire starlink fleet orbiting the world can fit all together in one single row of servers in an existing datacenter.
And yes, a space-based computing node would not need quite as much of some of these things but they'll still need them in some way. It's not like you can just plug in a power and ethernet cable into them.
I doubt this will scale to a level that is actually useful. It's a nice experiment, just like Microsoft when they threw a datacenter container into the ocean. But not practical in the current conditions: https://news.microsoft.com/source/features/sustainability/pr...
Yes they say it is amazing and sustainable there in that blog post, yet somehow they've never bothered to do it again.
They can't make them a little bit bigger? But also I found this interesting image on the scale at hand here of a given launch of starlinks, no clue how many are included in a single launch, and it is a substantial amount of rack space they have been sending up at once:
So to me they have solved the issue of having a space based compute array network interfacing with the earth. They have solved the issue of launching and deploying this array. And their given launches seem to have a substantial payload of compute going up at once just in sheer volume. And right now the only real difference is that the nodes they are launching are just pretty weakly specced. Everything else is in place and turnkey.
But most of that is nothing to do with compute. A rack of servers is all compute. Starlink sats are antennas, RF amplifiers, solar panels, laser links, shielding, even maneuvering thrusters and fuel for those (they could be electrical ion thrusters but they'll still need reaction mass, not sure what type they use). Probably some inertial reaction wheels too, they are used for spacecraft orientation, not positioning.
They are sending a few racks of stuff up every launch but the problem is not that it's underspecced. It's that most of it is just needed for equipment survival and communication in space.
You're talking about an environment that's full of radiation and goes from -200C to +200C every 90 minutes. That needs to be orbit managed and cooled (and sometimes also heated) without any airflow. Just sticking a few servers in a barrel isn't going to do the job.
A datacenter is about data. Your network of space router is in no way something a reasonable person would consider a datacenter... Even less an inference datacenter.
Why, because on board storage is too small and the compute nodes are underpowered? And that can't ever change? A reasonable person doesn't understand technology usually. That is increasingly an understanding left to the wizard class.
I mean people make clusters out of raspberry pis and minipcs.
There is nothing magical here, you definitely don’t need to be a wizard to understand the hardware necessary for AI inference. You can make nice little clusters with rpis, yes, there is nothing magical about that, it’s pretty much baremetal 101. But no, you cannot run any meaningful inference on that cluster.
Maybe look at what is inside a datacenters, the amount of power required is very large, and the hardware to run the inference + network isn’t small. Then try to see how much sending that to space cost
> Why, because on board storage is too small and the compute nodes are underpowered?
Yes
> And that can't ever change?
It can, but but not for free. As the comment earlier in the thread was referring to, more computing power needs more cooling, and cooling in space is hard.
> A reasonable person doesn't understand technology usually.
What? Do you mean a layperson? Why does that matter when discussing the feasibility of space-based AI datacenters?
> That is increasingly an understanding left to the wizard class.
No, you can get there with a bachelor's degree in a relevant subject. Or just reading informative news sources.
> I mean people make clusters out of raspberry pis and minipcs.
> Anything in space is more expensive and way harder to do, for a datacenter there is no benefit.
If we pick an extremely fast orbit, then relativity means the hardware will age out (slightly) slower, so I'm sure that'll help with the maintenance issue.
It's the wrong way around though. Ideally we want to speed up our current compute ability not slow it down; if it experiences more time than we do then it can do more. Relative-MHz means my slower hardware becomes tangibly fast again.
General Relativity says mass warps space time, so we need to get these datacentres out of the Earth's gravity well. And the Sun's, and the Milky Way's; out into the deepest void of intergalactic space. The good news is that a maintenance callout is still quicker than some of the earth based DC's I've had gear in, but the bad news is that it doesn't get us much of anything at all.
Special Relativity lets us abuse time with speed (something I discovered as a teenager). Going faster than Earth means we experience less time, so we just need to try and slow down comparative to our home base. The earth is orbiting the Sun at ~30km/s, the solar system is orbiting the centre of the Milky Way at ~230km/s and our local group of galaxies is moving relative to the Cosmic Microwave Background at ~600km/s. We can easily get our DataSpaceCentre up to 1,000km/s or more, so we just need to point it relative to all that movement we mentioned above making stationary relative to the universe. It's completely doable, but (as well as far more variable response times to callouts) only gets us an extra second of compute over a human lifetime.
Fundamentally, we're attacking this problem in the wrong direction. Earth's gravity is comparatively minor, and our piddly ~600km/s relative movement is a tiny fraction of the speed of light. We should be filling The Earth with compute, and then decamping humanity into space and travelling at relativistic speeds. Or put the compute in space and move the Earth into the event horizon of a black hole. You can't do the inverse of Interstellar keeping Earth where it is, the maths isn't in our favour. If everyone lived on (a less moist) Miller's Planet, we'd get 7 years of compute every hour. It puts Moore's Law to shame; the relative MHz are obscene.
There's the obvious problem of communications. I'm led to believe there's issues with radio and light, so this probably isn't a job for fibre. Veritasium seemed to imply a battery, switch, lightbulb and a wire stretching around the globe would light instantaneously, so I'm sure we can come up with a new copper Ethernet standard for low latency over solar distances.
I get that they're using liquid coolant at higher than usual temperatures, but why couldn't they do that before? Most of the comparison in the article is for air cooled datacenters but what about other liquid cooled ones?
Surely in all the previous datacenters that have been designed there has been someone doing the math and determining what temperature things need to run at, how much energy it will use, how much heat it all will produce, etc.
edit: just saw this:
>Previous liquid-cooled servers were hybrid: GPUs and CPUs got cold plates, but the rest of the system stayed air-cooled, with finned heat sinks designed to shed heat into moving air. In a fully liquid-cooled server, the cooling for these components needed to be completely redesigned to use liquid.
The rest is marketing: The Cray supercomputer were fluid cooled back in the 1980's, the entire board had an inert liquid flowing across it.
He showed me their Cray, which had its own dedicated computer room, and they set it up with the coolant pump and fountain unit right in the middle in front of a glass wall facing the hallway so everyone could gawk at it.
It seemed like a pretty big deal ~ 2011 when big companies were running their (air cooled) datacenters closer to 95F (35C) vs the traditional 72F (22C). So jumping up a little more is maybe not super exciting, but it's still innovation.
Even air-cooled datacenters work somewhat the same way, but instead of water to chips, it's air. The air goes into hot aisles then exchanges heat with water, after which, see above.
* Other datacenter marketing materials talk about how they have a "closed loop system that uses no water" and they do still use water in the evap towers. I was half expecting this article to be that again, glad it wasn't.
"NVIDIA’s thermal engineering team reworked how those components handle heat, designing cooling loops that simplify how liquid is routed to multiple high-power chips on the board using a single inlet and outlet, resulting in a cleaner tray-level cooling architecture"
The innovation may be in the speed or volume flow of the coolant through different parts of the data centre to regulate the temperature. And of course, redesigning every component to be compatible with this fan-less design.
I think it’s only possibly because NVIDIA is much more vertically integrated than ever before.
Summer is still an issue, but fun solutions are possible. With the right geology, I think it’s possible to heat an underground volume in the summer and recapture (some of) that heat in the winter. In many, many climates, annual heating costs are far higher than cooling costs, at least if people aren’t stupid with skylights. [0]
[0] As a back-of-the-envelope heuristic, heating or cooling load due to conduction and air exchange is proportional to the difference between indoor and outdoor temperature. Outdoor temperatures of -10F to 30F are not unusual in the winter and are 40-80F away from an indoor temp of 70F. But outdoor temperatures in these climates rarely exceed 95F and are mostly lower in the summer, so that’s 15-25F of cooling. And heat pumps are more efficient at smaller temperature differences.
Radiative heating is an entirely different story.
(Seasonal heat storage is also a thing, Espoo's neighbours have tens of GWh of storage, with a new 90 GWh cavern in the works. Not sure if the systems are interlinked.)
Then 45 or below is sent back on the return.
DC inlet is 45°C, outlet is 55°C assuming a 10°C ΔT. By the time that's travelled 500m–1km through pipework you've lost a few degrees, so you're arriving at the HIU at maybe 50–52°C. The home radiator circuit then takes that down by around say 12°C, returning ~38°C. Factor in pipe losses on the return leg and you're back at the data centre with maybe 35°C inlet rather than 45°C — meaning the DC output is now only 45°C rather than 55°C, and the whole system gradually degrades each cycle. You could address this by mixing some hot output back into the return to keep the DC inlet stable at 45°C, but eh.
Surely having the input fluid being colder is a benefit, not a problem? Just run the fluid more slowly through the system?
It's got a "heat energy to/from campus" exchange in there.
That's a link in March and the air temperature was 31°F.
https://web.archive.org/web/20210708150410/https://www.nrel.... is later with air temperature of 68°F.
Property values have consistently gone up in that region for decades, and are up to $6 million an acre if there's enough contiguous land to put another data center on.
Many of the people complaining about datacenters would also complain about literally any kind of development.
Power plants are all over, even in populated areas. They’re not so bad either (except perhaps coal).
There is no fundamental reason that datacenters need to be especially unpleasant to their neighbors.
But that is not how corporations roll. They want the cheapest shit that they can get away with. No regulations only corruption. Which is middle of nowhere America.
What I don't understand is putting these things in populated areas.
Sure there is, being a good neighbor costs more than being a bad neighbor
I don't have any datacenters near me but I can hear some heavy hums from the washing machine 3 floors up when it put my head on my pillow, for some reason it just propagates through the building physically. When I walk around I don't hear it. Datacenter noise can be the same.
IMO they should be put away from habitation, there's no reason for them to be near there anyway
Right. Vibration propagates through solid (and liquid) materials.
But this can all be measured and controlled, and there's nothing special about datacenters. A building that is hundreds of feet away will couple to your pillow much less strongly than a washing machine in your building. And the washing machine often has a wildly unbalanced load and minimal decoupling between itself and the floor, whereas a big fan in a datacenter or other industrial building ought to be balanced and also ought to be installed on decoupling mounts.
If datacenter operators (cough xAI) are being lazy about properly selecting, installing and maintaining equipment, then you can have a problem. Otherwise you have a much smaller problem.
I agree, but that's a hard problem (in the US anyway). Unless you're plopping data centers in the middle of national parks, or in the middle of the desert where water is going to be a problem, you are nearly always going to be within some small mile radius of civilization. Plus the cost of trenching new fiber out in the middle of nowhere.
The same reasons humans want to concentrate in a particular area (access to jobs, infrastructure) are the same things that data centers need.
Once water-less cooling tech like this improves then yeah, just plopping them in the middle of the unpopulated desert becomes viable (assuming you can get the fiber out there and latency is tolerable), so long as they generate their own power.
Imo we should just solve the problems with data centers being near cities. Manage/regulate the noise and any waste (heat included, it shouldn't drastically impact the neighbors) and make them pay for any utility capacity/reliability upgrades needed. If this article is right and water usage can be nearly eliminated then it seems like the rest should be solvable? Especially if we can take the extra heat and use it for local power or heating needs.
If you cycle between 45 C and 55 C water temperature (as mentioned by the press release), you are only getting a 10 C delta. That isn't even enough for district heating, probably not even with heat pumps.
Now if you have something like a steel foundry, that have much hotter cooling water, you can absolutely use the heat for district heating, but even then it usually isn't enough for cost effective electricity generation. Even when it is waste heat, as the equipment to handle it still costs money and requires maintenance.
You are calculating the wrong delta T. To heat a space, you need your working fluid to be warmer that the space you’re heating by an appropriate amount.
55°C is certainly on the cool side to heat a building, but it’s entirely workable with a high-area, highish-thermal-conductivity system. Here’s an actual chart:
https://www.warmboard.com/wp-content/uploads/2022/04/WaterTe...
You don’t actually want an absurdly warm floor.
Even for buildings that need warmer fluid, water at 45-55°C is a fantastic source for a heat pump.
People said this about high voltage electric lines and wind turbines. Blind tests proved they were imagining things.
Nearly 10x more people die from the cold than from the heat.
"...9.43% of global deaths were attributable to non-optimal temperatures, with 8.52% from cold and 0.91% from heat."
https://www.thelancet.com/journals/lanplh/article/PIIS2542-5...
I don't think this is climate change propaganda, but your application of this study by evoking it in a discussion about climate change feels like it.
No one bats an eye when an air conditioner runs.
I find their noise pretty obnoxious. Out of respect for my neighbours, when I get around to installing one, I'll be getting the absolute quietest model available.
In the US
My GPUs at Hurricane Electric in Fremont are also completely unnoticeable outside the building. Inside, when I'm working at the cabinets it's obviously deafening. Outside you wouldn't even know. Realistically, the predominant sounds at my home are from the traffic on the Bay Bridge so it's nice when there's congestion because it's quiet.
Honestly, I wish there were more urban datacenters. It's getting quite annoying having to make a 1 hr trek to Fremont every time I want to rack a new server.
There's a lot of them in high rise buildings... but they come with high rise rent.
With that said, by the standards of industrial sites data centres are quiet, low traffic and smell free. An industrial area that can’t build a data centre certainly can’t build a steelworks or oil refinery or leather tannery.
I think we are going to need heating.
I don't live next to one but I'd take constant humming over the constant stop/go traffic noise, honking, squeaky brakes, slamming doors and revving engines I now have on my western side of the apartment, thanks to the unemployment office the city opened on my street not too long ago.
So how come constant humming is somehow an illegal nuisance, but we've been expected to put up with the much more annoying urban traffic noise for decades just fine?
My parents apartment have constant humming anyway thanks to the HVAC system on the roof of the nearby supermarket and white/brown noise is far more tolerable and easy to tune out than traffic noises.
For one, there tends to be little traffic at night when most people want quiet in order to sleep. Driving is also something (nearly) everyone does and benefits directly from, so negative externalities are easier to accept. It is much harder to accept a new source of noise near your home you haven't asked for and don't directly benefit from.
This reads a little too close to driving being an inherently good thing or some sort of objective requirement, but it's only that way in certain urban places because the built environment makes it as arduous as possible to do those things without.
Something that pisses me off about many urban places that don't even otherwise require people to drive, is that many who do use their cars the most often have their neighborhoods protected from the noise they contribute to everywhere else. This whole thing of putting apartments only where there's already the most disgusting car-infested thoroughfares; "sorry, can't have an apartment one street in off the main drag, that's only for bungalows! Don't like it? Get richer. Excuse me while I drive through your bedroom and park for free in front."
https://xcancel.com/BrianEntin/status/2067930868191035474?s=...
So what, winters would be no more? Snow will disappear, no more ice-men and christmas trees, and subzero conditions in general, too?
You do eat, don't you?
What's a favorable climate, apart from, obviously, Greenland? The piece is a little light on details on the correlation between outside temperatures and efficiency & cost. It'd be nice to see even a broad-strokes discussion of that.
https://www.kit.edu/kit/english/pi_2024_038_kit-supercompute...
I assume Germany is the same, many years ago really is different to today.
For e.g you might think of the outskirts of London as fairly moderate, but this week it's been hot enough that supplemental cooling would likely have been needed at points. For a data centre here you'd typically design the cooling system to cope with outdoor temps in excess of 40°C, which is not a conservative number anymore.
Also, while Nvidia might be happy with you supplying water at 45°C I suspect you will get better longevity of the hardware at lower temps like say 35°C. GPUs are expensive, and extending longevity may well be 'worth' a bit more water or energy to you. In practice you are also likely to have air cooled systems that sit 'beside' the AI compute like storage severs, any extra CPU compute and network switches. So you are likely to need a separate room and cooling system for that. Great progress though.
https://www.nasa.gov/centers-and-facilities/ames/doing-more-...
https://www.nas.nasa.gov/assets/nas/pdf/ModularSupercomputin...
They’ve made this claim numerous times in the article and I really don’t understand it. The building has tons of water being recirculated through it. That water came from somewhere in the surrounding natural world. How is that 0 water consumption?
But it's still misleading. The major source of water use in datacenters, by _far_, is the water used in power generation. This improves PUE, which reduces power draw, but the savings are almost certainly under about 20% given that many modern datacenters already operate at a PUE of under 1.2. So if you're running on coal or gas, you're still consuming quite a bit of water indirectly.
Now that said again - the water consumption part of this equation is generally overhyped. The power draw is the problem, as are the really bad temporary hacks to the power problem (e.g., what x.ai is doing with "temporary" gas turbines).
Imagine a residential building that reclaims and reprocesses and purifies 100% of all the water it uses. This would be dramatically more difficult and better than the status quo, and would be called 'net zero' by any sensible accounting method.
---
Obviously, evaporative cooling is net-zero water use when accounted across the entire globe (the water falls as rain, somewhere, eventually), but it is net-negative for a local community.
Does that mean the whole server board is running in liquid? If true, how do they do maintenance? Replacing parts must be extremely difficult.
But it did not sound like they were describing a Cray 2 style liquid immersion cooling system.
Or are they for some unfathomable reason using evaporative cooling in data centers?
East of the 100 degree W line of longitude, there is more than enough water to use evaporative cooling if needed.
The loss of material must be included
If water is evaporated or spent out of the system.. it is not more efficient
You can certainly argue DCs should pay more for water than other uses, but who gets to decide what is a good vs bad use of water? Pricing in externalities is tricky, and water usage rights are especially complicated. I don't know what a good&fair solution is.
1gw of power converts approx 400 liters of cold water into steam _per second_.
currently.
and also more energy efficient, because evaporating water away takes a lot of energy with it. you have to raise radiators to a higher temperature to keep up with that, or have much more surface area.
A closed circuit cooling tower still has water spraying onto the closed loop process water heat exchanger coil and mixing with atmospheric air to evaporate and cool the process water indirectly instead of evaporating and recirculating the process water that doesn’t evaporate directly like in an open-loop cooling tower.
I suppose you could condense the evaporated water somehow by using a chilled umbrella or some other ridiculous contraption above the cooling tower, but why would you do that?
FWIW I sell and run commercial electrical work, primarily to mechanical contractors who are installing boilers, chillers, cooling towers, and pumps. I spend my professional life immersed in this type of equipment.
https://www.sciencedirect.com/science/article/abs/pii/S13594...
There are some systems that pipe refrigerant around the building, but they’re relatively uncommon (VRF or variable refrigerant flow if you want more details).
Glycol and water is cheaper than refrigerant so there’s usually a chilled water loop that passes thru a heat exchanger that interfaces with a chiller (vapor compression refrigeration) to reject the heat from the chilled water loop.
This eliminates the need for evaporative cooling towers.
On the other hand: the heat has to go somewhere. So… where? Datacenters already create a warm microclimate in their vicinity, is that getting even worse?
It’s kind of like how brine from desalination is not a global problem for the oceans at all — all that matters is diluting it enough that it doesn’t poison the local ecosystem.
It's not clear to me what changes are happening here. The siblings to your post seem to be indicating an overall improvement.
More on it at [0], but it doesn't take anything beyond a basic energy calculation to know that 1GW of energy is not going to have a significant effect "6.2 miles away".
[0] https://andymasley.com/writing/data-centers-heat-exhaust-is-...
If you manage to use the waste heat to avoid generating heat somewhere else (that the article calls heat recovery) then there’s a further reduction in total heat output.
Nvidia has so much money and they can’t afford to pay a human for a day of their time to write a blog post?
> Nvidia has so much money and they can’t afford to pay a human for a day of their time to write a blog post?
The shareholders desperately need that money.
We are all fucked.
And it’s sad because Jensen seems like one of the rare good CEOs when I listen to him speak.
But even Dario says he doesn’t let Claude actually write his blog.
Have we been listening to the same person speak for the last few years? Jensen rarely even sounds sane anymore.
The same as technical docs for any codebase, humans will not read them anymore, only AIs which then translate it to human on-demand, it's already happening, I've worked recently with many new frameworks/codebases without even opening the doc (not even the Github page) and solely asking the agent to gather info for me about it.
PS: The reason I feel it will be this way is that it will allow to legitimatize mass data collection indirectly, instead of doing telemetry on page and software level, we will just send all the content automatically to some inference providers (probably provided for free by Google, MS and so-on)
Watch for cases where content has been through two layers of LLMs. It's not good.
[1] https://news.ycombinator.com/item?id=48665916
I do think ground based centers will have better economics when they can be built though, and this addresses noise and water complaints which are the big 2 regional complaints.
It seems like lots of bottlenecks are getting solved quickly, except for maybe memory.
https://spacenews.com/china-backs-orbital-data-center-startu...
https://www.reuters.com/science/china-vows-develop-space-tou...
Plans for space data centers should be seen with skepticism. However when they are backed by different parties who have stakes in the game, that's more credible. More than HN crowd for sure.
1. Space is terrible for heat regulation. It's a perfect insulator for everything except radiative cooling, which is the least efficient. Hot things stay hot.
2. Space is full of radiation. Everything has to be radiation hardened, which makes it heavier, more expensive, and, yes, more difficult to cool.
3. Space is far away. Well, farther than a data center on Earth can be. I know China hasn't solved the speed of light.
We put up with it with satellites because it still has some advantages over trying to run cables literally everywhere, but we do, in fact, still use cables laid on the bottoms of the oceans.
So, is physics wrong, or is a country known for making dumb decisions some times making a dumb decision?
To answer your first question, yes. Even if they were US companies. Remember when each Big Tech player was claiming to build their own blockchain, except centralized which was a contradiction.
I do really think that if large numbers of jobs are indeed going to be displaced by AI, movements will pop up of people attacking datacenters (and honestly I wouldn't blame them even though it won't really accomplish anything). Having them in space keeps them out of reach of anyone but state actors.
To be precise, heat rejection via radiative cooling scales with the fourth power of the temperature (in K) the radiator operates at, all else constant.
And yes, a space-based computing node would not need quite as much of some of these things but they'll still need them in some way. It's not like you can just plug in a power and ethernet cable into them.
I doubt this will scale to a level that is actually useful. It's a nice experiment, just like Microsoft when they threw a datacenter container into the ocean. But not practical in the current conditions: https://news.microsoft.com/source/features/sustainability/pr...
Yes they say it is amazing and sustainable there in that blog post, yet somehow they've never bothered to do it again.
https://i.redd.it/zh7qvyfqgvx21.jpg
So to me they have solved the issue of having a space based compute array network interfacing with the earth. They have solved the issue of launching and deploying this array. And their given launches seem to have a substantial payload of compute going up at once just in sheer volume. And right now the only real difference is that the nodes they are launching are just pretty weakly specced. Everything else is in place and turnkey.
They are sending a few racks of stuff up every launch but the problem is not that it's underspecced. It's that most of it is just needed for equipment survival and communication in space.
You're talking about an environment that's full of radiation and goes from -200C to +200C every 90 minutes. That needs to be orbit managed and cooled (and sometimes also heated) without any airflow. Just sticking a few servers in a barrel isn't going to do the job.
I mean people make clusters out of raspberry pis and minipcs.
Maybe look at what is inside a datacenters, the amount of power required is very large, and the hardware to run the inference + network isn’t small. Then try to see how much sending that to space cost
Yes
> And that can't ever change?
It can, but but not for free. As the comment earlier in the thread was referring to, more computing power needs more cooling, and cooling in space is hard.
> A reasonable person doesn't understand technology usually.
What? Do you mean a layperson? Why does that matter when discussing the feasibility of space-based AI datacenters?
> That is increasingly an understanding left to the wizard class.
No, you can get there with a bachelor's degree in a relevant subject. Or just reading informative news sources.
> I mean people make clusters out of raspberry pis and minipcs.
So? What does that have to do with anything?
If we pick an extremely fast orbit, then relativity means the hardware will age out (slightly) slower, so I'm sure that'll help with the maintenance issue.
It's the wrong way around though. Ideally we want to speed up our current compute ability not slow it down; if it experiences more time than we do then it can do more. Relative-MHz means my slower hardware becomes tangibly fast again.
General Relativity says mass warps space time, so we need to get these datacentres out of the Earth's gravity well. And the Sun's, and the Milky Way's; out into the deepest void of intergalactic space. The good news is that a maintenance callout is still quicker than some of the earth based DC's I've had gear in, but the bad news is that it doesn't get us much of anything at all.
Special Relativity lets us abuse time with speed (something I discovered as a teenager). Going faster than Earth means we experience less time, so we just need to try and slow down comparative to our home base. The earth is orbiting the Sun at ~30km/s, the solar system is orbiting the centre of the Milky Way at ~230km/s and our local group of galaxies is moving relative to the Cosmic Microwave Background at ~600km/s. We can easily get our DataSpaceCentre up to 1,000km/s or more, so we just need to point it relative to all that movement we mentioned above making stationary relative to the universe. It's completely doable, but (as well as far more variable response times to callouts) only gets us an extra second of compute over a human lifetime.
Fundamentally, we're attacking this problem in the wrong direction. Earth's gravity is comparatively minor, and our piddly ~600km/s relative movement is a tiny fraction of the speed of light. We should be filling The Earth with compute, and then decamping humanity into space and travelling at relativistic speeds. Or put the compute in space and move the Earth into the event horizon of a black hole. You can't do the inverse of Interstellar keeping Earth where it is, the maths isn't in our favour. If everyone lived on (a less moist) Miller's Planet, we'd get 7 years of compute every hour. It puts Moore's Law to shame; the relative MHz are obscene.
There's the obvious problem of communications. I'm led to believe there's issues with radio and light, so this probably isn't a job for fibre. Veritasium seemed to imply a battery, switch, lightbulb and a wire stretching around the globe would light instantaneously, so I'm sure we can come up with a new copper Ethernet standard for low latency over solar distances.
Invest early, we're going straight past the moon!