I understand the vision, but how does this work on a global scale. e.g. American employees refuse to build this, but China's don't.
Edit: I originally ended with "What would have happened if Germany had a nuclear bomb and America didn't?", but I think it distracted from the point I was trying to make so moving this to an edit. I'm not trying to ask "is the US the bad guy". I'm trying to ask how to balance personal anti war sentiments with the realities of the world (specifically in this case keeping up in an arms race).
Do the same thing we did with the nuclear arms race: Treaties to limit and control it.
Obviously, we would have had more political leverage if our leaders had started working on a treaty before they crossed enough moral red lines to start a tech revolt, but we did not elect the sort of leaders that would do that.
The obvious countries sign those treaties for political reasons. There are countries officially pretending they don’t have any nuclear warheads, but it is well known by everyone they have plenty and a capacity to deliver them. They sandbag for political, but also for religious reasons and that’s scary.
treaties worked because both sides had large quantity of bombs. In this case, certain people do not want US to have AI bomb, while China and others will have it.
>American employees refuse to build this, but China's don't.
How about you articulate the threat from an AI powered China to people outside of AI powered China and discuss potential methods to counter that, instead of insisting capabilities be developed just in case.
>is the US the bad guy
Yes
>I'm trying to ask how to balance personal anti war sentiments with the realities of the world
Insist on open information, never surrender consent willingly and demand justification for everything. As always.
PRC isn't going to do any of the things you are asking for, and no one expects them to. The threat of an AI powered China is really obvious to me, but apparently the idea of "IP theft and industrial sabotage, but at scale with AI agents instead of human meat sacks" is hard to clearly articulate.
One method, beyond AI powered kill chains, to counter an AI powered China is of course strategic weapons.
That’s exactly why I think the principled position is naive in a tragedy of the commons situation we’re in - it isn’t a sci fi story with a happy ending, it’s the Manhattan project and 70+ years ago nazi and japanese data centers doing foundational model training would’ve been bombed to smithereens at any cost.
The reason it works is when you have less participants in an effort you have slower progress in that endeavor. Brilliant employees prohibiting their entire org to not support the development of bad things prevents less brilliant employees from doing bad things.
It is sort of like computers are amazing but can also be a privacy nightmare. Software engineers don’t help or coordinate with black hat hackers. So black hat hackers have a harder time refining their systems.
I'm going to give a shout out here to an episode of the excellent podcast Hardcore History, specifically Episode 59: The Destroyer of Worlds [1].
The development of the atomic bomb created a debate in American policy circles about how the US should react. Within a few years, the same debate occurred over developing thermonuclear weapons. The same question kept coming up: what if the enemy has these weapons and we don't?
Dan Carlin's position, which I happen to agree with, is that America chose wrong. It became both belligerent and paranoid to a degree that just wasn't the case before WW2. If you look up the history of regime changes at the hands of the US [2] then you can see it went into overdrive after 1945.
Part of the problem here I think is projection, the psychological phenomenon. It's also a cultural phenomenon. So, for example, when you have a historically oppressed people who are being potentially freed, the oppressors will fret that the formerly oppressed will rise up and kill them. This is projection.
We saw this exact thing play out with Emancipation. There was no mass revenge violence by the former slaves. If anything, there was more violence by the former oppressors against freed slaves and a system that excuded the violence (eg the Colfax massacre [3]).
I think nations can be guilty of this too. The US sees any other global power as a potential hegemonic, imperialist power that will dominate and exploit everyone around them because, well, that's what we do.
We also see this in how we view AI as a resource. We see it as something to be owned and gatekept such that some US company will become insanely wealthy further extracting every last dollar from every person on Earth.
So your comment belays a common fear that China will displace us as a global hegemonic, imperialist power despite there being zero evidence that China behaves in that fashion. American propaganda runs deep and the projection is strong so this will immediately cause some to say "but Tibet" or "but Taiwan" without really knowing anything any of those situations.
As just one example, the One China policy is the official policy of the US, the EU and almost every nation on Earth. "They might invade" I preemptively hear. They won't, partly because they can't but really because they don't need to. If the world already has the One China policy, why do anything? Oh and I said they can't because they can't. They don't have that military capability. If you think that, you don't know anything about war. Crossing 100 miles of ocean to invade an island with a army of over 500,000 is simply not possible.
Let me put it this way: the 17 or so miles of the English Channel stopped the German war machine despite having millions of soldiers.
Anyway, back to the point: this whole argument of "what if China does military AI?" is (IMHO) projection. If anything, China has shown that they won't allow a US tech company to control and gatekeep AI (eg by rreleasing DeepSeek). And if China gets AI, they're more than likely to use it to further raise people out of poverty and automate away more menial jobs without making those displaced workers homeless.
You seem to be laboring under the naive belief that mainland China is a rational actor which will refrain from attacking Taiwan over fear of heavy losses and possible defeat. You might have been correct at some point, but that situation no longer obtains. Xi Jinping has successfully purged all potential rivals and personally taken over centralized control of all important decisions. We have no visibility into his thinking, so we have to assume the worst. If he orders the PLA to go then they'll go, regardless of consequences. Part of preparing for the eventuality involves building more effective autonomous weapons. There is no realistic alternative.
So I follow a number of China scholars and experts and I've yet to see any consensus about what these military purges actually mean.
It could be about corruption. You see this in the Russian military where paid-for tanks didn't exist because the generals had pocketed the money. It could be to have an expansionist policy. It could well be to not have an expansionist policy. The point is that nobody really knows yet.
But the string I really wanted to pull at was this idea that China isn't a "rational actor". It's lazy and really a thought-terminating cliche. It's certainly no basis for analysis or policy-making. It's kind of the final boss of justification. "Putin/Saddam/Xi/Castro/Maduro is crazy". That really just means you don't understand what's going on or want to ignore the facts.
We now have 50+ years (since really the end of the Cultural Revolution) of China acting in a very rational, very intentional and very long-term way. Xi's own history here is pretty interesting. He went from privileged child (his father was one of Mao's lieutenants) to being banished to working his way up through the party's ranks over decades.
It's a mistake (IMHO) to view Xi as a singular actor, let alone as a irrational autocrat. While the PRC and the CCP might be relatively new the systems and political structures can probably be traced back thousands of years. I'm thinking particularly of the bureaucratic reforms of the Qin Dynasty some ~2300 years ago.
What cannot be ignored is that a billion Chinese have seen a massive improvement in their living conditions during their lifetimes. Almost all of the people pulled out of extreme poverty in the 20th century were because of China (~800M). So although China is authoritarian, the government is extremely popular because of that increase in living conditions. It's something that we in the West have a hard time fathoming because our living conditions have been in decline since at least the 1970s.
> And if China gets AI, they're more than likely to use it to further raise people out of poverty and automate away more menial jobs without making those displaced workers homeless.
Your comment is very optimistic. But the quoted part reminded me of something I heard (again) about China using slave labor in their lithium mines:
You seem to really like history. Maybe you're ready to graduate from podcasts to reading books and primary sources. Fair warning: you might end up with a picture of history that is less cartoonish and motivated.
> The US sees any other global power as a potential hegemonic, imperialist power that will dominate and exploit everyone around them because, well, that's what we do.
In the Cold War, this was the correct approach, the USSR was that.
> American employees refuse to build this, but China's don't.
It's not American employees vs. China employees. No need to villainize China at every opportunity. Most Chinese employees are more similar to American employees than you think.
It's {top candidates who have their pick of employers} have the luxury to refuse to build this.
Mid-tier dude who can't land a job at any of the top AI companies and can code with Cursor and trying to pay their rent or medical bills will absolutely build AI for the military in return for having their rent paid.
This is regardless of whether it is in the US or China.
This is not answering the question.. and HN ain't US only.
You can say the same for any other country... What if Japan employee refuse, but American want that anyway? What if China employee refuse, but Russia employee want that anyway?
The implication are still the same -- social, culture, jurisdiction, national interest, company interest don't share the same boundary and don't align on their priorities.
Is there any reason to think that autonomous weapons are a critical strategic capability? It's hard to see what an unpiloted drone can do that a remotely piloted drone can't, other than perhaps human rights violations.
The simple version: Weapons systems are quickly advancing to the point where many of them can navigate and operate independent of human control. The obvious question here is at which point do we give these platforms release authority for lethal weapons. It becomes impractical to require (or even imagine, really) there to be a human "pilot" operating every single drone when you have hundreds or thousands of them operating in theater. That's really what this is about.
Think of it this way: mines installed in the seabed in wars past were "dumb", in that a passing ship had to happen into it. Imagine systems deployed underwater that were mobile, contained multiple torpedoes, and could strike warships with little to no warning given their small acoustic signature. It's the same principal as a mine (you leave it one spot, hope an enemy ship comes by), but the capabilities are far more advanced. If the system is not at least semi-autonomous than it might as well be a dumb mine again.
Remotely piloted drones can't operate at long ranges in a conflict against a near-peer adversary such as China. All of the high-bandwidth communications links will be degraded by a combination of jamming, cyber attacks, and anti-satellite weapons. Remote piloting will only be reliable using fiber optic cables (very short range) or direct line-of-sight transmission. So hardly practical in the Pacific theater of operations.
In an existential conflict no one cares about human rights. That's something for the winners to worry about after the shooting stops.
There is only self regulation, ultimately, at the top. I think it's still progress to see these groups specifically call out their moral hesitations, even if it doesn't go anywhere - it gives people ground to realize that others share their concerns. All movements, all progress starts from people putting their stance out there and getting a conversation going around the topic; that builds mindshare and eventually a demand for change.
There's always this comment, saying that its useless to possibly govern or resist advancement or development or use of weapons capable of indiscriminate killing.
If the world actually worked like they believe it does, if restraint were just not possible, the world would have been destroyed at least 3 documented times over.
We've banned the other account for repeatedly breaking the site guidelines, but - WTF? I don't want to ban you because it doesn't look like you have a habit of posting like this, but please don't do this kind of thing here again.
Although in the context of the parent comment, majority of Googlers probably aren't working on things directly related to controversial topics, instead they are probably working on mundane and non-external facing projects like "how do I migrate my libraries from this deprecated dependency to this other shiny new thing".
Why is there any controversy about defending one's nation being "good" or "bad"?
I can not believe what I am reading here, and how the single comment supporting defending one's country is so heavily downvoted. Qatar has poisoned Western online communities such that all defence of the United States is considered taboo? I don't even live in the US and I am frightened by what I see here.
The controversy isn't about defending one's country, it's about you and the parent comment author assuming what this is all about without reading the article.
The core of the issue about autonomous use of AI in mass surveillance of Americans and autonomous use of AI in automated weapons that make kill decisions. Anthropic is perfectly fine with working with the War Department and "defending one's nation".
But they are not okay with their AI being used to make a mockery of the 4th amendment and making automated kill/no-kill decisions about actual human lives.
Oh I believe it’s important to defend the country, but not because it’s a popular opinion. I dislike any statement that believes truth is based on consensus.
The resistance goes out the window the first time an American is gunned down by an autonomous system. They should do whatever possible to prevent that outcome.
We already forgotten about this already? [0] Where was the open letter then?
Both companies (Google, OpenAI [0]) have defense contracts. At this point, the best course of action is to leave Google and OpenAI if you disagree with that (they won't).
Piggybacking on deeply integrated information and connectivity within society that was marketed and adopted under the guise of trust and an ethos of not being evil is pathetic.
Build, train, develop and maintain an AI for military if needed. When a government is scared of individuals they've clearly lost their edge.
This gets a giant eye roll from me. Are you really so naive that you thought working on AI for a giant tech company, creating software that is capable of finding deep patterns in massive amounts of data... and it wasn't going to used by the Defense / Intelligence industry? If you are so against the US government, and you are working for ANY big tech company you are aiding the Intelligence and Defense industry. Government uses AWS and Azure. Intelligence agencies use the data and tools of Meta / Google / Apple / etc.
Am I the only one who remembers the prime directive of google, much easier to understand than 'organizing the worlds information' etc. etc. It was simpler.
Google employees must think this is pre 2024. The employer has the power and doesn’t mind laying off people who don’t tow the company line and all of the CEOs bend over and bribe the President - ie “settling” frivolous lawsuits brought by Trump himself over “censorship” when he was out of office
I think a lot of software companies are going to learn just how much employee power remains tomorrow, in the very likely event that the Pentagon issues an order purporting to ban all defense contractors from using Claude.
It is when "defense" means invasion and subjugation of other countries. All countries pose their military operations as "defense." Inquiring minds should ask if a country surrounded on sides by two oceans with two pacified neighbors has any real threats or merely opportunities for cheap labor, market access, and mineral rights abroad.
This has been going on for a very long time (read what Smedley Butler said in "War is a Racket"), but after the Iraq War, the credibility of the US should be somewhere in hell.
I remember they successfully got Google out of a military contract in the first admin (and briefly vilified by the right for that). that's not going to work now. Workers have a lot less power and the CEO is buddies with Trump
As the article says, the workers didn't petition the CEO, they petitioned the head of Google AI who's already expressed solidarity with Anthropic. If they can convince Jeff Dean, I don't think Sundar necessarily gets a say; it's a lot easier to stick your head in the sand and ignore things than to fire one of your most widely respected engineers because he won't help the Pentagon build Terminator robots.
My one concern in this whole thing is that if these slightly less benevolent, but still have some morality, companies don't engage, we'll be left with companies like OAI and xAI engaging and you just know that's not going to make things better for anyone.
Which companies are these? Google and Facebook already bribed Trump under the cover of “settling” a frivolous case he personally brought forward. Tim Cook personally donated to his inauguration fund and gave him an expensive trinket. The Netflix CEO is now kissing up to him trying to get the WB acquisition approved. Even companies that are hurt by the tariffs won’t say anything bad about him. The only CEO that has spoken out against any of his policies is Chase’s CEO.
> it's a lot easier to stick your head in the sand and ignore things than to fire one of your most widely respected engineers because he won't help the Pentagon build Terminator robots.
Wouldn't it be more like he would leave on his own and the company would keep moving along? Why would they fire him?
I mean, right. Why would they fire him? The Pentagon isn't demanding some concrete technical action that Jeff Dean has to personally perform or could personally obstruct, so it wouldn't make any sense. That's why I don't think Google executives can realistically stop him from announcing a similar policy if he wants to.
Edit: I originally ended with "What would have happened if Germany had a nuclear bomb and America didn't?", but I think it distracted from the point I was trying to make so moving this to an edit. I'm not trying to ask "is the US the bad guy". I'm trying to ask how to balance personal anti war sentiments with the realities of the world (specifically in this case keeping up in an arms race).
Obviously, we would have had more political leverage if our leaders had started working on a treaty before they crossed enough moral red lines to start a tech revolt, but we did not elect the sort of leaders that would do that.
How about you articulate the threat from an AI powered China to people outside of AI powered China and discuss potential methods to counter that, instead of insisting capabilities be developed just in case.
>is the US the bad guy
Yes
>I'm trying to ask how to balance personal anti war sentiments with the realities of the world
Insist on open information, never surrender consent willingly and demand justification for everything. As always.
PRC isn't going to do any of the things you are asking for, and no one expects them to. The threat of an AI powered China is really obvious to me, but apparently the idea of "IP theft and industrial sabotage, but at scale with AI agents instead of human meat sacks" is hard to clearly articulate.
One method, beyond AI powered kill chains, to counter an AI powered China is of course strategic weapons.
Keep your shit patched. You dont need LLM targeting of Drone based weapons to patch your servers.
Also, Anthropic didn't actually refuse to work on all military stuff. They have some conditions, which isn't the same thing.
It is sort of like computers are amazing but can also be a privacy nightmare. Software engineers don’t help or coordinate with black hat hackers. So black hat hackers have a harder time refining their systems.
The development of the atomic bomb created a debate in American policy circles about how the US should react. Within a few years, the same debate occurred over developing thermonuclear weapons. The same question kept coming up: what if the enemy has these weapons and we don't?
Dan Carlin's position, which I happen to agree with, is that America chose wrong. It became both belligerent and paranoid to a degree that just wasn't the case before WW2. If you look up the history of regime changes at the hands of the US [2] then you can see it went into overdrive after 1945.
Part of the problem here I think is projection, the psychological phenomenon. It's also a cultural phenomenon. So, for example, when you have a historically oppressed people who are being potentially freed, the oppressors will fret that the formerly oppressed will rise up and kill them. This is projection.
We saw this exact thing play out with Emancipation. There was no mass revenge violence by the former slaves. If anything, there was more violence by the former oppressors against freed slaves and a system that excuded the violence (eg the Colfax massacre [3]).
I think nations can be guilty of this too. The US sees any other global power as a potential hegemonic, imperialist power that will dominate and exploit everyone around them because, well, that's what we do.
We also see this in how we view AI as a resource. We see it as something to be owned and gatekept such that some US company will become insanely wealthy further extracting every last dollar from every person on Earth.
So your comment belays a common fear that China will displace us as a global hegemonic, imperialist power despite there being zero evidence that China behaves in that fashion. American propaganda runs deep and the projection is strong so this will immediately cause some to say "but Tibet" or "but Taiwan" without really knowing anything any of those situations.
As just one example, the One China policy is the official policy of the US, the EU and almost every nation on Earth. "They might invade" I preemptively hear. They won't, partly because they can't but really because they don't need to. If the world already has the One China policy, why do anything? Oh and I said they can't because they can't. They don't have that military capability. If you think that, you don't know anything about war. Crossing 100 miles of ocean to invade an island with a army of over 500,000 is simply not possible.
Let me put it this way: the 17 or so miles of the English Channel stopped the German war machine despite having millions of soldiers.
Anyway, back to the point: this whole argument of "what if China does military AI?" is (IMHO) projection. If anything, China has shown that they won't allow a US tech company to control and gatekeep AI (eg by rreleasing DeepSeek). And if China gets AI, they're more than likely to use it to further raise people out of poverty and automate away more menial jobs without making those displaced workers homeless.
[1]: https://www.dancarlin.com/product/hardcore-history-59-the-de...
[2]: https://en.wikipedia.org/wiki/United_States_involvement_in_r...
[3]: https://en.wikipedia.org/wiki/Colfax_massacre
It could be about corruption. You see this in the Russian military where paid-for tanks didn't exist because the generals had pocketed the money. It could be to have an expansionist policy. It could well be to not have an expansionist policy. The point is that nobody really knows yet.
But the string I really wanted to pull at was this idea that China isn't a "rational actor". It's lazy and really a thought-terminating cliche. It's certainly no basis for analysis or policy-making. It's kind of the final boss of justification. "Putin/Saddam/Xi/Castro/Maduro is crazy". That really just means you don't understand what's going on or want to ignore the facts.
We now have 50+ years (since really the end of the Cultural Revolution) of China acting in a very rational, very intentional and very long-term way. Xi's own history here is pretty interesting. He went from privileged child (his father was one of Mao's lieutenants) to being banished to working his way up through the party's ranks over decades.
It's a mistake (IMHO) to view Xi as a singular actor, let alone as a irrational autocrat. While the PRC and the CCP might be relatively new the systems and political structures can probably be traced back thousands of years. I'm thinking particularly of the bureaucratic reforms of the Qin Dynasty some ~2300 years ago.
What cannot be ignored is that a billion Chinese have seen a massive improvement in their living conditions during their lifetimes. Almost all of the people pulled out of extreme poverty in the 20th century were because of China (~800M). So although China is authoritarian, the government is extremely popular because of that increase in living conditions. It's something that we in the West have a hard time fathoming because our living conditions have been in decline since at least the 1970s.
Your comment is very optimistic. But the quoted part reminded me of something I heard (again) about China using slave labor in their lithium mines:
https://www.state.gov/forced-labor-in-chinas-xinjiang-region...
In the Cold War, this was the correct approach, the USSR was that.
It's not American employees vs. China employees. No need to villainize China at every opportunity. Most Chinese employees are more similar to American employees than you think.
It's {top candidates who have their pick of employers} have the luxury to refuse to build this.
Mid-tier dude who can't land a job at any of the top AI companies and can code with Cursor and trying to pay their rent or medical bills will absolutely build AI for the military in return for having their rent paid.
This is regardless of whether it is in the US or China.
You can say the same for any other country... What if Japan employee refuse, but American want that anyway? What if China employee refuse, but Russia employee want that anyway?
The implication are still the same -- social, culture, jurisdiction, national interest, company interest don't share the same boundary and don't align on their priorities.
Think of it this way: mines installed in the seabed in wars past were "dumb", in that a passing ship had to happen into it. Imagine systems deployed underwater that were mobile, contained multiple torpedoes, and could strike warships with little to no warning given their small acoustic signature. It's the same principal as a mine (you leave it one spot, hope an enemy ship comes by), but the capabilities are far more advanced. If the system is not at least semi-autonomous than it might as well be a dumb mine again.
In an existential conflict no one cares about human rights. That's something for the winners to worry about after the shooting stops.
Modern tools lend themselves more to information warfare and deobfuscation.
If the world actually worked like they believe it does, if restraint were just not possible, the world would have been destroyed at least 3 documented times over.
Don't listen to them.
Soviet Union 1949
United Kingdom 1952
France 1960
China 1964
Israel 1966
India 1974
South Africa 1979
Pakistan 1998
North Korea 2006
also https://en.wikipedia.org/wiki/Nuclear_latency
also, see what Ukraine and Iran got for their "restraint".
https://news.ycombinator.com/newsguidelines.html
Probably.
https://xkcd.com/1170/
Although in the context of the parent comment, majority of Googlers probably aren't working on things directly related to controversial topics, instead they are probably working on mundane and non-external facing projects like "how do I migrate my libraries from this deprecated dependency to this other shiny new thing".
I can not believe what I am reading here, and how the single comment supporting defending one's country is so heavily downvoted. Qatar has poisoned Western online communities such that all defence of the United States is considered taboo? I don't even live in the US and I am frightened by what I see here.
The core of the issue about autonomous use of AI in mass surveillance of Americans and autonomous use of AI in automated weapons that make kill decisions. Anthropic is perfectly fine with working with the War Department and "defending one's nation".
But they are not okay with their AI being used to make a mockery of the 4th amendment and making automated kill/no-kill decisions about actual human lives.
Both companies (Google, OpenAI [0]) have defense contracts. At this point, the best course of action is to leave Google and OpenAI if you disagree with that (they won't).
[0] https://www.theguardian.com/technology/2025/jun/17/openai-mi...
Build, train, develop and maintain an AI for military if needed. When a government is scared of individuals they've clearly lost their edge.
This is just pigslop masquerading as a moral stand.
What happened to the OG Google that cared about users, prioritized honest search, fast performance, and didn't murder pages with ads?
They never removed "don't be evil", they just changed where it is in the document.
Don't be evil.
Oh, wait...
This has been going on for a very long time (read what Smedley Butler said in "War is a Racket"), but after the Iraq War, the credibility of the US should be somewhere in hell.
It's Demis they need to convince, not Jeff Dean.
Wouldn't it be more like he would leave on his own and the company would keep moving along? Why would they fire him?
I assume by red lines they are referring to a life-sized tic-tac-toe game board painted in a hallway.