> It would be foolish to deny the effectiveness of image recognizers based on generalized adversarial networks (GANs), the key neural network technology underlying LLMs
I could be misreading this, but I hope the author doesn't think GANs are used in LLMs. They are cool, though.
It’s unfortunate - he’s an author, it would have been fine to stick with an authors perspective that LLMs can’t write (which is true), as well as the copyright stuff (which I don’t agree with but he certainly has standing to give an opinion on).
But he’s made the error of trying to come at it from a technical perspective, when he clearly knows nothing about that side of things, which discredits the rest.
> But he’s made the error of trying to come at it from a technical perspective
Huh? That quote appears to be the entirety of the "technical perspective" of the post and is an aside from his larger points that you have blessed as "fine". Literally nothing in the rest of the post relies on that incorrect statement.
Let me quibble with what is discredited here, given the entirety of your point is built upon an error.
Things that are vaguely GAN-shaped might play some role in post training. Though no one would call them GANs or identify them as the key underlying technology.
There is RLHF with a model that's trained to emulate a human evaluator, but the evaluator is not really trained jointly with the main model to adapt to its distribution and tell it from real text. Though I'm sure there are some niche cases when this is done. But definitely not a prominent thing.
This is a pivot almost all of these "AI is junk and everyone using AI is a fool" pieces -- these things are pablum to many HN users -- resorts to. It's a bit of a "don't presume I'm not actually an expert on such systems, and here I hint to my deep, profound understanding of their inner workings".
It's almost always ignorant nonsense, but somehow it works on some subset of users.
Yeah. Ironically, if he used an LLM to proofread his work, it would have told him that GAN's are generative not "generalized". That LLMs are primarily built on Transformers rather than GANs. And that they're more famous for image generation rather than image recognition.
sorry but I see incorrect AI responses almost every day in things I query on Google, generally because I am often querying on something I am an expert in and I just want a linking source. There will be a good link in the result, generally, but the AI almost always messes up some elemental fact, sometimes it is because the AI has "misunderstood" what I asked which is reasonable, but most of the time the result shows that it has "understood" what I said but there will be a statement of fact that is just wrong.
Crazy to assume that an individual of vision has so thoroughly dropped the plot simply because their position is at odds with tech industry orthodoxy. If anything that should be an indication to more closely examine your assumptions.
> I will concede that using LLMs in software development is a different category from fiction, insofar as the languages and APIs are much smaller and more tightly constrained and in software you actively don't want long-range dependencies between elements (something you often do want in fiction, i.e. put a gun on the mantlepiece in part 1 of the book, pull the trigger in part 3).
It can definitely plan such things. Because agentic systems don't have to output a book all at once. The mainstream still lives 1-2 years in the past.
Agents can plan out a narrative draft, write it out section by section, maybe out of order, then tweak it, plant clues if it wants to, then trigger it in another chapter file etc. Then read the whole thing, reconsider, etc. It can iterate. I'm not saying it will be great literature, but the problem isn't the inability to create long range references.
My bigger point would be this: arguing from such mistaken technical ideas, when not having any idea about the tools is a sign that people's real problem is none of those technical limitations, but Western culture has lost the vocabulary to express their true sentiment. They are grasping for words but can't reach for "soul" and other religious terms because that would sound ridiculous in a secular modern world. But they clearly want to express that and we'd be ahead if we could discuss in such actual terms. Would you want the machine to write literature if, hypothetically it could write indistinguishably from any great writer? I guess not. Because there is something mysterious and important about the human spirit. (the same vocabulary-inaccessibility is plaguing other culture war issues too). People will rather talk about fake water consumption problems than touch the topic of the human soul or spirit.
What are you talking about… You can use the word soul without believing in a religious concept of soul. It’s done all the time in secular societies. It’s even a very common term used to describe the problem with LLM generated content, that it lacks soul or human spirit, it lacks the human meaning, intent
It's rare. It's mostly, "but it will never be able to do X" (when it can already do X), or it uses too much water (while having no idea or reference point to what amount is much), or other climate things, or non-progressive bias, or copyright (when training is fair use) etc.
Just be upfront. 1) It threatens your livelihood that people can get the thing done faster and cheaper than what they used to pay you, 2) our relationship to an AI shaped entity is spiritually risky and a esthetically off putting, and we have to talk about golems and the tower of babel and stuff. If you think I'm being incoherent, you're in for a ride. Society will have to come to terms with major major issues, not just job replacement and bad slop art.
I think it is true that people who obviously dislike AI for ideological reasons often lead with ecological concerns or doubt about the capabilities, leading to an unproductive discussion.
I don't think it ever gets old to restate that the singularity and all of the gradiose promises of a glorious 21st century have last mile problems.
But this post seems to mostly channel Harlan Ellison. And while he was amazing in his prime, with some absolutely legendary rants, he didn't age well in the end.
I do like his challenge to make LLMs useful to himself and to other authors though. Someone needs to make that happen and nearly exactly how he described it.
For some value of "work out" anywhere between "human utopia that spreads across the stars" to "we destroy everything in a ball of nuclear fire and only microbes survive", yes, it will. The universe will keep on going either way.
Roughly a third of the market is digging a $1T-a-year-deep-hole with <5yr amortization schedule on all of it and passing around the same $100B like it's in all of their bank accounts simultaneously. Yeah, shit's going to work out for sure, in all of the same ways that flying a plane into the side of a mountain has a clear ending.
Speaking as a fucker, I think the use of em-dashes has been pretentious since the 1890s. The rise of typewriters destroyed em-dashes already. The LLMs subsequently created an accidental parody of sophisticated writing.
It's like saying I am intimately involved in the printer's art. This is usually not true, unless you're involved with a very niche publisher who still has a box of sorts and inky fingers. Hyphen-minus all the way.
No it doesn't. I happened to have read "design and typography in easy steps" as a (bored) child of the 90s, but that's the extent of my "printer's art" knowledge.
I don't understand the celebration of ignorance myself. There are lots of things I don't know how to do or how to use, but I'd never dream of complaining because somebody else does.
People seem to LOVE parens, although I was taught that they are almost 100% never the correct thing to use in prose. But apparently "some fucker" will accuse me of being pretentious ¯\_(ツ)_/¯
Otherwise, I see people just use a regular dash a lot. I love the em dash myself, but of course stopped using it so I won't get accused of AI (which is sad). It's easy to type, at least on a Mac.
Parens and em-dashes do different things. A simple parenthetical is an secondary aside (it's a half step towards a footnote). An interrupting phrase is intended to (abruptly) command attention.
I mean, make the case em-dashes are overused and used lazily, I'm right there with you. I just don't see the pretense.
> I'd quite like a tool (running entirely locally on my own hardware, with no cloud service and no copyright-thieving grifters making bank on it via subscription fees) that digests a manuscript and derives a scene-by-scene timeline, that I could then query interactively and use to plan my next round of edits. Being able to map out where and when each protagonist and minor character shows up, and see a frequency distribution heat map of names in the manuscript, would be useful.
Interestingly enough, I do this with various books I'm reading. E.g, I'm currently rereading Accelerando and had Claude generate a wiki-like timeline of key events, characters, and salient plot points. That makes it easier to jump around when I want to re-read a section and grok a plot thread that is scattered across chapters. Ironically, it also exposes inconsistencies (or "hallucinations" as some might call them) in the text because the author didn't have an AI proofread the text.
You could build a tool like that and on a MBP with 48+ GB of RAM all of that can happen locally in terms of keeping your content locally. I am sort of an outliner and planner and I heavily use AI when writing documents for work. I don't see why fiction / prose would be any different? Indexing your content with a vector RAG and having a little 27B model locally could do everything he wants with some help from a big model to implement it all. It sounds like a weekend of coding for a functional prototype to me.
> Ironically, it also exposes inconsistencies (or "hallucinations" as some might call them) in the text because the author didn't have an AI proofread the text.
That's not irony, or even cute. He's literally describing a tool he'd like to use to plan edits to his writing.
While Stross is currently a neo-Luddite who doesn't believe in the Singularity, his book Accelerando is probably the best work of Singularitarian fiction ever written. It was written in the 2000s, starts in the 2010s, and covers the various decades of this century. I highly recommend it for anyone who wants to know how weird this century will be. It's also available for free online. [1]
It's one of my favorite books and the main reason I'm e/acc and many other pro-AI people love it because we view it as utopian sci-fi rather than the dystopian world Stross invented. This might be the best case of the Torment Nexus meme [2] in action.
> This might be the best case of the Torment Nexus meme in action.
At least the author seems to think[1] it is one, if in not so many words:
> And don't ever let anyone tell you that Accelerando is techno-optimistic or pro-AI; by the end of the book our entire species is extinct, surviving only as simulations/memories recalled by something arguably not alive.
1) Having met Stross, and briefly spoken to him way back when, he never actually believed in the singularity as a possible future.
2) Turns out it was the sci-fi author who was wrong all along! Fascinating to see someone confidently citing a meme, while actively evidencing the fallacy it exposes.
>he never actually believed in the singularity as a possible future.
I think my favorite moment in the book is when one of the characters (at the time, a digitized consciousness traveling at near light speed in a space ship the size of a coke can to explore an anomalous signal sent by alien life) argues that the singularity is a ridiculous concept and will never happen. I always thought that was a really clever observation on the part of Stross that our perspectives are so narrow and provincial, we often can’t see what’s happening in front of our own faces. Your comment does kind of recontextualize that for me and I sort of wonder if that was the author speaking directly to the audience?
You think accelerando is utopian sci fi!? Which characters perspective are you identifying with so strongly? They all have pretty severe societal challenges that makes me surprised to see the word Utopia.
Manfred and Amber’s lives don’t seem particularly dystopian. Human society, up until the Field Circus encounters the router and later returns home, seems pretty great. Chaotic and challenging, yes, sad for Microsoft shareholders, sure, but I would certainly rather live in a world with access to the technology they have than our own. Space exploration, extended life spans and rejuvenation, new methods of relating more closely with other human beings are all things I would love to experience.
The protagonist who believes that information should be free and that scarcity is mostly artificial due to bad economic models. And that it would be possible to implement proper pro-human capitalism or communism if we had enough compute to solve standard CS problems like filling knapsacks or maximizing flow through a graph.
Belief in the Singularity and belief in God strike me as extremely similar.
Both, I think, are extremely interesting ideas where very few theories, e.g. pro, anti, or anywhere in between, can be written off as ridiculous -- probably due to a lot of ambiguity and difficulty in defining proof.
For me, I find the Singularity very implausible due to the fact that we talk about "general intelligence" quite a bit, and the more we do, I'm finding that I don't think it exists. There are a set of low-level basic skills that I think -- if you don't have, you're not intelligent. And then there are higher level skills that vary wildly.
So, will we make AIs that can say and do very clever and amazing things; yes, this is happening. Will they be, like ascendant, or something? I would say -- yes, but perhaps not in the way we think? Like, humans made "The Bible." And that's obviously a big huge, impactful, maybe superhuman thing. But feels different from this idea of a "Singularity."
There's a reason the Singularty was called "the Rapture for nerds" before LLMs were even a thing.
You can find a lot of neo-religious behavior within the AI accelerationist community. Cults coming out of rationalism and fear of the wrath of an angry AI God, making the world right for AI's coming (and even killing the nonbelievers.) Evangelism. Promises of infinite plenty in a post-scarcity heaven. Even a persecution complex - "Luddites" are increasingly demonized, hated, accused of moral depravity for their doubt, and for not abandoning the world, taking up their prompt and following the AI.
How many people post about how AI seems to be draining them of meaning and purpose, that it doesn't live up to the hype, that keeping up with the rituals is exhausting, but feeling that the problem must be with them and not AI? That's a religious person having doubts about the nature of their faith, unable to accept that the God they believe in might not be real, and framing their issues as their own moral failing.
And all of this before the "Singularity" even shows up, and people start uploading their mind to the cloud.
It's religion, very specifically neo-Christian ideologically. Regardless of any arguments one might make about the practical benefits of using AI in any particular field, it is obviously also religion.
> In the background of what looks like a Panglossian techno-optimist novel, horrible things are happening. Most of humanity is wiped out, then arbitrarily resurrected in mutilated form by the Vile Offspring. Capitalism eats everything then the logic of competition pushes it so far that merely human entities can no longer compete; we're a fat, slow-moving, tasty resource – like the dodo. Our narrative perspective, Aineko, is not a talking cat: it's a vastly superintelligent AI, coolly calculating, that has worked out that human beings are more easily manipulated if they think they're dealing with a furry toy. The cat body is a sock puppet wielded by an abusive monster.
https://www.antipope.org/charlie/blog-static/2013/05/crib-sh...
Regardless of believing in the “singularity” or not, please explain how this sounds utopian in the slightest without borderline (if not literal) sociopathic anti-human worldviews.
I don't find it to be utopian, but in trying to imagine how, it occurred to me that maybe the idea that we even survive to be entangled in the wider universal future, maybe that is utopian? To matter.
I think there are a lot of paths where we just rock ourselves back to a regressed agency societal state, and just never have the capacity to have any affect on the wider universe. Which is maybe good for the universe, as we are certainly not a role-model for certain forms of balance. But maybe the balanced view doesn't seek to dominate and influence and become ubiquitous, after all
> Regardless of believing in the “singularity” or not, please explain how this sounds utopian in the slightest without borderline (if not literal) sociopathic anti-human worldviews.
There was a guy on here the other day trying to argue that Brave New World wasn't a dystopian novel. There are some absolutely wiiild world views out there.
I'm almost 50, but I absolutely could see how someone younger could view it that way -- precisely because the present is much more similar to than novel than it used to be?
> LLMs sprinkle em-dashes in their output because LLMs exist to stochastically emit strings of text that approximate the form of their inputs
This is wrong, LLMs don't emit text on their own, it's always a sequence of back and forth with a human, usually a long sequence. They are more like pianos than parrots. But you got to put some original spices in the soup you want original taste.
> Did I mention the "stealing" thing? This isn't hyperbole: I'm one of the parties to the settlement in theclass action lawsuit against Anthropic AIfor pirating ebooks to train their LLMs.
I see this as a power grab by authors who want now to own copyrights over all content even if not substantially similar if it is downstream of a LLM. Copyright used to be about expression not abstracted ideas. Accomplished authors like the OP want to cut the ladder an declare vibes and abstractions copyrighted so nobody else can climb after them.
I could be misreading this, but I hope the author doesn't think GANs are used in LLMs. They are cool, though.
But he’s made the error of trying to come at it from a technical perspective, when he clearly knows nothing about that side of things, which discredits the rest.
Huh? That quote appears to be the entirety of the "technical perspective" of the post and is an aside from his larger points that you have blessed as "fine". Literally nothing in the rest of the post relies on that incorrect statement.
Let me quibble with what is discredited here, given the entirety of your point is built upon an error.
It's almost always ignorant nonsense, but somehow it works on some subset of users.
The fact that you got another dead account outright calling for murder is perfectly on brand. Own it. That's who you are.
AI is so good that if you see an hallucination as obviously wrong like this one it’s a sign it’s written by a human.
> I will concede that using LLMs in software development is a different category from fiction, insofar as the languages and APIs are much smaller and more tightly constrained and in software you actively don't want long-range dependencies between elements (something you often do want in fiction, i.e. put a gun on the mantlepiece in part 1 of the book, pull the trigger in part 3).
Agents can plan out a narrative draft, write it out section by section, maybe out of order, then tweak it, plant clues if it wants to, then trigger it in another chapter file etc. Then read the whole thing, reconsider, etc. It can iterate. I'm not saying it will be great literature, but the problem isn't the inability to create long range references.
My bigger point would be this: arguing from such mistaken technical ideas, when not having any idea about the tools is a sign that people's real problem is none of those technical limitations, but Western culture has lost the vocabulary to express their true sentiment. They are grasping for words but can't reach for "soul" and other religious terms because that would sound ridiculous in a secular modern world. But they clearly want to express that and we'd be ahead if we could discuss in such actual terms. Would you want the machine to write literature if, hypothetically it could write indistinguishably from any great writer? I guess not. Because there is something mysterious and important about the human spirit. (the same vocabulary-inaccessibility is plaguing other culture war issues too). People will rather talk about fake water consumption problems than touch the topic of the human soul or spirit.
Just be upfront. 1) It threatens your livelihood that people can get the thing done faster and cheaper than what they used to pay you, 2) our relationship to an AI shaped entity is spiritually risky and a esthetically off putting, and we have to talk about golems and the tower of babel and stuff. If you think I'm being incoherent, you're in for a ride. Society will have to come to terms with major major issues, not just job replacement and bad slop art.
But this post seems to mostly channel Harlan Ellison. And while he was amazing in his prime, with some absolutely legendary rants, he didn't age well in the end.
I do like his challenge to make LLMs useful to himself and to other authors though. Someone needs to make that happen and nearly exactly how he described it.
I don't understand the celebration of ignorance myself. There are lots of things I don't know how to do or how to use, but I'd never dream of complaining because somebody else does.
Otherwise, I see people just use a regular dash a lot. I love the em dash myself, but of course stopped using it so I won't get accused of AI (which is sad). It's easy to type, at least on a Mac.
I mean, make the case em-dashes are overused and used lazily, I'm right there with you. I just don't see the pretense.
Interestingly enough, I do this with various books I'm reading. E.g, I'm currently rereading Accelerando and had Claude generate a wiki-like timeline of key events, characters, and salient plot points. That makes it easier to jump around when I want to re-read a section and grok a plot thread that is scattered across chapters. Ironically, it also exposes inconsistencies (or "hallucinations" as some might call them) in the text because the author didn't have an AI proofread the text.
That's not irony, or even cute. He's literally describing a tool he'd like to use to plan edits to his writing.
If you want to build your own, you can feed this DESIGN.md to your favorite clankie: https://pastebin.com/huRkGnba
It's one of my favorite books and the main reason I'm e/acc and many other pro-AI people love it because we view it as utopian sci-fi rather than the dystopian world Stross invented. This might be the best case of the Torment Nexus meme [2] in action.
1. https://www.antipope.org/charlie/blog-static/fiction/acceler...
2. https://en.wikipedia.org/wiki/Torment_Nexus
At least the author seems to think[1] it is one, if in not so many words:
> And don't ever let anyone tell you that Accelerando is techno-optimistic or pro-AI; by the end of the book our entire species is extinct, surviving only as simulations/memories recalled by something arguably not alive.
[1] https://news.ycombinator.com/item?id=48163630
2) Turns out it was the sci-fi author who was wrong all along! Fascinating to see someone confidently citing a meme, while actively evidencing the fallacy it exposes.
I think my favorite moment in the book is when one of the characters (at the time, a digitized consciousness traveling at near light speed in a space ship the size of a coke can to explore an anomalous signal sent by alien life) argues that the singularity is a ridiculous concept and will never happen. I always thought that was a really clever observation on the part of Stross that our perspectives are so narrow and provincial, we often can’t see what’s happening in front of our own faces. Your comment does kind of recontextualize that for me and I sort of wonder if that was the author speaking directly to the audience?
You’ve lost the plot. There is no singularity, your machine-god is not coming to save you.
Both, I think, are extremely interesting ideas where very few theories, e.g. pro, anti, or anywhere in between, can be written off as ridiculous -- probably due to a lot of ambiguity and difficulty in defining proof.
For me, I find the Singularity very implausible due to the fact that we talk about "general intelligence" quite a bit, and the more we do, I'm finding that I don't think it exists. There are a set of low-level basic skills that I think -- if you don't have, you're not intelligent. And then there are higher level skills that vary wildly.
So, will we make AIs that can say and do very clever and amazing things; yes, this is happening. Will they be, like ascendant, or something? I would say -- yes, but perhaps not in the way we think? Like, humans made "The Bible." And that's obviously a big huge, impactful, maybe superhuman thing. But feels different from this idea of a "Singularity."
You can find a lot of neo-religious behavior within the AI accelerationist community. Cults coming out of rationalism and fear of the wrath of an angry AI God, making the world right for AI's coming (and even killing the nonbelievers.) Evangelism. Promises of infinite plenty in a post-scarcity heaven. Even a persecution complex - "Luddites" are increasingly demonized, hated, accused of moral depravity for their doubt, and for not abandoning the world, taking up their prompt and following the AI.
How many people post about how AI seems to be draining them of meaning and purpose, that it doesn't live up to the hype, that keeping up with the rituals is exhausting, but feeling that the problem must be with them and not AI? That's a religious person having doubts about the nature of their faith, unable to accept that the God they believe in might not be real, and framing their issues as their own moral failing.
And all of this before the "Singularity" even shows up, and people start uploading their mind to the cloud.
It's religion, very specifically neo-Christian ideologically. Regardless of any arguments one might make about the practical benefits of using AI in any particular field, it is obviously also religion.
https://www.antipope.org/charlie/blog-static/2011/06/reality...
Regardless of believing in the “singularity” or not, please explain how this sounds utopian in the slightest without borderline (if not literal) sociopathic anti-human worldviews.
I think there are a lot of paths where we just rock ourselves back to a regressed agency societal state, and just never have the capacity to have any affect on the wider universe. Which is maybe good for the universe, as we are certainly not a role-model for certain forms of balance. But maybe the balanced view doesn't seek to dominate and influence and become ubiquitous, after all
There was a guy on here the other day trying to argue that Brave New World wasn't a dystopian novel. There are some absolutely wiiild world views out there.
Like, "cautionary" but not full on "dystopian."
This is wrong, LLMs don't emit text on their own, it's always a sequence of back and forth with a human, usually a long sequence. They are more like pianos than parrots. But you got to put some original spices in the soup you want original taste.
> Did I mention the "stealing" thing? This isn't hyperbole: I'm one of the parties to the settlement in theclass action lawsuit against Anthropic AIfor pirating ebooks to train their LLMs.
I see this as a power grab by authors who want now to own copyrights over all content even if not substantially similar if it is downstream of a LLM. Copyright used to be about expression not abstracted ideas. Accomplished authors like the OP want to cut the ladder an declare vibes and abstractions copyrighted so nobody else can climb after them.