Ask HN: What are your worst war stories bringing agentic applications into prod

For a bit of context, I’m currently creating a team of AI agents at work to generate reports by fanning out into a large amount of subagents to process a large amount of transcript data. When the analysis fails mid-way because of some individual step like an API call returns an error or the machine is out of memory, it would create cascading errors that break the entire generation with almost no visibility. I’ve just spent the past month rewriting the individual jobs as durable execution jobs on DBOS but just wondering if there are better solutions out there and if others encountered similar issues? And then there is the issue to reflect back the progress to the users which I’ve just been coding ad-hoc honestly…

When an agent fails at step 9 of 12, how do you handle that?

Roughly how many engineer-weeks have you sunk into agent infrastructure (durability, monitoring, human-in-the-loop, live UI) vs. the actual agent logic? Curious if my ratio is normal.

For those who built this stuff in-house: was it ever a build-vs-buy conversation? What would a tool have had to do for you to buy instead of build?

Do you currently pay for anything in your agent stack (LangSmith, Temporal, Braintrust, etc.)? What made that one worth a line item when others weren't and should I look into it too?

8 points | by yaoke259 20 hours ago

6 comments

  • eb0la 3 hours ago
    Be careful about the culture. Higher management had a massive panic because "AI stopped working" in a product we make. They started an all-hands meeting. Everything worked except the AI responses. Turns out we just ran out of credits. The bad part is we told everybody we would run out of credits in less that 3 days. Nobody cared until we had production outage. Now procurement is complaining about token usage.
  • Raed667 9 hours ago
    Definitely similar experience, building the "boring infra stuff" around it is a time-sink no one prepares you for...
  • vgudur297 19 hours ago
    [flagged]
  • frictasolver 18 hours ago
    [flagged]
  • Kwouz 8 hours ago
    [flagged]
  • cyanydeez 11 hours ago
    sounds like busy work you give to someone you dont expect to be here in 6 months.
    [-]
    • ahahs 8 hours ago
      lol I was hired to build agents. It's all some companies can think and/or dream about doing right now. This is a F100 company too. The feeling is "how can we turn everything into some sort of AI tool, but hey lets also change the scope constantly and use this week's #latest technology to do so".