GitHub Incident with Actions, Pages, and Webhooks

(githubstatus.com)

56 points | by damontal 1 hour ago

16 comments

  • peri-cl 32 minutes ago
    The shape of this HN duplicate detector actually works as a GitHub uptime chart:

    https://news.ycombinator.com/from?site=githubstatus.com

    It's driven by aggregated user complaints, so it's pretty high-quality. Comment counts are a proxy for "how bad was the outage" + "how long was the outage thread topical for".

  • montroser 29 minutes ago
    It bugs me how there can be an acknowledged live incident happening today with Actions, and yet down the page where it shows last 30d uptime for actions, today's little rectangle is a chipper green.
    • hoppp 24 minutes ago
      It's hard coded probably.
      • deathanatos 5 minutes ago
        If you hover over the green bar for today, the tooltip knows there's an incident.
  • pksunkara 52 minutes ago
    I think all they have to do is stop giving free CI to everyone and this reliability issue will be solved immediately.

    Disclaimer: I am building a code forge myself at https://juju.bi

    • m4rtink 18 minutes ago
      Isn't that why most people are on GitHub in the first place ? Which then, Windows style eventually turns into people wanting to use GitHub also for company work, where the real money comes from for GitHub.

      Without free CI in GitHub, you might as well say:

      "they can just shut the whole thing down and this reliability issue will be solved immediately"

      • jjice 1 minute ago
        Maybe it is, but I know that lots of us used GitHub way before Actions launched. I think even if Actions wasn't free, GitHub would still have a massive advantage because it is the most well known place to host code, especially by less technical users (which is where I see a lot of finger pointing going with the increase in GitHub usage).

        I would not be surprised if a serious amount of this usage is not user-intended and instead the LLM just inserts some sort of CI job off the bat and people roll with it.

    • willio58 41 minutes ago
      I've heard this claimed dozens of times. I wonder if anyone at Github can confirm this theory and maybe give insight into why they haven't just done this?
      • tmpz22 36 minutes ago
        Im guessing the lock-in is strong enough that they can eat the reputational hit while getting meaningful training data by subsidizing the free users.

        Github has a decade of positive reputation to deal out, I don't see Grok or OpenAI jumping in and competing with that.

    • oleg_antonyan 32 minutes ago
      free CI on github is so ridiculously overpowered it's hard to believe they still provide it
  • senderista 6 minutes ago
    Lesson for every infra provider: physically separate your free tier infra from your paid tier infra.
  • Nedomas 24 minutes ago
    Self-hosted GitHub Actions on AWS spot are still working.

    After many outages we had to build our own free runner https://github.com/supercorp-ai/superci

  • m4rtink 21 minutes ago
    I'm sure there is a good meme to use for such a frequently occurring problem, but it would be too over used by this point.
  • SoftTalker 31 minutes ago
    Nice graphic on that page of a bunch of cartoon workers tending to what looks like a Rube Goldberg contraption. It's all just lighthearted fun and games.
  • enjoyyourlife 12 minutes ago
    Giving Github more attention
  • nerdypepper 48 minutes ago
    like clockwork! right now on the frontpage:

    The federated code forge (https://news.ycombinator.com/item?id=50305573)

  • uneekname 1 hour ago
    My Actions are stuck waiting :-/
    • jefb 59 minutes ago
      same here
  • talon8635 47 minutes ago
    No way I don’t believe it
  • rvz 55 minutes ago
    Last time these intermittent incidents happened was days ago and for this exact same incident was 11 days ago. [0]

    After everything announced in GitHub Universe and with that to improve the reliability of GitHub overall, yet we have another incident afterwards.

    It doesn't matter what GitHub does, only you can decide for yourself to self-host rather than going all in on GitHub and it continues to go down.

    Predicted this 6 years ago and self hosting has aged perfectly well. [1]

    [0] https://news.ycombinator.com/item?id=50154406

    [1] https://news.ycombinator.com/item?id=22868406

    • tempest_ 48 minutes ago
      We use a self hosted gitlab-ce instance which has pretty high up time running on a 10 year old server (and its backup)

      Takes a few minutes a month to update and babysit. We only basically use the CI and the repository functionality though so gitlab has been "feature complete" for us for a while. Probably I we could swap to something like gitea or forgeo but the gitlab CI seems like a nice balance between powerful when it needs to be and simple otherwise for our use case so there is nothing driving us to swap.

    • kriztw 49 minutes ago
      Yep. Ironically this downtime is currently delaying our ongoing migration to self-hosted forgejo...
  • mosesman 48 minutes ago
    [flagged]
  • parkerhiggins 48 minutes ago
    [dead]
  • OsamaJaber 58 minutes ago
    [dead]
  • dc3k 58 minutes ago
    Another day another GHA incident