Ask HN: Are systems ready for the first negative leap second?

It’s been 10 years since we had the last leap second and it looks like we will get the first negative one soonish. Are systems ready for that?

78 points | by Asmod4n 5 days ago

22 comments

  • _alternator_ 20 hours ago
    You may be relived to know that Unix epoch time does not have this problem. But you may also be horrified to know why.

    First, epoch seconds are not the numbers of seconds since 1970/01/01. This is a lie we tell to children. Rather, epoch seconds are the number of days since 1970/01/01 * 86400 plus the number of seconds since midnight.

    Leap seconds, to epoch time, don't exist. Or maybe they are double counted. Or maybe we smear them over 12 or 24 hours (but which 12 or 24 hours depends on whether you are Google, Microsoft, or Oracle; I can't even make this stuff up). The point is, it's not defined, and this means implementations do it differently.

    A negative leap second might be easier though. The spec suggests (though lack of speaking) that a correct implementation will just skip it since number of seconds stays less than 86400 for that day. But of course the smear-organizations still smear it.

    So what if you really want to know how many seconds were between two different epochs? Subtracting epoch seconds is wrong because you need to correct for the number of leap seconds between the two times.

    And the smears.

    • FabHK 19 hours ago
      Indeed. I like to say, we would like three things to be true:

      1) Every day has 86400 seconds.

      2) Every day is from noon (sun exactly above) to noon (sun exactly above).

      3) We use SI seconds.

      You can't have all three. Pick any two:

      1, 2, not 3: What you describe. Day has 86400 seconds, we keep in sync with the sun, but we tweak the seconds a bit. There are different versions, like epoch time, or UT1.

      1, not 2, 3: Every day has 86400 SI seconds, and we slowly go out of sync with the sun. That's TAI.

      not 1, 2, 3: We use SI seconds and keep in sync with the sun +/- 1 second, but need to add/drop seconds occasionally. That's UTC.

      https://en.wikipedia.org/wiki/Universal_Time

      https://en.wikipedia.org/wiki/International_Atomic_Time

      https://en.wikipedia.org/wiki/Coordinated_Universal_Time

      • nick238 13 hours ago
        The length of a noon-to-noon day (synodic day) varies from +29.9 seconds near the winter solstice to −21.3 seconds near the autumnal equinox[1]. If you account for the seasonal changes, you get closer to the ideal solar day which only deviates by milliseconds from 86400 s, but that deviation does pile up and "forces" leap seconds. I say "forces" because it's a legal requirement that the time tracks the sun.

        Going to leap hours I think is a sufficient kick of the can down the road, one is only needed every few centuries. Pretty sure something'll happen in the next 600 years that obviates the need for that though (nuclear war, asteroid, technological singularity, need for a unified solar system time...), so us hack programmers can assume all three of those things.

        [1]: https://en.wikipedia.org/wiki/Synodic_day

        • throw0101a 5 hours ago
          > Going to leap hours I think is a sufficient kick of the can down the road, one is only needed every few centuries.

          The drama of rolling out the Gregorian calendar to replace the Julian so that the seasons went back to where they 'should' be shows how convoluted things could get in coördinating things.

        • DeluluDon 12 hours ago
          Legal only in the sense that the requirement hasn't changed since the Romans invented the concept of time.

          If you really want to think about it, were it not for them, we wouldn't have many of the things we have today.

          Philip K. Dick was correct, the Roman Empire never truely fell.

      • duskwuff 16 hours ago
        Unless you're an astronomer, you probably don't care about condition 2. Many countries still shift wall time around by an hour twice a year for DST - if people are okay with that, solar time being offset by a few seconds is insignificant.

        Even for astronomers, I suspect that preserving conditions 1 and 3 is probably more valuable than 2. It's easier to correct for the uneven rotation of our planet than for an uneven timescale.

        • Telaneo 13 hours ago
          I agree. The difference in how much we're out of sync with the sun is tiny and also completely inconsequential in the vast majority of cases, especially compared to the trouble of the alternatives. By the time the difference is relevant or noticeable, I sure hope we have a better way of tracking time.

          If anything, it's probably for the better to completely disconnect the measurement from it's old standard, the same way imperial measurements are now defined using SI units as a baseline, and how 1 AD isn't actually the year of Christ's birth. It's OK that the actual measurement is off from what it was once based of, so long as we all agree how we're defining our units.

        • Charon77 13 hours ago
          Condition 2 are pretty much hard to get. Happens everytime in the equator, happens only twice a year in most countries, hopefully never happens in the n/s pole
    • SahAssar 20 hours ago
      Yes, it's wrong. But we have also agreed to all be wrong in the same way. Except during the specific day that we are wrong in different ways.

      Correction: We are mostly right, most of the time, but wrong in ways most people don't notice except if they try to talk while everyone is wrong.

      Clarification: Human perception of time is not understandable, and the machine abstraction even less so.

    • stingraycharles 14 hours ago
      And this, ladies and gentlemen, is why you use a proper datetime library rather than doing epoch arithmetic yourself.
      • fragmede 5 hours ago
        "How hard could it be?"
  • Bender 5 days ago
    Google's proposal is a smear. [1] Most time servers do not use smear. No idea what behavior it may introduce in places where sub-second time is important. Curious if all these bugs [2] were fixed specifically to deal with going backwards.

    [1] - https://developers.google.com/time/smear

    [2] - https://rivassec.com/leap-second-chaos-2012.html

    • eqvinox 5 hours ago
      Leap smearing is amazing exactly until the moment you need to know whether you're leap smearing (because your seconds are 1.0000116s long, or the reverse). That instant, it turns from amazing into the worst godawful mess you can imagine.

      Leap seconds can be stored and communicated, if you need it you can do the work and get it. Leap smearing, you're shit out of luck, especially if you're a developer and need to deal with systems operated by others.

      If you're sure you'll never cross that moment and need to know you're in a smear, go ahead, it's great. Otherwise test yo' stuff with leap seconds and get bugs reported & fixed.

    • yen223 4 days ago
      The brilliant thing about the smear is that it distributes the new second across each second of the day, so that each second differed by 1/86400 seconds, well within the margin of error for NTP.

      As far as the computers were concerned, nothing was different.

      • toast0 4 days ago
        The less brilliant thing about the smear is that if your ntpd syncs from smeared and unsmeared servers, the results aren't great.

        It would have been better if they would have kept the time on the wire accurate or added mandatory protocol stuff to avoid confusing things for ntpds configured to different leap second handling.

        • lozf 4 hours ago
          Better still, if Unix time tracked TAI (monotonically increasing every SI second), and Leap Seconds were handled in a distributed database much like time zones - possibly even the same one we all use anyway.
        • not-a-llm 22 hours ago
          if you need below 1 millisecond time accuracy, probably you know what you are doing and you wont mix NTP servers (and I think you need PTP for that)
          • lesam 18 hours ago
            Until you work in financial markets and you want to measure latency vs a bunch of other servers you don't own, all of whom pick different smears (and some of whom aren't sure in advance which smears).
            • not-a-llm 18 hours ago
              financial markets dont trade when the smears happen (NYE)
              • lesam 5 minutes ago
                Early in my career, I had to babysit our FX quoting app through the July 2015 leap second.

                And some things do trade 24/7 (though I wish they wouldn't, one auction per minute during US east coast business hours would be totally sufficient for most US markets).

              • _alternator_ 16 hours ago
                They can happen mid-year too. They usually don't for this reason, but they can.
              • weakened_malloc 16 hours ago
                Depends on the market. Crypto and prediction markets wont stop.
          • chaps 21 hours ago
            You'd think so. Legacy servers that've been running for ten years really puts a hitch in that.
        • ikiris 21 hours ago
          User error
    • ikiris 21 hours ago
      Yeah 90% of the time the simple solution is just use Google time and these problems are smeared away because they got burned enough internally they did it themselves
  • wmf 5 days ago
    Systems are absolutely not ready. Leap seconds are a bad idea and negative leap seconds are worse. Just don't do it and let the drift cancel out.
    • voxic11 21 hours ago
      Explain why negative leap seconds are worse? Intuitively it seems like normal leap seconds would cause way more issues, or at least not more.
      • bluefirebrand 18 hours ago
        I am not an expert in Time systems but I imagine the problem is similar to daylight savings problems. With a leap second it's like putting the clocks forward an hour. With daylight savings, In your time series database you suddenly go from 0059 to 0200. 0100-0159 are missing, but that's fine. Most systems are fine with missing data, because we assume there will be occasional outages and such

        When you go the other direction there's a potential problem. You have records with times from 0100-0159, then the daylight savings shift happens and you repeat 0100-0159. Now you have records with (apparently) duplicate timestamps.

        I think it's probably the same problem with leap seconds and negative leap seconds, but I'm not 100% sure

        • voxic11 11 hours ago
          Negative leap seconds mean we skip a second, normal leap seconds are the ones where we repeate a second.
          • bluefirebrand 2 hours ago
            Huh. TIL

            That's really unintuitive. Thank you for letting me know

    • toast0 4 days ago
      negative leap seconds aren't too bad. jumping forward a second won't lead to a time loop like jumping back did on several systems (some twice!)
      • lokar 16 hours ago
        Unless you have something scheduled for that second.
    • knorker 21 hours ago
      What's worse about negative leap seconds? The "experienced" time by systems will just look like they froze for a second. Added leap seconds are worse, surely, as time goes backwards.
      • gmuslera 21 hours ago
        Suppose being charged by time used to run your tasks. Then your task takes -1 second, you will be charged zero, 1 second, or 18 quintillon seconds?
        • thomashabets2 1 minute ago
          That's exactly what I see happening when a leap second is ADDED.

          A leap second is added. The system goes "oh shit, I'm a second ahead", and subtracts a second. And that's how you get a negative duration and exactly this problem.

        • OneDeuxTriSeiGo 18 hours ago
          That shouldn't be the case?

          When a negative leap second is applied it just means that 23:59:59 is skipped. The seconds go 56 57 58 0 1 2 3.

          vs a positive leap second where time goes 56 57 58 59 60 0 1 2 3.

          So if you are billed by time and it's tracked via a timer then your time is still accurate but if it's tracked by reading your start and end time then your billing will just read an extra second.

          Positive leap seconds are harder as they require you to reason about a clock that includes a 60th second but systems view negative leap seconds as if nothing happened for an entire second.

          Now there are some cases where this runs into issues. If you have jobs that fire off based on the time then you can get jobs running a second early. Normally this shouldn't be a problem but in some cases this could result in contention of resources.

          You could also accidentally trigger watchdogs but any watchdog worth its salt will use a real timer not wall time.

          But overall negative leap seconds are way easier as they are still monotonically increasing and don't require downstream systems even understanding the concept of a leap second in the first place.

          • hunter2_ 16 hours ago
            > just read an extra second [...] accidentally trigger watchdogs

            Well, this seems to touch heavily on the concern: if your job normally takes 0.1s and costs $100, it could accidentally be considered to have taken 1.1s and therefore cost $1100. This is quite contrived, but if you consider that some people put very infrequent cron jobs at the top of the hour (which isn't typically the best idea, but I digress) and it might start just ahead of what the billing system considers to be the top of the hour, theoretically there's something to this concern.

        • flexagoon 20 hours ago
          If you're calculating time that a task took by subtracting two UTC datetimes instead of just using a monotonic timer or at least unix timestamps, you're doing it wrong either way
          • gmuslera 5 hours ago
            If systems are not ready, then there is plenty of room of systems doing it the wrong way.
  • SeanAnderson 21 hours ago
    I don't think we're going to do any negative leap seconds.

    https://www.yahoo.com/news/science/articles/international-ti...

    I think we're voting to change to a leap hour in early 2027. Or I'd assume we're going to go that route instead of continuing to entertain the tech nightmares.

    • ssl-3 21 hours ago
      We've on track to do something different before the end of 2035: https://en.wikipedia.org/wiki/Leap_second#Phase-out_and_futu...

      Shifting to a leap-minute feels close-enough to me: We might get one every 50 or 100 years. A lot of us reading this today will never live to see a leap-minute, but it's close enough that we'll still have it collectively in-mind when it it needs to happen. (And if we screw it up at that time, the outliers will only be off by a minute. Not so bad.)

      A leap-hour, meanwhile: That kicks the can so far down the road that we'll probably lose track of it completely. ~6,000 years is a very long time; society will be a very different thing by then. Leap-hours seem to me to be moral equivalent to the "fuck it, let's just give up" option.

      edit: accidentally a word, and fixed an off-by-an-order-of-magnitude error on the approximate years required for a leap hour

      • DSMan195276 20 hours ago
        > A leap-hour, meanwhile: That kicks the can so far down the road that we'll probably lose track of it completely. 600 years is a very long time; society will be a very thing by then. Leap-hours seem to me to be moral equivalent to the "fuck it, let's just give up" option.

        Something to consider: The use of timezones in mostly 1-hour increments over inconsistently placed areas means that the vast majority of people are already living many minutes off from the "actual" time at their precise location, in some cases even more than an hour. "Giving-up" implies that this is something important worth maintaining, where-as for the vast majority of people they gain nothing from leap seconds or even leap minutes. The most important thing for people is simply that everybody agrees on what time it is, which is easier when leap-Xs aren't done.

        That said it's also probably true that a leap-hour would never actually happen, but that's not some big issue. By the time we got to the point that a leap-hour would make sense people would have already adjusted their habits and it probably wouldn't be worth it.

        • schoen 20 hours ago
          An extreme example is Xinjiang, where solar time can be more than three hours off from civil time (due to the PRC policy of having only one time zone for all of China).
        • ssl-3 20 hours ago
          Indeed. On a long-enough timeline, people will adjust to whatever it is that the sun is doing regardless of what the clock on the wall says. That's the way it has always been.

          So when we're talking about one second every once in awhile, I'm not sure that [effectively] giving up by adopting leap hours instead of leap seconds isn't the right option -- as long as we agree to do it uniformly.

      • Telaneo 13 hours ago
        > Leap-hours seem to me to be moral equivalent to the "fuck it, let's just give up" option.

        Sometimes, doing nothing is the right option. Sometimes, giving up is the right option. I believe this is one of those times. The alternatives seem worse to me, and the fact that it's even being considered by the curators of time would seem to indicate it's not a completely invalid option.

      • repsilat 5 hours ago
        One way of thinking about ignoring leap seconds is that it's like letting the effective prime meridian drift east/west. And a "leap hour" is kind of like a timezone change. In fact, you could just not have a centralized/coordinated leap hour, and let individual jurisdictions change their own timezones as they wanted, if they wanted noon to be closer to the daily solar peak.
  • aomix 22 hours ago
    Not responsible for those systems.

    The last time this came up I thought “smearing” the second over the course of a day kind of solved the problem a discrete +/- 1 second suddenly appearing on the clocks.

    • repsilat 5 hours ago
      I had an issue in a CI pipeline once where clocks on two different machines drifted out of sync, and it looked like an HTTP request went back in time (it had some validity from now until now plus some delta, and the second machine thought it arrived "before now"), validation failed, pipeline failed.

      An unusual case, fragile design, but indicative of potential issues with more interesting timekeeping choices.

  • ls65536 5 hours ago
    Fortunately, in the past couple months, the likelihood of actually needing a negative leap second in the next few years has been trending down. This is based on measurements and predictions done by the IERS, the data for which they publish weekly in their Bulletin A [0]. I've been tracking this data for a while, and their DUT1 predictions have been trending more negative over the past few months, suggesting that the anticipated negative leap second is likely to be delayed, or it may very well not happen at all.

    [0] https://datacenter.iers.org/data/latestVersion/bulletinA.txt

  • al_borland 4 days ago
    If we have positive and negative leap seconds, why are we doing anything at all? 1 second forward, just to go 1 second back 10 years later…
    • yen223 4 days ago
      I don't think we can predict ahead of time whether we'll need a leap second or not

      If the question is "why bother syncing time to Earth's orbit around the sun at all", I don't have a good answer for that except at this point, it's tradition.

      • jMyles 22 hours ago
        We can set some rasterization floor, such as like 3 minutes or something, and live with that.

        Correcting for a 3-minute offset every few millenia seems easier than trying to understand all this minutia about wobble and aquifer management and whatever else goes into a leap second.

        • StevenWaterman 21 hours ago
          I'm pretty sure that's the plan. Currently they're legally bound to keep it within 0.9s of solar noon (or something like that) but in 2035 it's changing to +-1 minute, which basically kicks the can down the road for another century or so

          I say we let it reach 15 minutes then countries can solve it themselves by shifting timezone by 15 mins. Since making sure solar noon matches noon on the clock, is the point of timezones existing in the first place

        • repsilat 5 hours ago
          > every few millennia

          We'll feel awful silly having made any leap seconds if the robots kill us all in the next decade.

          More seriously -- all of today's computers and probably all of today's software (not to mention many scientific and governmental institutions) will be gone in 500 years. Does it make sense to plan too far ahead?

    • happytoexplain 22 hours ago
      The Earth's rotation is slowing down in the long term, hence the need to adjust. In the short term (where 10 years is "short") it can speed up or slow down, but long-term it is slowing down.

      Note that this is not an argument for leap seconds - just my understanding of their rationale.

  • dgrin91 22 hours ago
    I wonder how many systems actually care? I presume the core NTP servers handle this well, and most systems just feed off of that?

    GPS satellites probably handle it well too, but maybe some consumer or even industrial GPS receivers don't? Maybe some trading systems? I don't think crypto systems care too much.

    • netsharc 22 hours ago
      I wonder if there's things that run 24/7 and need to be monitored.. e.g. if you have oil flowing through some pipeline at 100 liters/second, one particular minute will have 6100 liters, and someone will want to get paid for that 100 extra liters.

      But the meter/reporting tool would say "Well, we measure every second, and the meter reported a constant rate of 100 liter/second, and as we know we have 60 seconds in a minute, so we got 6000 liters!".

      Or a database for "measurements every second for this minute" that has 60 fields, and don't have a field for the 61st measurement.

      • blaze33 21 hours ago
        I once worked with smart meters for electricity consumption that do run 24/7. Leap seconds were not an issue but we had a very similar one now that I think about it: DST shenanigans!

        Like how much time is there between 2 and 3 am? Usually one hour, but sometimes 2 and also sometimes 0. It looks simple at first but it creates a lot of edge cases that your business logic now needs to handle and we had a fairly complex system for this.

        • adev_ 18 hours ago
          I can relate:

          My son is born at 02h30 right DST change time.

          It tooks 3 f*cking days to get his birth registered in the Hospital new birth registration system.

          The hospital (at the time) just got digitalized and most systems where not able to agree if he was born after or before DST changes.

          PS: I guess he is born fated to be a software QA later on.

      • mike_hock 6 hours ago
        No matter what the 100 liters of mystery fluid are worth, if the pump is running continually, the billing for that day would only be off by .0011574%, so either side can probably afford to absorb the loss.

        Though if it's valuable enough it could of course be "worth" paying top dollar to have engineers solve a ludicrous non-issue.

      • imhoguy 18 hours ago
        Now imagine blood pump during surgery.
        • enneff 18 hours ago
          I’m horrified to imagine a blood pump or any life support system being network connected, or to rely on the precise time of day in order to function correctly. Accurate time keeping for anything in this realm can and should be done without a network.
    • chaps 21 hours ago
      Systems definitely care, especially in finance and trading systems.

      Was involved in rolling out a large NTP annealing patch about ten years ago. We missed a couple and the effect was largely overall muted, but we did have one server with an old JVM hard crashing the server right at the second shift.

      That specific server was already hobbling along so it wasn't a surprise. But it required a bit of firefighting.

    • TrueDuality 22 hours ago
      The problem frequently crops up in order-deterministic systems that use time and haven't accounted for the edge case of all the vagaries related to time-keeping of this being only one.

      I've worked on some extremely sensitive systems that had thousands of lines of C dedicated to handling skewing a time gap across an hour-per-second when necessary. I know that code assumed only "missing" time (jump-forwards)... Even knowing what I know as a developer now, if I was re-implementing that system from scratch and didn't have this top-of-mind, I'd bet I would miss "overlapping" or "duplicate" time entirely.

      Maybe that is more of a me problem than others, but I'd bet there are some safety critical systems out there where the responsible engineers, QA, and specs all missed this as well.

    • wat10000 22 hours ago
      GPS uses its own time base that doesn't do leap seconds. For display purposes, the leap second offset to UTC is transmitted to the receivers and added to the displayed time if needed.
    • not-a-llm 22 hours ago
      traditional markets are closed when time changes happen
  • BadBadJellyBean 21 hours ago
    Since the whole leap second system will be phased out by 2035 anyways I doubt that anyone will test it. No need to rock the boat over a second.
  • thomashabets2 21 hours ago
    I'd say yes we are ready. gettimeofday() should never be used to measure time[1], but at least with a negative leap second it's monotonic.

    We'll just get some poorly coded stuff claim that an operation took 1100ms instead of 100ms. Not great, but not -900ms.

    Well, I say that, but per my link here F5 load balancers at least used to keep track of TCP connections using gettimeofday. And it's annoying that libpcap delivers metadata in wallclock time.

    [1] https://blog.habets.se/2010/09/gettimeofday-should-never-be-...

  • d00d0ff000 5 days ago
    NTP.

    By any other standard, most manually set clocks are up to a full minute off all the time.

    • subscribed 4 days ago
      Yeah, but we're thinking of systems where nanoseconds matter.

      MiFID 2 alone forces sub-μs precision. Million times less than the leap 1 second.

      NTP minute away is good for displaying date on the workstation, not for many of the devices that are critical to the modern world.

      • dmurray 21 hours ago
        MiFID 2 does not require nanosecond accuracy. It's something like 100 microseconds in the strictest case.

        Some MiFID reports require microsecond or perhaps nanosecond precision, but that's really just a formatting requirement "please write your timestamps with six figures after the decimal point."

        • subscribed 1 hour ago
          I said, quote "MiFID 2 alone forces sub-μs precision", and not that it "requires nanoseconds accuracy". Put the strawman away.

          It's not "something like 100μs", it's 100 times less: 1μs.

          From a convenient QA:

          > Algorithmic and high-frequency trading (HFT) firms must timestamp to 1 microsecond (μs) accuracy, synchronized to UTC. For venues and systematic internalisers executing algorithmic strategies, this microsecond-level precision is non-negotiable. Other investment firms trading on venues must achieve 1 millisecond (ms) accuracy.

          No, precision means actually precision, not a length of a number.

      • cyanydeez 4 days ago
        dont most systems that rely on sharp timing simply manage it themselves.
        • subscribed 4 days ago
          Yesno.

          Sure they have their own time servers fed from the GPS, but they need to be _accurate_ in relation to the world.

          But timestamps used by companies forced to use very accurate timing must be synchronised to UTC.

  • jaggederest 20 hours ago
    I'm rooting for pure UTC never adjusted to actual sidereal days. Let me wake up at 1600 and go to sleep at 800, it's fine.
  • deepspace 22 hours ago
    Wasn't there a recent discussion here, where it was pointed out that leap seconds are about to be phased out in less than 10 years' time? I would be extremely surprised if a negative leap second was implemented before then, given that IERS already balked at doing that several years ago.
    • ls65536 5 hours ago
      There's an agreement to do something about leap seconds before 2035, either by allowing DUT1 to exceed its currently specified bounds (this is most likely) or by agreeing to simply not insert/delete leap seconds going forward (this is less likely). For certain bounds on DUT1 and at some timescale, these would be practically the same in terms of their effects on civilization, computing, etc. In any case, a decision will almost certainly be made much sooner than 2035, perhaps even before the next leap second insertion/deletion would need to take place under the present rules.
  • ak217 13 hours ago
    No and they shouldn’t be. Leap seconds were never a great idea and they become an increasingly worse idea with every passing year. They are a great example of a standards making decision made without sufficient negative feedback from its implementation costs.
  • throw0101a 20 hours ago
    The FreeBSD folks test their code for these things and it works:

    * https://lists.freebsd.org/pipermail/freebsd-stable/2020-Nove...

    Of course third-party userland code understanding what happens is another thing.

  • OsrsNeedsf2P 21 hours ago
    Don't systems regularly have this issue when they do time server syncs?
  • epsteingpt 17 hours ago
    This is gonna be so much better than Y2K!
  • Faha_2011 11 hours ago
    [flagged]
  • rezvovmobile 4 days ago
    [flagged]
  • its_not_a_bug 16 hours ago
    [dead]
  • m_m_carvalho 20 hours ago
    [dead]