The work done on real-time Linux has benefitted the open-source OS for years, but it was only this week that Linus Torvalds admitted its last piece into the mainline kernel. Exactly what took so long?

After 20 years, Real-Time Linux (PREEMPT_RT) is finally – finally – in the mainline kernel. Linus Torvalds blessed the code while he was at Open Source Summit Europe. […] The real-time Linux code is now baked into all Linux distros as of the forthcoming Linux 6.12 kernel. This means Linux will soon start appearing in more mission-critical devices and industrial hardware. But it took its sweet time getting here. An RTOS is a specialized operating system designed to handle time-critical tasks with precision and reliability. Unlike general-purpose operating systems like Windows or macOS, an RTOS is built to respond to events and process data within strict time constraints, often measured in milliseconds or microseconds. As Steven Rostedt, a prominent real-time Linux developer and Google engineer, put it, “Real-time is the fastest worst-case scenario.” He means that the essential characteristic of an RTOS is its deterministic behavior. An RTOS guarantees that critical tasks will be completed within specified deadlines. […]

So, why is Real-Time Linux only now completely blessed in the kernel? “We actually would not push something up unless we thought it was ready,” Rostedt explained. “Almost everything was usually rewritten at least three times before it went into mainline because we had such a high bar for what would go in.” In addition, the path to the mainline wasn’t just about technical challenges. Politics and perception also played a role. “In the beginning, we couldn’t even mention real-time,” Rostedt recalled. “Everyone said, ‘Oh, we don’t care about real-time.’” Another problem was money. For many years funding for real-time Linux was erratic. In 2015, the Linux Foundation established the Real-Time Linux (RTL) collaborative project to coordinate efforts around mainlining PREEMPT_RT.

The final hurdle for full integration was reworking the kernel’s print_k function, a critical debugging tool dating back to 1991. Torvalds was particularly protective of print_k --He wrote the original code and still uses it for debugging. However, print_k also puts a hard delay in a Linux program whenever it’s called. That kind of slowdown is unacceptable in real-time systems. Rostedt explained: “Print_k has a thousand hacks to handle a thousand different situations. Whenever we modified print_k to do something, it would break one of these cases. The thing about print_k that’s great about debugging is you can know exactly where you were when a process crashed. When I would be hammering the system really, really hard, and the latency was mostly around maybe 30 microseconds, and then suddenly it would jump to five milliseconds.” That delay was the print_k message. After much work, many heated discussions, and several rejected proposals, a compromise was reached earlier this year. Torvalds is happy, the real-time Linux developers are happy, print_K users are happy, and, at long last, real-time Linux is real.

  •  nebeker   ( @nebeker@programming.dev ) English
    27•2 months ago

    This is super interesting. I’ll admit I wasn’t even aware of this effort. Even real-time usage of Windows relies on a parallel kernel.

    This sounds like it’ll create a lot of cool opportunities and reduce friction.

      •  melroy   ( @melroy@kbin.melroy.org ) 
        9•2 months ago

        I do understand why. Due to all the heated discussions with Rust in Linux, it does makes a lot of sense to rebuild a kernel in Rust from scratch. However, it will take a long time to get on the same level of Linux (read: 30+ years).

        •  Telorand   ( @Telorand@reddthat.com ) 
          5•2 months ago

          Definitely will take time, though let’s not discount the fact that Linux came about before the internet was the internet. I don’t know if it will take 30 years, but certainly ten years or more doesn’t seem unreasonable.

          •  KeriKitty (They(/It))   ( @RiikkaTheIcePrincess@pawb.social ) English
            4•2 months ago

            There’s also free, easy access to a major open source OS and its devs! … Maybe it’s just wishful thinking that they’ll be able to kang a bunch of useful stuff off’ Linux 😅 Useful design structures or something, despite the different overall kernel design.

            I don’t even like Rust, I just like new things 😅 Am excited to see where, if anywhere, it goes. Will be glad to see some success getting past C. … Which I don’t dislike; I just like new things 😁 I hope and expect that good comes of Redox regardless of its success, which definitely is a possibility.

    •  Admiral Patrick   ( @ptz@dubvee.org ) OPEnglish
      17•2 months ago

      Probably some use cases for “regular” users. Someone mentioned music production, though that’s probably more professional than hobby.

      To my understanding, you mostly need real time performance for specialty cases where timing is absolutely critical. So I guess if you were building custom drones or custom control boards for drones, you could use real time Linux for that now since the timing could be guaranteed.

      •  moody   ( @moody@lemmings.world ) 
        5•2 months ago

        So what about 3D printing? Currently, input shaping uses an accelerometer to calculate resonances and uses that data to adjust movement and reduce flaws in the printing process. For anyone with knowledge of both fields, would this allow a built-in or add-on accelerometer to be used in real time to compensate for momentum and resonances even further?

        •  RustyWizard   ( @RustyWizard@programming.dev ) 
          2•2 months ago

          What’s preventing that from working now? If it’s indeterminate latency, then yeah, absolutely. Theoretically this will give you the ability to have a very deterministic loop around the accelerometer data, but 3d printers don’t move all that fast to begin with so having unbounded latency might not matter. The determinism we’re talking about here is on the order of tens of microseconds or less.

    •  BCsven   ( @BCsven@lemmy.ca ) 
      4•2 months ago

      My guess is not a whole lot to the average user, but it would allow for things to still respond when other things have bogged down resources. I am assuming real world applications would be industry like a machine safery stop should always have a quick turn around, and not be delayed by harddrive writes. But may like how they write special OS code for spacecraft where sending and receiving instructions on board has special states and if response isn’t given in timely manner the system can recognize, so malfunctions are prevented. There was an artivle/podcast somewhere abouy how this all had to work in realtime and not be queued waiting

  •  noddy   ( @noddy@beehaw.org ) 
    3•2 months ago

    Anyone know what real time means here? Does it mean that sleeping a thread is more accurate (as in the thread is resumed at the correct time after calling sleep)? Or is there an API that implements some functionality for something that should run in real time?

    •  lemming934   ( @lemming934@lemmy.sdf.org ) 
      7•2 months ago

      I’m not an expert, but have used a real time kernel for scientific research, using rtxi. My understanding is that the real time threads allows the computations to occur in a deterministic amount of time. This is necessary if you want to quickly respond to changes in personal membrane voltage with injections of current, and don’t want it to sometimes take longer to calculate how much current to inject.

    •  gandalf_der_12te   ( @gandalf_der_12te@lemmy.blahaj.zone ) 
      4•2 months ago

      RT systems are all about responsibility.

      When you’re driving a vehicle, the vehicle has a lot of CPUs inside that help the steering.

      When these CPUs pause - for even 0.4 seconds - something bad can happen. Somebody can get seriously injured, or even killed. So it is very important that these systems never freeze or pause for any reason. That is what RT is all about.