2w ago

A noticeable difference in kernels?

I have the following kernels installed:

linux-zen (Zen)
linux-rt (RealTime)
linux-hardened (Security Hardened)
linux-lts (Long Term Support)
linux-tr-lts (Realtime LTS)

When I boot up, I try the different kernels from time to time just to see if anything interesting happens. It never does.

My question: How do I actually physically notice the difference between these kernels? If I use RT, does Firefox spawn quicker (in my testing, no, not really)?

What are some use cases when I can really see the difference in these kernels?

22 comments

Realtime is not about being fast, it's about time guarantees. It helps with or is required for workloads that require realtime, which I think includes audio production, but might also be helpful for things like controllers etc. where you need to make sure incoming data is processed in a guaranteed time or else fail. Browsing the web isn't part of these, so an RT kernel will most likely be a hindrance.
- I think realtime is on mainline now since 6.12 though so anyone with at least 6.12 should be able to use rt functionality.
- Ok this discussion reminds me of a gripe: Is there a Linux distribution or kernel that prioritizes the UI over everything else, including an OOM situation?
  I’ve never had (modern) Windows kernel panic on me, or completely slow to the point I can’t get Ctrl+Alt+Del registered.
  Let me know if I’m just using Linux stupidly though…
  
  I've definitely had Windows hard lock before and stop responding to the keyboard, from Win95 all the way to Win10. I have no experience with Win11 so I can't speak for that, but all others have situations where it can happen.
  In fact, Windows is bad enough that the disk usage being high can cause the system to stop responding until it's done and drops back down.
  
  There's a systemd OOM service that can be setup. I use CachyOS and they have it as a checkbox to turn it on or off.
  
  I thought that's what Zen was for
  
  Yeah I've had multiple times where a silly process (usually a game running via WINE) will shit itself and lock the whole desktop. it's my only gripe even if it's rare
- Yes, aircraft for example. If the pilot says "gear down", the gear must go down in short order. You can't say "well I think I'll check the airspeed sensor a few times first".
  
  aircraft for example
  due to DO-178C requirements, Linux can't be used there... yet.
- I think banking is the old standard example of real time kernel needs. Money goes in, numbers go up, no time to explain the tide.
  
  I don’t know if you’re being serious, but I can confirm from my time at as a developer at a banking software company, we didn’t use a hard RT OS even for like Mosler or Hitachi high speed check sorters. Just fast C++ code. (On Windows XP still, when I left in 2016)
  (Work load is basically: batch of checks is loaded into an input hopper, along with check sized pieces of paper which are headers and footers, machine rapidly scans MICR lines and they go flying towards output pockets, and our code has something like 20 ms to receive the MICR data and pass back a sorting decision.)

My question: How do I actually physically notice the difference between these kernels?
Generally, you don't. You can look for some benchmark to try and find a difference between them, but if you don't notice a difference in your day to day tasks, then it's all the same. In my experience you should pick a kernel based on your desired experience. For my needs this is how the kernels differ:
Generic kernel: a sane default for most regular users
LTS: only makes sense if you're worried about regressions in the generic kernel causing issues, and only viable if you can afford to stay behind on hardware driver updates, ie you use old hardware and/or optimal performance is not required
Zen: sometimes better for gaming, but often indistinguishable from the generic kernel
Realtime: rarely what you want, it sounds "faster" but it's basically optimized for very specific use cases and if you're not among them you'll see the same or worse performance
- I'm trying to tinker with my system and replace a perfectly good and well optimized default kernel for some kernel made for specific niche use cases and I don't see any performance increase. Why would it be?
  Yes, surprisingly the default kernel is optimized well rather than just being a badly written placeholder that users should manually replace for their system to become usable.
  It's 2025 and stuff is designed to just work out of the box.

Difference goes from negligible to imperceptible. They were only really necessary for back in the day specific use cases like digital audio latency.

From what I recall the completely fair scheduler (CFS) used by default on most Linux systems has a lower average latency than the RT kernel. The RT kernel just gives you more consistency, hence the CFS having lower latency “on average”
So honestly for opening Firefox it’ll probably depend depending more on your SSD data rate, but in theory it’ll open faster on a “regular” distribution most of the time.
Real time is good for things like audio processing where having better guarantees that a processor will get its share of the CPU is a benefit.

Most of them won't be that different when you're not running anything that's pushing your system to its limits. Zen might be a bit faster in games or benchmarks, RT really won't do much unless you're running software that needs Real-time processing (you shouldn't use it for general use).
Hardened and zen are the only ones you might benefit from, but not really massively.

Zen and "mainline" (default/vanilla) are generally fine for "desktop use" and gaming. Zen is basically the mainline kernel with some tweaks. They are mostly concerned with latency, reducing the maximum time a process can spend blocking the processor - among other things.
This can lead to less input lag or a "smoother" desktop experience, but overall performance is as good as mainline at most. Slightly worse in some scenarios.
Hardned is a tradeoff afaik. You will stay behind mainline a bit, but get extra hardening. This can also impact performance, but rarely does in a meaningful way. If you don't have any specific reason to use it, e.g. you carry it around on a laptop with sensitive data, I would look at other ways to harden my system first (firewall, encryption, access control, anti-virus, sandboxing, VPN...).
Pretty much the same goes for LTS, but with the focus more on stability than security.
RT is only for special applications.

22 comments