I'd like to use io_uring, but as long as it bypasses seccomp it should be disabled whenever seccomp is in use. As such, I use epoll, and find it annoying when kernel APIs like ublk require io_uring. The places I'd want to use ublk are inside sandboxes using seccomp. Given that container runtimes, hardened kernels, chromeos, etc., disable io_uring, using it means needing an epoll fallback anyways, so might as well just use epoll and not maintain two async backends for your application.
ublk, specifically, is something I'd expect to be primarily used in privileged contexts anyway, because the primary use of the resulting block device is to mount it, which requires privileges for most interesting filesystems. If you want an unprivileged mechanism, you may be interested in the upcoming uring-accelerated FUSE support.
For other uses, uring has a "restriction" mechanism that does part of what you want. See REGISTER_RESTRICTIONS in the documentation. Any process that's setting up its own seccomp restrictions can also set up a uring with restrictions, limiting the opcodes it can use.
That said, that mechanism would benefit from a way to apply such restrictions to a process that isn't doing the setup itself, such as when setting up seccomp restrictions on a container or daemon. For instance, a way to set restrictions on all rings created by child processes, or a way for seccomp to enforce that any uring created has restrictions applied to it.
The main problem I have with fuse is inotify not working. If inotify just worked for fuse, I'd just use it. Ideally I could just run the software in a mount namespace with a fuse fs, but I need inotify.
I mainly was trying to use ublk to implement a sort of fuse like thing with the kernel handling the fs and thus having inotify support.
Interesting, I didn't realize inotify didn't work with FUSE. Is this a flaw in the FUSE interface, or is it just a deficiency in certain FUSE filesystems?
I think the key problem is that mapping from FUSE requests to inotify events requires information that only the FUSE daemon has. For example, lets say you open a file with O_CREAT. Whether this should trigger IN_CREATE depends on whether the file already exists. The kernel doesn't know this, and so couldn't be responsible for generating the IN_CREATE event.
Now, the FUSE daemon could generate the event, but correctly generating events (especially handling edge cases) is difficult.
> For instance, a way to set restrictions on all rings created by child processes, or a way for seccomp to enforce that any uring created has restrictions applied to it.
SELinux or your favorite MAC is there to solve this exact problem.
Is there a specific io_uring opcode you would like disabled in your sandboxes? It's not like io_uring is a complete seccomp bypass, just another syscall that provides an alternative way to do many things. I doubt you block "read" or "accept" in docker, for example. You can't execute a sysctl or mount a filesystem using io_uring, which are things that are actually blocked in Docker by default.
edit: on the other hand, a good reason to disable uring in containers is that it's infested with vulnerabilities. It's new, complex, and does a whole lot of things - all of which make serious security bugs there quite common right now.
Current io_uring is not particularly prone to vulnerabilities. The original version of it had a design that often led to them (a kernel thread doing operations on behalf of the process and not always remembering to set the appropriate privileges), but it no longer uses that design, and the current design is much more resilient. Unfortunately, the original design led to a reputation that it's still trying to shake.
20 CVEs in 2024. Yes, some of them are not (exploitable) vulnerabilities, probably, because Linux CNA is being difficult. But many of them are, just ctrl+f privilege.
That's a lot like saying "the syscall interface is the most exploited interface to the kernel". io_uring is an entire syscall interface itself; the right point of comparison would be "every other syscall".
How do the exploits for io_uring compare to the exploits for the rest of the kernel?
It's not only potentially infested with vulnerabilities. It's also not possible to filter io_uring using seccomp at all. So if you allow io_uring, you allow all that is possible with it.
Out of current ones, at a quick glance: connect, openat, openat2, renameat, mkdirat, and bind. More importantly, I'd like to block any opcode I haven't whitelisted, even when my software runs on future kernels with more opcodes available.
Now that I think about it, how does io_uring interact with landlock?
> find it annoying when kernel APIs like ublk require io_uring
Good. That's a forcing function for making io_uring work in your environment.
> bypasses seccomp
Seccomp sucks.
We shouldn't be enforcing security by filtering system calls, the set of which will grow forever, but instead by describing access control rules on objects, e.g. with SELinux. If your security policy is that your sandbox should be able to read from some file but not write to it, you should do that with real MAC, which applies to all operations , il_uring included. You shouldn't just filter read(2) and write(2) in particular.
We shouldn't hold back evolution in systems interfaces because some people are stuck on bad ways of doing things and won't move.
seccomp is a mitigation. Once you have already been exploited, if further escalation is prevented by seccomp, or ASLR, or NX stack, or ....... then you got lucky.
I was actually glued to that page for a few days recently, it's a great write-up.
io_uring is such a tremendous improvement over epoll, in both speed and user experience. With sqpoll, vectored ops and proper batching you can get some crazyy speed. I am definitely looking forward to seeing some of these seccomp and privilege issues getting fixed and getting container support in the future.
I'm realizing from the title of this that the intended pronunciation of "uring" is probably "yoo-ring"; for some reason I had mentally been reading it as "yurr-ring" all this time, and I guess I never heard anyone say it out loud before. In retrospect, I probably could have guessed that I might be missing something given that I had no clue what "uring" was supposed to mean.
There are examples of cat and cp using io_uring. What are the chances of having io_uring utilised by standard commands to improve overall Linux performance? I presume GNU utils are not Linux specific hence such commands are programmed for a generic *nix.
Another one is I could not find a benchmark with io_uring - this would confirm the benefit of going from epoll.
>Another one is I could not find a benchmark with io_uring - this would confirm the benefit of going from epoll.
One of the advantages of io_uring, unrelated to performance, is that it supports non-blocking operations on blocking file descriptors.
Using io_uring is the only method I recall to bypass https://gitlab.freedesktop.org/wayland/wayland/-/issues/296. This issue deals with having to operate on untrusted file descriptors where the blocking/non-blocking state of the file descriptions might be manipulated by an adversary at any time.
So does the FIONREAD ioctl, but it's not a general solution. (According to https://news.ycombinator.com/item?id=42617719, neither is io_uring yet.) Thanks for the link to the horrifying security problem!
I thought for sure this was wrong, but when I actually checked the docs, it turns out that `RWF_NOWAIT` is only valid for `preadv2` not `pwritev2`. This should probably be fixed.
For sockets, `MSG_DONTWAIT` works with both `recv` and `send`.
For pipes you should be able to do this with `SPLICE_F_NONBLOCK` and the `splice` family, but there are weird restrictions for those.
GNU coreutils already has tons of Linux-specific code. But it would be a bit of a kernel fail if io_uring were faster or other preferable to copy_file_range for cp (at least for files that do not have holes).
On a hard disk, copying multiple files in parallel is likely to make the copy run slower because it spends more time seeking back and forth between the files (except for small files). Perhaps that isn't a problem with SSDs? It seems like you'd still end up with the data from the different files interleaved in the erase blocks currently being written instead of contiguous, which seems like it would slow down all subsequent reads of those files (unless they're less than a page in size).
> On a hard disk, copying multiple files in parallel is likely to make the copy run slower because it spends more time seeking back and forth between the files (except for small files).
Certainly not; it's likely to make it run faster, since you can use the elevator algorithm more efficiently instead of seeking back and forth between the files. You can easily measure this yourself by using comparing wcp, which uses io_uring, and GNU cp (remember to empty the cache between each run).
I don't think it's a troll (though not a particularly useful comment); Linux has had no true async story thus far. poll, epoll, et. al. are all synchronous behind-the-scenes.
What Linux still lacks is an OVERLAPPED data structure.
NT has supported async I/O since it's inception. It was a design principle of the kernel -- all I/O operations in the kernel are async'ed.
I wasn't referring to async I/O. I'm talking about the ability to make system calls using a user/kernel shared memory buffer, without having to enter and exit the kernel. (This is particularly important with all the security mitigations that make kernel entry/exit more expensive.)
Technically Windows has Registered I/O in Windows 8/Server 2012 for networking which provided this functionality. I/O Rings in Windows extended this to other types of I/O, but of course with a separate API.
For other uses, uring has a "restriction" mechanism that does part of what you want. See REGISTER_RESTRICTIONS in the documentation. Any process that's setting up its own seccomp restrictions can also set up a uring with restrictions, limiting the opcodes it can use.
That said, that mechanism would benefit from a way to apply such restrictions to a process that isn't doing the setup itself, such as when setting up seccomp restrictions on a container or daemon. For instance, a way to set restrictions on all rings created by child processes, or a way for seccomp to enforce that any uring created has restrictions applied to it.
I mainly was trying to use ublk to implement a sort of fuse like thing with the kernel handling the fs and thus having inotify support.
Now, the FUSE daemon could generate the event, but correctly generating events (especially handling edge cases) is difficult.
Do you have a reference for this? What is the anticipated timeframe?
I don't know when it'll be merged, but it seems like it's getting close to ready.
SELinux or your favorite MAC is there to solve this exact problem.
edit: it does seem it is disabled there now: https://github.com/containerd/containerd/pull/9320 (thanks to sibling comment for an adjancent link)
Unfortunately decided it's not worth it.
edit: on the other hand, a good reason to disable uring in containers is that it's infested with vulnerabilities. It's new, complex, and does a whole lot of things - all of which make serious security bugs there quite common right now.
Current io_uring is not particularly prone to vulnerabilities. The original version of it had a design that often led to them (a kernel thread doing operations on behalf of the process and not always remembering to set the appropriate privileges), but it no longer uses that design, and the current design is much more resilient. Unfortunately, the original design led to a reputation that it's still trying to shake.
20 CVEs in 2024. Yes, some of them are not (exploitable) vulnerabilities, probably, because Linux CNA is being difficult. But many of them are, just ctrl+f privilege.
The tech industry: launch early! Develop in public! Many eyes make all bugs shallow!
Also the tech industry: we will never forgive you for that one segfault you had ten years ago.
How do the exploits for io_uring compare to the exploits for the rest of the kernel?
https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=%22linux%20...
Now that I think about it, how does io_uring interact with landlock?
Good. That's a forcing function for making io_uring work in your environment.
> bypasses seccomp
Seccomp sucks.
We shouldn't be enforcing security by filtering system calls, the set of which will grow forever, but instead by describing access control rules on objects, e.g. with SELinux. If your security policy is that your sandbox should be able to read from some file but not write to it, you should do that with real MAC, which applies to all operations , il_uring included. You shouldn't just filter read(2) and write(2) in particular.
We shouldn't hold back evolution in systems interfaces because some people are stuck on bad ways of doing things and won't move.
You can use io_uring with epoll to monitor eventfd to wake up your sleeping with io_uring wait for completions.
I have implemented a barrier and thread safe techniques that I am trying to turn into a command line tool
My goal is that thread safe performant servers are easy to write.
I am using bloom filters for fast set intersection. I intend to use Simd instructions with the bloom hashes.
io_uring is such a tremendous improvement over epoll, in both speed and user experience. With sqpoll, vectored ops and proper batching you can get some crazyy speed. I am definitely looking forward to seeing some of these seccomp and privilege issues getting fixed and getting container support in the future.
Another one is I could not find a benchmark with io_uring - this would confirm the benefit of going from epoll.
One of the advantages of io_uring, unrelated to performance, is that it supports non-blocking operations on blocking file descriptors.
Using io_uring is the only method I recall to bypass https://gitlab.freedesktop.org/wayland/wayland/-/issues/296. This issue deals with having to operate on untrusted file descriptors where the blocking/non-blocking state of the file descriptions might be manipulated by an adversary at any time.
For sockets, `MSG_DONTWAIT` works with both `recv` and `send`.
For pipes you should be able to do this with `SPLICE_F_NONBLOCK` and the `splice` family, but there are weird restrictions for those.
Certainly not; it's likely to make it run faster, since you can use the elevator algorithm more efficiently instead of seeking back and forth between the files. You can easily measure this yourself by using comparing wcp, which uses io_uring, and GNU cp (remember to empty the cache between each run).
In practice io_uring can be used in many different ways, and it can be challenging to find the most efficient one.
also https://www.phoronix.com/news/Linux-6.0-IO-Block-IO_uring
Issues with io_uring security mostly stemmed from an old architecture and just the fact that there's a ton of surface area.
There's nothing wrong with the general concept.
https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=io_uring
Previous discussion https://news.ycombinator.com/item?id=23132549
What Linux still lacks is an OVERLAPPED data structure.
NT has supported async I/O since it's inception. It was a design principle of the kernel -- all I/O operations in the kernel are async'ed.
https://windows-internals.com/ioring-vs-io_uring-a-compariso...
https://serverframework.com/asynchronousevents/2011/10/windo...
So if we're talking about concepts... NT first, again ;-)