Contributed by tbert on from the pthreading-the-needle dept.
Philip Guenther(guenther@), the man who got this hackathon rolling, takes the time to organize his post-its:
Sometimes you go into a hackathon knowing exactly what you're going to tackle and it all follows the plan. No, wait, that never happens. There's always something that comes up that ends up eating more of the hackathon than you would have expected.
Over the last few years, we've been working through a long list of issues in the kernel and threading library. When rthreads were originally added, OpenBSD took the tack of using the pre-existing 'proc' structure as the thread object with a new process structure 'above' that. This contrasts with how some other BSDs did it, where struct proc remained as the process structure and a new structure was added beneath that to serve as the thread object. The result is that we've had a gradual migration of information from struct proc to struct process. At the start of the year, there were only a handful that still needed to migrate:
- ptrace(2) information
- resource usage tracking
- NPROC rlimit handling
- interval timers
- profil(2) information
- accounting information
- sysctl(2)/ps(1) thread handling
- systrace(1) handling
That's not too much, so we figured that a hackathon would let us push through these items any others that cropped up immediately. The sense was that if something completely explodes when we try this then we can still make progress before rolling back to the old user-threads for 5.2. So we started planning the logistics of r2k12, and when the tree unlocked after 5.1, I switched the build over from uthread to rthreads.
What followed wasn't a big boom, but rather a series of small ones. The ports people worked very quickly to identify and start fixing ports with problems. At the same time, they were quite vocal about the NPROC rlimit handling and the lack of real resource usage information from threaded processes. The former meant a desktop running gnome would hit the NPROC limit very quickly, making it useless, while the latter was making it harder to (re)balance the build order or track down hog processes. It turned out the resource usage bits closely associated with the itimer and profil(2) bits, so we tackled those as a group and got them and the NPROC limit item fixed there in February, instead of waiting until the hackathon, to help the ports work move forward more quickly.
Meanwhile, kettenis@ at that point was teaching gdb how to handle coredumps from programs that had multiple threads and kurt@ worked out that we could drop the thread suspension bits from the library: the ports that were using it could use the same signal-based code that they used on Linux.
What did turn up before the hackathon was a set of issues around file descriptor handling. For example, close(2) would block if another thread was concurrently performing an operation, and if the open(2) of a FIFO blocked then many other fd operations would block until the open(2) was unblocked. Investigating those turned up issues in the error handling of UNIX-domain file-descriptor passing.
The result was that for me, the hackathon ended up being centered on understanding the file descriptor handling in the kernel with krw@. We finally figured out how we wanted to handle the close(2) issue towards the end (and it's now in) and have an understanding of what the solution to the FIFO issue will be. Fortunately, mikeb@ was able to pick up the accounting bits and pirofti@ tackled the sysctl(2)/ps(1) bits.
One thing that did go as planned was that kettenis@ got additional ptrace(2) functionality added to support using gdb on running processes with multiple threads, with the per-architecture bits trickling in in the weeks since the hackathon.
Our hosts, David and the Institute Henri Poincare', provided a great facility, with good access, good net, free exercise climbing the stairs to the 5th floor, and a coffee maker and more ground coffee than we could drink. Extra thanks to Ann, the local network admin, who dealt with firewall issues for us and even stayed on site overnight in case of emergencies.
Thanks to all the devs for their great work on moving our favorite OS forward! Remember that hackathons are funded by users like you, so please take the time to visit the OpenBSD store to pick up some swag and become the envy of your peers, or make a donation and feel the inner peace of a smug sense of superiority!
(Comments are closed)