OpenBSD Journal

[p2k9] Ports Hackathon - Budapest, Hungary

Contributed by jcroberts on from the slacking-undeadly-editors dept.

Although more than a bit late getting posted here, you should know about the accomplishments of the p2k9 Ports Hackathon. Sure, it might look like a party, but when you get a bunch of people together with similar ideas of having fun porting programs, amazing amounts of supposed "work" gets done.

p2k9: Ports hackathon
Oct 9 - 16, 2009
Budapest, Hungary
14 developers



Front: nicm, robert, aja, sthen
Back: phessler, jasper, steven, pirofti, espie
  1. Antoine Jacoutot (ajacoutot@)
  2. Christian Weisgerber (naddy@)
  3. Edd Barrett (edd@)
  4. Florent Tribouilloy (guest)
  5. Jasper Lievisse Adriaanse (jasper@)
  6. Landry Breuil (landry@)
  7. Marc Espie (espie@)
  8. Matthias Kilian (kili@)
  9. Nicholas Marriott (nicm@)
  10. Paul Irofti (pirofti@)
  11. Peter Hessler (phessler@)
  12. Robert Nagy (robert@)
  13. Steven Mestdagh (steven@)
  14. Stuart Henderson (sthen@)


More Photos

Edd Barrett (edd@) - a.k.a. "Edd the Vagabond"


Irrational Fear of Snakes?
I arrived in the building of the hackroom on the Friday afternoon. I walked around the building blindly for a while, ending up in some data centre, where bearded UNIXy looking people were speaking at me in Hungarian (probably wanting to know what I was doing in their data centre).

After finding Robert, I set up in the hackroom and started working on some ports I had hanging around in my tree as well as some new stuff:

Some of this I did not fully complete. The TeX Live work was based upon a nightly snap, as it has not yet been released. The good news is that it requires far fewer patches than before and should be a bit easier to integrate.


landry@, edd@

Myself and espie@ are still working in QT-creator, as it has various issues. Hopefully we can nail that soon.

On the Sunday naddy@ created my CVS account! My first commit was shortly after. (Thanks naddy!)

In the evenings we usually went to a restaurant for food and beer. Early in the week we had a great curry, then later in the week we went to the legendary Sir Lancelot's for a medieval feast. One afternoon we slacked at the spa for a few hours and got wrinkly.

The hackathon was both productive and fun. I would like to take the opportunity to thank the project for allowing the event to happen, Robert for planning everything and also the other porters for having me there. Thanks

Matthias Kilian (kili@)

  • Update lang/guile and depending ports, mainly to be able to update print/lilypond. After some initial attempts earlier this year, where okan@ and me started to test-build all the depending ports and looking at the results of make regress, jasper@ helped a lot some weeks ago by putting the diffs into a bulk build. I didn't want to commit it before p2k9, though, because of possible breakage. Only at an event like this hackathon you're able to find and fix problems quickly.
  • Update print/poppler, and help edd@ to let his latest work on texlive to build with it (it did build with poppler-0.10, but poppler-0.12 broke it, so I fixed it).
  • After edd@ committed the update of xpdf, I started to merge the patches into ports based on or including xpdf sources, like kdegraphics (x11/kde/graphics3), texlive (the version currently on ports) and poppler. Some older xpdf changes went also in. The texlive aren't committed yet, and I'm also working on merging the patches into the stable branch.
  • The `traditional' print/ghostscript update (only sent to ports@ but not yet committed, because I want to get people a chance to test it).
  • Some work on updating net/kismet to the latest version. This is still work in progress, but I think it will be finished early enough for 4.7.
  • Last but not least, I did some work on getting lang/ghc back into shape. It's still not bootstrappable from scratch, but I think I got a little bit closer.

Well, no thrilling non-ports stories from me, except you want to read a big lament about travelling from Braunschweig to Budapest and back by train ;-)

Nicholas Marriott (nicm@)


jasper, aja, landry, nicm
I was only able to be at p2k9 for four days but it was great to meet everyone and see all the ports stuff going on and, after some initial setup tribulations, it was very productive - I was able to drastically reduce the number of mails in my inbox and diffs lying around in source trees.

I was working on tmux, mainly on cleaning up some longstanding bugs, queued diffs or todo items rather than adding any huge features. Of the 39 tmux commits, these were the most significant:

  • Added simple job scheduling so that when you run tmux itself from inside (such as from #() in status-left) it doesn't hang and so that commands are run only every status-interval rather than every redraw.
  • Finished off and committed a diff to add "grouped sessions", allowing a session to share windows but have an independent current window and options, this is similar behaviour to "screen -x" and has been much requested.
  • Stop tmux outputting a hard newline at the end of every line even if it wrapped at the screen edge. Instead, let the terminal wrap naturally, allowing xterm mouse selection to correctly find the end of the line. This was another MUCH requested fix, but was quite difficult and frustrating to get right - without p2k9 I probably wouldn't have been able to spend the time it needed.
  • A "mouse-select-pane" option to allow the active pane to be chosen by clicking on it with the mouse. Suggested before by someone but brought up again at p2k9 by sthen@.
  • Committed the diff for a "pipe-pane" command to send any output in a pane to a command. Requested by espie@.
  • Scroll lines into the history even when the scroll region doesn't cover the whole screen, this makes ircII users like naddy@ happy.
  • Permit attributes to be turned off in #[] by prefixing "no", such as "noblink".
  • Use relative cursor movement when the terminal supports it. This makes things a bit more efficient, and shook out a number of redraw bugs.
  • Committed a diff a user submitted extending the lock timer to work independently for individual sessions.
  • Lots of code cleanup, bug hunting, and some work on making tmux and vttest play more nicely together. And beer.

I also sent off some minor fixes for some other areas, made a couple of trivial ports changes.

Christian Weisgerber (naddy@)


Sunshine Girl
Due to the bad weather we were pretty much limited to hacking and eating. But we did discover this AMAZING Swedish Eurodance artist Gunther and his Sunshine Girls...

Reminded by some recent questions on misc@, I finally sat down and fixed luit(1), which is not a port but part of our X11 installation. By itself, xterm(1) only supports the ISO 8859-1 and UTF-8 character encodings. For other encodings required to write many languages, such as ISO 8859-2 for Czech or KOI8-R for Russian, xterm will run in UTF-8 mode and spawn luit(1) as a wrapper that translates from and to the respective encoding. luit lacked proper support to allocate a pseudo-TTY on OpenBSD and would barely run, issuing a scary message: "Warning: could not change ownership of tty -- pty is insecure!". I fixed this by adding proper support for openpty(3), so you can now safely use xterm with a large number of traditional character encodings.

If you had to learn French as a foreign language, you are painfully aware that there are a few hundred irregular verbs with unpredicatble stem changes in the many inflectional forms. The new education/verbiste port can demonstrate the conjugation of even the most tricky French verbs. And if you come across a strange verb form in a text, verbiste can also tell you which tense/mood/person of which verb(s) you are looking at, so you know what to look up in a dictionary.

Larger port updates I committed at p2k9 were audio/mpeg123, which now uses optimized assembly code on more platforms, and in particular graphics/jpeg v7. A lot of ports directly or indirectly depend on jpeg, so I needed to run a full bulk build, but happily only two ports required special fixes.

phessler@ suggested that it would be nice if audio/openal, a 3D audio API used by a number of games, supported our native sndio audio interface instead of having to resort to OSS emulation. As usual, there was no documentation on the internal API, so I had to spend a silly amount of time reading the backends for other operating systems before I felt confident to write what turned out to be a surprisingly simple sndio backend. This still requires some testing before it can be committed.

And, as usual, I also took the opportunity to commit a number of minor maintenance updates and bug fixes too boring to list in detail.

Marc Espie (espie@)


Florent

There was a student from 1st year Epitech, Florent Tribouilloy, whom I invited, and who did tasks such as helping Antoine on a editors/subtitleeditor, update and Landry on some x11/xfce4/terminal, update. He's been learning quite a few ropes of the ports tree. Before that, I assigned him to work on some internal school project: write a specific bsd.rd that can pass their rather unusual proxy system (which he did) so that people at epita/epitech can install OpenBSD over the network without having first to download all the sets using another OS. I've also asked him to have a look at writing a BSD-licenced devel/cflow based on the lint parser... no need to reinvent the wheel, since we already have a C language parser in the tree. This might help in finding horrible code in lint, to boot ;-)

As far as I'm concerned, I mostly did work on pkg_add. Most of the changes don't have a lot of user impact, so far. I'm cleaning up the code to allow for two things:

  • merging the update system with the dependency solver. So that updates will eventually happen "on the fly", instead of computing all updates first, and doing them later. This allows me to be more correct in the dependency order, also to do partial updates correctly, and to install new packages while updating their dependencies. If you're in a hurry, this means you will be able to start an update and have it do stuff quickly, instead of waiting forever until it has scanned your whole list of packages. I've had this in my tree for ages, but there were bugs to fix. I had to clean up dependency handling an to add a new "tracker" object that knows about all pending updates just so that the code doesn't get any messier.
  • merged updatesets. That is, updating two or three packages at once, in cases where it's necessary: tight dependencies (mysql-client/server), files moving from package to package (kdelibs/kdebase), or even dependency reversal (somewhere in cairo/pango a few releases back). Most of the infrastructure is now there. What's left to do is to recognize those situations, and check that an updateset with newer > 1 works. This is fairly important for correctness, since some update scenarios currently do cheat and work by accident.
The user-visible changes:
  • stems with flavors, ala screen-- or star--static. Actually trivial to do.
  • -z/-l option, to duplicate an install (more or less) quickly. Best effort so far. Will work much better once the "update-on-the-fly" thingy works.
  • cosmetic clean-ups of install messages: less lines during updates. Restoring dependency handling. Showing signed packages in a cleaner way.
  • recognizing some cases where repositories don't have packages (wrong path) that we didn't get so far.

I also worked on ports a bit, mostly with Edd, who's trying to make progress on cmake (which is a tough issue), and on qt-creator, for which you should expect a port sometime in the near future: the actual issue is just a question of path to the plugins, which is recorded nowhere except on linux.

There was also a pending patch from Nicm that I validated over p2k9: the pipe-pane command, which can be used with a variation of portslogger to log all activity in a tmux window that's related to the ports tree. Very cool.

I was happy to meet edd, phessler, and nicm. And of course, to see again the usual band of suspects.

Landry Breuil (landry@)


cmixer
Regarding ports, i imported some standalone stuff like audio/cmixer, a quick hack written in perl-curses by yours truly to display a console mixer, or games/jbrickshooter, a stupid reflexion game which made me dreaming of moving coloured blocks, ajacoutot@ can witness it's addictive (and stupid).. i fixed some x11/gtk+2, update breakage, mainly in www/kazehakaze, (which required some awful hackery with axe and rocketlaunchers, some parts of this code are really horrible and made me produce a lot of grumblings and rants).

I also updated www/webkit, to a more recent version, now that we have a recent devel/libsoup, and updated various x11/xfce, components (that's a background task). And the two most important items :


Thou Shalt Not Pass!
This required working on a lot of dependencies : i took some from djm@'s work on multimedia/coherence and audio/py-tagpy, and worked on net/libmms support in gstreamer. devel/py-cssutils gave me some work too, but the hardest part was definitely graphics/pigment and graphics/py-pigment, which required some help from oga, guenther and kurt to understand how py-pigment & python were supposed to load and use libGL. I spent two days on it, but finally figured out a "clever" workaround, and was able to successfully run it on my atom netbook with decent performance.

I had a quick (one night) look too at XBMC, another HTPC/Media player software, and there's a huge amount of work before having this beast running on OpenBSD... maybe someday; it's a huge pile (160Mb tarball) of awful code from various sources, linking tons of homemade patched copies of ffmpeg, libdvd* & various video libs, an embedded ms-windows dll virtual loader, a homemade VFS layer, and tons of linux/osx/windows-only functions. A good looking app for sure, but not what i'd call portable to OpenBSD... a make clean in the wrong dir ruined my patching efforts, so it's pushed for another day :)

And most importantly, i enjoyed sharing some good french & dutch cheese and homemade mirabelle with the porters crew, eating to death at sir lancelot, trofea or in other really awsum' restaurants serving crazy liver-based meals.. that was a really cool edition, way too short !

Paul Irofti (pirofti@)

WORK: (i.e. FUN)
  • take over landisk package builds from robert
  • update about 40-50 ruby outdated ports (including www/ruby-rails and friends)
  • work on bringing math/octave up to date (this is still WIP but all the major parts were done during c2k9)
  • do some OT stuff like ACPI suspend/resume work and investigation (a big patch was commited during p2k9 actually)
  • bring in some new ports like pdq and other ruby utilities
  • plus lots of small stuff that I don't remember at the moment
FUN: (i.e. Other FUN)
  • drink beer and toss ideas at night
  • revisit pubs from the last hackathon
  • cool restaurants like the indian one we went to last year
  • Sir Lancelot
  • Spa (I didn't attend, but others went)
  • I spent the weekend after p2k9 with my girlfriend visiting the old center and some of the other interesting parts of Budapest, going to the spa, going to the dead baby heads pub and eating at some of the nice restaurants I found during the hackathon

Steven Mestdagh (steven@)


The Plate
I spent most time on devel/libtool. Code cleaning, and separation into smaller, more understandable pieces, as well as fixing a few corner cases. Right after p2k9, I was happy to see that it can now finally complete a build of xenocara. It is also able to build some more other ports correctly. I need to do some more testing with it, and if the result is good enough, we can enable it for general use.

The new libtool allows us to more easily build programs or libraries even when other versions are installed on the system, and will also speed up ports bulk builds (about 20% of the ports tree uses libtool, many of which are big ports), so it helps to reduce the time needed to build a package snapshot.

I also spent some time helping pirofti@ with getting math/octave to build; the gfortran compiler and associated fortran libraries are needed there, and this required some fixes in the lang/gcc/4.2 packages. Some more minor things, helping out new people a bit, and the activities... I think we finished the plate this time :)

Jasper Lievisse Adriaanse (jasper@)


jasper landry nicm

p2k9 had a pretty clear goal for Antoine and myself; update our x11/gnome ports to their 2.28.x release.

Pretty early on I made the "mistake" of queuing some fine Swedish cultural gems...and pretty soon thereafter everyone was familiar with Guenther's tralala and ding-ding-dong. Oh, and having a beamer in the hacking room didn't really help either.

Some time before p2k9 major updates to devel/glib2 and x11/gtk+2 had already been committed, which allowed us to really focus on GNOME. Instead of having to worry about those important libraries too. We set off right from the beginning at updating and pretty soon we ran into "old friends". Being HAL, PolicyKit and GStreamer. Stuart is doing some great groundwork which will eventually allow us to update GStreamer. He'll elaborate on that though.

As it turned out, even with two people hacking on GNOME for one week, almost 24 hours a day, we still weren't able to finish all the things we wanted to. There are some regressions that we noticed, but we continue to work on/debug it.

Besides the work on GNOME I also did some work on devel/libffi, multimedia/gstreamer, net/farsight (audio/video conferencing) and general tweaking on dozens of other ports.

Five of us had decided to go for some serious and well deserved slacking after p2k9, and on Friday we headed to Hévíz. Where we slept, swam and drank to prevent post-hackathon burn-out...

As was written on ports@ we would appreciate your testing and feedback! And thanks again to Robert for organizing yet another superb edition of the ports hackathon in Budapest.

Stuart Henderson (sthen@)


As well as updating some ports I maintain (and a few which I'll need in order to update some others sometime in the future), I spent some time working with a newer version of devel/flex which is needed in order to update various ports (including gstreamer, which itself blocks many other port updates).

My initial aim was to update the version in base and started on that basis (beinning with a diff from millert@ and working towards updating the manual from the info pages) while running a bulk package build. But after looking over the failure logs, it doesn't seem quite ready to make it the only version in the system, so I've imported it as a port for now.

And of course we had the usual mix of awesome music, and a really nice time in Budapest (and then a few of us stayed the following weekend near the lake in Hévíz for some much-needed recovery).

Due to slacking on the part of the undeadly editors, the article lags behind reality a bit, so some of the changes described here have been committed a long time ago. None the less, it's always nice to see work and fun behind the `sudo pkg_add whatever` magic.

EDIT: And by popular demand...
Guenther and The Sunshine Girls

And another one

EDIT2: sthen wants MORE!
Hopefully the last

NOTE: I am the one responsible for putting embedded youtube files on undeadly, and there have been more than a few questions about it, particularly regarding Adobe/MacroMedia shockwave. First of all, the actual videos are a running joke with some of the devs, and even the act of embedding youtube videos on undeadly is also a bit of a joke, but most importantly, you should already know how to handle video sites and formats with OpenBSD. If not, then send me hate mail until I write an article explaining how to deal with it.

Now please excuse me while I go watch sthen's lattest addition, and then promptly go running off into the weeds, screaming.

(Comments are closed)


Comments
  1. By Timo Myyrä (zmyrgel) timo.myyra@gmail.com on

    Great article as usual.

    It's good to see that lang/ghc has not been forgotten.

    Comments
    1. By Matthias Kilian (kili) on

      > Great article as usual.

      Well, it's a little bit embarrasing if you see your promises for 4.7 that you didn't done in time ;-)

      > It's good to see that lang/ghc has not been forgotten.

      Of course not. I tried to get GHC boostrapping work since it has been broken. But ir's incredibly frustrating; whenever you get some small step done (like `stage1 finally compiles from .hc files) you run into the next problem (like `but the linker fails'), and have to dive into the still overcomplicated build system and into its dependency hell.

      But after I still didn't manage to even get a working ghc bootstrapped from .hc files on the very same platform I created the .hc files, I know decided to join the dark side of the power[tm]. I recently started to prepare "minimal" precompiled binaries (for i386 and amd64) that then can be used to run the main GHC build. Of course, testing wether those minimal precompiled binaries (with debugging and profiling libraries as well as GHCi removed) are enough to start a full GHC build revealed, that it does not yet work. Surprise! Some library currently included in GHC (dph) can't be built without GHCi available. After my usual rant on the GHC lists, I'm now running some builds with dph removed from GHC (since dph isn't required at all by GHC), and I hope to get a patch ready for upstream.

      Anyway, switching to the dark side of the power means that lang/ghc is stuck to i386 amd amd64 forever. .hc bootstrapping didn't work for more than three years and is a dead-end (IMHO), crosscompiling still doesn't work (and even if it worked, I wouldn't have the time nor the resources to port the rts and maintain binary builds for all OpenBSD architectures), and there's no other Haskell compiler that can be used to build GHC.

      So what can I do?

  2. By jirib (jirib) jirib@mailinator.com on

    thanks for report. there's typo in url for net/flickcurl.

    Comments
    1. By J.C. Roberts (jcroberts) on http://www.designtools.org

      > thanks for report. there's typo in url for net/flickcurl.

      Thanks. Fixed. There are moments when my typing skills leave a lot to be desired.

  3. By BentuP (BentuP) on

    Always great to hear what's going on. Thanks for the write up.

  4. By jirib (jirib) jirib@mailinator.com on

    Little offtopic but anyway...

    I'd like to write a specific bsd.rd just for one simple task - to boot and scan harddisk(s) and filesystem(s) for 'something' :)

    What is the best documenation to follow?

    And... is it possible to just copy this bsd.rd to a flesh drive and to boot it from some boot loader? (grrr... just like linux kernel and its ramdisk).

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]