OpenBSD Journal

syspatch: yet another way to 'binary patch' OpenBSD boxes

Contributed by dhartmei on from the binary-patches dept.

Olivier Lemaire writes:
I know the subject of 'binary patching' has already been discussed numerous time on Undeadly... Anyway, syspatch is the way I manage to keep numerous OpenBSD boxes quickly up to date with erratas applied.

After some lurking, I found nothing that satisfied my (OAPS, tepatch, 'cd / and tgz mypatch.tgz', ...). I decided to write these few lines (perl beginner, please don't shoot too hard...).

I wanted something simple and stupid, which will not mask in any way the process of patching (you still have to do so on a buildhost and understand what is patched and how), and could save my (and my colleages) time patching numerous hosts (once a patch 'binary package' is done, you just have to install it on target hosts).

Code, some docs and samples for OpenBSD are available at: http://sourceforge.net/projects/syspatch (see the cvs) and http://sholari.free.fr/syspatch/ (docs and examples).

Hope it may helps other than us too. Thank's to my company to allow me a BSD publication out there.

(Comments are closed)

Comments

  1. By BB (194.84.21.41) openbsd@yandex.ru on

    Nice Work, but You need make a little correction in:

    4- Grab the latest collection of SPMS packages for your arch (i386) and release (3.7) there: http://sholari.free.fr/download/spms/3.7/i386/ (use wget mirror, lynx, or whatever method you're addicted to...)

    change URL from http://sholari.free.fr/download/spms/3.7/i386/ to http://sholari.free.fr/syspatch/download/spms/3.7/i386/

    Comments

    1. By mortalblow (80.81.48.133) on

      very interesting, why there are two seprate patches for zlib bug 004 and 005. They should be united together. because, booth security issues are affected in same libraries. So, if you apply one, you discard another? it would be half a smart only if in binary patch005 is included patch004 too.

      Comments

      1. By LEM (195.167.202.151) sholari@free.fr on

        Hi,

        I made two distincts packages because there is two different patches... and that at a given date, there's only one new patch to be considered.

        You're right, if you apply 37-004_libz _after_ 37-005_libz, you'll discard 005_libz errata, which is of course not a good thing to do. Anyway, applying an older patch after an newer one is not a good thing to do.

        You're also right saying 37-005_libz deliver both 005_libz and 005_libz erratas.

        But I prefered to stick to the erratas process and history, than diverging making somthing which seems interesting, but which should not happen in real life: In fact, in real life, for each errata comming, a corresponding SPMS package should come. So, we'll have the case again.

        Anyway, why not creating a "meta-spms" which will "embed" all patches (up to 'localtime') for a release? I think I'm gonna work on a "apt-get upgrade" feature soon which will add to syspatch the following features:
        1- checking for "new patches to be applied" on a given repository
        2- grabing new patches
        3- applying them using syspatch

        This will deliver more or less the sames results as your described (have the host quickly patched no matter what the erratas have done through the time). But, I think this way is more the way real life will go, isn't it?

    2. By LEM (195.167.202.151) sholari@free.fr on

      Thx for the BR, I've fixed it in cvs.

      The web site should be updated tonight (gr... no ssh access to the website... btw, if anyone know were I can host the SPMS/syspatch homepage & SPMS repository, with a shell account, drop me a note :-) .

      Cheers,
      --
      LEM

  2. By Chas (147.154.235.52) on

    1. The end-user agent should be dead-simple like RedHat up2date.

    2. This end-user agent should use ctorrent.sourceforge.net to down/upload, saving server bandwidth. The binary is tiny, but it is written in C++, so some extra libraries will have to be around. I don't think that it plays well with firewalls.

    3. The end-user agent should be configured by default to leave ctorrent running for 24 hours as an uploader after the download is finished.

    4. The end-user agent should be able to be configured as client, server, or relay.

    4. Lots of md5 and gpg verification is always nice.

    5. Best-effort support for old releases would be FANTASTIC. At the very least, the last zlib bugs applied cleanly to unsupported 3.5. A patch like that should work with the agent. Community-contribued and verified patches against unsupported releases should appear to the agent with a warning advisory. OpenBSD's biggest downfall is that you are only allowed 1 year of uptime if you want to be fully supported.

    5. Somebody should lobby like hell to get these features into the OpenBSD base. There was an interview some time ago that implied that they are looking at binary patches (but obviously not too hard).

    Comments

    1. By Anonymous Coward (81.57.42.108) on


      1. The end-user agent should be dead-simple like RedHat up2date.

      Yeah, I've always thought that an upgrade process running all the time is
      better than a crontab fired one, for this purpose.

      2. This end-user agent should use ctorrent.sourceforge.net to down/upload,
      saving server bandwidth. The binary is tiny, but it is written in C++, so some
      extra libraries will have to be around. I don't think that it plays well with
      firewalls.

      About bandwidth consumption: 2-clauses BSD licenced bsdiff
      (http://www.daemonology.net/bsdiff/) is worth considering here.

      Ctorrent as too many downsides imho (like being overly complex, firewall
      nightmares, c++ code -think about ABI changes-, etc.). So for large LAN
      ability to multicast may be a better option. For internet (WAN) patches
      distribution, having a centralized distribution point is still the best
      compromise for security and simplicity (this matter!).

      4. The end-user agent should be able to be configured as client, server, or
      relay.

      Still, firewall and security nightmare: too much unpredictable traffic,
      too much multipurpose code running on every hosts ...

      4. Lots of md5 and gpg verification is always nice.

      Gpg isn't in base (and is GPL software). OpenSSL API should covers
      the needs as well for that purpose (except if you need in-depth key
      trust management support, but it's pointless here).

      5. Somebody should lobby like hell to get these features into the OpenBSD
      base. There was an interview some time ago that implied that they are looking
      at binary patches (but obviously not too hard).

      Distributing binary patches & allowing automatic updates the right way may
      be harder you think. For instance: how do you handle needs like restarting
      daemons and the like ?

      While I beleive that the lack of automated binaries updates is (now we have
      raid management, thanks !) the weakest point of OpenBSD (I mean the point
      where it doesn't shine facing Linux distros and even ms windows), I'm
      sure they're won't be any official support for this until they've solved
      all those issues the right way (with, as usual, corectness, simplicity,
      and security in mind). And what's sure is that lobbying won't help, here ;).

      By the way, the Espie work on packages management and updates is amazing !
      Sad that the standards OpenBSD tarballs aren't packages too :(

      Comments

      1. By LEM (82.229.192.54) sholari@free.fr on http://sholari.free.fr/syspatch

        "Gpg isn't in base (and is GPL software)."
        => So it won't be used as syspatch aims at keeping things simple and stupid (if not in baseXX.tgz, should not be used if not forced to do so).

        "OpenSSL API should covers the needs as well for that purpose (except if you need in-depth key trust management support, but it's pointless here)."
        => I was thinking about OpenSSL too (as in base, can deal with X509 certs too, have tons of perl modules to handle (not all in baseXX.tgz anyway...),...).
        => I was thinking about gzsig, if in 3.8: should be simple and stupid too...

        Anyway, signing/verifying packages (SPMS) should be better than relying on strong authentication on repositories, but should not be exclusive too.

      2. By LEM (82.229.192.54) sholari@free.fr on http://sholari.free.fr/syspatch

        "The end-user agent should be able to be configured as client, server, or relay.

        Still, firewall and security nightmare: too much unpredictable traffic,
        too much multipurpose code running on every hosts ..."

        => I agree and don't see why all this sci-fi. A client, a server, an http(s) protocol for transport, and that's fine.

        Anyway, checking and grabing is probably not the role of syspatch, but of some other tools to code (an 'spms-get check/update' ?).

        --
        lem

      3. By Anonymous Coward (82.229.192.54) on

        "This end-user agent should use ctorrent.sourceforge.net to down/upload,
        saving server bandwidth. The binary is tiny, but it is written in C++, so some extra libraries will have to be around. I don't think that it plays well with firewalls.

        About bandwidth consumption: 2-clauses BSD licenced bsdiff
        (http://www.daemonology.net/bsdiff/) is worth considering here.

        Ctorrent as too many downsides imho (like being overly complex, firewall
        nightmares, c++ code -think about ABI changes-, etc.). So for large LAN
        ability to multicast may be a better option. For internet (WAN) patches
        distribution, having a centralized distribution point is still the best
        compromise for security and simplicity (this matter!)."

        => I fully agree. Remember: Keep It Simple and Stupid. An http(s) transport and everything will be ok.

        --
        lem

      4. By LEM (82.229.192.54) sholari@free.fr on http://sholari.free.fr/syspatch

        "1. The end-user agent should be dead-simple like RedHat up2date.

        Yeah, I've always thought that an upgrade process running all the time is better than a crontab fired one, for this purpose. "

        Remember we're talking about OpenBSD erratas which are:
        . not so frequent (see history)
        . not so bing (the biggest are the kernel ones)

        So, imho, a cronjob is sufficient to deal with packages/erratas checks and retrieval (but not automatic application).

        Remember too that some OpenBSD erratas are kernel based, and that kernel may differs from users (not every users use the GENERIC), so automatic application of patches is not always a thing to do.

        I guess we must identify which SPMS may be automatically applied and which one must not be automatically applied.

        I guess the sysadm should have the choice to ask for automatic application or not (for SPMS that can by automatically applied).

        Finally, I think we must separate the check (checking there is no new patch to apply from a central repository) and grab (grabing the SPMS to localhost) features than the syspatch (check if patch is for the localhost release and arch, backup files and apply patch) features.

        We could have 'syspatch [check,update-grab]' to deal with check/grab features.

        Hum... personnaly, I already disabled RH up2date to install apt-rpm on my RH[EL] boxes... A matter of taste :)
        --
        lem

      5. By LEM (82.229.192.54) sholari@free.fr on http://sholari.free.fr/syspatch/

        "5. Best-effort support for old releases would be FANTASTIC. At the very least, the last zlib bugs applied cleanly to unsupported 3.5. A patch like that should work with the agent. Community-contribued and verified patches against unsupported releases should appear to the agent with a warning advisory. OpenBSD's biggest downfall is that you are only allowed 1 year of uptime if you want to be fully supported. "

        A longer support cycle means lots of peoples to maintain old releases.

        A 1-year support cycle is not that difficult to deal with: if patching your hosts is not painfull (syspatch aims at helping in this way), upgrading your packages if fairly easy (Marc Espie did a tremendous work to provide us such features!) and upgrading to next release is smoothed (the way syspatch help in the patching process, not black magic, just some automations).

        Personnally, I often use OpenBSD for firewalls usages. Having a 6-12 month cycle after upgrading is not that tremendous work and help maintaining a good security level. Off course, you have to invest a little bit more time in sysadm than for other boxes.

        Anyway, this point (supporting old releases) is out of scope of 'binary patching', imho.

      6. By m0rf (68.104.17.51) on

        from the bsdiff page
        "bsdiff and bspatch use bzip2; by default they assume it is in /usr/bin.

        bsdiff is quite memory-hungry. It requires max(17*n,9*n+m)+O(1) bytes of memory, where n is the size of the old file and m is the size of the new file. bspatch requires n+m+O(1) bytes. "

        so out for the low end then.

    2. By LEM (82.229.192.54) sholari@free.fr on

      "Somebody should lobby like hell to get these features into the OpenBSD base. There was an interview some time ago that implied that they are looking at binary patches (but obviously not too hard)."

      Hum... there will perhaps have room for policits later. For now, it's only time to code :)

      --
      lem

  3. By LEM (82.229.192.54) olem@metawire.org on http://metawire.org/~olem

    Greetings all,

    I've just released syspatch-0.3 which took over some bugfixes evoqued on undeadly and the project's tracker. Thank's to all bug reporters.

    I whish to clean-up the code a little bit more (using 'eval') before working on automatic retrieval of new spms packages as evoqued on undeadly (and as we use it at work).

    The project homepage have just changed to http://metawire.org/~olem/syspatch/ . Thank's to this free hosting service and teams for the excellent job: they run OpenBSD and provides free shell accounts!

    Feel free to report any idea about spms retrieval and checks. I had in mind to try to 'stick' to what is going on with pkg_add if possible.

    Cheers,
    --
    lem

    Comments

    1. By LEM (82.229.192.54) olem@metawire.org on http://metawire.org/~olem/syspatch

      More code cleanup done.

      Now, non-root user may list applied patchs and the same code may be used on any archs (please, test & report).

      Time to code the update (check/get/verify) method, isn'it?

    2. By Olivier Lemaire (195.167.202.151) olem@metawire.org on http://metawire.org/~olem/syspatch

      Greetings all,

      0.5 release of syspatch is out.

      Now, any SPMS package signature will be verified using gzsig (using SSH Identity keys or X509.v3 certificates). This verification is only done if gzsig is present (shipped in the base set of 3.8) and if a public key is supplied (syspatch ship the sample public key of the http://metawire.org/~olem/syspatch repository).

      Next step with syspatch is to code the "update & upgrade" part; I mean checking a (potentially distant) SPMS repository for new SPMS and warn if new patches are to be applied (eventually apply new patches, but I'm not sure a totally-automatic process is really a good thing).

      See ya,
      --
      lem

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]