OpenBSD Journal

Ask undeadly: Superports?

Contributed by grey on from the searching for more than new flavors dept.

Chad Loder writes:

I'm wondering if anyone has done work on putting together what I'll call (for lack of a better term) superports -- preconfigured collections of ports along with detailed configuration information. This would make it easier to install something like: postfix + tls + amavisd + spamassassin + razor + dcc.

I doubt the ports framework can handle all of the post-install configuration tasks that would need to be done, but it seems that we should be able to do better than a plethora of HOWTO articles that become out of date within weeks of having been written. Does anything like this exist (in other BSD's, even)? Is there any desire to see it?

(Comments are closed)


Comments
  1. By m0rfich (68.104.14.15) on

    in freebsd:

    from /usr/ports/misc/instant-server/Makefile:
    RUN_DEPENDS= ${LOCALBASE}/sbin/postfix:${PORTSDIR}/mail/postfix \
    ${LOCALBASE}/sbin/cvsupd:${PORTSDIR}/net/cvsup \
    ${LOCALBASE}/sbin/dhcpd:${PORTSDIR}/net/isc-dhcp3-server \
    ${LOCALBASE}/sbin/smbd:${PORTSDIR}/net/samba \
    ${LOCALBASE}/sbin/traceroute:${PORTSDIR}/net/traceroute \
    cnewsdo:${PORTSDIR}/news/cnews \
    ${LOCALBASE}/sbin/httpd:${PORTSDIR}/${APACHE_PORT} \
    ${LOCALBASE}/sbin/squid:${PORTSDIR}/www/squid

    /usr/ports/misc/instant-workstation/Makefile:

    RUN_DEPENDS= acroread:${PORTSDIR}/print/acroread5 \
    bash:${PORTSDIR}/shells/bash2 \
    cdrecord:${PORTSDIR}/sysutils/cdrtools \
    dos2unix:${PORTSDIR}/converters/unix2dos \
    emacs:${PORTSDIR}/editors/emacs20 \
    fetchmail:${PORTSDIR}/mail/fetchmail \
    grip:${PORTSDIR}/audio/grip \
    gimp:${PORTSDIR}/graphics/gimp1 \
    gv:${PORTSDIR}/print/gv \
    gpg:${PORTSDIR}/security/gnupg \
    ispell:${PORTSDIR}/textproc/ispell \
    startkde:${PORTSDIR}/x11/kde3 \
    mutt:${PORTSDIR}/mail/mutt \
    mozilla:${PORTSDIR}/www/mozilla \
    ${LOCALBASE}/sbin/postfix:${PORTSDIR}/mail/postfix \
    xtset:${PORTSDIR}/x11/xtset \
    xmms:${PORTSDIR}/multimedia/xmms \
    xv:${PORTSDIR}/graphics/xv

    OpenBSD could implement it similarly.

    I don't know how you could make a set of services or user programs
    that would be useful to a sufficiently large group of people to
    justify this though.

    Comments
    1. By Anonymous Coward (67.70.74.251) on

      Wow, I never knew about this. This in 5.x series only?

      Comments
      1. By Jonathan (217.86.145.132) on

        In FreeBSD the ports have no tag, they are same same for 4.X, 5.X and 6.X. No STABLE or CURRENT ports.

        So this is avaliable in all versions.

        Comments
        1. By Anonymous Coward (67.70.74.251) on

          I think it'd be cool to see something like this in OpenBSD too, just links to existing ports.

      2. By Asenchi (68.77.101.238) asenchi@asenchi.com on www.asenchi.com`

        No those have been around for a really long time. I started using FreeBSD at 4.6, I think, and they were there.

  2. By Bas Keur (213.84.93.41) bas.keur@dmrt.net on http://www.dmrt.net

    Just curious, since OpenBSD doesn't give much about *speed (as in fs io/i686 flag hacks etc) as it does for security. (You can't have both as we all know) Why suffer through all the cpu cycles/io crunches if one whould install lets say /usr/ports/x11/kde ? Try doing so on a Sparcstation 5, unless you mount your fs async (and risk fs death) it will take about -+ 6 hours minimum to finish. What does it offer *more* that the packages online ? Precompiled packages are also updated if bugs are found/fixed.

    $ PKG="ftp://ftp.nl.uu.net/pub/OpenBSD/`uname -r`/packages/`uname -m`/"
    $ for LEECH in `ncftpls $PKG`;do echo "pkg_add -v $PKG$LEECH";done|grep .tgz > list
    $ grep mysql-server list
    pkg_add -v ftp://ftp.nl.uu.net/pub/OpenBSD/3.5/packages/i386/mysql-server-4.0.18.tgz
    pkg_add -v ftp://ftp.nl.uu.net/pub/OpenBSD/3.5/packages/i386/mysql-server-4.0.18p1.tgz
    pkg_add -v ftp://ftp.nl.uu.net/pub/OpenBSD/3.5/packages/i386/mysql-server-4.0.18p2.tgz
    

    For some kind of reason it handles it's DEPENDS much better with static packages. I might be missing something, please do enlighten me.

    Comments
    1. By JOS (84.27.70.72) on

      Setting PKG_PATH to 'ftp://<yourfavoritemirror>/pub/OpenBSD/3.6/packages/i386/' is also interesting. You can just type "pkg_add <something> and it will download and install it, including all dependancies.

      Comments
      1. By Bas Keur (213.84.93.41) bas.keur@dmrt.net on http://www.dmrt.net

        That's basically what i tried before.
        The idea is ok, but it did not work for me :)
        # export FTP="ftp://ftp.nl.uu.net/pub/OpenBSD/3.5/packages/i386"
        # pkg_add -n $FTP/mplayer*
        Pretending to add ftp://ftp.nl.uu.net/pub/OpenBSD/3.5/packages/i386/mplayer*
        Can't find ftp://ftp.nl.uu.net/pub/OpenBSD/3.5/packages/i386/mplayer*
        # ncftpls $FTP/ | grep mplayer | wc -l
              11
        
        And besides, i feel like a 8 year old in toys'r'us when i cat the list dump. Who said the copy/paste generation will not enjoy the all mighty OpenBSD =]

        Comments
        1. By Anonymous Coward (68.224.187.79) on

          It works for me daily. You can also ssh in,sudo the pkg_add. you could also copy and paste the url to the exact package on the mirror in ssh.

        2. By Anonymous Coward (204.209.209.129) on

          that won't work. When you use the pkg_add this way, you have to use the full package name. Why is this, well what if there where more than one mplayer. Say mplayer-sdl and mplayer-arts, then which one should it choose. Unless you can find a way to make it scan your brain for the correct answer, you won't find a solution. Later.

        3. By Marc Espie (62.212.102.210) on

          For wildcards to work through ftp, you have to handle one stupid issue: quite a few ftp mirrors won't allow simultaneous connections from the same site. which means that you must have just one connection active at the one time... which is the main reason for wildcards and other nifty features not yet working: I need to set up some caching that will interrupt/restart the right connection to ensure just one connection to a given ftp site at one time. It's on my list of things to do, not high-priority because pkg updates take precedence... ;-)

    2. By Anonymous Coward (69.197.92.181) on

      You realize you are on crack right? Stupid compiler flags slow things down as often as they speed things up, and not enough to notice the difference anyways. You don't have to give up performance for security, and openbsd performs just as well as netbsd and linux in my real world (apache, sendmail, postgresql) benchmarks.

      Comments
      1. By Bas Keur (213.84.93.41) bas.keur@dmrt.net on http://www.dmrt.net

        > You realize you are on crack right ?
        I prefer chewing Coca plants.
        Want some ?

        > Stupid compiler flags slow things down as often as they speed things up

        I suggest you send a mailto:gcc@gnu.org telling them how stupid they are.
        (While your at it, tell them to keep the next release under the 10mb)

        > And not enough to notice the difference anyways.
        [viper@allevil viper]$ curl www.dmrt.net/rcutest.c > rcutest.c
        [viper@allevil viper]$ wc -l rcutest.c
              68 rcutest.c
        [viper@allevil viper]$ gcc -O3 -march=i686 -fomit-frame-pointer rcutest.c -o rcutest-i386
        [viper@allevil viper]$ gcc -march=i386 rcutest.c -o rcutest-i686
        
        [viper@allevil viper]$ ./rcutest-i386 1
        [1 Proc x 5000000 Loop] Time: 10.4056
        
        [viper@allevil viper]$ ./rcutest-i686 1
        [1 Proc x 5000000 Loop] Time: 9.0979
        
        *kuch*
        > You don't have to give up performance for security

        Really ? Ever seen a Hummer outrun a Lamborghini Murcielago on the highway ? You pay a price for security, always. The art is to balance it to your needs. (Result: 25" rims offroad lambo with bullbar)

        > performs just as well as netbsd and linux in my
        > real world (apache, sendmail, postgresql) benchmarks


        If your real world is a web blog writing in a 40mb
        sql db sure. Mine is not.

        Comments
        1. By Ray (216.254.116.107) ray@cyth.net on

          Unless you copy and pasted your compile lines incorrectly, you just proved your opposition’s point.

          Comments
          1. By Bas Keur (213.84.93.41) bas.keur@dmrt.net on http://www.dmrt.net

            I did *sign*

        2. By Anonymous Coward (69.197.92.181) on

          That's great, next time I need to loop 5 million times and that's it, I'll be certain to use "optimizations" to save myself all that time. Unfortunately, I tend to use actual applications, not microbenchmarks. Apply those same flags to apache and see what a big performance improvment you get. And what does a hummer have to do with security? You should try comparing a Lamborghini with seatbelts to one without seatbelts, and see what performance loss you have to trade for the security. And I don't know what you are talking about with the blog comment, I said real world benchmarks. Do you understand the concept? Not seeing how fast looping 5 million times is, but seeing how many pages/sec you can serve. Try it yourself, compile apache with the exact same flags on netbsd, openbsd, freebsd and linux, then run http_load on them and see how they perform. The only difference you'll see if freebsd 5 is slower than the rest.

        3. By Anonymous Coward (69.197.92.181) on

          That's great, next time I need to loop 5 million times and that's it, I'll be certain to use "optimizations" to save myself all that time. Unfortunately, I tend to use actual applications, not microbenchmarks. Apply those same flags to apache and see what a big performance improvment you get.

          And what does a hummer have to do with security? You should try comparing a Lamborghini with seatbelts to one without seatbelts, and see what performance loss you have to trade for the security.

          And I don't know what you are talking about with the blog comment, I said real world benchmarks. Do you understand the concept? Not seeing how fast looping 5 million times is, but seeing how many pages/sec you can serve. Try it yourself, compile apache with the exact same flags on netbsd, openbsd, freebsd and linux, then run http_load on them and see how they perform. The only difference you'll see if freebsd 5 is slower than the rest.

        4. By Anonymous Coward (143.127.131.4) on

          SQL is not a database. It is a query language.

    3. By RC (4.11.46.164) on

      > What does it offer *more* that the packages online ?

      The abitity to select the dependencies.

      Several GTK programs depend on GDK-PIXBUF, and (by default) GDK-PIXBUF depends on GNOME. So, to install a GTK-based image viewer, or something similar, you have to install pretty much ALL of GNOME.

      In ports, I just set FLAVOR="no_gnome" then build GDK-PIXBUF, and I'm set.

      This gets really crazy with the more complex packages like MPlayer. It has nearly a dozen different flavors, which will require you to, or not to have another set of dependencies install.

      You'd need to have a binary package for each and every possible combination of dependencies.

  3. By aht (80.58.4.235) on

    [please, avoid flames]

    I am not kidding, but Debian's .deb/dpkg/apt-get systems does exactly that; and more. It is far from perfect, but does that job flawlessly.

    It handles dependencies, allow for installing pre-compiled packages or make-your-own packages, and have pre and post install facilities (and much more, I am sure everyone knows how it works).

    What about porting (or re-writing) it to OpenBSD?

    Comments
    1. By Brad (216.138.200.42) brad at comstyle dot com on

      No way about Debian apt crap. A lot of work has been done in -current to improve the package tools and what has been done so far has been pretty damn good.

      Comments
      1. By Anonymous Coward (68.224.187.79) on

        I agree! RTFM if you can't understand how to use ports and packages.

        Comments
        1. By Anonymous Coward (212.238.188.197) on

          It is no question of RTFM. apt-get is really nice from a user perpective, bashing it is very childish. Even is ports/packages are improved it's hard to beat apt-get ... However OpenBSD with the ease of apt-get would be a killer OS ....

  4. By submicron (68.89.197.247) submicron@inherently-evil.net on

    The problem with making a generic set of configurations for a large number of packages is that it does away with the flexibility of the solution and ultimately makes your "superport" less functional for a wide group of users. Having a diverse number of semi-out of date how-tos helps someone put together exactly the solution they need. Case in point: I just spent a month building a spam/virus filtering mail gateway appliance for the company I work for. In doing it, I ended up reading about 20 different how-tos along with the appropriate documentation for each package I intended to use. As a result, I got exactly the solution I wanted and I understood all the of the design tradeoffs I was making and their impact. If I had been able to install some super package, I would have saved time, but I'd have been hard pressed to suppor the solution as efficiently because I wouldn't understand the solution nearly as well. Additionally, I'd have missed out on some very interesting/useful functionality because it wasn't well documented and probably wouldn't have been included in the "super package". The whole point of modular design is to give maximum flexibility. Monolithic solutions based packages are inferior.

    Comments
    1. By Anonymous Coward (213.118.102.119) on

      The same thing applies when installing any pre-compiled binary.

      Comments
      1. By Anonymous Coward (67.137.236.205) on

        You must be a gentoo user!

        Comments
        1. Comments
          1. By Nickus (213.146.107.225) on

            Thanks for the wonderful Sunday entertainment. :-)

  5. By JIMMY (68.58.135.1) on

    I just use SUBDIRLIST

    $ cat > /tmp/plist
    mail/postfix/stable
    mail/amavisd-new
    mail/p5-Mail-SpamAssassin
    mail/razor-agents

    $ cd /usr/ports
    $ SUBDIRLIST=/tmp/plist BIN_PACKAGES=Yes TRUST_PACKAGES=Yes BULK=Yes \
    make install

  6. By hubertf (217.84.18.56) hubertf@pkgsrc.org on www.feyrer.de

    pkgsrc supports the concept similar to all it's ancestors and siblings: Make a package (port :) that pulls in everything needed as dependencies, and then -- if you want -- make it adjust configuration as the "superport"s special feature.

    Right now pkgsrc only prints what changes need to be made: E.g. if you install the ap-php package, it will tell you what changes to make in httpd.conf. If you install php-gd, it will tell you what to change in php.init. This behaviour is standard in pkgsrc. Making "superports" (or meta-pkgs, as can be found in pkgsrc/meta-pkgs) do that work shouldn't be hard.

    pkgsrc runs on NetBSD, FreeBSD, OpenBSD, Darwin, MacOS X and some others (Solaris, Linux, Interix/MS Windows, Irix). See http://www.pkgsrc.org/ for more information!


    - Hubert

  7. By r0sk (80.58.15.107) r0sk@userlinux.net on http://www.userlinux.net

    Hums, at this point.. anyone was able to configure a Postfix with SASL2 and MySQL support? Is there a binary on ftp or anywhere to install with pkg_add?. Thanks in advice.

  8. By jose (82.108.34.130) on http://monkey.org/~jose/

    chad: quick answer. the ports tree and package subsystem definitely support it. no one that i have seen has any of these assembled. making one for yourself based on the freebsd ones is a good idea. the use of a SUBDIRLIST looks like it requires you to build stuff. i don't always have the time or the disk to build stuff when i have a laptop i just built or upgraded.

    personally i think they should be in-tree, despite the fact that many people differ on what they want on their workstation.

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]