OpenBSD Journal

Update on Filesystem Mini-Hackathon

Contributed by jason on from the quotas-are-for-cops-not-filesystems dept.

Nikolay Sturm writes in to remind us about an upcoming event:

f2k7, the OpenBSD Filesystem Mini-Hackathon, will take place from April
10th to April 15th 2007 in Vienna. We expect about 15 hackers from
Europe (Austria, France, Germany, Hungary, Iceland, and Sweden) and Americas
(Brazil, Canada, and the USA). The event's focus will be on improving
OpenBSD's filesystem support, which includes topics like FFS2 (support
for filesystems of more than 1TB), NFSv4 (next generation NFS), Software
RAID, and the buffer cache.

Due to the nature of hackathons, this list cannot be extensive. People
go out for a beer the first evening, discuss their ideas and suddenly
everyone is working on something completely different than they
originally thought.

To make this event a success and actually permit people to work on these
topics, needs hardware most developers don't have lying around. It takes
RAIDs of more than 1TB to figure out the issues and test FFS2, fast
build machines will help compiling kernels, as most of the work takes
place in the kernel and we will compile a lot of them, and last but not
least we need machines with more than 4GB or RAM to sort out problems
with the buffer cache.

If you want to support this event, we are still looking for fast build
machines like Sun Fire X2100 M2 or comparable and big SATA and IDE
drives (>160GB, preferrably 250GB) as loans or donations. In order to be
able to buy the missing pieces for the project, we also take PayPal
donations to martin@catai.org (excess money will be forwarded as regular
OpenBSD donations).

[Editor's Note]: The event is getting close, so let's make sure the developers have the hardware they need to bring these goals to fruition. For all those folks clamoring for FFS2 and large disk support, now is the time to make your donations count! If you have any further questions, please contact Nikolay Sturm.

(Comments are closed)

Comments

  1. By Anonymous Coward (155.212.93.230) on

    Anyone know the prospects of having native file system ACLs added to OpenBSD's file system? That, along with RBAC or MAC would be awesome, although I would take FS ACLs for now. I'll be donating regardless.

    Comments

    1. By tedu (71.139.164.111) on

      > Anyone know the prospects of having native file system ACLs added to OpenBSD's file system? That, along with RBAC or MAC would be awesome, although I would take FS ACLs for now. I'll be donating regardless.

      poor.

  2. By Anonymous Coward (216.68.198.57) on

    FFS2 = ? According to wikipedia, an FFS2 from Amiga. Or some hacked
    up FFS, for 64 bit? Or meaning a FFS / with UFS2 bit 32 or 64 ?

    Would be nice to know where OpenBSD is going.

    Wish I could influence those with real $ to contribute for XFS support
    if OpenBSD team thought that best...

    Thanks again for all your hard work and quality.

    Comments

    1. By Anonymous Coward (216.68.198.57) on

      FF2 = UFS2, searched htdig, undeadly.
      Original asker 2, brain fart.

  3. By novas kopas (68.165.27.172) on

    Okay, it's something I've been looking for for a long time but why doesn't undeadly or openbsd setup some page: "bounties for features".

    Everybody can paypal or promise a bounty for a specific feature such as java in ports or whatever and people would bid/promise money for each feature.

    Once a feature is labeled 'stable' the dev(s) gets 25% of the bid (simply because whichever project may have required the dev to buy some hardware?) and 75% goes to openbsd.

    Features could be anything, for instance:
    "Port OpenBSD to Sun's niagara servers" $1500
    "Support 3ware-9500 raid card in openbsd" $2500
    "Add Zork I to ports/games" $5
    or whatever.

    I know my company would be interested bidding on some features, it's like outsourcing but cheaper and money goes to the right cause.

    Comments

    1. By David Gwynne (dlg) on

      > "Support 3ware-9500 raid card in openbsd" $2500

      is that how much you value this support? or is it how much you think a developer needs to be compensated for the pain?

      Comments

      1. By Joao Baptista (81.134.117.212) joaobaptista@hotmail.com on

        > > "Support 3ware-9500 raid card in openbsd" $2500
        >
        > is that how much you value this support? or is it how much you think a developer needs to be compensated for the pain?

        I think that the figures are just to explain the idea, and NOT what he thinks they are worth!

        New ideas are always good, don't try to skew them into something that it isn't.

        Comments

        1. By Anonymous Coward (151.136.100.2) on

          > > > "Support 3ware-9500 raid card in openbsd" $2500
          > >
          > > is that how much you value this support? or is it how much you think a developer needs to be compensated for the pain?
          >
          > I think that the figures are just to explain the idea, and NOT what he thinks they are worth!
          >
          > New ideas are always good, don't try to skew them into something that it isn't.

          new 3ware cards are bullshit and are not worth the effort.
          not the old ones were really worth anything either...

        2. By David Gwynne (dlg) on

          > > > "Support 3ware-9500 raid card in openbsd" $2500
          > >
          > > is that how much you value this support? or is it how much you think a developer needs to be compensated for the pain?
          >
          > I think that the figures are just to explain the idea, and NOT what he thinks they are worth!
          >
          > New ideas are always good, don't try to skew them into something that it isn't.

          ah, yeah. i guess i was just surprised at the example.

          looking back i am a bit amused by this. as a developer i do hear a lot of requests from users along the lines of "omg i would love it if openbsd had more buzzword compliant filesystems". and now along come some developers going "we need some help to do some work on cool filesystem stuff". then a user goes "it would be nice if users had a mechanism to help get their requests worked on, such as some filesystem stuff".

          the filesystem guys are giving us an easy way to help get development going, and you're asking how you can help get development going. seems like a bit of a loop to me.

          on a related note though, donations do help get things done. for example, someone sent me an arc(4) when i asked for completely different hardware, and now we have a driver. it seems we do agree that sometimes people have to put their money where their mouths are, just the how is still open to discussion.

  4. By jirib (195.212.29.163) on

    software raid - wow, maybe something which we could control via bioctl :) that would be great...

    Comments

    1. By Anonymous Coward (87.79.237.121) on

      > software raid - wow, maybe something which we could control via bioctl :) that would be great...

      What's wrong with raidctl(8)? Works great for me.
      Of course I use -A yes and -A root... keeping the
      raid<n>.cfg files around is just stupid.

  5. By gwyllion (134.58.253.130) on

    By buffer cache do you mean UBC (Unified Buffer Cache)?

  6. By Anonymous Coward (151.136.100.2) on

    this whole thing is obviously a covert attempt to revive LFS!

    Comments

    1. By Anonymous Coward (87.79.237.121) on

      > this whole thing is obviously a covert attempt to revive LFS!

      I *do* hope it _is_ because LFS rocks (in theory).

      RAIDframe rocks too, but due to the Y2038 problem,
      we'll need UFS2 (for both ffs and lfs ;) soon...
      the storage size problem has already been explained.

  7. By Anonymous Coward (156.34.239.21) on

    I'm sure this is a naive post, so hold back with the flame-throwers. I have noticed that OpenBSD becomes very unresponsive if any process does a long disk I/O operation. I don't claim to understand how or even if this actually relates to the filesystem, or if it is in anyway fixable, but I certainly think it would be a significant improvement to workstation/desktop usability if this behavior could be altered to keep the system more responsive ....

    Comments

    1. By Anonymous Coward (82.135.30.22) on

      did you read the article really?
      it's not about what you want.
      it's about what developers NEED to do the work.
      without what they NEED nothing will happen of
      what you WANT. besides you can do it yourself...

    2. By Anonymous Coward (198.175.14.5) on

      > I'm sure this is a naive post, so hold back with the flame-throwers. I have noticed that OpenBSD becomes very unresponsive if any process does a long disk I/O operation. I don't claim to understand how or even if this actually relates to the filesystem, or if it is in anyway fixable, but I certainly think it would be a significant improvement to workstation/desktop usability if this behavior could be altered to keep the system more responsive ....

      How do you think OpenBSD can fix your excessive I/O problem??
      If you are depending on the hard disk for data, and the disk is busy, you're fucked. The operating system is not going to make a difference here, unless it does a lot of caching or uses other techniques to stave down the required disk i/o. The only real solution is to either change how you use the machine, or get more disks.

      Although NCQ support provides a 10-20% i/o boost, you ultimately need faster hard disks or RAID or both. You would be pleasantly surprised at what happens when you have enough disk i/o capability to support your usage demands.

      Comments

      1. By Anonymous Coward (156.34.212.120) on

        > How do you think OpenBSD can fix your excessive I/O problem??
        > If you are depending on the hard disk for data, and the disk is busy, you're fucked. The operating system is not going to make a difference here, unless it does a lot of caching or uses other techniques to stave down the required disk i/o. The only real solution is to either change how you use the machine, or get more disks.
        >
        > Although NCQ support provides a 10-20% i/o boost, you ultimately need faster hard disks or RAID or both. You would be pleasantly surprised at what happens when you have enough disk i/o capability to support your usage demands.

        Well, perhaps it just a limit of harddisk technology then, and falls in to the 'unfixable' (without better hardware) category. Like I said, I just don't know, and almost anything I say will probably just sound idiotic to someone that truely understands what issues are at play. I just noticed (and I think I this a fair characterization) that one process that is, say, copying a very large file, is able to monopolize the disk for a very long time before any another process requiring even heartbeat of access gets it. I really don't use other OSes enough to separate the influence of the OS from the limits of the hardware. Strange that with so much focus on multitasking when the CPU is concerned, that harddisks wouldn't do this a lot better than they seem to then!

        Comments

        1. By Anonymous Coward (66.39.179.5) on

          >
          > Strange that with so much focus on multitasking when the CPU is concerned, that harddisks wouldn't do this a lot better than they seem to then!

          Well, honestly, there's not magic going on here (at least not at the most basic level.)

          There's two issues here. First you have the "magic" level that you are not going to understand very well unless you research it. This amounts to "what happens in openbsd?", perhaps in relation to caching, request scheduling, file system layers, and so forth. There's certainly some interest to see how much performance improvement can be gained in OpenBSD with different disk I/O scheduling algorithms. There's also interest in NCQ and other techniques for streamlining disk access. These ideas are not holy grail fixes, just potentially modest (20-40%) performance improvements. So, CAN openbsd use improvement? Sure. But that's only part of the story. The rest is computer 101.

          What it really comes down to is moving parts, including heads that have to move around and seek out data. If you are spending a significant amount of time waiting for your hard disk to do this, then either:

          1. You have heavy or very heavy disk I/O requirements. Time to see what's causing it! Maybe you thrash the disks with large data sets. So, if that's it, then you should invest in a RAID controller (or even just try out ccd across multiple disks.) What you say? No big data munching? Your heavy disk I/O could be a sign of another problem. For instance, you just need to add more RAM because you've been digging into your swap space every time you fire up mozilla or openoffice.

          or

          2. You have an old hard disk and you should invest in a modern one!

          or

          3. Some combination of both 1 & 2

          With 160GB disks retailing for USD $60, you can build an inexpensive RAID that's really quite fast. Areca 1110/1210 controllers are around USD $300---not cheap, but very fast with RAID 5 on 3 drives, and very, very fast when you go to RAID 6 on 8+ drives. And well supported by OpenBSD. WD Raptor SATA disks are not cheap, but certainly some of the fastest SATA disks to build a RAID set with. I've used plain Seagate SATA+NCQ drives and the Areca controller is very impressive. It's notably faster than the LSI 150-4 and kicks the shit out of any of the older stuff!

      2. By Jussi Heino (MunOBSD) on

        >> I have noticed that OpenBSD becomes very unresponsive if any process does a long disk I/O operation.
        > How do you think OpenBSD can fix your excessive I/O problem??
        > If you are depending on the hard disk for data, and the disk is busy, you're fucked. The operating system is not going to make a difference

        OS can fix the responsivity issue, since (G)UI - in general - is not depending the HDD for data, it's the O.P.'s one process that is.

        NCQ is great, yes, but on the OS level there interrupt handling, DMA, process/thread priorities, IO queues, inter-process message handling etc.

        Server applications focus on throughput, but the user IRL is annoyed when/if Pine is not responsive during 'cp -r /alllegal/movies /backup/'.

  8. By Anonymous Coward (85.201.63.39) on

    I know I may already have said this. But OpenBSD is one of the nicest OS and has some capabilities no other OS has. I think more particularly of CARP wich is really great. However, if one would like to make a redundant web server with CARP he would have to get the pages data elsewhere than on the hosts, adding another single point of failure. So I think something very interesting almost no other OS has is a multi master real time replicating file system.

    Comments

    1. By Anonymous Coward (82.195.149.9) on

      > I know I may already have said this. But OpenBSD is one of the nicest OS and has some capabilities no other OS has. I think more particularly of CARP wich is really great. However, if one would like to make a redundant web server with CARP he would have to get the pages data elsewhere than on the hosts, adding another single point of failure. So I think something very interesting almost no other OS has is a multi master real time replicating file system.

      Yes, this would be very nice. It would also take a huge amount of work.

      Comments

      1. By David Gwynne (dlg) on

        > Yes, this would be very nice. It would also take a huge amount of work.

        like the work people do at hackathons?

        Comments

        1. By Anonymous Coward (151.136.100.2) on

          > like the work people do at hackathons?

          like the work people do instead of telling others
          what to do...

    2. By Chris (194.193.52.249) chriswareham@chriswareham.demon.co.uk on

      > I know I may already have said this. But OpenBSD is one of the nicest OS and has some capabilities no other OS has. I think more particularly of CARP wich is really great. However, if one would like to make a redundant web server with CARP he would have to get the pages data elsewhere than on the hosts, adding another single point of failure. So I think something very interesting almost no other OS has is a multi master real time replicating file system.

      Matt Dillon of DragonFlyBSD fame is hoping to implement this kind of filesystem. Given the major shift towards a message passing style of kernel with things being shunted out to userland, and talk of changes to the VFS layer as part of the work on the new filesystem, I don't expect it to be easily portable to the other BSDs.

      Chris

    3. By Anonymous Coward (198.175.14.5) on

      > I know I may already have said this. But OpenBSD is one of the nicest OS and has some capabilities no other OS has. I think more particularly of CARP wich is really great. However, if one would like to make a redundant web server with CARP he would have to get the pages data elsewhere than on the hosts, adding another single point of failure. So I think something very interesting almost no other OS has is a multi master real time replicating file system.

      nfsv4 is listed on the task list and it has a replication feature. check it out.

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]