OpenBSD Journal

True VPN Failover with IPsec

Contributed by mbalmer on from the if-it-fails-it-does-not-fail dept.

Mark Uemura, mtu@, reports on his work to setup failover VPNs:

Many great things came out of the c2k6 hackathon, one very important achievement is the support for true failover VPNs using IPsec. The story of sasyncd(8) and ipsecctl(8) must be told!

Late in 2005 when OpenBSD 3.8 came out, a sasyncd daemon first appeared. It was written by Håkan Olsson and funded by Multicom Security AB. As many know by now, sasyncd brought us true failover of VPNs with state. Håkan raised the bar considerably and to my knowledge OpenBSD is again the first to ever achieve this. Since many of us maintain VPNs, this was a blessing that we only dreamed of having come true.

We soon discovered that sasyncd was not quite polished and we also hoped that most of it would be finished during this year's hackathon. Ryan McBride (mcbride@) worked on fixing sasyncd while Hans Hoexer (hshoexer@) and Mathieu Sauve-Frankel (msf@) worked on ipsecctl (the isakmpd.conf killer). My role was simply to test and give feedback. I was also one of those guys who resisted change (isakmpd.conf --> ipsec.conf) even though Theo told me well in advance that isakmpd.conf was going the way of the Dodo bird real soon (not in those same words, of course). So it was an awkward situation when I kept telling Ryan that his diffs weren't working and he then finally replied, "You're using ipsec.conf, right?" Theo was in spitting distance to me and I thought for sure that I was going to get 50 lashes with a wet tip when I said, "ah, no ...I'm still using isakmpd.conf." *ducks for cover* Theo did warn me about changes forthcoming and I saw signs of this when I noticed that ipsecadm was taken out of the tree. I asked how do we manually flush tunnels now that ipsecadm was gone. Reyk Floeter (reyk@) turns to me and says, "ipsecctl -F". I kept thinking to myself, "duh, of course, even without reading the man page, that should have been obvious." Well, needless to say, that was the turning point for me but I needed help as I could never get ipsec.conf working for CARP'd VPNs.

Todd Fries (todd@) helped me to get ipsec.conf setup properly but it took us into the wee hours of the morning to do so. After a lot of troubleshooting we realized that we overlooked one very important thing when setting up ipsec.conf for use with CARP'd VPN Servers. We needed to specify the "local <localip>" to listen on the carp interface. Again, a silly oversight as I had specified this in my isakmpd.conf files. Anyhow, when we made that addition, it just magically worked and my huge isakmpd.conf file was turned into a one line ipsec.conf file! Even though we were both exhausted, it was like we just won the Super Bowl :)

The next day, I tackled sasyncd with my four box VPN system using the new ipsec.conf configuration and tried to fail over the VPN but without success. I noticed that when I failed over the MASTER VPN Server, the Security Associations (SA) were deleted on the other end (VPN peers). However, if I pulled the power on the MASTER, then the VPN failed over as expected. I sat down with Hans Hoexer and he told me that he knew how to fix it and that it would be a simple fix. Shortly thereafter I received a diff from Hans and it did the job! His change made sure that the flows were not deleted on a graceful failover. I could now failover the VPN with state using ipsec.conf rather than isakmpd.conf. Yet, there was one last piece of the puzzle that needed to be fixed -- sasyncd. By the time the hackathon was almost over, on the last day in fact, Ryan did his usual magic and had finished sasyncd to start and synchronize the SA only after the rc scripts were completed on boot taking into account if it was MASTER or BACKUP.

True VPN failover while keeping state is now a reality!

(Comments are closed)


  1. By Anonymous Coward () on

    About killing isakmpd.conf as ipsecadm was killed: is it really yet the time ?

    ipsecctl(8) is a great tool, ultra promising, but it's not like he could, just now, be a drop in replacement isakmpd.conf(8) + keynote(5) for all supported cases.
    Some examples of features that still requires isakmpd.conf & keynote :

    * Accept connexions for client we don't yet know the adress of (road warriors, under isakmpd.conf, the "Default" keyword, for [Phase 1] setup).

    * Uses of X509 certificates properties, for eg. authenticating from certs CA subject, or use subjectAltName as USER_FQDN ID ...

    * Allowing proposals with multiples cryptos suites (eg. to propose and accept "AES or Blowfish or 3DES or ..."). This is useful because many implementations have limited crypto support, so our peers can sometime come up with only MD5+3DES (Windows), or without SHA2 (Greenbow, isakmpd on Linux, racoon, Cisco IOS), or no AES for phase1 (racoon) etc. So when you don't know or can't control you peers setup, isakmpd.conf remains useful. Or maybe should we have a "relaxed mode" option for ipsecctl, that woud propose and accept combinations of (md5 or sha1 or sha2) and (AES or BF or 3DES) ?

    * AH is not yet supported (well, that's not a huge lost anyway ;).

    * Can't be used to choose the lifetimes (eg. too please braindeads peers like Cisco PIX), or activate/deactivate PFS, choosing the PFS group ...

    * As far as I know, can't setup an IKE tunnel with both ipcomp and esp (or am I wrong ?)

    * You talked about flushing with ipsectl -F: ipsecadm allowed us to flush a specific SA (from SPI #), that was nice and I'll miss this.

    So isakmpd.conf and isakmpd.policy remains useful. In the meantime, ipsecctl(8) Just Work, and with the one-line-is-enough policy, it's even dumb-admins compliant. Cheers guys !

    1. By David Krause () on

      Some of the things you mentioned are recently supported in -current ipsecctl and more work is still happening.

      > * Accept connexions for client we don't yet know the adress of (road
      > warriors, under isakmpd.conf, the "Default" keyword, for [Phase 1]
      > setup).

      date: 2006/06/01 02:20:44; author: hshoexer; state: Exp; lines: +9 -24
      When no peer is specified, make this rule a "catch-all" rule for any remote
      peer. Similar to isakmpd(8)s "Default=" tag.

      > * Can't be used to choose the lifetimes (eg. too please braindeads peers
      > like Cisco PIX), or activate/deactivate PFS, choosing the PFS group
      > ...
      >

      date: 2006/06/02 05:59:31; author: hshoexer; state: Exp; lines: +30 -3
      allow to specify phase 1 and 2 lifetimes. Right now, these values
      can only be set globally (ie. Default-phase-[12]-lifetime).

      date: 2006/05/27 17:21:40; author: hshoexer; state: Exp; lines: +34 -2
      allow to specify groups to be used IKE

      date: 2006/06/18 18:18:01; author: hshoexer; state: Exp; lines: +2 -1
      add group "none"; when choosen, pfs will be disabled.


      > * You talked about flushing with ipsectl -F: ipsecadm allowed us to flush a
      > specific SA (from SPI #), that was nice and I'll miss this.

      ipsecctl -d?

      1. By cruel () on

        > date: 2006/06/02 05:59:31; author: hshoexer; state: Exp; lines: +30 -3 > allow to specify phase 1 and 2 lifetimes. Right now, these values > can only be set globally (ie. Default-phase-[12]-lifetime). "main life" and "quick life" are not documented in ipsec.conf(5). this is because some people can't see them.

    2. By kokamomi () on

      > About killing isakmpd.conf as ipsecadm was killed: is it really yet the time ?

      well, as has been shown before in openbsd history, proactively deleting bad implementations can actually encourage development of their replacements. if you really need the borderline scenarios, you're probably fine sticking with 3.9 for a while stable while ipsecctl matures.

      1. By Anonymous Coward () on

        > > About killing isakmpd.conf as ipsecadm was killed: is it really yet the time ? > > well, as has been shown before in openbsd history, proactively deleting bad implementations can actually encourage development of their replacements. if you really need the borderline scenarios, you're probably fine sticking with 3.9 for a while stable while ipsecctl matures. Isakmpd(8) isn't by any way a "bad implementation" !
        The case for ipsecctl(8) is that, for simple cases, IPsec setups could be made easier.

        1. By Anonymous Coward () on

          > About killing isakmpd.conf as ipsecadm was killed: is it really yet the time ?
          >
          > well, as has been shown before in openbsd history, proactively deleting bad implementations can actually encourage development of their replacements. if you really need the borderline scenarios, you're probably fine sticking with 3.9 for a while stable while ipsecctl matures.
          >
          > Isakmpd(8) isn't by any way a "bad implementation" !
          > The case for ipsecctl(8) is that, for simple cases, IPsec setups could be made easier.

          Thats good that for simple cases IPsec setups will be made easier. I would expect that more advanced users will still be able to drill down and do manual more complex configurations.

      2. By Anonymous Coward () on

        > > About killing isakmpd.conf as ipsecadm was killed: is it really yet the time ? > > well, as has been shown before in openbsd history, proactively deleting bad implementations can actually encourage development of their replacements. if you really need the borderline scenarios, you're probably fine sticking with 3.9 for a while stable while ipsecctl matures.
        ipsec.conf is to isakmpd.conf as sendmail.mc is to sendmail.cf. At least, that was the original idea.
        Our plan was to get someone else write a ``real'' configuration file format that could be translated into our style. So far no one has taken the bait. Note that ideally, very little configuration should be needed for isakmpd; most of the information should be provided on-the-fly by the kernel (at least in the end-to-end case), or through some security policy discovery mechanism.

  2. By Anonymous Coward () on

    ipsecctl(8) link is pointing to the sasyncd(8) link...

    1. By Mike Jacobi () on

      This is awesome stuff. A quick note: You state that you aren't aware of any other solutions that can do stateful VPN failover -- according to Cisco, the PIX does stateful VPN failover too.

      http://www.cisco.com/univercd/cc/td/doc/product/multisec/asa_sw/v_70/config/failover.htm#wp1051178

      1. By stuff () on

        The first FREE solution perhaps ?

      2. By wob () wob@bonch.org on

        > This is awesome stuff. A quick note: You state that you aren't aware of any other solutions that can do stateful VPN failover -- according to Cisco, the PIX does stateful VPN failover too.
        >
        > http://www.cisco.com/univercd/cc/td/doc/product/multisec/asa_sw/v_70/config/failover.htm#wp1051178

        Same with Juniper/Netscreen. Their higher models from the NS204 and up will do stateful failover. Not that they're any good at it. :)

      3. By Anonymous Coward () on

        > This is awesome stuff. A quick note: You state that you aren't aware of any other solutions that can do stateful VPN failover -- according to Cisco, the PIX does stateful VPN failover too.
        >
        > http://www.cisco.com/univercd/cc/td/doc/product/multisec/asa_sw/v_70/config/failover.htm#wp1051178

        True...but for *LARGE* dollar. We use PIX 535's at the Internet gateway and several smaller ones (525's and such) throughout the enterprise, and while they work, Cisco's increasing frequency of bugs is concerning us.

        Now, with OpenBSD, enterprises can purchase a copy for $40 and be able to do nearly everything that a Cisco PIX can do, and we have the very important security of the source code being Free and, thus, auditable. If I were in a position to do so, I would take half the money saved from a PIX solution and shoot that to the OpenBSD project.

      4. By scot bontrager () firstname at indievisible.org on

        > This is awesome stuff. A quick note: You state that you aren't aware of any other solutions that can do stateful VPN failover -- according to Cisco, the PIX does stateful VPN failover too.
        >
        > http://www.cisco.com/univercd/cc/td/doc/product/multisec/asa_sw/v_70/config/failover.htm#wp1051178

        I am doing stateful failover of IPSEC tunnels with PIX 535's but it requires a private network for the states to be sync'd (Unless they've changed something since last time I've reconfigured them) IP states sync over the failover cable, VPN states require a dedicated ethernet interface on each system. Using the old-school failover cable you are limited to two PIXies to a cluster.

        I've been running PIX's for 10 years now and OpenBSD since 2.2... guess which one I run at home?

        I'm in the process of changing from isakmpd.conf to ipsec.conf now. Everything looks good so far.

  3. By reader () on

    Do I understand correctly that this is possible with 3.8 or must we wait for a 3.9-current or 4.0 ? And, well, a Howto saves me some time as well :-)

    great job, I use ipsec.conf for some time now, works like a charm for a quick and easy vpn setup between networks (VPN in 4 minutes :-)

    1. By ciph3r () on

      All this changes are in -current and will be if all goes OK in 4.0.

    2. By Anonymous Coward () on

      > Do I understand correctly that this is possible with 3.8 or must we wait for a 3.9-current or 4.0 ? And, well, a Howto saves me some time as well :-)

      it's possible to failover in 3.8, but failing back when the machine comes back up again is borked. you need current code for that to work, and to pick up some other nice changes. testers are always welcome.

  4. By mcbride@ () on

    As a note: Much of the credit for the challenging and tweaky fixes to sasyncd, isakmpd, and the kernel goes to Nathanael Rensen.

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]