Contributed by merdely on from the embedded-packet-passing dept.
I recently purchased a new Soekris net5501 to replace my beige box firewall. I had previously set up a net4501 but I wasn't happy with it and sold it to a friend. Large file transfers would grind it to a halt and the performance wasn't as good as my beige box. The net5501 has increased horsepower (faster processor, more memory) and a better network chipset (vr(4)). And, most of all, because there are 4 network ports! At my house I have 3 network + my FiOS connection. I have my internal LAN (wired with full access to everything), my wireless network (requires authentication and has limited access to the LAN) and my DMZ (for my web server; no access to the LAN or wireless networks).
For my installation, I use Yaifo so I don't have to deal with a serial console or setting up pxeboot. (I actually did hook up a serial console to update the bios, which I'll discuss later). Also, I use a custom rc and a flashdist-like system so I can mount my CF read-only. My "fdlite" script doesn't rely on a customized install like flashdist. It does use some of the device modifications Chris uses to make the read-only / work properly, though.
Preparing the CompactFlash card
- Download the (yet to be released) Yaifo 0.5 for OpenBSD 4.2 or check out the latest yaifo from CVS:
- Run: cvs -d:pserver:anonymous@yaifo.cvs.sourceforge.net:/cvsroot/yaifo login
- Run: cvs -z3 -d:pserver:anonymous@yaifo.cvs.sourceforge.net:/cvsroot/yaifo co -P yaifo
- Copy your ~/.ssh/authorized_keys to the yaifo directory.
- If upgrading or migrating, copy the /etc/ssh/ssh_host_* files to the yaifo directory.
- Uncomment the two lines in yaifo/boot.conf to enable booting to the serial console. The Soekris will not boot without this.
- Edit the yaifo/config file to specify the NIC to use for Yaifo and optionally static IP information.
- With current CVS sources checked out to /usr/src, build Yaifo (make obj; make).
- Connect a CompactFlash card reader (with the CF card).
- Copy the yaifo.fs image to the CF card: dd if=yaifo.fs of=/dev/rsd0c (substitute the correct device).
Installing OpenBSD 4.2
- Choose either locally mirrored files or an official mirror.
- Go through the OpenBSD installation process normally with the following exceptions:
- During the label editing phase, create / with $SIZE - 1 blocks and a swap with 1 block.
- Limit the installation sets. I chose bsd, bsd.rd, base42.tgz, etc42.tgz and man42.tgz. It's taking up 172MB of my 256MB CF.
- When prompted at the end, choose to redirect output to a serial console. It's best to choose 19200 baud because that's what the net5501 defaults to.
- When the installation is complete, wait before rebooting.
Setting up the Read-only /
- Download my fdlite.sh script.
- Download my fdlite.rc to "./rc".
- Run: sh ./fdlite.sh
- Assuming you renamed fdlite.rc to rc, press Enter when prompted.
- Reboot when finished.
- Expect things to break. In fact, something is seriously wrong if things didn't break!
- When the system boots, log in as root and mount / read-write; run: mount -uw /
- Edit /etc/rc for the system's configuration.
- I configure /etc/syslog.conf to send syslogs to a remote server. For a firewall, consider using the pflogrotate method described in the PF FAQ. The existence of /etc/pflogrotate automatically sets up newsyslog and pflogrotate in root's crontab.
- Make sure /etc/resolv.conf exists and is correct. Same with /etc/mygate.
- Consider setting a profile for root (or the whole system) by downloading something like my example profile for either /root/.profile or /etc/profile.
- Edit /etc/pf.conf
- It's probably best to reboot at this point to test the boot process. Or, set root back to read-only with: mount -ur /
After following these steps, my Soekris net5501 is running perfectly with a read-only root, logging to a remote host.
Updating My Soekris's BIOS
- Download the newest BIOS file.
- Set up a serial connection and use tip to connect: tip -19200 ttyU0 (for example)
- Install, if necessary, the lrzsz port: pkg_add lrzsz
- Reboot the Soekris into Monitor mode (Ctrl+P).
- Type: download
- Type: ~C (that's tilde, shift+c)
- Type: lsz -X b5501_132d.bin
- After the file is downloaded, type: flashupdate
- Reboot the Soekris into the newest BIOS version.
(Comments are closed)
By Anonymous Coward (76.87.252.90) on
Comments
By Mike Erdely (merdely) on http://erdelynet.com/
I can max out my LAN to LAN (100 Mbps) connection. It is performing as well as my old beige box did. So, yes, I can max out my FiOS connection.
Comments
By eax (82.236.190.62) on
OpenBSD 4.2 (GENERIC) #375: Tue Aug 28 10:38:44 MDT 2007
deraadt@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
RTC BIOS diagnostic error 80<clock_battery>
cpu0: Geode(TM) Integrated Processor by AMD PCS ("AuthenticAMD" 586-class) 499 MHz
cpu0: FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CFLUSH,MMX
real mem = 268009472 (255MB)
avail mem = 251506688 (239MB)
RTC BIOS diagnostic error 80<clock_battery>
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 08/01/07, BIOS32 rev. 0 @ 0xfcc1a
pcibios0 at bios0: rev 2.1 @ 0xf0000/0x10000
pcibios0: pcibios_get_intr_routing - function not supported
pcibios0: PCI IRQ Routing information unavailable.
pcibios0: PCI bus #0 is the last bus
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 1 function 0 "AMD Geode LX" rev 0x31
glxsb0 at pci0 dev 1 function 2 "AMD Geode LX Crypto" rev 0x00: RNG AES
vr0 at pci0 dev 12 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 15, address 00:0d:b9:12:50:bc
ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI 0x004063, model 0x0034
vr1 at pci0 dev 13 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 11, address 00:0d:b9:12:50:bc
ukphy1 at vr1 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI 0x004063, model 0x0034
pcib0 at pci0 dev 15 function 0 "AMD CS5536 ISA" rev 0x03
pciide0 at pci0 dev 15 function 2 "AMD CS5536 IDE" rev 0x01: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: <TRANSCEND>
wd0: 1-sector PIO, LBA, 495MB, 1014048 sectors
wd0(pciide0:0:0): using PIO mode 4, DMA mode 2
pciide0: channel 1 ignored (disabled)
"AMD CS5536 Audio" rev 0x01 at pci0 dev 15 function 3 not configured
ohci0 at pci0 dev 15 function 4 "AMD CS5536 USB" rev 0x02: irq 9, version 1.0, legacy support
ehci0 at pci0 dev 15 function 5 "AMD CS5536 USB" rev 0x02: irq 9
usb0 at ehci0: USB revision 2.0
uhub0 at usb0: AMD EHCI root hub, rev 2.00/1.00, addr 1
isa0 at pcib0
isadma0 at isa0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pccom0: console
usb1 at ohci0: USB revision 1.0
uhub1 at usb1: AMD OHCI root hub, rev 1.00/1.00, addr 1
biomask 77ef netmask ffef ttymask ffef
pctr: user-level cycle counter enabled
mtrr: K6-family MTRR support (2 registers)
nvram: invalid checksum
dkcsum: wd0 matches BIOS drive 0x80
root on wd0a swap on wd0b dump on wd0b
clock: unknown CMOS layout
WARNING: clock time much less than file system time
WARNING: using file system time
WARNING: CHECK AND RESET THE DATE!
By Anonymous Coward (167.202.221.228) on
my DMZ (for my web server; no access to the LAN or wireless networks).
Unquote
Does this also mean that your LAN or Wireless Networks have no access to the DMZ? Is it Isolated or not?
I'm thinking of setting up something similarly and was glad you gave some detais about your network.
Thanks
Comments
By Anonymous Cow (193.53.59.134) on
> my DMZ (for my web server; no access to the LAN or wireless networks).
> Unquote
>
> Does this also mean that your LAN or Wireless Networks have no access to the DMZ? Is it Isolated or not?
>
> I'm thinking of setting up something similarly and was glad you gave some detais about your network.
>
> Thanks
In big lines it means:
(Some) Traffic is allowed from Internet & LAN/WLAN to DMZ
(Most) Traffic is denied from Internet & DMZ to LAN/WLAN
As a starting point, see: http://en.wikipedia.org/wiki/Demilitarized_zone_%28computing%29
By Mike Erdely (merdely) on http://erdelynet.com/
> to the DMZ? Is it Isolated or not?
>
> I'm thinking of setting up something similarly and was glad you gave
> some detais about your network.
My LAN has access to anything.
My Wireless LAN, authenticated, has very limited access to the LAN (specific http, ldp, dhcp, dns and secure ports like https, ssh). It has the same access to the DMZ'd server as connections from the Internet would.
My DMZ has unrestricted access to the Internet and *NO* access to any other network.
The Internet has access to http, https, imaps, pop3s, smtp, submission, smtps, ftp and ssh to my DMZ and ssh to certain hosts on my LAN.
Hope that helps.
By sthen (85.158.44.149) on
I'm not sure I've heard vr(4) called "better" before (-:
Comments
By Mike Erdely (merdely) on http://erdelynet.com/
>
> I'm not sure I've heard vr(4) called "better" before (-:
Of course I meant "better than sis(4)".
By Anonymous Coward (151.43.200.129) on
Tank's a lot !
What about the real throughput ?
May you give some example of bandwidth-per-pf.conf ?
By QuailRider (74.110.140.116) on
Comments
By Mike Erdely (merdely) on http://erdelynet.com/
> I'm still debating the best way to restrict my own wireless DMZ segment.
Be gentle:
http://erdelynet.com/downloads/pf/pf.conf
Comments
By Mike Erdely (merdely) on http://erdelynet.com/
> http://erdelynet.com/downloads/pf/pf.conf
Forgot to add http://erdelynet.com/downloads/pf/authpf.rules for my wifi connections.
By joe (204.176.49.45) on
Comments
By Brad (2001:4830:122b:3:216:41ff:fe17:6933) brad at comstyle dot com on
For 2.5" drives PATA is still very common. But the board also has a SATA
port which until recently wasn't usable due to a BIOS bug.
Comments
By Anonymous Coward (85.178.110.194) on
>
> For 2.5" drives PATA is still very common. But the board also has a SATA
> port which until recently wasn't usable due to a BIOS bug.
Why don't they test their HW before they start selling...?
Sorry but that is just.... well... anyway...
The customer gets what the customer deserves...
Comments
By Mark (192.61.201.164) markpeloquin@gmail.com on
> >
> > For 2.5" drives PATA is still very common. But the board also has a SATA
> > port which until recently wasn't usable due to a BIOS bug.
>
> Why don't they test their HW before they start selling...?
> Sorry but that is just.... well... anyway...
>
> The customer gets what the customer deserves...
It's not a hardware bug. It's a BIOS bug. It can be fixed with a BIOS update, I'm sure.
Perhaps they thought that it would be better to not hold up the release for something that can be patched. It makes more sense, in a way. I could wait for them to send me a board with an updated BIOS, and all that assembly, packaging, shipping, etc. Or I could get my board now and wait on that new BIOS.
But then I don't know the details. How long was the SATA port unusable? That makes all the difference.
Comments
By sthen (85.158.44.149) on
Actually, they held back on selling the SATA kits...
By Anonymous Coward (24.37.242.64) on
I do it on Soekris, WRAP, SD, USB Flash, etc., works like a charm for me - running -stable (no custom rc, just one line commented) and I use rsync to sync MFS to Flash via a crontab and /etc/rc.shutdown.
A sample, from a WRAP box:
$ mount
/dev/wd0a on / type ffs (local, noatime, read-only)
mfs:25084 on /dev type mfs (asynchronous, local, noexec, nosuid, size=1200 512-blocks)
mfs:877 on /var type mfs (asynchronous, local, nodev, noexec, nosuid, size=32768 512-blocks)
mfs:3140 on /tmp type mfs (asynchronous, local, nodev, nosuid, size=2048 512-blocks)
I've created something like flashdist a long time ago that helps install stock OpenBSD on such devices but not a 'modified' version; mainly for stock OpenBSD -stable and read-only flash memory devices as mentioned above, with the rest in MFS... BSD licensed shell script of course. =)
I'm still debating cleaning it up and releasing it here or somewhere - hopefully others can help clean it up and maybe add things to it, but I don't have anywhere to host it yet.
Also another nifty script for CARP with DHCP environments - especially for those ISP's that only dish out IP's for reserved/registered MAC addresses - works flawlessly so far and if I can release it, it'll be BSD licensed too - been using this for a while too without problems.
Comments
By Anonymous Coward (203.20.79.132) on
Are you using rsync to minimize writes to flash?
Comments
By Anonymous Coward (24.37.242.64) on
>
> Are you using rsync to minimize writes to flash?
>
It's always read-only except when I sync, so I use it mainly to sync my log files and such periodically or on a graceful shutdown. It's like keeping a hard-copy of things too.
Comments
By Anonymous Coward (203.20.79.132) on
> >
> > Are you using rsync to minimize writes to flash?
> >
>
> It's always read-only except when I sync, so I use it mainly to sync my log files and such periodically or on a graceful shutdown. It's like keeping a hard-copy of things too.
Okay. I thought you may have been using rsync instead of a plain copy, since rsync reads through blocks on the destination and mostly only writes to those that are different from the source.
As opposed to updating all blocks with a copy.
I run my Sun firewalls off CF. I only use softdep and noatime. They've been going great for years (fingers crossed).
Comments
By baldusi (200.68.102.49) on
>
> As opposed to updating all blocks with a copy.
>
> I run my Sun firewalls off CF. I only use softdep and noatime. They've been going great for years (fingers crossed).
>
Please remember that reading a flash card is a destructive process. So rsync would actually be worse for the worn off.
Comments
By Anonymous Coward (204.80.187.5) on
> >
> > As opposed to updating all blocks with a copy.
> >
> > I run my Sun firewalls off CF. I only use softdep and noatime. They've been going great for years (fingers crossed).
> >
>
> Please remember that reading a flash card is a destructive process. So rsync would actually be worse for the worn off.
hunh? reading a flash card is not a destructive process. bullshit.
Comments
By Anonymous Coward (200.68.102.49) on
> > Please remember that reading a flash card is a destructive process. So rsync would actually be worse for the worn off.
>
> hunh? reading a flash card is not a destructive process. bullshit.
When you read you actually erase and thus you have to rewrite the value. At least with NAND Flash.
Comments
By Anonymous Coward (206.248.190.11) on
> > > Please remember that reading a flash card is a destructive process. So rsync would actually be worse for the worn off.
> >
> > hunh? reading a flash card is not a destructive process. bullshit.
>
> When you read you actually erase and thus you have to rewrite the value. At least with NAND Flash.
No you don't. WTF are you smoking?
Comments
By Anonymous Coward (200.68.102.49) on
> > > > Please remember that reading a flash card is a destructive process. So rsync would actually be worse for the worn off.
> > >
> > > hunh? reading a flash card is not a destructive process. bullshit.
> >
> > When you read you actually erase and thus you have to rewrite the value. At least with NAND Flash.
>
> No you don't. WTF are you smoking?
Yes you do. It's just hidden by the hardware of the flash chip. That's why they rate for 100,000 "cycles" instead of "writes". So using rsync means a read+write, while a straight cp will do just a write and spread the write to new sectors. Which is good since CF cards not always have a wear levelling.
Comments
By sthen (85.158.44.149) on
> > > > > Please remember that reading a flash card is a destructive process. So rsync would actually be worse for the worn off.
> > > >
> > > > hunh? reading a flash card is not a destructive process. bullshit.
> > >
> > > When you read you actually erase and thus you have to rewrite the value. At least with NAND Flash.
>
> > No you don't. WTF are you smoking?
>
> Yes you do. It's just hidden by the hardware of the flash chip. That's why they rate for 100,000 "cycles" instead of "writes".
Bullsh*t. That's *write/erase* cycles.
Also, 100,000 cycles sounds like NOR flash, which hasn't been used in CF for years. NAND flash is generally quoted being *at least* 10x that, usually more. And that's per-block, without taking wear levelling into account.
> Which is good since CF cards not always have a wear levelling.
Care to offer an example? While it's not required by the CF spec, I think you'll have to try hard to find a card with no form of wear levelling. Especially for NAND based CF since the controller generally already needs to support remapping (looks like it's assumed that bad blocks will exist, otherwise the yield is too low).
By Anonymous Coward (218.214.194.113) on
> > > > > Please remember that reading a flash card is a destructive process. So rsync would actually be worse for the worn off.
> > > >
> > > > hunh? reading a flash card is not a destructive process. bullshit.
> > >
> > > When you read you actually erase and thus you have to rewrite the value. At least with NAND Flash.
> >
> > No you don't. WTF are you smoking?
>
> Yes you do. It's just hidden by the hardware of the flash chip. That's why they rate for 100,000 "cycles" instead of "writes". So using rsync means a read+write, while a straight cp will do just a write and spread the write to new sectors. Which is good since CF cards not always have a wear levelling.
No. They ARE rated in write cycles. And BTW I haven't seen a CF card without wear-levelling in years. I imagine there are some old ones that haven't been written much in somebody's museum somewhere.
Anyway here is a quote from some Kingston marketing blurb quoting Toshiba research:
"Flash Cell Endurance of Kingston 256MB CF Memory Card: Up to 10,000 Multi- Level Cell (MLC) Flash or up to 100,000 Single-Level Cell (SLC Flash) write cycles per physical sector in Kingston 256MB CF Memory Card. According to Toshiba, the inventor of flash memory: “the 10,000 cycles of MLC NAND is more than sufficient for a wide range of consumer applications, from storing documents to digital photos. For example, if a 256MB MLC NAND Flashbased card can typically store 250 pictures from a 4-megapixel camera (a conservative estimate), its 10,000 write/erase cycles, combined with wear-leveling algorithms in the controller, will enable the user to store and/or view approximately 2.5 million pictures within the expected useful life of the card.” 1 For USB flash drives, Toshiba calculated that a 10,000 write cycle endurance would enable customers to “completely write and erase the entire contents once per day for 27 years, well beyond the life of the hardware.” SLC flash based-products, typically found in Kingston 256MB CF Memory Card and DataTraveler 2.0 USB flash drives, offer both high-performance and high endurance. "
Note: "write cycles" and "write/erase cycles" are the quoted endurance metrics.
As far as use for an OS is concerned try this:
I loaded a 512MB Apacer PhotoSteno II CF with OpenBSD as if it was a hard disk. It sat in front of a mailserver running spamd, ntpd and named with the highest levels of logging verbosity I could turn on. It went through two OS re-installs and in 14 months of thrashing nothing failed.
I then moved the Soekris to firewall duty elsewhere with the same CF and it is still running 10 months later. Go figure.
By Anonymous Coward (2001:6f8:124a::6) on
By default, rsync copies the target file to a temporary file, updates the copy, and finally moves the copy into place. If you want actual in-place updating, you need to specify the --inplace option. See the man page for caveats.
Comments
By Anonymous Coward (203.20.79.132) on
>
> By default, rsync copies the target file to a temporary file, updates the copy, and finally moves the copy into place. If you want actual in-place updating, you need to specify the --inplace option. See the man page for caveats.
>
Thanks. I forgot about rsync using a temp file. Glad to hear of that option.
By dingo (192.85.50.2) af.dingo@gmail.com on http://1984.ws
>
> Are you using rsync to minimize writes to flash?
>
>
This all seems to be a complete waste of time to me. Especially for upgrades ;(
Comments
By Anonymous Coward (24.37.242.64) on
> >
> > Are you using rsync to minimize writes to flash?
> >
> >
>
> This all seems to be a complete waste of time to me. Especially for upgrades ;(
What's a waste of time? Syncing PF log files, in case the unit goes down? :)
By jason (TheDudeAbides) on http://www.snakelegs.org
Is that you, Bill?
Bill Maas' mfsmount script is what I use.
Comments
By Anonymous Coward (24.37.242.64) on
>
> Bill Maas' mfsmount script is what I use.
No, I'm not Bill, but that script does looks pretty nice.
I'm going to check it in more detail and maybe get some more ideas.
Thanks for the link!
By Anonymous Coward (62.224.239.123) on
Comments
By sthen (85.158.44.149) on
The forthcoming PowerPC-based ones look a *lot* more interesting...
http://forum.mikrotik.com/viewtopic.php?f=3&t=15246
Comments
By Anonymous Coward (85.178.110.194) on
>
> The forthcoming PowerPC-based ones look a *lot* more interesting...
>
> http://forum.mikrotik.com/viewtopic.php?f=3&t=15246
4 GBit Nics and 3Gbit thoroughbred?!?!
Seriously: Doesn#t sounds that great at all. AlsoI found nod detailed plan how these NICs are connected. Don#t get me wrong but at the end they're connected via PCI.... *shrugs*
Comments
By sthen (85.158.44.149) on
> >
> > The forthcoming PowerPC-based ones look a *lot* more interesting...
> >
> > http://forum.mikrotik.com/viewtopic.php?f=3&t=15246
>
> 4 GBit Nics and 3Gbit thoroughbred?!?!
>
> Seriously: Doesn#t sounds that great at all.
If it's reasonably {small, not-too-expensive, cool-running} I can see a lot of uses (not just for routing) for something that can use CF, 1-2Gb of RAM, even if it can only push 0.5-1 Gb/s. Soekris 7501 might have something there too (always seemed more interesting than 5501 to me..).
> AlsoI found nod detailed plan how these NICs are connected.
For RB333, it's like http://www.freescale.com/files/graphic/block_diagram/MPC8321_BLKDIAG.jpg i.e. it doesn't use discrete network controllers, the main processor has a separate RISC core handling comms (I guess parts of the network stack would need to run on this rather than the main CPU core to get good speeds). I would guess at RB1000 using a faster version of the same, but the interesting chip is hidden under the fan in the photograph.
By nuintari (64.246.109.26) on
I have several of these on my network, and I hate them. They are underpowered and unstable. They flake out and crash under high loads. Mind you, it might be the software, RouterOS is a piece of shit. But these have been nothing but trouble.
We have started replacing them with soekris net5501s with openbsd. Very happy with the results thus far.
By Anonymous Coward (216.204.206.146) on
By Anthony (68.145.117.155) on
-tmp and var can be ramdisks, with symlinks into a small rw filesystem where necessary (eg /var/cron/tabs). This drastically reduces writes without breaking conventions and causing unexpected behavior (eg crontabs not being persistent across reboots without copying the ramdisk copy to flash). It also leaves you with a bootable system in the event the filesystem is corrupted.
-keep /dev, but mount an exact copy as a ramdisk on top of it. This way tty's don't need to be symlinks, as some stuff doesn't like this (eg screen).
There's a few other minor tweaks needed, but it becomes an almost-standard OpenBSD install. Combined with cheap flash that's big enough for a complete install, this is a lot easier to deal with down the road than a heavily customized system.
Comments
By Anonymous Coward (64.246.109.26) on
Comments
By Anonymous Coward (68.145.117.155) on
> willing to do a quick right up on what I did. It pretty much addresses
> everything you just said.
I would be very interested if it handles things more unobtrusively than my own method. :)
By chris cappuccio (204.80.187.5) chris@nmedia.net on
>
i didn't have a goal with flashdist of making the system easily upgradeable or of making it easily compatible with existing conventions.
however it may be time to incorporate those types of changes. if you can send me your stuff, i'd love to see it.