OpenBSD Journal

Sun4v logical domain support coming up.

Contributed by jj on from the yo-dawg-I-heard-you-like-logical-domains dept.

So this stuff is in a usable state now. Below are some instructions on how to use things, and I encourage people with access to suitable hardware to play with it. If you do, you probably should be very current and rebuild your kernel and ldomctl(8) from absolutely -current sources before doing so.

Have fun!

Mark

Configuring guest domains on sun4v systems

These notes assume your system is in the "factory-default" configuration. You can check whether this is the case by using the "ldomcfg list" command. It should produce something like:

  # ldomcfg list
  factory-default [current]

If it lists the "factory-default" configuration as [current], you're good to go.

Start by dumping the "factory-default" configuration for future use. A configuration consists of several files, so it is best to keep each configuration in its own directory:

  # mkdir factory-default
  # cd factory-default
  # ldomctl dump
  # cd ..

Now start making a new configuration:

  # mkdir openbsd
  # cp factory-default/* openbsd/
  # cd openbsd

Now create a configuration file that describes your desired configuration. Here is an example:

  domain puffy {
          vcpu 12
          memory 4G
          vdisk "/home/puffy/vdisk0"
          vdisk "/home/puffy/vdisk1"
          vnet
  }

  domain salmah {
          vcpu 8
          memory 2G
          vdisk "/home/salmah/vdisk0"
          vdisk "/home/salmah/vdisk1"
          vnet
  }

This file creates two domains, one named "puffy" and one named "salmah". The "puffy" domain gets 12 "virtual" CPUs, 4 GB of memory, two virtual disks and a virtual network interface. The "salmah" domain gets 8 "virtual" CPUs, 2G of memory and again two virtual disks and a virtual network interface. The remaining "virtual" CPUs and memory get assigned to the primary domain. You should make sure there are some resources left fot the primary domain, otherwise the configuration won't boot. For the virtual disks you have to specify the name of the backing disk image.

Save the configuration file in a file named "ldom.conf", and do:

  # ldomctl init-system ldom.conf

This generates the required configuration files for your new configuration. The next step is to save your configuration to non-volatile storage on the service processor.

  # cd ..
  # ldomctl download openbsd

The name of the configuration is taken from the name of the directory that contains the configuration files. If a configuration with that name already exists, you'll need to remove it first.

  # cd ..
  # ldomctl delete openbsd
  # ldomctl download openbsd

Saving the configuration takes some time, so don't panic if the command doesn't complete immediately. If you now list the configurations again, things should look like:

  # ldomctl list
  factory-default [current]
  openbsd [next]

You'll need to create virtual disk images for your guest domains. These can be created using dd(1). For example:

  # dd if=/dev/zero of=/home/puffy/vdisk0 bs=1m count=8192

will create a 8G disk image.

You also need install media for your guest domain. For this purpose, the sparc64 miniroot52.fs file is usable as a disk image. So to make this the second virtual disk of the "puffy" domain, do someting like:

  # cp miniroot52.fs /home/puffy/vdisk1

Now enable ldomd(8), by adding the following line to your /etc/rc.conf.local:

  ldomd_flags=

Then halt the primary domain:

  # halt

and do a full reset of the machine. You need to be logged in to the service processor (ALOM or ILOM) to do this. If necessary enter "#." to switch from the console back to the service processor prompt. For ALOM (or ILOM in ALOM emulation mode), reset the machine with:

  sc> reset -c

For ILOM you'll need to RTFM, since I try to avoid using it ;).

The machine will now reset and boot into the new configuration. Your primary domain should have less CPUs and memory, since they're now assigned to the guest domains. The guest domains should be running:

  # ldomctl status
  primary      running      OpenBSD running                   1%
  puffy        running      OpenBoot Primary Boot Loader      8%
  salmah       running      OpenBoot Primary Boot Loader     12%

As you can see the guest domains are stuck in OBP. Not really surprising as you didn't install an OS on them yet. You get access to the serial console of the first guest domain do:

  # cu -l ttyV0

If you do this shortly after the system has booted, OBP might still be trying to netboot. In that case you might want to send a break character (~#) to interrupt it. If you followed the instructions above, you can boot the OpenBSD installer by doing:

  ok boot disk1

Before you proceed with installing OpenBSD, you might want to set up networking for the guest domain such that you can download the installation sets. You might have noticed that your primary domain now has a vsw(4) device and a couple of vnet(4) devices. These vnet(4) devices are ports of a virtual switch that you build by bridging them together with a physical interface. To do this, first bring up the vnet(4) interfaces. Since these interfaces don't have a proper MAC address, you propobably want to use -inet6 to prevent the generation of link-local IPv6 addresses:

  # ifconfig vnet0 -inet6 up
  # ifconfig vnet1 -inet6 up

Then, assuming the physical network interface you want to use for carrying the traffic to and from your guest domains is em0, just do:

  # ifconfig bridge0 add em0 add vnet0 add vnet1 up

You probably want to make this configuration permanent by creating the appropriate hostname.if(5) files.

Now you're ready to continue the installation.

Known problems

  • It's not possible to assign crypto units to a domain yet. OpenBSD doesn't support them anyway, but Solaris running in a guest domain could use them.

(Comments are closed)


Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]