The
Waxmill package provides a command-line interface and Java API for creating,
configuring, and running
bhyve
virtual machines. The package follows a number of design principles in order to ensure stability and
reliability.
The
Waxmill package stores configuration data for virtual machines in a strict,
strongly-typed, versioned XML format. This ensures that virtual machine configurations created by older versions
of the
Waxmill package can continue to be understood by newer versions
of
Waxmill, or can be safely migrated to newer formats without any loss of
information.
Waxmill unconditionally validates all input against a schema (including
data that the package wrote itself), and will refuse to work with malformed and/or invalid data. Additionally,
the
Waxmill package uses a hardened XML parser configuration that is expected to be
immune to
entity expansion attacks.
The
Waxmill package is written
in
Java 11, using only a couple of
native methods (
fork() and
execve()), and is therefore
likely to be immune to buffer overflows and other memory-safety related bugs that plague the typical present-day
system languages. The codebase is, additionally, subjected to extensive static checking on every build. Binary
builds are expected to be byte-for-byte reproducible. Extensive use of static typing is used throughout the
codebase to enforce invariants statically where possible, and the code is annotated with
precondition/invariant/postcondition checks in order to induce bugs to cause early failures instead of silently
corrupting state. The codebase also has a high-coverage test suite which exhaustively tests the various API
methods and the entire command-line interface.
The
Waxmill package is designed to work under process supervision. The supervision
of system services has been a solved problem for decades although many pieces of server software are still not
designed to work well under process supervision. Many virtual machine management systems still work with
incredibly fragile
rc-style scripts and pidfiles that instantly become stale and dangerous on virtual
machine crashes. The
Waxmill
package provides a single command to start a virtual machine in the foreground such that the virtual machine can
be supervised using systems such as
daemontools, or
runit. These tools can guarantee that a virtual
machine always starts from a clean slate, and will always be reliably restarted should it crash.
The primary means by which users use the Waxmill package is via the
waxmill
command-line tool.
The waxmill command-line tool will attempt to read a configuration file specified by
the WAXMILL_CONFIGURATION_FILE environment variable. The configuration file, by
convention, should be stored at
/etc/waxmill/config.xml. The default configuration file is as follows:
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration xmlns="urn:com.io7m.waxmill.config:1:0">
<Paths>
<Path type="ZFSExecutable"
value="/sbin/zfs"/>
<Path type="GRUBBhyveExecutable"
value="/usr/local/sbin/grub-bhyve"/>
<Path type="BhyveExecutable"
value="/usr/sbin/bhyve"/>
<Path type="BhyveCtlExecutable"
value="/usr/sbin/bhyvectl"/>
<Path type="CuExecutable"
value="/usr/bin/cu"/>
<Path type="VirtualMachineConfigurationDirectory"
value="/etc/waxmill/vm"/>
<Path type="IfconfigExecutable"
value="/sbin/ifconfig"/>
</Paths>
<ZFSFilesystems>
<ZFSFilesystem type="VirtualMachineRuntimeFilesystem"
name="storage/vm"
mountPoint="/storage/vm"/>
</ZFSFilesystems>
</Configuration>
The configuration file explicitly specifies the locations of various system utilities
, and also specifies the location of the
virtual machine configuration directory
and the
virtual machine runtime filesystem. The virtual machine configuration
directory
is a directory consisting of XML configuration files for individual virtual machines. The
virtual machine runtime filesystem
is a ZFS filesystem containing one ZFS filesystem per virtual machine, each of which stores resources needed by
virtual machines at run-time such as GRUB configuration files, sockets, and lock files. The
waxmill tools expect to take complete ownership of the
virtual machine runtime filesystem, and can be expected to create arbitrary
nested filesystems and volumes as needed.
A
virtual machine, in
Waxmill, carries a
universally-unique identifier
that is used to refer to the virtual machine during its entire lifetime. The identifier of a virtual machine
cannot be changed after it is defined; this property of identifiers is useful to track virtual machines if they
are migrated to new physical machines.
Defining
a virtual machine involves taking a description of the virtual machine and recording that description in
the
virtual machine configuration directory. This is typically accomplished using the
vm-define
command. Once a virtual machine is
defined, it will appear in the list of defined virtual
machines (which can be inspected using the
vm-list command).
$ waxmill vm-define \
--name 'com.io7m.example4' \
--cpu-count 4 \
--comment 'An example machine' \
--memory-gigabytes 1 \
--memory-megabytes 512
$ waxmill vm-list
# ID Name
800a2dad-3367-4a98-879a-0fac219f55f4 com.io7m.example2
a866f9e4-f6c1-4ecf-ad49-000925eefa0a com.io7m.example4
538a90e4-d50d-4511-8643-ae418279bac4 com.io7m.example
7c2bcb79-f20c-47fa-812b-8994f17f97b2 com.io7m.example5
The definition of the created machine can be
exported using the
vm-export
command. The command will simply print the XML configuration file to the standard output. This can be used to
manually inspect the configuration of the machine, or to copy the configuration to a new physical machine.
$ waxmill vm-export --machine a866f9e4-f6c1-4ecf-ad49-000925eefa0a
<?xml version="1.0" encoding="UTF-8"?>
<wxm:VirtualMachines xmlns:wxm="urn:com.io7m.waxmill.vm:1:0">
<wxm:VirtualMachine id="a866f9e4-f6c1-4ecf-ad49-000925eefa0a" name="com.io7m.example4">
<wxm:Comment>An example machine</wxm:Comment>
<wxm:CPUTopology sockets="1" threads="1" cores="4"/>
<wxm:Memory gigabytes="1" megabytes="512"/>
<wxm:Devices>
<wxm:HostBridge vendor="UNSPECIFIED">
<wxm:DeviceSlot bus="0" slot="0" function="0"/>
</wxm:HostBridge>
</wxm:Devices>
<wxm:BootConfigurations/>
<wxm:Flags>
<wxm:Flag name="DisableMPTableGeneration" enabled="false"/>
<wxm:Flag name="ForceVirtualIOPCIToUseMSI" enabled="false"/>
<wxm:Flag name="GenerateACPITables" enabled="true"/>
<wxm:Flag name="GuestAPICIsX2APIC" enabled="false"/>
<wxm:Flag name="IncludeGuestMemoryInCoreFiles" enabled="false"/>
<wxm:Flag name="RealTimeClockIsUTC" enabled="false"/>
<wxm:Flag name="WireGuestMemory" enabled="false"/>
<wxm:Flag name="ExitCPUOnPAUSE" enabled="true"/>
<wxm:Flag name="YieldCPUOnHLT" enabled="true"/>
</wxm:Flags>
</wxm:VirtualMachine>
</wxm:VirtualMachines>
Similarly, the
vm-import command can be used to import
one or more virtual machines from a given XML configuration file.
bhyve virtual machines have a number of configurable boolean flags. The
vm-set command can be used to turn these flags on and
off. For example, using PCI passthru devices requires wiring the memory of the guest virtual machine.
A virtual machine needs to be configured with one or more
devices in order to actually do
useful work. Typically, a virtual machine will have one or more
storage devices
(emulated disks), one or more
network devices
(emulations of ethernet interfaces), and a
console device
(to allow access to the operating system console from the host machine). A
device is
attached to exactly one
device slot at any given time. A
device slot
is a
PCI slot
value consisting of a 3-tuple
(bus, pcislot, function),
where
bus ∈ [0, 255],
pcislot ∈ [0,31], and
function ∈ [0,7]. The concrete
notation used to denote a device slot is
bus:pcislot:function, so the string
1:5:3
denotes a device slot on bus
1, slot
5,
function
3. In practice, virtual machines will typically specify all devices to use
bus
0 and function
0, and only vary the slot value.
The waxmill command-line tool contains various subcommands to create and attach
devices to virtual machines.
The types of devices attached to a virtual machine will depend on the level of support that the guest operating
system running on the virtual machine has for virtualization. Operating systems that have explicit support for
virtualization (typically every major operating system with a release more recent than about 2007) will perform
best using
Virtio
devices. Older operating systems may need to be presented with the illusion of having access to
AHCI
devices. Please see the documentation for each of the commands for details.
The
Waxmill package can optionally automatically manage storage devices backed by
ZFS volumes
if requested. If, on a virtual machine with ID
X, a storage device that is
specified to use a
zfs-volume backend is attached to slot
a:b:c, the
Waxmill
package will automatically create a ZFS volume at
/storage/vm/X/disk-a:b:c (where
/storage/vm
is the configured
virtual machine runtime filesystem)
when
realizing the virtual machine. This is merely a
convenience feature; if the ZFS volume already exists, it will be used and the
Waxmill
package will not attempt to modify it.
Realizing
a virtual machine involves reading the description of the virtual machine from the
virtual
machine configuration directory
and then creating any resources that are needed at run-time such
as
ZFS volumes.
The
vm-realize command is used to
realize
virtual machines:
$ waxmill vm-realize \
--machine a866f9e4-f6c1-4ecf-ad49-000925eefa0a \
--dry-run true
/sbin/zfs create -V 10gb storage/vm/a866f9e4-f6c1-4ecf-ad49-000925eefa0a/disk-0_1_0
$ waxmill vm-realize \
--machine a866f9e4-f6c1-4ecf-ad49-000925eefa0a
$ waxmill vm-realize \
--machine a866f9e4-f6c1-4ecf-ad49-000925eefa0a
INFO com.io7m.waxmill.cmdline.internal.WXMCommandVMRealize: ZFS volume storage/vm/a866f9e4-f6c1-4ecf-ad49-000925eefa0a/disk-0_1_0 exists and has the correct size
Before a virtual machine can actually be started, it is necessary to define at least one
boot configuration. A
boot configuration is a named set of
instructions that describe how the virtual machine is to be started up. The instructions may include the names of
disk images that are to be inserted into emulated
optical
drives, the type of boot loader to be used, and/or how the boot loader is to locate an operating system
kernel on any of the storage devices attached to the virtual machine. Because of the complexity of the information
described within a boot configuration, there is no terse command analogous to
vm-define
that can be used to create a boot configuration. Instead, the boot configuration must be written in an XML file
and added to the virtual machine with the
vm-update-boot-configurations
command. A simple configuration, named
install, that boots
OpenBSD
from an official installer image inserted into the optical drive at device slot
0:4:0
using
grub-bhyve might look as
follows:
<?xml version="1.0" encoding="UTF-8" ?>
<BootConfigurations xmlns="urn:com.io7m.waxmill.vm:1:0">
<BootConfigurationGRUBBhyve name="install">
<BootDiskAttachments>
<BootDiskAttachment>
<DeviceSlot bus="0"
slot="4"
function="0"/>
<StorageBackendFile path="/storage/images/openbsd/install67.iso"/>
</BootDiskAttachment>
</BootDiskAttachments>
<GRUBBhyveKernelOpenBSD>
<BSDBootDevice kernelPath="/6.7/amd64/bsd.rd">
<DeviceSlot bus="0"
slot="4"
function="0"/>
</BSDBootDevice>
</GRUBBhyveKernelOpenBSD>
</BootConfigurationGRUBBhyve>
</BootConfigurations>
Once a virtual machine has been
realized and has at least one boot configuration, it can
be
run using the
vm-run
command:
$ waxmill vm-run \
--machine 538a90e4-d50d-4511-8643-ae418279bac4 \
--boot-configuration install
rdmsr to register 0x140 on vcpu 0
rdmsr to register 0x34 on vcpu 0
Unhandled ps2 mouse command 0xe1
Unhandled ps2 mouse command 0x88
The above command attempts to run the virtual machine with ID
538a90e4-d50d-4511-8643-ae418279bac4, using the boot configuration
named run. The waxmill command-line tool process is
replaced with that of the bhyve virtual machine, and the
bhyve
process continues to run in the foreground until the user shuts it down. The vm-run command
is designed to be executed by a process supervision system.
The
Waxmill package is designed to be usable in environments that use
a process supervision system. For example, assuming a virtual machine
3bab9914-dfe2-4a37-9c19-c7e9b06c854b has already been configured, it's possible
to create
runit
run and
finish scripts such as the following:
#!/bin/sh
exec /usr/local/bin/waxmill vm-run \
--machine 3bab9914-dfe2-4a37-9c19-c7e9b06c854b \
--boot-configuration normal
#!/bin/sh
exec /usr/local/bin/waxmill vm-kill \
--machine 3bab9914-dfe2-4a37-9c19-c7e9b06c854b \
These scripts will cause 3bab9914-dfe2-4a37-9c19-c7e9b06c854b to be started
and supervised, and the kernel resources associated with the virtual machine will be properly cleaned up
when the service is shut down or restarted.
The
waxmill package provides a command-line interface and API
for creating, configuring, and running
bhyve
virtual machines. The base
waxmill command is broken into a number of subcommands
which are documented over the following sections.
INFO com.io7m.waxmill.cmdline.Main: Usage: waxmill [options] [command] [command options]
Options:
--verbose
Set the minimum logging verbosity level.
Default: info
Possible Values: [trace, debug, info, warn, error]
Use the "help" command to examine specific commands:
$ waxmill help help.
Command-line arguments can be placed one per line into a file, and the file
can be referenced using the @ symbol:
$ echo help > file.txt
$ echo help >> file.txt
$ waxmill @file.txt
Commands:
help Show detailed help messages for commands.
schema Export schemas.
version Show the application version.
vm-add-ahci-disk Add an AHCI disk to a virtual machine.
vm-add-ahci-optical Add an AHCI optical drive to a virtual machine.
vm-add-e1000-network-device Add an e1000 network device to a virtual machine.
vm-add-framebuffer-device Add a framebuffer device to a virtual machine.
vm-add-lpc-device Add an LPC device to a virtual machine.
vm-add-passthru-device Add a PCI passthru device to a virtual machine.
vm-add-virtio-disk Add a virtio disk to a virtual machine.
vm-add-virtio-network-device Add a virtio network device to a virtual machine.
vm-console Connect to the console of a virtual machine
vm-define Define a new virtual machine.
vm-delete Delete a virtual machine.
vm-delete-boot-configurations Delete boot configurations from a virtual machine.
vm-delete-devices Delete devices from virtual machines.
vm-export Export virtual machine descriptions.
vm-import Import virtual machine descriptions.
vm-kill Kill a virtual machine.
vm-list List the available virtual machines.
vm-list-with-name List the virtual machines with the given name.
vm-realize Realize a virtual machine.
vm-run Run a virtual machine.
vm-set Set virtual machine configuration flags.
vm-update-boot-configurations Add/replace boot configurations in a virtual machine.
Documentation:
https://www.io7m.com/software/waxmill/documentation/
All subcommands accept a --verbose parameter that may be set to one of
trace, debug, info,
warn, or error. This parameter sets the lower bound for
the severity of messages that will be logged. For example, at debug verbosity, only
messages of severity debug and above will be logged. Setting the verbosity to
trace
level effectively causes everything to be logged, and will produce large volumes of debugging output.
Name |
Description |
trace
|
All messages greater than or equal to trace level (highest) |
debug
|
All messages greater than or equal to debug level |
info
|
All messages greater than or equal to info level |
warn
|
All messages greater than or equal to warn level |
error
|
All messages greater than or equal to error level |
The
waxmill command-line tool uses
jcommander
to parse command-line arguments, and therefore supports placing command-line arguments into a file,
one argument per line, and then referencing that file with
@. For example:
$ waxmill vm-list
# ID Name
538a90e4-d50d-4511-8643-ae418279bac4 com.io7m.example
$ waxmill vm-define --name com.io7m.example2
$ waxmill vm-list
# ID Name
800a2dad-3367-4a98-879a-0fac219f55f4 com.io7m.example2
538a90e4-d50d-4511-8643-ae418279bac4 com.io7m.example
$ (cat <<EOF
vm-define
--name
com.io7m.example3
EOF
) > args.txt
$ waxmill @args.txt
$ waxmill vm-list
# ID Name
800a2dad-3367-4a98-879a-0fac219f55f4 com.io7m.example2
538a90e4-d50d-4511-8643-ae418279bac4 com.io7m.example
6730354e-bcba-4636-ab07-ad853c375f7d com.io7m.example3
All subcommands, unless otherwise specified, yield an exit code of 0 on success, and
a non-zero exit code on failure.
help
- Show help on any command
The help command shows extended help information for other commands.
Parameter |
Type |
Required |
Description |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
$ waxmill help help
INFO com.io7m.waxmill.cmdline.Main: Usage: help [options] command
Options:
--verbose
Set the minimum logging verbosity level.
Default: info
Possible Values: [trace, debug, info, warn, error]
The "help" command, executed without arguments, shows the names of all
commands including the names and types of all of their parameters.
The "help" command can also be passed the name of a command. Commands
may include extended help messages such as the one you are reading
right now.
Example: waxmill help help
$ ./waxmill help vm-console
INFO com.io7m.waxmill.cmdline.Main: Usage: vm-console [options]
Options:
--configuration
The path to the configuration file (environment variable:
$WAXMILL_CONFIGURATION_FILE)
--dry-run
Show the commands that would be executed, but do not execute them.
Default: false
* --machine
The ID of the virtual machine
--verbose
Set the minimum logging verbosity level.
Default: info
Possible Values: [trace, debug, info, warn, error]
The "vm-console" command attempts to connect to the primary console
for the given virtual machine. In practice, this means executing the
"cu"[0] program on the device node that represents the console for
the VM. The command will fail with a diagnostic error message if the
virtual machine has no console, or has multiple consoles.
[0] https://www.freebsd.org/cgi/man.cgi?query=cu&sektion=1
schema
- Export XSD schemas
The
schema command lists the available
XSD
schemas available for the various XML formats used by the waxmill tools. The command can dump the text of specific
schemas when passed the identifier of a schema.
Parameter |
Type |
Required |
Description |
--id
|
URI
|
false
|
The schema identifier |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
$ waxmill schema
urn:com.io7m.waxmill.config:1:0
urn:com.io7m.waxmill.vm:1:0
$ waxmill schema --id urn:com.io7m.waxmill.config:1:0
<?xml version="1.0" encoding="UTF-8" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:wxmc="urn:com.io7m.waxmill.config:1:0"
targetNamespace="urn:com.io7m.waxmill.config:1:0">
<xsd:annotation>
<xsd:documentation>
A schema describing the format of Waxmill configuration files.
</xsd:documentation>
</xsd:annotation>
<xsd:simpleType name="PathType">
<xsd:annotation>
<xsd:documentation>
The type of configuration paths.
</xsd:documentation>
</xsd:annotation>
<xsd:restriction base="xsd:string">
<xsd:enumeration value="BhyveExecutable">
<xsd:annotation>
<xsd:documentation>
The path to the bhyve executable. On FreeBSD, this is /usr/sbin/bhyve by default.
...
version
- Show the Waxmill version
The version command shows the Waxmill application version.
Parameter |
Type |
Required |
Description |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
$ waxmill version
Waxmill 0.0.1-SNAPSHOT
vm-add-ahci-disk
- Add an AHCI disk to a virtual machine
The
vm-add-ahci-disk command adds a disk to a virtual machine. The disk is an
emulation of an
AHCI
controller attached to a SATA hard-drive and, as such, requires a driver in the guest operating system that is
capable of supporting AHCI.
Parameter |
Type |
Required |
Description |
--backend
|
Storage Backend
|
true
|
A specification of the AHCI storage device backend to add |
--comment
|
String
|
false
|
A comment describing the new disk |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--device-slot
|
Device Slot
|
true
|
The slot to which the device will be attached. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--open-option
|
OpenOption
|
false
|
The options that will be used when opening the storage device. |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--replace
|
Boolean
|
false
|
true if an existing device in the given slot should be replaced. |
bus = ? [0 .. 255] ? ;
slot = ? [0 .. 31] ? ;
function = ? [0 .. 7] ? ;
deviceSlot = bus , ":" , slot , ":" , function ;
Examples:
0:0:0
255:31:7
path = ? Any UNIX-like path ? ;
size = ? Any non-negative multiple of 128000 (bytes) ? ;
fileBackend = "file" , ";" , path ;
zfsBackend = "zfs-volume" , [ ";" , size ] ;
storageBackend = fileBackend | zfsBackend ;
Examples:
file;/tmp/xyz
zfs-volume
zfs-volume;1280000000
Name |
Description |
NO_CACHE
|
Do not use caching (O_DIRECT). |
SYNCHRONOUS
|
Writes to the file are synchronous (O_SYNC). |
READ_ONLY
|
The file is opened read-only. |
$ waxmill vm-add-ahci-disk \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--backend file;/tmp/disk.img
vm-add-ahci-optical
- Add an AHCI optical drive to a virtual machine
The vm-add-ahci-optical command adds an optical drive to a virtual machine.
Optical drives do not have any storage backends assigned to them by default. The storage backend for an
optical drive must be specified by a
boot
configuration.
Parameter |
Type |
Required |
Description |
--comment
|
String
|
false
|
A comment describing the new disk |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--device-slot
|
Device Slot
|
true
|
The slot to which the device will be attached. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--replace
|
Boolean
|
false
|
true if an existing device in the given slot should be replaced. |
bus = ? [0 .. 255] ? ;
slot = ? [0 .. 31] ? ;
function = ? [0 .. 7] ? ;
deviceSlot = bus , ":" , slot , ":" , function ;
Examples:
0:0:0
255:31:7
$ waxmill vm-add-ahci-optical \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0
vm-add-e1000-network-device
- Add an e1000 network device to a virtual machine
The
vm-add-e1000-network-device command adds a network device to a virtual machine.
The device is an emulation of an
Intel e82545
network device and, as such, requires a driver in the guest operating system that is capable of supporting that
device.
Parameter |
Type |
Required |
Description |
--type
|
Network Backend Kind
|
true
|
The type of network device backend to add (such as WXM_TAP, WXM_VMNET) |
--guest-mac
|
MAC Address
|
true
|
The MAC address of the device on the guest side |
--host-mac
|
MAC Address
|
true
|
The MAC address of the device on the host side |
--interface-group
|
Interface group name
|
false
|
The interface groups to which the device belongs. May be repeated to specify multiple groups. |
--comment
|
String
|
false
|
A comment describing the new disk |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--device-slot
|
Device Slot
|
true
|
The slot to which the device will be attached. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--replace
|
Boolean
|
false
|
true if an existing device in the given slot should be replaced. |
bus = ? [0 .. 255] ? ;
slot = ? [0 .. 31] ? ;
function = ? [0 .. 7] ? ;
deviceSlot = bus , ":" , slot , ":" , function ;
Examples:
0:0:0
255:31:7
$ waxmill vm-add-e1000-network-device \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--type WXM_TAP \
--name tap23 \
--host-mac d9:63:c0:d9:09:e8 \
--guest-mac d9:63:c0:d9:09:e9 \
--interface-group wwwUsers \
--interface-group ntpUsers
$ waxmill vm-add-e1000-network-device \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--type WXM_VMNET \
--name vmnet23 \
--host-mac d9:63:c0:d9:09:ea \
--guest-mac d9:63:c0:d9:09:eb \
--interface-group wwwUsers
vm-add-framebuffer-device
- Add a framebuffer device to a virtual machine
The vm-add-framebuffer-device command adds a framebuffer device to a virtual machine.
Parameter |
Type |
Required |
Description |
--comment
|
String
|
false
|
A comment describing the new device |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--device-slot
|
Device Slot
|
true
|
The slot to which the device will be attached. |
--height
|
Integer
|
false
|
The framebuffer height |
--listen-address
|
String
|
false
|
The VNC server listen address |
--listen-port
|
Integer
|
false
|
The VNC server listen port |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--vga-configuration
|
VGAConfiguration
|
false
|
The guest VGA configuration |
--wait-for-vnc
|
Boolean
|
false
|
Will cause the machine to wait for a VNC connection before booting |
--width
|
Integer
|
false
|
The framebuffer width |
--replace
|
Boolean
|
false
|
true if an existing device in the given slot should be replaced. |
bus = ? [0 .. 255] ? ;
slot = ? [0 .. 31] ? ;
function = ? [0 .. 7] ? ;
deviceSlot = bus , ":" , slot , ":" , function ;
Examples:
0:0:0
255:31:7
ipv4Part = ? [0 .. 255]{1,3} ? ;
ipv4 = ipv4Part , ".", ipv4Part , "." , ipv4Part , "." , ipv4Part , ;
ipv6 = ? Any https://tools.ietf.org/html/rfc4291 IPv6 address ? ;
name = ? Any host name ? ;
address = ipv4 | ipv6 | name ;
Examples:
127.0.0.1
[::1]
[2600:1417:6000:198::255e]
localhost
www.example.com
$ waxmill vm-add-framebuffer-device \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--wait-for-vnc true \
--width 1024 \
--height 768 \
--listen-address [::1] \
--listen-port 5901 \
--vga-configuration OFF
vm-add-lpc-device
- Add an LPC device to a virtual machine
The
vm-add-lpc-device command adds an LPC device to a virtual machine. The
bhyve
documentation describes such devices as
"LPC PCI-ISA bridges with COM1 and COM2 16550 serial
ports and a boot ROM". In practice, LPC devices are used to provide console access to virtual machines, and
UEFI
firmware to virtual machines that are booting from UEFI.
Parameter |
Type |
Required |
Description |
--add-backend
|
TTY Backend
|
true
|
A specification of the TTY device backend to add |
--comment
|
String
|
false
|
A comment describing the new device |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--device-slot
|
Device Slot
|
true
|
The slot to which the device will be attached. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--replace
|
Boolean
|
false
|
true if an existing device in the given slot should be replaced. |
bus = ? [0 .. 255] ? ;
slot = ? [0 .. 31] ? ;
function = ? [0 .. 7] ? ;
deviceSlot = bus , ":" , slot , ":" , function ;
Examples:
0:0:0
255:31:7
path = ? Any unix-like path ? ;
port = "com1" | "com2" | "bootrom" ;
stdioBackend = "stdio" , ";" , port ;
fileBackend = "file" , ";" , port , ";" , path ;
nmdmBackend = "nmdm" , ";" , port ;
ttyBackend = stdioBackend | fileBackend | nmdmBackend ;
Examples:
stdio;com1
file;bootrom;/tmp/xyz
nmdm;com2
$ waxmill vm-add-lpc-device \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--add-backend stdio;com1
$ waxmill vm-add-lpc-device \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--add-backend file;bootrom;/usr/local/share/uefi-firmware/BHYVE_UEFI.fd
$ waxmill vm-add-lpc-device \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--add-backend nmdm;com1
vm-add-passthru-device
- Add a PCI passthru device to a virtual machine
The
vm-add-passthru-device command adds a PCI passthru device to the virtual
machine. Using a PCI passthru device requires wiring the guest virtual machine's memory - see the
vm-set command.
Parameter |
Type |
Required |
Description |
--add-backend
|
TTY Backend
|
true
|
A specification of the TTY device backend to add |
--comment
|
String
|
false
|
A comment describing the new device |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--device-slot
|
Device Slot
|
true
|
The slot to which the device will be attached. |
--host-device-slot
|
Device Slot
|
true
|
The host device. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--replace
|
Boolean
|
false
|
true if an existing device in the given slot should be replaced. |
bus = ? [0 .. 255] ? ;
slot = ? [0 .. 31] ? ;
function = ? [0 .. 7] ? ;
deviceSlot = bus , ":" , slot , ":" , function ;
Examples:
0:0:0
255:31:7
$ waxmill vm-add-passthru-device \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--host-device-slot 1:2:3
vm-add-virtio-disk
- Add a Virtio disk to a virtual machine
The
vm-add-virtio-disk command adds a disk to a virtual machine. The disk is a
Virtio block storage device and,
as such, requires a driver in the guest operating system that is capable of supporting Virtio.
Parameter |
Type |
Required |
Description |
--backend
|
Storage Backend
|
true
|
A specification of the Virtio storage device backend to add |
--comment
|
String
|
false
|
A comment describing the new disk |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--device-slot
|
Device Slot
|
true
|
The slot to which the device will be attached. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--open-option
|
OpenOption
|
false
|
The options that will be used when opening the storage device. |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--replace
|
Boolean
|
false
|
true if an existing device in the given slot should be replaced. |
bus = ? [0 .. 255] ? ;
slot = ? [0 .. 31] ? ;
function = ? [0 .. 7] ? ;
deviceSlot = bus , ":" , slot , ":" , function ;
Examples:
0:0:0
255:31:7
path = ? Any UNIX-like path ? ;
size = ? Any non-negative multiple of 128000 (bytes) ? ;
fileBackend = "file" , ";" , path ;
zfsBackend = "zfs-volume" , [ ";" , size ] ;
storageBackend = fileBackend | zfsBackend ;
Examples:
file;/tmp/xyz
zfs-volume
zfs-volume;1280000000
Name |
Description |
NO_CACHE
|
Do not use caching (O_DIRECT). |
SYNCHRONOUS
|
Writes to the file are synchronous (O_SYNC). |
READ_ONLY
|
The file is opened read-only. |
$ waxmill vm-add-virtio-disk \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--backend file;/tmp/disk.img
vm-add-virtio-network-device
- Add a Virtio network device to a virtual machine
The
vm-add-virtio-network-device command adds a network device to a virtual machine.
The device is a
Virtio
network device and, as such, requires a driver in the guest operating system that is capable of supporting Virtio.
Parameter |
Type |
Required |
Description |
--type
|
Network Backend Kind
|
true
|
The type of network device backend to add (such as WXM_TAP, WXM_VMNET) |
--guest-mac
|
MAC Address
|
true
|
The MAC address of the device on the guest side |
--host-mac
|
MAC Address
|
true
|
The MAC address of the device on the host side |
--interface-group
|
Interface group name
|
false
|
The interface groups to which the device belongs. May be repeated to specify multiple groups. |
--comment
|
String
|
false
|
A comment describing the new disk |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--device-slot
|
Device Slot
|
true
|
The slot to which the device will be attached. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--replace
|
Boolean
|
false
|
true if an existing device in the given slot should be replaced. |
bus = ? [0 .. 255] ? ;
slot = ? [0 .. 31] ? ;
function = ? [0 .. 7] ? ;
deviceSlot = bus , ":" , slot , ":" , function ;
Examples:
0:0:0
255:31:7
$ waxmill vm-add-virtio-network-device \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--type WXM_TAP \
--name tap23 \
--host-mac d9:63:c0:d9:09:e8 \
--guest-mac d9:63:c0:d9:09:e9 \
--interface-group wwwUsers \
--interface-group ntpUsers
$ waxmill vm-add-virtio-network-device \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--device-slot 0:4:0 \
--type WXM_VMNET \
--name vmnet23 \
--host-mac d9:63:c0:d9:09:ea \
--guest-mac d9:63:c0:d9:09:eb \
--interface-group wwwUsers
vm-console
- Connect to the console a virtual machine
The
vm-console command attempts to connect to the primary console for the given
virtual machine. In practice, this means executing the
cu
program on the host device node that represents the console for the VM. The command will fail with a diagnostic
error message if the virtual machine has no console, or has multiple consoles.
Parameter |
Type |
Required |
Description |
--comment
|
String
|
false
|
A comment describing the new disk |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--dry-run
|
Boolean
|
false
|
Show the commands that would be executed, but do not execute them. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
$ waxmill vm-console \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd \
--dry-run true
/usr/bin/cu -l /dev/nmdm_de17174a-1b3c-4ebc-80af-cb239aa18cbd_B
$ waxmill vm-console \
--machine de17174a-1b3c-4ebc-80af-cb239aa18cbd
Connected
OpenBSD/amd64 <931.461.60231.14.vt920> (tty00)
login:
vm-define
- Define virtual machines
The vm-define command defines a new virtual machine. The command will define the
configuration of the virtual machine but will not create any of the underlying resources such as ZFS filesystems
and/or volumes. The command will, by default, generate a new unique machine ID unless one is manually specified
with the --machine option.
Parameter |
Type |
Required |
Description |
--comment
|
java.lang.String
|
false
|
A comment describing the new virtual machine |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--cpu-count
|
int
|
false
|
The number of CPU cores in the virtual machine |
--machine
|
UUID
|
false
|
The ID of the new virtual machine |
--memory-gigabytes
|
long
|
false
|
The size in gigabytes of the virtual machine's memory (added to --memory-megabytes) |
--memory-megabytes
|
long
|
false
|
The size in megabytes of the virtual machine's memory (added to --memory-gigabytes) |
--name
|
String
|
true
|
The name of the new virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
$ waxmill vm-define \
--name 'com.io7m.example4' \
--cpu-count 4 \
--comment 'An example machine' \
--memory-gigabytes 1 \
--memory-megabytes 512
$ waxmill vm-define \
--name 'com.io7m.example5' \
--cpu-count 4 \
--comment 'An example machine' \
--memory-gigabytes 1 \
--memory-megabytes 512 \
--machine 7c2bcb79-f20c-47fa-812b-8994f17f97b2
$ waxmill vm-define \
--name 'com.io7m.example5' \
--cpu-count 4 \
--comment 'An example machine' \
--memory-gigabytes 1 \
--memory-megabytes 512 \
--machine 7c2bcb79-f20c-47fa-812b-8994f17f97b2
ERROR com.io7m.waxmill.cmdline.Main: com.io7m.waxmill.exceptions.WXMExceptionDuplicate: A virtual machine already exists with the given ID.
ID: 7c2bcb79-f20c-47fa-812b-8994f17f97b2
Machine (A) Name: com.io7m.example5
Machine (A) Source: <unspecified>
Machine (B) Name: com.io7m.example5
Machine (B) Source: file:///etc/waxmill/vm/7c2bcb79-f20c-47fa-812b-8994f17f97b2.wvmx
vm-delete-boot-configurations
- Delete boot configurations for a virtual machine
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--file
|
Path
|
true
|
A file containing boot configurations |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--name
|
Boot Configuration
|
false
|
The names of boot configurations |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
$ waxmill vm-delete-boot-configurations --machine 538a90e4-d50d-4511-8643-ae418279bac4 --name nonexistent
ERROR com.io7m.waxmill.cmdline.internal.WXMCommandVMDeleteBootConfigurations: No boot configuration exists with the given name: nonexistent
$ waxmill vm-delete-boot-configurations --machine 538a90e4-d50d-4511-8643-ae418279bac4 --name install
INFO com.io7m.waxmill.cmdline.internal.WXMCommandVMDeleteBootConfigurations: Deleted boot configuration: install
vm-delete-devices
- Delete devices from a virtual machine
The vm-delete-devices removes devices from a given virtual machine. If any of the
specified devices are referenced by a boot configuration on the virtual machine, the boot configuration must first
either be deleted, or modified such that it no longer refers to the device.
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--device-slot
|
Device Slot
|
false
|
The devices to remove |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
$ waxmill vm-delete-devices --machine 538a90e4-d50d-4511-8643-ae418279bac4 --device-slot 0:10:0
ERROR com.io7m.waxmill.cmdline.internal.WXMCommandVMDeleteDevice: Device does not exist: 0:10:0
$ waxmill vm-delete-devices --machine 538a90e4-d50d-4511-8643-ae418279bac4 --device-slot 0:3:0
INFO com.io7m.waxmill.cmdline.internal.WXMCommandVMDeleteDevice: Deleted device 0:3:0
$ waxmill vm-delete-devices --machine 538a90e4-d50d-4511-8643-ae418279bac4 --device-slot 0:2:0
ERROR com.io7m.waxmill.cmdline.internal.WXMCommandVMDeleteDevice: One or more boot configurations are using the device.
Device: 0:2:0
Boot configurations: [run]
vm-delete
- Delete defined virtual machines
The vm-delete command deletes the configuration for a virtual machine. It will not
stop any running instance of the given virtual machine, nor will it delete the underlying resources such as ZFS
volumes.
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--machine
|
UUID
|
true
|
The ID of the virtual machine. |
$ waxmill vm-list
# ID Name
800a2dad-3367-4a98-879a-0fac219f55f4 com.io7m.example2
538a90e4-d50d-4511-8643-ae418279bac4 com.io7m.example
6730354e-bcba-4636-ab07-ad853c375f7d com.io7m.example3
$ waxmill vm-delete --machine 6730354e-bcba-4636-ab07-ad853c375f7d
$ waxmill vm-list
# ID Name
800a2dad-3367-4a98-879a-0fac219f55f4 com.io7m.example2
538a90e4-d50d-4511-8643-ae418279bac4 com.io7m.example
$ waxmill vm-delete --machine 6730354e-bcba-4636-ab07-ad853c375f7d
ERROR com.io7m.waxmill.cmdline.Main: com.io7m.waxmill.exceptions.WXMExceptionNonexistent: The specified machine does not exist.
Machine: 6730354e-bcba-4636-ab07-ad853c375f7d
ERROR com.io7m.waxmill.cmdline.Main: Caused by: java.nio.file.NoSuchFileException: /etc/waxmill/vm/6730354e-bcba-4636-ab07-ad853c375f7d.wvmx
vm-export
- Export defined virtual machines
The
vm-export command exports the configuration data for a virtual machine. This
command is the inverse of the
vm-import command.
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--machine
|
UUID
|
true
|
The ID of the virtual machine. |
$ waxmill vm-export --machine 538a90e4-d50d-4511-8643-ae418279bac4
<?xml version="1.0" encoding="UTF-8"?>
<wxm:VirtualMachines xmlns:wxm="urn:com.io7m.waxmill.vm:1:0">
<wxm:VirtualMachine id="538a90e4-d50d-4511-8643-ae418279bac4" name="com.io7m.example">
<wxm:CPUTopology sockets="1" threads="1" cores="1"/>
<wxm:Memory gigabytes="0" megabytes="250"/>
<wxm:Devices>
<wxm:HostBridge vendor="UNSPECIFIED">
<wxm:DeviceSlot bus="0" slot="0" function="0"/>
</wxm:HostBridge>
<wxm:AHCIDiskDevice>
<wxm:DeviceSlot bus="0" slot="1" function="0"/>
<wxm:StorageBackendZFSVolume/>
</wxm:AHCIDiskDevice>
<wxm:AHCIOpticalDiskDevice>
<wxm:DeviceSlot bus="0" slot="2" function="0"/>
</wxm:AHCIOpticalDiskDevice>
<wxm:VirtioNetworkDevice>
<wxm:DeviceSlot bus="0" slot="4" function="0"/>
<wxm:TAPDevice name="tap23" address="a3:26:9c:74:79:34"/>
</wxm:VirtioNetworkDevice>
</wxm:Devices>
<wxm:BootConfigurations>
<wxm:BootConfigurationGRUBBhyve name="run">
<wxm:GRUBBhyveKernelOpenBSD>
<wxm:BSDBootDevice kernelPath="/bsd">
<wxm:DeviceSlot bus="0" slot="2" function="0"/>
</wxm:BSDBootDevice>
</wxm:GRUBBhyveKernelOpenBSD>
</wxm:BootConfigurationGRUBBhyve>
</wxm:BootConfigurations>
<wxm:Flags>
<wxm:Flag name="DisableMPTableGeneration" enabled="false"/>
<wxm:Flag name="ForceVirtualIOPCIToUseMSI" enabled="false"/>
<wxm:Flag name="GenerateACPITables" enabled="true"/>
<wxm:Flag name="GuestAPICIsX2APIC" enabled="false"/>
<wxm:Flag name="IncludeGuestMemoryInCoreFiles" enabled="false"/>
<wxm:Flag name="RealTimeClockIsUTC" enabled="false"/>
<wxm:Flag name="WireGuestMemory" enabled="false"/>
<wxm:Flag name="ExitCPUOnPAUSE" enabled="true"/>
<wxm:Flag name="YieldCPUOnHLT" enabled="true"/>
</wxm:Flags>
</wxm:VirtualMachine>
</wxm:VirtualMachines>
vm-id-of
- Find the ID of a virtual machine
The vm-id-of command finds the first virtual machine that has a given name
and prints the ID of the machine. If the --short option is used, the command
will print the ID in a shortened form consumable by bhyve.
Parameter |
Type |
Required |
Description |
--name
|
Machine name
|
true
|
The name of the machine. |
--short
|
Boolean
|
false
|
Specifies whether the output should be in shortened Bhyve-compatible form |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
$ waxmill vm-id-of --name example
538a90e4-d50d-4511-8643-ae418279bac4
$ waxmill vm-id-of --name example --short
U4qQ5NUNRRGGQ65Bgnm6xA
vm-import
- Import defined virtual machines
The
vm-import command imports configuration data for a virtual machine. This
command is the inverse of the
vm-export command.
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--file
|
Path
|
true
|
A file containing one or more virtual machine configurations. |
$ cat vm.xml
<?xml version="1.0" encoding="UTF-8"?>
<wxm:VirtualMachines xmlns:wxm="urn:com.io7m.waxmill.vm:1:0">
<wxm:VirtualMachine id="538a90e4-d50d-4511-8643-ae418279bac4" name="com.io7m.example">
<wxm:CPUTopology sockets="1" threads="1" cores="1"/>
<wxm:Memory gigabytes="0" megabytes="250"/>
<wxm:Devices>
<wxm:HostBridge vendor="UNSPECIFIED">
<wxm:DeviceSlot bus="0" slot="0" function="0"/>
</wxm:HostBridge>
<wxm:AHCIDiskDevice>
<wxm:DeviceSlot bus="0" slot="1" function="0"/>
<wxm:StorageBackendZFSVolume/>
</wxm:AHCIDiskDevice>
<wxm:AHCIOpticalDiskDevice>
<wxm:DeviceSlot bus="0" slot="2" function="0"/>
</wxm:AHCIOpticalDiskDevice>
<wxm:VirtioNetworkDevice>
<wxm:DeviceSlot bus="0" slot="4" function="0"/>
<wxm:TAPDevice name="tap23" address="a3:26:9c:74:79:34"/>
</wxm:VirtioNetworkDevice>
</wxm:Devices>
<wxm:BootConfigurations>
<wxm:BootConfigurationGRUBBhyve name="run">
<wxm:GRUBBhyveKernelOpenBSD>
<wxm:BSDBootDevice kernelPath="/bsd">
<wxm:DeviceSlot bus="0" slot="2" function="0"/>
</wxm:BSDBootDevice>
</wxm:GRUBBhyveKernelOpenBSD>
</wxm:BootConfigurationGRUBBhyve>
</wxm:BootConfigurations>
<wxm:Flags>
<wxm:Flag name="DisableMPTableGeneration" enabled="false"/>
<wxm:Flag name="ForceVirtualIOPCIToUseMSI" enabled="false"/>
<wxm:Flag name="GenerateACPITables" enabled="true"/>
<wxm:Flag name="GuestAPICIsX2APIC" enabled="false"/>
<wxm:Flag name="IncludeGuestMemoryInCoreFiles" enabled="false"/>
<wxm:Flag name="RealTimeClockIsUTC" enabled="false"/>
<wxm:Flag name="WireGuestMemory" enabled="false"/>
<wxm:Flag name="ExitCPUOnPAUSE" enabled="true"/>
<wxm:Flag name="YieldCPUOnHLT" enabled="true"/>
</wxm:Flags>
</wxm:VirtualMachine>
</wxm:VirtualMachines>
$ waxmill vm-import --file vm.xml
INFO com.io7m.waxmill.cmdline.internal.WXMCommandVMImport: Imported 1 virtual machines
$ waxmill vm-import --file vm.xml
ERROR com.io7m.waxmill.cmdline.Main: com.io7m.waxmill.exceptions.WXMExceptionDuplicate: A virtual machine already exists with the given ID.
ID: 538a90e4-d50d-4511-8643-ae418279bac4
Machine (A) Name: com.io7m.example
Machine (A) Source: file:///tmp/vm.xml
Machine (B) Name: com.io7m.example
Machine (B) Source: file:///etc/waxmill/vm/538a90e4-d50d-4511-8643-ae418279bac4.wvmx
vm-kill
- Kill a running virtual machine
The vm-kill command kills a running virtual machine.
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--dry-run
|
Boolean
|
false
|
Show the commands that would be executed, but do not execute them. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
(Formatted for legibility)
$ waxmill vm-kill \
--machine 538a90e4-d50d-4511-8643-ae418279bac4 \
--dry-run true
/usr/sbin/bhyvectl \
--vm=538a90e4-d50d-4511-8643-ae418279bac4 \
--destroy
vm-list-with-name
- List defined virtual machines with a given name
The vm-list-with-name command lists the identifiers of all of the virtual machines
that have a given name. The command raises an error if no virtual machines have the given name.
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--name
|
Machine Name
|
true
|
The name of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
$ waxmill vm-list-with-name --name com.io7m.example
538a90e4-d50d-4511-8643-ae418279bac4
$ waxmill vm-list-with-name --name nonexistent
ERROR com.io7m.waxmill.cmdline.internal.WXMCommandVMListWithName: No virtual machines exist with the given name.
Name: nonexistent
vm-list
- List defined virtual machines
The vm-list command lists the defined virtual machines.
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
$ waxmill vm-list
# ID Name
800a2dad-3367-4a98-879a-0fac219f55f4 com.io7m.example2
538a90e4-d50d-4511-8643-ae418279bac4 com.io7m.example
6730354e-bcba-4636-ab07-ad853c375f7d com.io7m.example3
vm-realize
- Realize a virtual machine
The
vm-realize command
realizes a virtual machine. Realizing a virtual
machine may be unnecessary if you have already manually created all of the resources (such as ZFS volumes)
that the machine needs to run. The
vm-run command
will validate that all of the required resources exist when attempting to start a virtual machine.
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--dry-run
|
Boolean
|
false
|
Show the commands that would be executed, but do not execute them. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
(Formatted for legibility)
$ waxmill vm-realize \
--machine 538a90e4-d50d-4511-8643-ae418279bac4 \
--dry-run true
vm-run
- Start a virtual machine
The
vm-run command starts a virtual machine. The command, internally, uses
execve()
to replace the waxmill process with that of the
bhyve virtual machine. The virtual
machine stays running in the foreground. The command is intended to be used to start virtual machines under
process supervision systems such as
daemontools
or
runit.
The
vm-run command will, when executed, will first generate and write the necessary
configuration files into the
virtual machine runtime directory
for the given machine, and then generate and execute the necessary shell commands to start the virtual machine.
For example, if the selected boot configuration uses
GRUB bhyve, the required
grub.cfg
and
device.map files will be generated and saved into the directory. The
--dry-run
option will cause the command to generate those required files, but will display the shell commands
necessary to start the virtual machine on the standard output instead of actually executing them.
Parameter |
Type |
Required |
Description |
--boot-configuration
|
Boot Configuration
|
true
|
The name of the boot configuration that will be used. |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--dry-run
|
Boolean
|
false
|
Show the commands that would be executed, but do not execute them. |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
(Formatted for legibility)
$ waxmill vm-run \
--machine 538a90e4-d50d-4511-8643-ae418279bac4 \
--boot-configuration install \
--dry-run true
/usr/local/sbin/grub-bhyve \
--device-map=/zroot/storage/vm/538a90e4-d50d-4511-8643-ae418279bac4/grub-device.map \
--root=host \
--directory=/zroot/storage/vm/538a90e4-d50d-4511-8643-ae418279bac4 \
--memory=250M \
538a90e4-d50d-4511-8643-ae418279bac4
/usr/sbin/bhyve \
/home/rm/git/com.github/io7m/waxmill/bhyve \
-P \
-A \
-H \
-c cpus=1,sockets=1,cores=1,threads=1 \
-m 250M \
-s 0:0:0,hostbridge \
-s 0:1:0,ahci-hd,/zroot/storage/vm/538a90e4-d50d-4511-8643-ae418279bac4/disk-0_1_0 \
-s 0:2:0,ahci-cd,/tmp/openbsd-6.6-amd64.iso \
-s 0:3:0,lpc \
-l com1,/dev/nmdm_538a90e4-d50d-4511-8643-ae418279bac4_B \
-s 0:4:0,virtio-net,tap23,mac=a3:26:9c:74:79:34 \
538a90e4-d50d-4511-8643-ae418279bac4
vm-set
- Set virtual machine configuration flags
The vm-set command sets configuration flags for virtual machines.
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
--include-guest-memory-cores
|
Boolean
|
false
|
Include guest memory in core files. |
--yield-on-HLT
|
Boolean
|
false
|
Yield the virtual CPU thread when a HLT instruction is detected. |
--exit-on-PAUSE
|
Boolean
|
false
|
Force the guest virtual CPU to exit when a PAUSE instruction is detected. |
--generate-acpi-tables
|
Boolean
|
false
|
Generate ACPI tables. Required for FreeBSD/amd64 guests. |
--disable-mptable-generation
|
Boolean
|
false
|
Disable MP table generation. |
--force-msi-interrupts
|
Boolean
|
false
|
Force virtio PCI device emulations to use MSI interrupts instead of MSI-X interrupts. |
--guest-apic-is-x2apic
|
Boolean
|
false
|
The guest's local APIC is configured in x2APIC mode. |
--rtc-is-utc
|
Boolean
|
false
|
RTC keeps UTC time. |
--ignore-unimplemented-msr
|
Boolean
|
false
|
Ignore accesses to unimplemented Model Specific Registers. |
--wire-guest-memory
|
Boolean
|
false
|
Enable/disable wiring of guest memory. |
$ waxmill vm-set \
--machine 538a90e4-d50d-4511-8643-ae418279bac4 \
--wire-guest-memory true
vm-update-boot-configurations
- Add or update boot configurations for a virtual machine
The
vm-update-boot-configurations adds or updates
boot configurations within a given
virtual machine. The boot configurations are specified using an XML format. The precise format of the boot
configurations elements are given in the
urn:com.io7m.waxmill.vm:1:0 schema
.
Parameter |
Type |
Required |
Description |
--configuration
|
Path
|
false
|
The path to the configuration file (environment variable: $WAXMILL_CONFIGURATION_FILE) |
--file
|
Path
|
true
|
A file containing boot configurations |
--machine
|
UUID
|
true
|
The ID of the virtual machine |
--update
|
Boolean
|
false
|
Update existing boot configurations |
--verbose
|
Log Level
|
false
|
Set the minimum logging verbosity level. |
The
Waxmill package currently supports booting virtual machines using
UEFI
or
grub-bhyve.
An example boot configuration that attaches an
OpenBSD
installer CD to device
0:4:0, and boots the installation kernel using
grub-bhyve:
<?xml version="1.0" encoding="UTF-8" ?>
<BootConfigurations xmlns="urn:com.io7m.waxmill.vm:1:0">
<BootConfigurationGRUBBhyve name="install">
<BootDiskAttachments>
<BootDiskAttachment>
<DeviceSlot bus="0"
slot="4"
function="0"/>
<StorageBackendFile path="/storage/images/openbsd/install67.iso"/>
</BootDiskAttachment>
</BootDiskAttachments>
<GRUBBhyveKernelOpenBSD>
<BSDBootDevice kernelPath="/6.7/amd64/bsd.rd">
<DeviceSlot bus="0"
slot="4"
function="0"/>
</BSDBootDevice>
</GRUBBhyveKernelOpenBSD>
</BootConfigurationGRUBBhyve>
</BootConfigurations>
An example boot configuration that attaches an
OpenBSD
installer CD to device
0:4:0, and boots machine using UEFI:
<?xml version="1.0" encoding="UTF-8" ?>
<BootConfigurations xmlns="urn:com.io7m.waxmill.vm:1:0">
<BootConfigurationUEFI name="install"
firmware="/usr/local/share/uefi-firmware/BHYVE_UEFI.fd">
<BootDiskAttachments>
<BootDiskAttachment>
<DeviceSlot bus="0"
slot="4"
function="0"/>
<StorageBackendFile path="/storage/images/openbsd/install67.iso"/>
</BootDiskAttachment>
</BootDiskAttachments>
</BootConfigurationUEFI>
</BootConfigurations>
$ cat boot.xml
<?xml version="1.0" encoding="UTF-8" ?>
<BootConfigurations xmlns="urn:com.io7m.waxmill.vm:1:0">
<BootConfigurationGRUBBhyve name="install">
<GRUBBhyveKernelOpenBSD>
<BSDBootDevice kernelPath="/bsd.rd">
<DeviceSlot bus="0"
slot="4"
function="0"/>
</BSDBootDevice>
</GRUBBhyveKernelOpenBSD>
</BootConfigurationGRUBBhyve>
</BootConfigurations>
$ waxmill vm-update-boot-configurations --machine 538a90e4-d50d-4511-8643-ae418279bac4 --file boot.xml
INFO com.io7m.waxmill.cmdline.internal.WXMCommandVMUpdateBootConfigurations: Parsed 1 boot configurations
INFO com.io7m.waxmill.cmdline.internal.WXMCommandVMUpdateBootConfigurations: Added new boot configuration: install
$ waxmill vm-update-boot-configurations --machine 538a90e4-d50d-4511-8643-ae418279bac4 --file boot.xml
INFO com.io7m.waxmill.cmdline.internal.WXMCommandVMUpdateBootConfigurations: Parsed 1 boot configurations
ERROR com.io7m.waxmill.cmdline.internal.WXMCommandVMUpdateBootConfigurations: A boot configuration already exists with the given name: install
$ waxmill vm-update-boot-configurations --machine 538a90e4-d50d-4511-8643-ae418279bac4 --file boot.xml --update
INFO com.io7m.waxmill.cmdline.internal.WXMCommandVMUpdateBootConfigurations: Parsed 1 boot configurations
INFO com.io7m.waxmill.cmdline.internal.WXMCommandVMUpdateBootConfigurations: Updated boot configuration: install
Please see the included
JavaDoc
for API reference documentation.