#############
Using libvirt
#############

I (nwf) highly suggest against it if at all possible. What a pain in the rump.
You'll probably be spending a lot of time on over here_ learning how to write
domain files. Here's a collection of site-specific notes. The only reason it's
here is that this turns out to be the best easy-and-mostly-standard way of
wrangling QEMUs into running at boot for the world's benefit, which
unfortunately we must do in order for afs to be reasonably manageable. Use
``virsh autostart ${DOMAIN}`` once you've got the domain created, to make that
auto-run happen.

.. _here: http://libvirt.org/formatdomain.html


Debugging XML
-------------

Egads this is painful.  You can get a good/no-good bit result out of the tool
``virt-xml-validate``, but you should ignore the error messages it prints, most
of the time, as it is a remarkably poor producer.  I suggest the usual
divide-and-conquer approach of commenting out half the file and narrowing from
there.

You can also compare a round-trip through libvirt:  use ``virsh define
${YOUR_XML_DOCUMENT}`` and ``virsh dumpxml ${YOUR_DOMAIN_NAME}``.

Using Ceph
----------

http://ceph.com/docs/master/rbd/libvirt/ will tell you what's up, but for
ease of administration, here are the steps:

* You'll want a ceph principal for your host; may I suggest
  ``client.$(HOSTNAME).libvirt``.  Go use :ref:`ceph_new-user` to make one and
  extract its key.  For caps, you'll probably want something like::

    "osd" "allow class-read object_prefix rbd_children, \
           allow rwx pool=volumes_2, allow rw pool=images_2,
           allow rwx pool=scratch"
    "mon" "allow r"

If you're standing up an AFS server using Ceph partitions, you probably need
to augment those pool lists.

* You need to register this secret with libvirt.  Create an XML file (of
  course)::

    cat >virsh-secret.xml <<HERE
    <secret ephemeral='no' private='yes'>
        <usage type='ceph'>
                <name>client.$(HOSTNAME).libvirt secret</name>
        </usage>
    </secret>
    HERE

  Then, ``virsh secret-define --file virsh-secret.xml`` which will print out
  a UUID for you, which you can also retrieve with ``virsh secret-list``.
  Then ``virsh secret-set-value ${UUID} --base64 ${EXTRACTED_KEY}``.  Then
  ``rm ${EXTRACTED_KEY} virsh-secret.xml``.

* Now you need to actually use that key with your RBD stanzas in domain XML.
  Try something like::

    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <auth username='magellan.libvirt'>
        <secret type='ceph' usage='client.magellan.libvirt secret'/>
      </auth>
      <source protocol='rbd' name='...'/>
      <target dev='vd...' bus='virtio'/>
    </disk>

  It appears that the virtio drivers are more robustly tested with Ceph
  than, for example, the IDE drivers.  We have had I/O timeouts and guest
  kernel panics with the latter but have not seen them with the former.
  Modern QEMU BIOSes are up to snuff enough to boot off of virtio devices.

Serial Console
--------------

In your domain XML, stick::

  <serial type="pty"><target port="0"/></serial>
  <console type="pty"><target type="serial" port="0"/></console>

And on the guest, follow the instructions in :ref:`install-common_linux-serial`.
Then you can use ``virsh console ${DOMAIN}`` once it's up and running to
attach to the console from the host.

.. note::  It'd be better if we could run ``-curses`` mode inside screen or
   something, but this is not to be, so here we are.  This probably means
   that you should do the install of a libvirt guest separately from the
   libvirt side of things, which necessitates learning not only how to write
   a domain file but how to write QEMU arguments directly.

Libvirt Events
--------------

Your domain file should probably contain these hooks::

  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <on_lockfailure>poweroff</on_lockfailure>