KVM - User contributions [en]

Nested Guests

2018-02-12T09:24:45Z

Kashyapc:

= Nested Guests =

Nested guests are KVM guests run in a KVM guest. As of Feb 2018 this feature is considered working but experimental, and some limitations apply.

When describing nested guests, we will use the following terminology:

* "L0" – the bare metal host, running KVM
* "L1" – a VM running on L0; also called the "guest hypervisor" — as it ''itself'' is capable of running KVM
* "L2" – a VM running on L1, also called the "nested guest"

= Detailed =
== Why use it? ==

An additional layer of virtualization sometimes comes in handy. You might have access to a large virtual machine in a cloud environment that you want to compartmentalize into multiple workloads. You might be running a lab environment in a training session.

== How to run ==

The KVM kernel modules do not enable nesting by default (though your distribution may override this default). To enable nesting, set the <code>nested</code> module parameter to <code>Y</code> or <code>1</code>. You may set this parameter persistently in a file in <code>/etc/modprobe.d</code> in the L0 host, for example:

# If you have an Intel CPU, use this:
$ cat /etc/modprobe.d/kvm_intel.conf
options kvm-intel nested=Y

# If you have an AMD CPU, then this:
$ cat /etc/modprobe.d/kvm_amd.conf
options kvm-amd nested=1

Once your L0 host is capable of nesting, you should be able to start an L1 guest with the <code>-cpu host</code> option (or for better live migration compatibility, use a named CPU model supported by QEMU, such as: <code>-cpu Haswell-noTSX-IBRS,vmx=on</code>) and the guest will subsequently be capable of running an L2 guest with accelerated KVM.

== Limitations ==

Once an L1 guest has started an L2 guest, it is no longer capable of being migrated, saved, or loaded (see [[Migration]] for details on these actions) until the L2 guest shuts down. This is currently an inherent limitation (that is being worked on, as of Feb 2018) of the KVM implementation on all architectures except s390x.

Attempting to migrate or save & load an L1 guest while an L2 guest is running will result in undefined behavior. You might see a <code>kernel BUG!</code> entry in <code>dmesg</code>, a kernel oops, or an outright kernel panic. At any rate, a thus migrated or loaded L1 guest can no longer be considered stable or secure, and must be restarted.

Migrating an L1 guest merely ''configured to support'' nesting, while not ''actually'' running L2 guests, is expected to function normally. Live-migrating an L2 guest from one L1 guest to another is also expected to succeed.

= Links =
* [https://www.redhat.com/archives/libvirt-users/2018-February/msg00014.html Discussion on limitations of nested guests] (<code>libvirt-users</code> mailing list, February 2018)
* [https://bugzilla.redhat.com/show_bug.cgi?id=1076294 Red Hat Bugzilla Bug 1076294 ]
* [https://bugzilla.kernel.org/show_bug.cgi?id=198621 kernel.org Bugzilla Bug 198621]
* [https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/virtual/kvm/nested-vmx.txt Intel nVMX upstream specification]
[[Category:Docs]]

Nested Guests

2018-02-12T09:23:30Z

Kashyapc:

= Nested Guests =

Nested guests are KVM guests run in a KVM guest. As of Feb 2018 this feature is considered working but experimental, and some limitations apply.

When describing nested guests, we will use the following terminology:

* "L0" -- the bare metal host, running KVM
* "L1" -- a VM running on L0; also called the "guest hypervisor" -- as it ''itself'' is capable of running KVM
* "L2" -- a VM running on L1, also called the "nested guest"

= Detailed =
== Why use it? ==

An additional layer of virtualization sometimes comes in handy. You might have access to a large virtual machine in a cloud environment that you want to compartmentalize into multiple workloads. You might be running a lab environment in a training session.

== How to run ==

The KVM kernel modules do not enable nesting by default (though your distribution may override this default). To enable nesting, set the <code>nested</code> module parameter to <code>Y</code> or <code>1</code>. You may set this parameter persistently in a file in <code>/etc/modprobe.d</code> in the L0 host, for example:

# If you have an Intel CPU, use this:
$ cat /etc/modprobe.d/kvm_intel.conf
options kvm-intel nested=Y

# If you have an AMD CPU, then this:
$ cat /etc/modprobe.d/kvm_amd.conf
options kvm-amd nested=1

Once your L0 host is capable of nesting, you should be able to start an L1 guest with the <code>-cpu host</code> option (or for better live migration compatibility, use a named CPU model supported by QEMU, such as: <code>-cpu Haswell-noTSX-IBRS,vmx=on</code>) and the guest will subsequently be capable of running an L2 guest with accelerated KVM.

== Limitations ==

Once an L1 guest has started an L2 guest, it is no longer capable of being migrated, saved, or loaded (see [[Migration]] for details on these actions) until the L2 guest shuts down. This is currently an inherent limitation (that is being worked on, as of Feb 2018) of the KVM implementation on all architectures except s390x.

Attempting to migrate or save & load an L1 guest while an L2 guest is running will result in undefined behavior. You might see a <code>kernel BUG!</code> entry in <code>dmesg</code>, a kernel oops, or an outright kernel panic. At any rate, a thus migrated or loaded L1 guest can no longer be considered stable or secure, and must be restarted.

Migrating an L1 guest merely ''configured to support'' nesting, while not ''actually'' running L2 guests, is expected to function normally. Live-migrating an L2 guest from one L1 guest to another is also expected to succeed.

= Links =
* [https://www.redhat.com/archives/libvirt-users/2018-February/msg00014.html Discussion on limitations of nested guests] (<code>libvirt-users</code> mailing list, February 2018)
* [https://bugzilla.redhat.com/show_bug.cgi?id=1076294 Red Hat Bugzilla Bug 1076294 ]
* [https://bugzilla.kernel.org/show_bug.cgi?id=198621 kernel.org Bugzilla Bug 198621]
* [https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/virtual/kvm/nested-vmx.txt Intel nVMX upstream specification]
[[Category:Docs]]

KVM Forum 2012

2013-05-07T05:53:33Z

Kashyapc:

= KVM Forum 2012 =

The [https://events.linuxfoundation.org/events/kvm-forum KVM Forum 2012] was held from November 7-9, 2012 at the Hotel Fira Palace in Barcelona. It was co-located with LinuxCon Europe 2012

== Schedule ==

'''Wednesday, November 7th'''
{|
! Time !! Title !! Speaker
|-
|09:00am || [[Media:2012-forum-avi-keynote.pdf | Keynote]] || Avi Kivity, Red Hat
|-
|09:15am || [[Media:2012-forum-Jan-Kiszka-BQL.pdf | For Performance and Latency, Not for Fun]] || Jan Kiszka, Siemens
|-
|10:30am || [[Media:2012-forum-Andrew-Theurer-Big-SMP-VMs.pdf | KVM Scalability: Preparing for big SMP VMs]] || Andrew Theurer, IBM
|-
|11:15am || [[Media:2012-forum-Liguori-qtest.pdf | Integrated Testing in QEMU - An Overview of qtest and qemu-test]] || Anthony Liguori, IBM
|}

{|
! !! colspan="2"|Track 1 !! colspan="2"|Track 2
|-
! Time !! Title !! Speaker !! Title !! Speaker
|-
|1:00pm || [[Media:2012-forum-kvm hyperv.pdf | KVM and Microsoft Hyper-V Enlightenments]] || Vadim Rozenfeld || [[Media:2012-forum-QOM CPU.pdf | QOM Vadis? Taking Objects to CPU and Beyond]] || Andreas Färber, SUSE
|-
|1:45pm || [[Media:2012-forum-Hemminger-New-Bridge.pdf | Linux bridging: Teaching an Old Dog New Tricks]] || Stephen Hemminger || [[Media:2012-forum-qidl-talk.pdf | QIDL: An Embedded Language to Serialize Guest Data Structures for Live Migration]] || Michael Roth, IBM
|-
|2:30pm || [[Media:2012-forum-Fernando-NTT-KVM-early-adopter.pdf | KVM in the Enterprise: An Early Adopter's Take]] || Fernando Luis Vazquez Cao, NTT || [[Media:2012-forum-live-migration.pdf | Live Migration: Even faster, Now With a Dedicated Thread!]] || Orit Wasserman, Red Hat
|-
|3:30pm || [[Media:2012-forum-qemu-usb-status-update.pdf | Qemu USB Status Report 2012]] || Gerd Hoffman, Red Hat || [[Media:2012-forum-postcopy.pdf | Yabusame Update on Postcopy Live Migration for QEMU/KVM]] || Isaku Yamahata, VA Linux Systems Japan K.K.
|-
|4:00pm || [[Media:2012-forum-yoshikawa.pdf | How to Use KVM's Reverse Mappings to Improve Scalability]] || Takuya Yoshikawa, NTT || [[Media:Spice-kvm-forum-2012.pdf | Spice Status Update]] || Hans de Goede, Red Hat
|-
|4:30pm || [[Media:2012-forum-Q35.pdf | A New Chipset for Qemu - Intel's Q35]] || Jason Baron, Red Hat || [[Media:2012-forum-rjones-libguestfs.pdf | New Features in libguestfs and the Virt Tools]] || Richard Jones, Red Hat
|-
|5:00pm || [[Media:2012-forum-libvirt-sandbox-kvm.pdf | Building Application Sandboxes on Top of KVM or LXC Using libvirt]] || Daniel Berrange, Red Hat KVM || [[Media:2012-forum-Kvm-autotest.pdf | Autotest - It's not Just a QA Tool Anymore]] || Lucas Meneghel Rodrigues, Red Hat
|}

'''Thursday, November 8th'''
{|
! Time !! Title !! Speaker
|-
|09:00am || [[Media:2012-forum-Keynote-Liguori.pdf | Keynote]] || Anthony Liguori, IBM
|-
|09:15am || [[Media:2012-forum-Virtio_net_status_challenges_2012.pdf | Virtio & Networking Status and Challenges]] || Michael S. Tsirkin, Red Hat
|-
|10:30am || [[Media:2012-forum-memory-mgmt.pdf | KVM Memory Management Update & Plans]] || Rik van Riel, Red Hat
|-
|11:15am || [[Media:2012-forum-memory-api.pdf | Revamping the QEMU Memory API]] || Avi Kivity, Red Hat
|}
{|
! !! colspan="2"|Track 1 !! colspan="2"|Track 2
|-
! Time !! Title !! Speaker !! Title !! Speaker
|-
|1:00pm || [[Media:2012-fourm-block-overview.pdf | A Block Layer Overview]] || Kevin Wolf, Red Hat || [[Media:2012-forum-MMIO Tuning.pdf | Partial Device Port Acceleration]] || Alexander Graf, SUSE
|-
|1:45pm || [[Media:2012-forum-thin-provisioning.pdf | The Road to Effective Thin Provisioning in QEMU]] || Paolo Bonzini, Red Hat || [[Media:2012-forum-VFIO.pdf | VFIO: A User's Perspective]] || Alex Williamson, Red Hat
|-
|2:30pm || [[Media:2012-forum-glusterfs-stefanha.pdf | GlusterFS for KVM Users and Developers]] || Stefan Hajnoczi, Red Hat || [[Media:2012-forum-megasas-vfio.pdf | Megasas on Steroids: Qemu Device-Passthrough With VFIO]] || Hannes Reinecke, SUSE
|-
|3:30pm || [[Media:2012-forum-nakajima apicv.pdf | Enabling Optimized Interrupt/APIC Virtualization in KVM]] || Jun Nakajima, Intel || QEMU Live Block Operations: Snapshots, Merging, and Mirroring || Jeff Cody, Red Hat
|-
|4:00pm || [[Media:2012-forum-device-sharing-Bharat.pdf | Efficient Sharing of Physical Devices Between KVM Guests and Host]] || Bharat Bhushan, Freescale || [[Media:2012-forum-virtio-blk-performance-improvement.pdf | Virtio-blk Performance Improvement]] || Asias He, Red Hat
|-
|4:30pm || [[Media:2012-forum-multiqueue-networking-for-kvm.pdf | Multiqueue Networking for KVM]] || Jason Wang, Red Hat || [[Media:2012-forum-WHQLAndTheCommunity.pdf | WHQL Pprocess for Windows Drivers and What the Community Can Learn From It]] || Yan Vugenfirer, Daynix
|-
|5:00pm || Lightning Talks: [[Media:2012-forum-Brugger-lightningtalk.pdf | PV Disk I/O]] * [[Media:2012-forum-ovmf.pdf | OVMF]] * [[Media:2012-forum NetWare as KVM Guest.pdf | Netware as KVM Guest]] || Matthias Brugger, Laszlo Ersek, Bruce Rogers || ||
|}

'''Friday, November 9th'''
{|
! !! colspan="2"|Track 1 !! colspan="2"|Track 2
|-
! Time !! Title !! Speaker !! Title !! Speaker
|-
|9:15am || [[Media:2012-forum-arm-christoffer-dall.pdf | ARM Virtualization for the Masses]] || Christoffer Dall, Virtual Open Systems || [[Media:2012-forum-Hecatonchire Benoit Hudzia.pdf | Memory Aggregation / Cloud with KVM]] || Benoit Hudzia, SAP
|-
|10:30am || [[Media:2012-forum-channel-io-kvm-forum.pdf | KVM on IBM System z: Channel I/O And How To Virtualize It]] || Cornelia Huck, IBM || [[Media:2012-forum-OpenStack-KVM-for-the-Masses.pdf | OpenStack: KVM for the Masses]] || Mark McLoughlin, Red Hat
|-
|11:15am || Boxes: A Box for Everyone! || Zeeshan Ali, Red Hat || [[Media:2012-forum-PowerKVM2012.pdf | KVM on IBM POWER: Update & IO Architecture]] || Benjamin Herrenschmidt, IBM & Stuart Yoder, Freescale
|}

== Community Team Photo ==

Jeff Cody managed to get a photograph of (a large part of) the KVM / QEMU community development team. [http://www.linux-kvm.org/static/kvm-forum-2012-barcelona-team-photo.html View the photo and name key]

== Notes ==

Some notes from the talks -- http://kashyapc.fedorapeople.org/virt/kvm-forum-2012-notes.txt