Latest revision as of 05:51, 23 August 2018

Automatic Ballooning

NOTE: This page describes an experimental development project from 2013 that was never completed. It is left here as a historical record. The feature described does not exist in any currently shipping version of QEMU

Introduction

The virtio balloon device allows KVM guests to reduce their memory size (thus relinquishing memory to the host) and to increase it back (thus taking memory from the host).

This feature is mainly intended to support over-committing memory on KVM hosts. That is, hosts that are running VMs whose total memory size is greater than what the host has physically available. For example, a 2G host running two VMs each with 2G would be over-committed.

The balloon device is important to support memory over-commitment because it allows for reducing a guest's memory size if needed. Suppose, in the previous example, that one of the guests is using 1G and its other 1G is free. We could use the balloon device to reduce this guest's size from 2G to 1G, this would free 1G in the host allowing the other VM to use it. Of course, if the reduced guest wants to run an application that consumes more than the 1G it currently has, it has to grow again.

That's the problem with the current balloon device, it's entirely manual. Someone (or some tool) is supposed to be watching the pressure in the host and guest and then operate the balloon accordingly. This doesn't work well in practice (if doable at all).

The balloon has to be automatic in order to be really useful. It could like this: when the host is under pressure, it asks guests to relinquish some megas if they can. When/if a guest gets into memory pressure, it gets some megas back from the host. That's what the automatic ballooning series is about.

KVM Forum 2013 presentation slides

They can be found here, but note that much has changed since this talk.

Patches and Git trees

Latest RFC version posted upstream

Guest kernel: RFC PATCH 0/4 virtio_balloon: add pressure notification via a new virtqueue

QEMU: RFC PATCH balloon: add automatic ballooning support

Git trees

Guest kernel:

git://repo.or.cz/linux-2.6/luiz-linux-2.6.git virtio-balloon/pressure-notification/rfc/v1

QEMU:

git://repo.or.cz/qemu/qmp-unstable.git balloon-automatic/handle-all-events/rfc/v1

TODO

More performance tests
Make it dynamic to enable/disable automatic ballooning
Do kernel work required in order to have non-cgroups code using in-kernel memory pressure notification

Testing

Setup:

Install kernel 3.10 or higher in your host. Make sure the kernel options CONFIG_CGROUPS and CONFIG_MEMCG are enabled
Build QEMU from the Git trees section
Build and install the guest kernel from the Git trees section

Then start qemu with:

# qemu [...] -device virtio-balloon,automatic=true

To see automatic ballooning in action, do the following:

Generate pressure in your host (like running several VMs in parallel); or connect to QMP and issue the balloon-inject-host-pressure command
You can monitor automatic ballooning activity by issuing info balloon in QEMU's monitor (also, I have debug code enabled that will dump a lot of info to stdout when "info balloon" is issued)
After the guest has shrinked a bit, you can generate pressure in the guest to see it increasing its size again

@@ Line 1: / Line 1: @@
-= Auto-ballooning =
+= Automatic Ballooning =
-When a kvm host is running out of memory, the host kernel will take action to reclaim memory. This action may be detrimental to kvm guests performace (eg. swapping) or even extreme to the point where the kernel kills user-space processes (which could be a kvm guest itself or a virt stack component).
+'''NOTE: This page describes an experimental development project from 2013 that was never completed. It is left here as a historical record. The feature described does not exist in any currently shipping version of QEMU'''
+== Introduction ==
-To help avoiding those scenarios, the hypervisor (eg. QEMU) could automatically inflate the guest's balloon (thus returning memory to the host). This will also require a method for automatically deflating the guest's memory, so that fairness is ensured and the guest kernel doesn't run itself into trouble.
+The virtio balloon device allows KVM guests to reduce their memory size (thus relinquishing memory to the host) and to increase it back (thus taking memory from the host).
-The object of this page is to collect ideas on how to implement auto-ballooning.
+This feature is mainly intended to support over-committing memory on KVM hosts. That is, hosts that are running VMs whose total memory size is greater than what the host has physically available. For example, a 2G host running two VMs each with 2G would be over-committed.
-Currently, there is only one proposal, which is the "automatic balance" idea,
-mostly designed by Rik van Riel.
-== Automatic balance ==
+The balloon device is important to support memory over-commitment because it allows for reducing a guest's memory size if needed. Suppose, in the previous example, that one of the guests is using 1G and its other 1G is free. We could use the balloon device to reduce this guest's size from 2G to 1G, this would free 1G in the host allowing the other VM to use it. Of course, if the reduced guest wants to run an application that consumes more than the 1G it currently has, it has to grow again.
-In automatic balance, the hypervisor (eg. QEMU) automatically inflates the guest balloon when the host kernel is under memory pressure. On the other hand, when the guest kernel is under memory pressure, it will automatically try to deflate its balloon (which possibly may have been inflated by the hypervisor).
+That's the problem with the current balloon device, it's entirely manual. Someone (or some tool) is supposed to be watching the pressure in the host and guest and then operate the balloon accordingly. This doesn't work well in practice (if doable at all).
-It's expected that automatic inflate and deflate can balance each other.
+The balloon has to be automatic in order to be really useful. It could like this: when the host is under pressure, it asks guests to relinquish some megas if they can. When/if a guest gets into memory pressure, it gets some megas back from the host. That's what the automatic ballooning series is about.
-Next sections describe the implementation details of this idea.
+=== KVM Forum 2013 presentation slides ===
-=== auto-inflate ===
+[https://www.linux-kvm.org/images/5/58/Kvm-forum-2013-automatic-ballooning.pdf They can be found here], but note that much has changed since this talk.
-There are three components needed to make auto-inflate work.
+== Patches and Git trees ==
-. A host kernel API to let user-space processes know when the kernel is facing memory pressure
+=== Latest RFC version posted upstream ===
-. QEMU changes to use the host kernel API, and to inflate the guest's balloon when the host kernel is facing memory pressure
+'''Guest kernel:''' [http://marc.info/?l=kvm&m=138988948715638&w=2 RFC PATCH 0/4 virtio_balloon: add pressure notification via a new virtqueue]
-=== auto-deflate ===
+'''QEMU:''' [http://marc.info/?l=kvm&m=138988966315690&w=2 RFC PATCH balloon: add automatic ballooning support]
-Only one component is needed for auto-deflate:
+=== Git trees ===
-. The guest virtio-balloon driver must be changed to automatically deflate the balloon when the guest is facing memory pressure
+'''Guest kernel:'''
-=== Status ===
+[http://repo.or.cz/w/linux-2.6/luiz-linux-2.6.git git://repo.or.cz/linux-2.6/luiz-linux-2.6.git] virtio-balloon/pressure-notification/rfc/v1
+'''QEMU:'''
+[http://repo.or.cz/w/qemu/qmp-unstable.git git://repo.or.cz/qemu/qmp-unstable.git] balloon-automatic/handle-all-events/rfc/v1
+== TODO ==
+* More performance tests
+* Make it dynamic to enable/disable automatic ballooning
+* Do kernel work required in order to have non-cgroups code using in-kernel memory pressure notification
+== Testing ==
+Setup:
+# Install kernel 3.10 or higher in your host. Make sure the kernel options CONFIG_CGROUPS and CONFIG_MEMCG are enabled
+# Build QEMU from [[#Git trees| the Git trees section]]
+# Build and install the guest kernel from [[#Git trees| the Git trees section]]
+Then start qemu with:
+<nowiki>   # qemu [...] -device virtio-balloon,automatic=true</nowiki>
+To see automatic ballooning in action, do the following:
+# Generate pressure in your host (like running several VMs in parallel); or connect to QMP and issue the ''balloon-inject-host-pressure'' command
+# You can monitor automatic ballooning activity by issuing ''info balloon'' in QEMU's monitor (also, I have debug code enabled that will dump a lot of info to stdout when "info balloon" is issued)
+# After the guest has shrinked a bit, you can generate pressure in the guest to see it increasing its size again

Projects/auto-ballooning: Difference between revisions