Projects/auto-ballooning

From KVM
Revision as of 15:00, 16 January 2014 by LuizCapitulino (talk | contribs)

Automatic Ballooning

Introduction

The balloon feature allows KVM guests to reduce their memory size (thus relinquishing memory to the host) and to increase back (thus taking memory from the host).

This feature is mainly intended to support memory over-committed hosts. That is, hosts that are running VMs whose the total memory is greater than what the host has physically available. For example, a host with 2G free of memory is running two VMs each with 2G would be over-committed.

The balloon feature is important to support memory over-commitment because it allows for reducing a guest's memory size if needed. Suppose, in the example above, one of the guests is using 1G and its other 1G free. We could use the balloon to reduce this guest's size to 1G, this would free 1G in the host allowing the other VM to use it. Of course, if the reduced guest wants to run an application that consumes more than 1G it currently has, it has to grow again.

That's the problem with the current balloon feature, it's entirely manual. Someone (or some tool) is supposed to be watching the pressure in the host and guest and then operate the balloon accordingly. This is just not doable in real life.

In order to solve this problem we have to make the balloon automatic. It could like this: when the host is under pressure, VMs with spare memory automatically reduce their size by some megas. When a VM get into pressure (maybe because they relinquished memory to host) it increases its size by some megas. That's exactly what the automatic ballooning feature does.

Design

KVM guests have a driver called the balloon driver. This driver allows guests to shrink and grow their memory. The balloon driver supports two operations:

  • Inflate: memory is taken from the guest and given to the host (guest shrinks)
  • Deflate: memory is returned from the host to the guest (guest grows)

Today, both operations are manual. The automatic ballooning project is about making them completely automatic, based on host and guest needs.

KVM Forum 2013 presentation slides

They can be found here

Automatic Inflate

Automatic inflate is performed by QEMU (ie. the KVM host). QEMU registers for memory pressure events so that it's notified when the host is under memory pressure.

Current patches have pre-defined values to be used by QEMU when it receives a memory pressure notification from the host kernel. Those values are:

  • 1MB on LOW pressure
  • 2MB on MEDIUM pressure
  • 4MB on CRITICAL pressure

For example, suppose the host is facing MEDIUM pressure and notifies QEMU. When QEMU receives the event, it asks the guest to inflate its balloon by 2MB. The guest in turn will shrink itself by 2MB and give that memory to the host.

Automatic Deflate

Automatic deflate is performed by the guest or, more specifically, by the balloon driver.

The virtio-balloon driver registers a callback with the shrinker API. That callback is called when the guest kernel is facing memory pressure, and the number of pages to be returned to the kernel is passed to the callback. The balloon driver shrink callback deflates the guest's balloon.

For example, suppose the guest shrunk itself because of pressure in the host. But after some time, the guest is running more applications which causes memory pressure in the guest due its reduced size. The guest kernel then starts reclaiming memory and calls all shrink callbacks. That's when the balloon driver's shrink callback runs, and deflates the balloon by the number of pages specified. This causes the guest to grow again.

Git trees

Latest RFC version posted upstream

QEMU:

git://repo.or.cz/qemu/qmp-unstable.git balloon/auto-ballooning/rfc.v2 (or grab the patch here)

Guest kernel:

git://repo.or.cz/linux-2.6/luiz-linux-2.6.git virtio-balloon/auto-deflate/rfc (or grab the first two patches from the web interface)

Development branches

QEMU:

git://repo.or.cz/qemu/qmp-unstable.git balloon/auto-ballooning/current

Guest kernel:

git://repo.or.cz/linux-2.6/luiz-linux-2.6.git virtio-balloon/auto-deflate/current

Testing

You have to setup the following before experimenting with auto-ballooning:

  1. Install kernel 3.10 or higher in your host. Make sure the kernel options CONFIG_CGROUPS and CONFIG_MEMCG are enabled
  2. Build and install QEMU from the Git trees section (suggest the RFC one)
  3. Build and install the guest kernel from the Git trees section (suggest the RFC one)

After setting up the above, do the following to experiment with automatic deflate:

  1. Pass -balloon virtio,auto-balloon=true when starting QEMU
  2. Wait for the guest to boot, then generate some memory pressure in the guest (say a kernel build with lots of jobs)
  3. Switch to QEMU's monitor and shrink the guest (say from 1G to 200MB)
  4. Watch the guest increase its memory by running "free" within the guest (or "info balloon" in QEMU)

There are two ways to play with automatic inflate:

  1. The simplest thing is to create a memory constrained cgroup, but this will require code changes in QEMU because it's hardcoded to use the root cgroup (yes, we need a command-line option for that)
  2. If you don't want to play with cgroups, you can overcommit your host by running several VMs in parallel. The VMs have to run a heavy memory workload
Retrieved from "https://linux-kvm.org/index.php?title=Projects/auto-ballooning&oldid=5012"