UsingLargePages

To utilize Larges Pages in the host to back your KVM guest one needs to use hugetlbfs. A little bit of setup is needed to get this going.

Host Kernel Config

Your host kernel config needs to have:

CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y

Host /etc/fstab

hugetlbfs is indeed a pseudo-filesystem, like tmpfs, so you need to mount it somewhere and add an entry to fstab if you want to automatically have it mounted on boot.

% sudo mkdir /hugetlbfs

Add an entry to fstab like this:

hugetlbfs       /hugetlbfs  hugetlbfs       defaults        0 0

Calculate amount number of pages to reserve

On x86, large pages are 2M, so determine the max size of the guest, and with some fudge factor (about 30 pages, completely arbitrary value); we have something like:

HPAGES = $(($GUEST_SIZE_IN_MB / 2) + 30)

For example, a 2G guest would yield 1054 2MB pages.

Reserve large pages after boot

Take your calculated value and echo that into proc quickly after a fresh boot of the machine to ensure you haven't fragmented your memory such that you can't reserve enough large pages.

echo $HPAGES >  /proc/sys/vm/nr_hugepages

Check your reservation

You can check whether or not your reservation was successful by looking at /proc/meminfo

% tail -n 5 /proc/meminfo
HugePages_Total:  1054
HugePages_Free:   1054
HugePages_Rsvd:      0
HugePages_Surp:      0
Hugepagesize:     2048 kB

Using large page reservation

To use your large page reservation you need to add an option to your guest command line. '-mem-path <path/to/hugetlbfs/mount/point>' If you followed the above commands, then you would add '-mem-path /hugetlbfs'

Checking large page usage

After launching your guest with large pages, you can check that your guest is using the large pages by sampling the HugePages_Free value from /proc/meminfo. This value should go down as your guest pages in memory.