Corner cases are slightly non-obvious problems implementing huge pages. If you think of one, please archive it here so it doesn't come back and bite us later.

• NUMA-aware allocation - don't want all the memory accesses going through one controller! See IBM's page about tuning Stream.

  • STREAM and hugetlb is a huge issue unless you have chipset level interleave no matter what
• Fork of an application with huge pages when no huge pages are left - must fail over to small pages.
• mmap of a file that is partially in cache already
• mmap of a file that then gets replaced on disk
  • (say glibc text and then glibc gets upgraded to a newer version)
• MAP_GROWSDOWN for stacks including automatic growth
• mprotect to executable (scenario: dlopen of a shared lib that
  • requires executable stack, causing all thread stacks to be mprotected executable)
• write() of mmaped area to the same file at an offset either
  • inside or outside the same 2Mb page
• same but for an O_DIRECT write
• same but for AIO write
• sendfile() of a non-4Kb region
• truncate of 4Mb mmaped file to 1Mb, then truncate to 1.5Mb
  • then 0.5Mb (creates a sparse file)
• mmap of a 2Mb region in 2 parts: first 1Mb gets mmaped writable private,
  • second 2Mb gets mmaped shared writable; then write a few bytes to each part
• Running a large .bss segment workload, specifying only a portion of the large page requirements to be made available to the workload... ie: app defines 10GB in arrays, but only 5GB are
• whether an entire segment must be dedicated to huge pages (256MB in the ppc environment) if one huge page is specified Should huge pages be allocated by segment?
• can we define stages of transparent functionality to enable customer production usage in the near future?
• can a non-root userid be specified to have access to huge pages? in a production environment, admins may require ability to designate huge page users..