Slab internals for exploitation #1: from the power button to pages

I’ve been away from kernel exploitation for a while. Trying to jump back in, I hit the usual wall: Linux memory management is huge, and trying to hold all the allocator internals, edge cases, and exploitation primitives in my head at once doesn’t work.

Rather than grepping mm/ from scratch every time I forget a struct layout, I’m writing down notes for myself based on Linux 6.8.9:

• Part 1 (this post): Power-on to a working page allocator e820 -> memblock -> buddy allocator).

• Part 2: SLUB (kmem_cache / kmalloc) sitting on top of the buddy allocator.

• Part 3: Exploitation primitives and modern mitigations.

Bootstrapping the Allocator

From the CPU’s perspective, physical RAM is just a flat byte array from 0 to N. There’s no hardware instruction for “give me 64 bytes”, software has to maintain the ledger of what’s free.

The annoying part is that the ledger itself has to live in RAM, so you can’t use the real allocator before its own metadata is allocated. Linux handles this in stages:

memblock  ──►  buddy (pages)  ──►  SLUB (kmalloc)  ──►  everything else
 (early)         (4K - 4M)          (small objs)

From Power-On to start_kernel()

When an x86 CPU comes out of reset, it starts in 16-bit real mode with CS:IP pointing to 0xFFFFFFF0 (16 bytes below the top of the 32-bit address space). The chipset wires that address straight to firmware flash. No paging, no protection, 1 MB of addressable space.

UEFI runs POST, initializes the memory controller, trains DRAM, enumerates PCIe, and builds ACPI tables. By the time it hands off to GRUB, systemd-boot, or the EFI stub, RAM works and the firmware has built a memory map separating usable DRAM from hardware-reserved regions.

bzImage

/boot/vmlinuz-6.8.9 isn’t a raw ELF binary; it’s a self-extracting container (bzImage):

┌────────────────────────────────────────────────────────┐
│ Boot sector + setup code (arch/x86/boot/, 16-bit)      │
│   └─ Setup header (boot protocol ABI, load addresses)  │
├────────────────────────────────────────────────────────┤
│ Compressed payload (arch/x86/boot/compressed/)         │
│   ├─ Decompressor stub (head_64.S + misc.c)            │
│   └─ Compressed vmlinux (gzip / zstd / lz4)            │
└────────────────────────────────────────────────────────┘

The setup header (arch/x86/boot/header.S) defines the x86 boot protocol. The bootloader parses it, drops the kernel and initrd into RAM, populates fields like the command-line pointer and the firmware’s e820 map, and jumps in.

Mode Transitions and KASLR

Getting into 64-bit mode takes three steps inside the early boot stub:

1. Real mode -> Protected mode: Load a flat GDT, set CR0.PE.

2. Protected mode -> Long mode: Long mode won’t run without paging enabled. startup_32 (arch/x86/boot/compressed/head_64.S) builds throwaway identity-mapped page tables, sets CR4.PAE and EFER.LME, and flips CR0.PG.

3. Decompression: extract_kernel() (arch/x86/boot/compressed/misc.c) unpacks vmlinux into memory.

KASLR happens right here, before the main kernel executes a single instruction. choose_random_location() picks a random physical and virtual base inside a valid RAM region from the e820 map. Everything in the kernel image lives at a fixed offset from that base, which is why leaking a single text or data pointer at runtime breaks KASLR completely.

After decompression, execution jumps to startup_64 (arch/x86/kernel/head_64.S), sets up real early page tables and %gs for per-CPU variables, and calls start_kernel() in init/main.c.

start_kernel()

start_kernel() brings up subsystems sequentially. Three calls form the memory initialization spine:

// init/main.c (trimmed)
asmlinkage **visible void** init start_kernel(void)
{
    setup_arch(&command_line);   // Parse e820, init memblock, early paging
    ...
    build_all_zonelists(NULL);   // Set up zone fallback order for buddy
    ...
    mm_core_init();              // mem_init() hands memblock -> buddy,
                                 // kmem_cache_init() brings SLUB online
    ...
}

e820 and memblock

Inside setup_arch(), the kernel parses the physical memory map (arch/x86/kernel/e820.c). Historically this came from INT 0x15, EAX=0xE820 on BIOS, and the name stuck on UEFI.

Entries are (start, size, type):

• E820_TYPE_RAM - Usable physical RAM.

• E820_TYPE_RESERVED - Firmware or MMIO regions.

• E820_TYPE_ACPI / E820_TYPE_NVS - ACPI tables and non-volatile storage.

• E820_TYPE_UNUSABLE - Faulty RAM.

dmesg | grep -i e820 shows the holes and reserved stripes carved out of physical address space:

BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
BIOS-e820: [mem 0x0000000000100000-0x00000000bffdffff] usable
BIOS-e820: [mem 0x00000000bffe0000-0x00000000bfffffff] reserved
BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved

To allocate memory before the real allocators exist, the kernel uses memblock (mm/memblock.c), which is just two arrays of (base, size) ranges:

• memblock.memory: Usable RAM regions copied from e820.

• memblock.reserved: Everything already occupying RAM (kernel image, initrd, early page tables, the e820 table itself).

memblock_alloc() linearly scans memblock.memory,top-down by default so low memory stays open for legacy DMA, finds an unreserved gap, and appends it to memblock.reserved. No freelists, no per-CPU caches. Once the struct page array and buddy metadata are allocated, mem_init() (mm/mm_init.c) calls memblock_free_all() to dump every remaining unreserved page into the buddy allocator, and memblock is done.

Pages, Zones, and struct page

Below SLUB, the kernel manages memory in 4 KiB pages (PAGE_SIZE on x86-64). Physical RAM is divided into 4 KiB frames indexed by PFN (physical_addr >> 12).

Every physical frame has a 64-byte struct page include/linux/mm_types.h). On a 16 GB system that’s 4 million structs (256 MB of metadata), so the struct is packed with overlapping C unions, the same 64 bytes get read differently depending on whether the frame backs userspace memory, page cache, or a SLUB slab. When SLUB claims a page, it overlays struct slab directly onto that struct page memory, enforced at compile time with static_assert.

Under SPARSEMEM_VMEMMAP on x86-64, all struct page entries sit in a contiguous virtual array (vmemmap), so pfn <-> struct page is just pointer arithmetic (&vmemmap[pfn]). The actual contents of physical frames are linearly mapped into kernel virtual space via the direct map / physmap (page_offset_base).

Frames are organized as Node -> Zone -> Page:

• Node (pg_data_t): NUMA memory bank (usually just Node 0 on desktop/VM setups).

• Zone (struct zone): Address-range buckets inside a node:

  • ZONE_DMA: Lowest 16 MB (ISA devices).

  • ZONE_DMA32: Below 4 GB (32-bit DMA devices).

  • ZONE_NORMAL: Above 4 GB.

The Buddy Allocator

The buddy allocator (mm/page_alloc.c) hands out physically contiguous blocks of pages in power-of-two sizes called orders, from order 0 ($2^0 = 1$ page, 4 KiB) up to order 10 ($2^{10} = 1024$ pages, 4 MiB). In 6.8.9, MAX_PAGE_ORDER is 10 (NR_PAGE_ORDERS = 11 in include/linux/mmzone.h).

Each struct zone keeps an array free_area[11]:

// include/linux/mmzone.h
struct free_area {
    struct list_head free_list[MIGRATE_TYPES];
    unsigned long    nr_free;
};

If an order-0 allocation hits an empty free_area[0], the allocator grabs a block from the next non-empty order up (e.g., order-3), splits it into two order-2 “buddies,” parks one half on free_area[2], and keeps halving down until it has a single page to hand out.

On free (__free_one_page()), it calculates the buddy’s PFN with buddy_pfn = pfn ^ (1 << order). If that buddy is also free, the two merge into an order + 1 block and try to merge again up the chain. When enough objects are freed to empty an entire slab page, SLUB returns the page to the buddy allocator where it can coalesce or get handed out to another subsystem, the root cause for cross-cache attacks.

Per-CPU Pagesets (pcp)

Acquiring the zone spinlock on every page allocation doesn’t scale across cores, so each CPU maintains a local cache: struct per_cpu_pages pcp). Allocations up to PAGE_ALLOC_COSTLY_ORDER (order 3, plus THP orders) pop directly off the local pcp list without touching the zone lock, only hitting free_area during bulk refills or drains. Local LIFO ordering on these per-CPU lists is what makes freeing a page and immediately reclaiming it on the same CPU reliable.

Migratetypes

free_list is split by MIGRATE_TYPES include/linux/mmzone.h). The allocator groups pages into 4 MiB pageblocks tagged by movability so pinned kernel allocations don’t fragment large contiguous regions of movable user memory:

• MIGRATE_UNMOVABLE: Kernel objects and page tables that cannot be relocated.

• MIGRATE_MOVABLE: Userspace pages and page cache (can be migrated to defragment).

• MIGRATE_RECLAIMABLE: Caches like dentries and inodes that can be dropped under pressure.

If a zone runs dry on one type, it falls back to stealing from another (fallbacks[] in mm/page_alloc.c). Both slab pages and page tables pull from MIGRATE_UNMOVABLE, so draining a slab page back to the buddy allocator and spraying page tables (or another slab cache) fishes out of the exact same pool.

Since a single buddy page is 4096 bytes, allocating a 12-byte filename or a 192-byte struct cred directly from the buddy allocator would waste almost the entire page. Part 2 covers SLUB (the only remaining slab allocator in 6.8 after SLOB was removed in 6.4 and SLAB in 6.8), how it carves buddy pages into object caches, how kmem_cache and kmalloc-* buckets are set up, and how the lockless per-CPU freelist works.