menios icon indicating copy to clipboard operation
menios copied to clipboard

Extend heap arena with page unmapping and diagnostics

Open pbalduino opened this issue 4 months ago • 0 comments

Goal

Target the kernel heap's VM arena so that once blocks are freed, the backing pages are unmapped (and frames returned) and tooling reports how much of the arena is actually committed. Currently the code tracks regions but doesn't reclaim physical pages or surface those metrics.

Current State

The kernel heap manages virtual memory but leaks physical frames:

  • heap_release_region_if_unused doesn't free frames: The function decides when an entire managed region is free, unlinks the list node, and calls heap_unmap_pages plus heap_virtual_release, but it never frees the underlying frames (src/kernel/mem/kmalloc.c:430-481). The region still carries phys_base/page_count that go unused before dropping the descriptor, leaking RAM even after the virtual range goes idle.
  • No accounting for committed vs reserved: Because descriptors are removed via heap_unregister_region immediately afterwards (src/kernel/mem/kmalloc.c:320-360), any new accounting will need to snapshot the bytes and counts before unregistering. Right now there's no hook that keeps committed-vs-reserved stats once the region disappears.
  • Basic diagnostics: heap_get_stats only reports total/free/used bytes and region count by walking the region list and free nodes (src/kernel/mem/kmalloc.c:870-903, include/kernel/heap.h:39-57). There's no visibility into reserved-but-unmapped space, release counters, or guard-page events, so consumers can't tell whether shrinkage actually occurred.
  • Single 256 MiB virtual window: The allocator relies on a single window (KHEAP_BASE/KHEAP_LIMIT) served by heap_virtual_acquire/heap_virtual_release (src/kernel/mem/kmalloc.c:17-307). Once shrinking is in place, these helpers must cope with holes being returned so subsequent growth can reuse them.

Definition of Done

1. Fix heap_release_region_if_unused to Free Physical Frames

  • Update heap_release_region_if_unused in src/kernel/mem/kmalloc.c:430-481
  • Call pmm_free_pages(region->phys_base, region->page_count) before dropping the descriptor
  • Reset the metadata (phys_base, page_count) as part of the release path
  • Ensure frames are actually returned to the physical memory allocator

2. Implement Committed vs Reserved Accounting

  • Snapshot bytes and counts before calling heap_unregister_region (src/kernel/mem/kmalloc.c:320-360)
  • Add tracking for:
    • Reserved space (total virtual window size)
    • Committed space (actually mapped pages)
    • Released frames (pages unmapped and returned)
  • Maintain counters that survive region deletion

3. Enhance Diagnostics and Stats

  • Extend heap_get_stats in src/kernel/mem/kmalloc.c:870-903 and include/kernel/heap.h:39-57
  • Add fields for:
    • Reserved memory (virtual address space allocated)
    • Committed memory (pages actually backed by physical frames)
    • Released page count
    • Guard page events (if applicable)
  • Expose both "reserved" (window size) and "committed" (actually mapped) totals
  • Let consumers verify that page unmapping works

4. Update Virtual Window Management

  • Modify heap_virtual_acquire/heap_virtual_release in src/kernel/mem/kmalloc.c:17-307
  • Enable these helpers to cope with holes being returned
  • Allow subsequent growth to reuse freed virtual ranges
  • Maintain efficiency while handling fragmented virtual space

5. Add Shrink Event Logging (Optional)

  • Optionally log shrink events for debugging
  • Track when regions are unmapped and frames returned
  • Help diagnose memory management behavior

6. Update Documentation

  • Update docs/architecture/kmalloc_vm.md to reflect new diagnostics
  • Document the committed vs reserved distinction
  • Explain the page unmapping and frame reclaim process

Next Steps After Frame Reclaim Lands

  1. Expand heap_get_stats (or create new API) to report reserved/committed counts
  2. Add debug logging for shrink events
  3. Update kernel memory monitoring tools to use new metrics
  4. Add regression tests to verify frame reclamation

Dependencies

  • Issue #6: kmalloc virtual memory arena (current implementation)
  • Physical memory manager (PMM) for frame allocation/deallocation
  • Virtual memory management infrastructure

Related Issues

This enhances the kernel heap implementation from Issue #6 with proper resource reclamation.

pbalduino avatar Sep 27 '25 02:09 pbalduino