| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367 |
- <!doctype html>
- <html lang="en">
- <head>
- <meta charset="utf-8">
- <meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Memory | NebOS Architecture</title>
- <meta name="description" content="NebOS memory architecture: fixed low-memory addresses, E820 parsing, bitmap frame allocator with lowmem/highmem zones, identity paging, kmap, MTRR write-combining, user arenas and boot-pack lifetime.">
- <link rel="icon" href="favicon.svg" type="image/svg+xml">
- <link rel="stylesheet" href="style.css">
- </head>
- <body>
- <a class="skip" href="#main">Skip to content</a>
- <header class="site-header">
- <a class="brand" href="index.html">
- <span class="brand-mark" aria-hidden="true"></span>
- <span>NebOS <span class="brand-sub">Architecture</span></span>
- </a>
- <nav class="site-nav" aria-label="Architecture sections">
- <a href="index.html">Overview</a>
- <a href="boot.html">Boot</a>
- <a href="memory.html" aria-current="page">Memory</a>
- <a href="cpu.html">CPU</a>
- <a href="drivers.html">Drivers</a>
- <a href="filesystems.html">Filesystems</a>
- <a href="graphics.html">Graphics</a>
- <a href="network.html">Network</a>
- <a href="userland.html">Userland</a>
- <a href="development.html">Development</a>
- </nav>
- </header>
- <main id="main" class="page">
- <div class="page-head">
- <p class="kicker">Memory & paging</p>
- <h1>A fixed ladder and a 4 KB frame allocator.</h1>
- <p class="lede">
- NebOS identity-maps the first 1 GB of RAM (lowmem) with 4 MB pages
- and hands out 4 KB frames from a bitmap; RAM from 1 GB up to 4 GB
- (highmem) backs ring-3 pages and is reached by the kernel through
- per-CPU <code>kmap</code> slots. A small set of low-memory addresses is
- fixed by the boot path and the ABI; everything else — process stacks,
- window canvases, the ring-3 heap and the rootfs copy — is dynamic.
- </p>
- <div class="meta">
- <span class="pill">1 GB lowmem · 4 GB highmem</span>
- <span class="pill">4 KB frames</span>
- <span class="pill">4 MB PSE pages</span>
- <span class="pill">identity map</span>
- </div>
- </div>
- <nav class="toc" aria-label="On this page">
- <h2>On this page</h2>
- <ul>
- <li><a href="#physical">Physical memory</a></li>
- <li><a href="#landmarks">Address landmarks</a></li>
- <li><a href="#paging">Paging & caching</a></li>
- <li><a href="#frame">Frame allocator</a></li>
- <li><a href="#heap">kmalloc</a></li>
- <li><a href="#user">Ring-3 memory</a></li>
- <li><a href="#pack">Rootfs pack lifetime</a></li>
- <li><a href="#limits">Limits & leaks</a></li>
- <li><a href="#sources">Source map</a></li>
- </ul>
- </nav>
- <section id="physical">
- <h2>Physical memory</h2>
- <p>
- The real-mode stage probes <code>INT 15h</code> E820 and writes the
- map to <code>0x1000</code>; the UEFI loader synthesises the same
- structure from <code>GetMemoryMap</code>. <span class="file">mm.c</span>
- starts with every page marked used, frees the E820 “usable RAM”
- ranges, then reserves the regions the kernel already owns.
- </p>
- <figure class="figure">
- <svg viewBox="0 0 920 180" role="img" aria-labelledby="phys-title phys-desc">
- <title id="phys-title">Physical memory composition</title>
- <desc id="phys-desc">Of the 1 GB addressable window, the low 1 MB and an 8 MB VGA shadow reserve are unavailable, leaving about 1015 MB for the frame allocator.</desc>
- <text class="svg-text" x="20" y="28">0x00000000</text>
- <text class="svg-text" x="900" y="28" text-anchor="end">0x40000000 · 1 GB</text>
- <rect class="svg-box blue" x="40" y="44" width="1.5" height="34" rx="1"/>
- <rect class="svg-box accent" x="41.5" y="44" width="6.7" height="34" rx="1"/>
- <rect class="svg-box green" x="48.2" y="44" width="851.8" height="34" rx="2"/>
- <text class="svg-tiny" x="40" y="100">low 1 MB</text>
- <text class="svg-tiny" x="64" y="100">VGA shadow · 8 MB</text>
- <text class="svg-tiny" x="477" y="100" text-anchor="middle">frame allocator · ~1015 MB</text>
- <line class="svg-line" x1="41" y1="78" x2="41" y2="88"/>
- <line class="svg-line" x1="45" y1="78" x2="45" y2="88"/>
- <line class="svg-line" x1="490" y1="78" x2="490" y2="88"/>
- <text class="svg-tiny" x="40" y="130">boot structures · kernel image · BIOS/UMA · DMA buffers</text>
- <text class="svg-tiny" x="40" y="150">The 8 MB reserve protects the shadow buffer; the allocator starts at 0x900000.</text>
- </svg>
- <figcaption>
- <strong>Lowmem ends at 1 GB</strong> (<code>MM_MAX_PHYS</code>,
- <code>include/phys.h</code>). The identity map covers
- <code>[0, 0x40000000)</code> and ring-3 space starts at
- <code>USER_BASE</code> = <code>0x40000000</code>; <code>paging.c</code>
- statically asserts the two never overlap. RAM above 1 GB (up to
- <code>MM_HIGH_LIMIT</code>, 4 GB) is highmem: tracked by the same
- bitmap, never identity-mapped, used only for ring-3 pages. The VGA shadow reserve is conservative: the
- actual buffer is smaller, but the gap keeps a future resolution bump
- from colliding with allocated pages.
- </figcaption>
- </figure>
- </section>
- <section id="landmarks">
- <h2>Address landmarks</h2>
- <p>
- These addresses are part of the boot contract or the kernel ABI.
- Changing one requires updating the assembly, linker scripts, headers
- and this table together.
- </p>
- <div class="table-wrap">
- <table>
- <thead>
- <tr><th>Address</th><th>Purpose</th><th>Written by / notes</th></tr>
- </thead>
- <tbody>
- <tr><td><code>0x500</code></td><td>Boot info, 12 dwords</td><td>VESA stage, UEFI loader or kexec; <code>bootinfo.h</code></td></tr>
- <tr><td><code>0x600</code></td><td>Boot framebuffer info</td><td>VESA stage or UEFI loader; <code>vga.h</code></td></tr>
- <tr><td><code>0x7C00</code></td><td>Boot-sector load address / UEFI handoff stack top</td><td>BIOS and UEFI trampoline</td></tr>
- <tr><td><code>0x7E00</code></td><td>Kernel load address and VESA-stage entry</td><td><code>kernel.ld</code></td></tr>
- <tr><td><code>0x800</code></td><td>kexec and UEFI trampoline</td><td>Relocated at runtime; below the kernel</td></tr>
- <tr><td><code>0x900</code></td><td>kexec/UEFI flat GDT and GDTR</td><td>Built at runtime; below the kernel</td></tr>
- <tr><td><code>0x1000</code></td><td>E820 memory map</td><td>VESA stage, UEFI loader or kexec; parsed by <code>mm.c</code></td></tr>
- <tr><td><code>0x3000</code></td><td>UHCI frame list</td><td>1024 × 4 bytes, 4 KB aligned</td></tr>
- <tr><td><code>0x4000</code></td><td>ISA DMA buffer</td><td>One sector, used by the floppy driver</td></tr>
- <tr><td><code>0x6000</code></td><td>AP real/protected-mode trampoline</td><td>SIPI vector <code>0x06</code>; image ends below <code>0x6C00</code></td></tr>
- <tr><td><code>0x6C00</code></td><td>AP trampoline real-mode stack top</td><td>Grows down</td></tr>
- <tr><td><code>0x6F00</code></td><td>AP startup info block</td><td>CR3, stack top, C entry, CPU index</td></tr>
- <tr><td><code>0x60000</code></td><td>Former rootfs staging</td><td>No longer used by the loader; kernel BSS/stack may use it</td></tr>
- <tr><td><code>_end</code></td><td>Kernel end</td><td><code>kernel.ld</code> asserts <code>_end ≤ 0x90000</code></td></tr>
- <tr><td><code>0x900000</code></td><td>Boot pack (written directly by the legacy stage), then the frame bitmap right after it; dynamic stacks and kexec staging</td><td>The pack is freed after <code>ramfs_init</code> takes its copy; the bitmap stays</td></tr>
- <tr><td><code>0x100000</code></td><td>VGA shadow buffer base</td><td><code>VGA_SHADOW_BASE</code>; 8 MB reserved</td></tr>
- <tr><td><code>0x40000000</code></td><td>Ring-3 ELF link/load address</td><td><code>USER_BASE</code>, 4 MB window up to <code>0x40400000</code></td></tr>
- <tr><td><code>0x4FC00000</code></td><td>kmap window (virtual)</td><td><code>MM_KMAP_VA</code>; supervisor page table shared by every directory, 4 slots per CPU</td></tr>
- <tr><td><code>0x50000000</code></td><td>Ring-3 window canvas</td><td><code>USER_WIN_BASE</code>, 4 MB slot</td></tr>
- <tr><td><code>0x58000000</code></td><td>WM shadow mapping</td><td><code>WM_SHADOW_VA</code>; ring-3 WM only</td></tr>
- <tr><td><code>0x59000000</code></td><td>WM content slots</td><td><code>WM_CONTENT_VA_BASE</code> + index × 4 MB (<code>WM_CONTENT_SLOT</code>)</td></tr>
- <tr><td><code>0x60000000</code></td><td>Ring-3 anonymous heap</td><td><code>USER_HEAP_BASE</code>, up to 64 MB</td></tr>
- </tbody>
- </table>
- </div>
- </section>
- <section id="paging">
- <h2>Paging & caching</h2>
- <p>
- <span class="file">paging.c</span> builds an identity map of RAM with
- 4 MB pages through <code>CR4.PSE</code>, adds the framebuffer's LFB
- range, and enables <code>CR0.PG</code> and <code>CR0.WP</code>. Before
- switching, it clears translation controls a BIOS protected-mode call
- can leave behind: <code>CR4.PAE/PGE/LA57/PCIDE/SMEP/SMAP</code> and
- <code>EFER.LME</code>.
- </p>
- <ul>
- <li><strong>Identity map.</strong> Physical and virtual addresses are equal below 1 GB (lowmem). Page-directory entries are shared with every process.</li>
- <li><strong>kmap window.</strong> The 4 MB slot at <code>MM_KMAP_VA</code> (<code>0x4FC00000</code>, in the unused gap below <code>USER_WIN_BASE</code>) has one supervisor page table that every process directory shares. <code>kmap(phys)</code> returns the identity address for lowmem, and for a highmem frame takes this CPU's next slot (4 per CPU, nesting LIFO) with preemption disabled until <code>kunmap()</code>; only the local TLB entry is flushed, because no other CPU uses that slot. <code>paging_map_ram</code>, <code>paging_map_mmio_cr3</code> and the LFB mapping refuse to overwrite that slot. It is not the last lowmem slot, because firmware puts the ACPI tables at the top of RAM, and not the top of the address space, where a VBE LFB may sit.</li>
- <li><strong>Kernel stacks mapped everywhere.</strong> Each process gets a page directory cloned from the kernel identity map, so CR3 can switch while still running on the old kernel stack.</li>
- <li><strong>LFB is uncacheable by default.</strong> If <code>mtrr_set_wc()</code> successfully covers the framebuffer, the LFB pages are mapped cacheable so the shadow-buffer flush can write-combine.</li>
- <li><strong>No process isolation.</strong> There is one address space with per-process page directories only for mappings; a wild ring-0 or ring-3 pointer can reach any mapped physical page.</li>
- </ul>
- <p>
- There is no cross-CPU TLB shootdown. MMIO mappings for another CPU's
- running process rely on not-present TLB entries not being cached;
- normal process teardown is unaffected because no two processes share
- an address space.
- </p>
- <div class="note">
- <p><strong>NVIDIA ION special case.</strong> On the ION the framebuffer is
- stolen DRAM and the VBE LFB is a BAR3 window into it. The BIOS leaves a
- 512 MB uncacheable MTRR over the PCIe apertures, so <span class="file">nvidia.c</span>
- locates the framebuffer in the stolen VRAM and writes through that
- alias once a WC MTRR covers the stolen range. Everything else keeps
- the uncacheable LFB.</p>
- </div>
- </section>
- <section id="frame">
- <h2>Frame allocator</h2>
- <p>
- <span class="file">mm.c</span> keeps one bit per 4 KB page for the
- RAM below 4 GB: the bitmap is sized to the highest E820 RAM
- address (32 KB at 1 GB, 96 KB at 3 GB) and is not part of the kernel
- image. <code>mm_init</code> places it in the first lowmem E820 RAM run at or above
- <code>0x900000</code> that avoids the boot rootfs pack and the LFB, and
- marks its own pages used (legacy boots put it right after the pack;
- a 16 MB static fallback covers a missing E820 map). The allocators
- disable interrupts around the bitmap and hand out the lowest free
- frames of a zone: each zone has a first-free word hint that skips its
- full low part, single pages are found with <code>bsf</code>, and
- <code>mm_alloc_pages(n)</code> steps over full and empty words 32 pages
- at a time. Everything the kernel dereferences by physical address
- (page tables, DMA buffers, canvases, kernel stacks, <code>kmalloc</code>)
- comes from lowmem; ring-3 program, stack and heap frames prefer
- highmem. The allocator returns:
- </p>
- <ul>
- <li><code>mm_alloc_page()</code> — one lowmem page (physical address 0 means failure).</li>
- <li><code>mm_alloc_user_page()</code> — one frame for a ring-3 mapping: highmem first, lowmem when highmem is exhausted. Touch it only through <code>kmap()</code> or the user mapping (<span class="file">elf.c</span>, the user stack in <span class="file">process.c</span>, the heap in <span class="file">sys_mem.c</span>).</li>
- <li><code>mm_alloc_pages(n)</code> — <code>n</code> contiguous lowmem pages.</li>
- <li><code>mm_free_page()</code> / <code>mm_free_range()</code> — release a page or a page-aligned boot reservation.</li>
- </ul>
- <p>
- At boot, <code>mm_init</code> reserves the low 1 MB, the 8 MB VGA
- shadow region, any LFB that overlaps RAM, the bitmap itself and the
- boot-time rootfs pack. The pack reservation is released by
- <code>ramfs_init</code> after it copies/inflates the pack into fresh
- pages.
- </p>
- <p>
- Fresh pages are <em>not</em> zeroed. Structures whose fields are read
- before being written must be cleared explicitly; <code>clear_proc()</code>
- in <span class="file">process.c</span> exists because a stale
- <code>cr3</code> faulted on real hardware while QEMU's cleared RAM hid
- the bug.
- </p>
- </section>
- <section id="heap">
- <h2>kmalloc</h2>
- <p>
- <span class="file">kmalloc.c</span> is a power-of-two heap allocator
- on top of the frame allocator. It is intentionally simple: allocation
- rounds to a size class, free lists are per class, and large objects
- become contiguous page runs. PCBs, command context and
- <span class="file">sys_fs.c</span> file loads use it.
- </p>
- <p>
- The kernel heap is separate from userspace. Ring-3 processes get one
- anonymous arena through <code>SYS_MEM_ALLOC</code>,
- <code>SYS_MEM_GROW</code> and <code>SYS_MEM_FREE</code>;
- commands manage it with <code>malloc</code>/<code>free</code>
- from <span class="file">libc/malloc.c</span>. The syscalls are the
- backend in <span class="file">libc/ucmd_mem.c</span> and are not
- called by application code.
- </p>
- </section>
- <section id="user">
- <h2>Ring-3 memory</h2>
- <p>
- A ring-3 ELF is linked at <code>0x40000000</code>. The image and its
- 32-page (128 KB) user stack live in a 4 MB window ending at
- <code>0x40400000</code>. The kernel window server maps a window's
- RGB565 back buffer at <code>0x50000000</code>, and the anonymous heap
- starts at <code>0x60000000</code>.
- </p>
- <figure class="figure">
- <div class="bars" role="img" aria-label="Ring-3 arena sizes relative to the 64 MB heap cap: image and stack 4 MB, canvas slot 4 MB, heap up to 64 MB.">
- <div class="bar-row">
- <span class="bar-label">image + stack</span>
- <div class="bar-track"><span class="bar-fill blue" style="--w:6.25%"></span></div>
- <span class="bar-value">4 MB</span>
- </div>
- <div class="bar-row">
- <span class="bar-label">canvas slot</span>
- <div class="bar-track"><span class="bar-fill" style="--w:6.25%"></span></div>
- <span class="bar-value">4 MB</span>
- </div>
- <div class="bar-row">
- <span class="bar-label">anonymous heap</span>
- <div class="bar-track"><span class="bar-fill green" style="--w:100%"></span></div>
- <span class="bar-value">64 MB</span>
- </div>
- </div>
- <figcaption>
- <strong>Three separate rings.</strong> The image/stack window is
- fixed at link time; the canvas slot is one 4 MB mapping; the heap is
- a growable arena. Bar widths are relative to the 64 MB heap cap
- (<code>USER_HEAP_MAX</code> in <span class="file">process.h</span>; the browser's
- DOM arenas and QuickJS heap use ~30 MB before images).
- </figcaption>
- </figure>
- <div class="table-wrap">
- <table>
- <thead>
- <tr><th>Range</th><th>Size</th><th>Contents</th></tr>
- </thead>
- <tbody>
- <tr><td><code>0x40000000</code>–<code>0x40400000</code></td><td>4 MB</td><td>ELF segments, user stack top at <code>0x40400000</code> (32 pages / 128 KB, grows down); highmem frames when available</td></tr>
- <tr><td><code>0x50000000</code>–<code>0x50400000</code></td><td>4 MB</td><td>One window canvas mapping (RW, per window)</td></tr>
- <tr><td><code>0x60000000</code>–<code>0x64000000</code></td><td>up to 64 MB</td><td>Anonymous heap (highmem frames when available, zeroed through <code>kmap</code>); initial allocation zeroed, growth pages are not</td></tr>
- </tbody>
- </table>
- </div>
- <p>
- Ring-3 commands treat that arena as a real heap:
- <span class="file">libc/malloc.c</span> maps 64 KB on
- first use, grows in page-rounded steps with 64 KB slack, and uses
- boundary-tag blocks with a first-fit free list and immediate
- coalescing. <code>heap_reset()</code> discards everything and unmaps
- the arena; the browser calls it on navigation, so page-scoped
- allocations (body, rules, image bitmaps and decode arenas) go back
- to the frame allocator. A single allocation near 64 MB fails because
- block headers count against the same cap.
- </p>
- <p>
- Kernel stacks are 4 KB pages from the frame allocator, with an 8-stack
- static fallback pool in BSS. The idle/bootstrap context uses the
- in-image 8 KB <code>.stack</code> section. The user stack is separate:
- 32 pages just below <code>0x40400000</code>.
- </p>
- </section>
- <section id="pack">
- <h2>Rootfs pack lifetime</h2>
- <ol>
- <li>The compressed pack is at most 2 MB (<code>PACK_MAX</code>), so the legacy copy ends below <code>0xB00000</code>; inflated it is about 2 MB with the WiFi firmware (the 8 MB <code>RAMFS_MAX_DECOMPRESSED</code> cap applies).</li>
- <li>The boot path places the gzip pack in RAM: the legacy stage copies it directly to <code>0x900000</code> with a protected-mode <code>rep movsb</code>, UEFI into a loader-data buffer below 1 GB, kexec in its staging buffer.</li>
- <li><code>mm_init</code> reserves <code>[pack_addr, pack_addr + pack_size)</code> so the frame allocator cannot reuse it.</li>
- <li><code>ramfs_init</code> takes a private page-allocated copy, validates the <code>NBROOT1</code> header and inflates the pack into it.</li>
- <li><code>mm_free_range</code> releases the boot region when it is above the VGA shadow reserve. The private copy lives until <code>ramfs_replace</code> or reboot.</li>
- </ol>
- <p>
- The ramfs holds at most 128 files and 8 MB of decompressed pack. A
- larger pack fails to load rather than partially mounting.
- </p>
- </section>
- <section id="limits">
- <h2>Limits & leaks</h2>
- <ul>
- <li><strong>No per-process address spaces.</strong> Paging is identity-mapped; there is no memory protection between processes.</li>
- <li><strong>4 GB ceiling, 64 MB per process.</strong> RAM above 4 GB is ignored (it would need PAE). Highmem raises the system total, but one process still has a 64 MB heap and a 4 MB image/stack window. MMIO is still identity-mapped at its physical address, which blocks growing ring-3 space toward the PCI hole. A wild kernel pointer between the identity map and the user windows is an instant <code>#PF</code>.</li>
- <li><strong>Highmem is user-only.</strong> Contiguous kernel buffers and every kernel structure stay in the 1 GB lowmem zone, so a machine with 4 GB can still run out of lowmem first.</li>
- <li><strong>Kernel heap fragmentation.</strong> <code>kmalloc</code>'s power-of-two size classes make long-running mixed-size kernel allocation wasteful; ring-3 commands use the separate first-fit libc heap described above.</li>
- <li><strong>No TLB shootdown.</strong> Cross-CPU mapping changes rely on not-present entries not being cached.</li>
- </ul>
- </section>
- <section id="sources">
- <h2>Source map</h2>
- <ul class="filelist">
- <li><code>kernel/mm.c</code>, <code>kernel/mm.h</code> — E820 + bitmap allocator</li>
- <li><code>include/phys.h</code> — lowmem ceiling, kmap window, highmem limit</li>
- <li><code>kernel/kmalloc.c</code>, <code>kernel/kmalloc.h</code> — kernel heap</li>
- <li><code>user/lib/libc/malloc.c</code>, <code>user/lib/libc/ucmd_mem.c</code> — ring-3 heap and <code>SYS_MEM_*</code> backend</li>
- <li><code>user/lib/libc/string.c</code>, <code>user/lib/libc/stdlib.c</code> — ring-3 libc support</li>
- <li><code>kernel/paging.c</code>, <code>kernel/paging.h</code> — identity map, kmap window</li>
- <li><code>kernel/mtrr.c</code> — write-combining MTRRs</li>
- <li><code>kernel/process.c</code>, <code>kernel/process.h</code> — stacks and user layout</li>
- <li><code>kernel/ramfs.c</code> — pack copy / inflate</li>
- <li><code>boot/efi/main.c</code> — firmware map conversion</li>
- <li><code>kernel/kernel.ld</code> — low-memory image limits</li>
- </ul>
- </section>
- </main>
- <footer class="site-footer">
- <div class="inner">
- <p><strong>NebOS 0.2.0</strong> · Architecture documentation · <a href="index.html">overview</a></p>
- <p>Maintain alongside <code>AGENTS.md</code> and the source files listed on each page.</p>
- </div>
- </footer>
- </body>
- </html>
|