memory.html 16 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327
  1. <!doctype html>
  2. <html lang="en">
  3. <head>
  4. <meta charset="utf-8">
  5. <meta name="viewport" content="width=device-width, initial-scale=1">
  6. <title>Memory | NebOS Architecture</title>
  7. <meta name="description" content="NebOS memory architecture: fixed low-memory addresses, E820 parsing, bitmap frame allocator, identity paging, MTRR write-combining, user arenas and boot-pack lifetime.">
  8. <link rel="icon" href="favicon.svg" type="image/svg+xml">
  9. <link rel="stylesheet" href="style.css">
  10. </head>
  11. <body>
  12. <a class="skip" href="#main">Skip to content</a>
  13. <header class="site-header">
  14. <a class="brand" href="index.html">
  15. <span class="brand-mark" aria-hidden="true"></span>
  16. <span>NebOS <span class="brand-sub">Architecture</span></span>
  17. </a>
  18. <nav class="site-nav" aria-label="Architecture sections">
  19. <a href="index.html">Overview</a>
  20. <a href="boot.html">Boot</a>
  21. <a href="memory.html" aria-current="page">Memory</a>
  22. <a href="cpu.html">CPU</a>
  23. <a href="drivers.html">Drivers</a>
  24. <a href="filesystems.html">Filesystems</a>
  25. <a href="graphics.html">Graphics</a>
  26. <a href="network.html">Network</a>
  27. <a href="userland.html">Userland</a>
  28. <a href="development.html">Development</a>
  29. </nav>
  30. </header>
  31. <main id="main" class="page">
  32. <div class="page-head">
  33. <p class="kicker">Memory &amp; paging</p>
  34. <h1>A fixed ladder and a 4 KB frame allocator.</h1>
  35. <p class="lede">
  36. NebOS identity-maps physical RAM with 4 MB pages and hands out
  37. 4 KB frames from a bitmap. A small set of low-memory addresses is
  38. fixed by the boot path and the ABI; everything else — process stacks,
  39. window canvases, the ring-3 heap and the rootfs copy — is dynamic.
  40. </p>
  41. <div class="meta">
  42. <span class="pill">128 MB physical ceiling</span>
  43. <span class="pill">4 KB frames</span>
  44. <span class="pill">4 MB PSE pages</span>
  45. <span class="pill">identity map</span>
  46. </div>
  47. </div>
  48. <nav class="toc" aria-label="On this page">
  49. <h2>On this page</h2>
  50. <ul>
  51. <li><a href="#physical">Physical memory</a></li>
  52. <li><a href="#landmarks">Address landmarks</a></li>
  53. <li><a href="#paging">Paging &amp; caching</a></li>
  54. <li><a href="#frame">Frame allocator</a></li>
  55. <li><a href="#heap">kmalloc</a></li>
  56. <li><a href="#user">Ring-3 memory</a></li>
  57. <li><a href="#pack">Rootfs pack lifetime</a></li>
  58. <li><a href="#limits">Limits &amp; leaks</a></li>
  59. <li><a href="#sources">Source map</a></li>
  60. </ul>
  61. </nav>
  62. <section id="physical">
  63. <h2>Physical memory</h2>
  64. <p>
  65. The real-mode stage probes <code>INT 15h</code> E820 and writes the
  66. map to <code>0x1000</code>; the UEFI loader synthesises the same
  67. structure from <code>GetMemoryMap</code>. <span class="file">mm.c</span>
  68. starts with every page marked used, frees the E820 “usable RAM”
  69. ranges, then reserves the regions the kernel already owns.
  70. </p>
  71. <figure class="figure">
  72. <svg viewBox="0 0 920 180" role="img" aria-labelledby="phys-title phys-desc">
  73. <title id="phys-title">Physical memory composition</title>
  74. <desc id="phys-desc">Of the 128 MB addressable window, the low 1 MB and an 8 MB VGA shadow reserve are unavailable, leaving about 119 MB for the frame allocator.</desc>
  75. <text class="svg-text" x="20" y="28">0x00000000</text>
  76. <text class="svg-text" x="900" y="28" text-anchor="end">0x08000000 · 128 MB</text>
  77. <rect class="svg-box blue" x="40" y="44" width="6.5" height="34" rx="1"/>
  78. <rect class="svg-box accent" x="46.5" y="44" width="52.5" height="34" rx="1"/>
  79. <rect class="svg-box green" x="99" y="44" width="781" height="34" rx="2"/>
  80. <text class="svg-tiny" x="40" y="100">low 1 MB</text>
  81. <text class="svg-tiny" x="105" y="100">VGA shadow · 8 MB</text>
  82. <text class="svg-tiny" x="420" y="100" text-anchor="middle">frame allocator · ~119 MB</text>
  83. <line class="svg-line" x1="43" y1="78" x2="43" y2="88"/>
  84. <line class="svg-line" x1="73" y1="78" x2="73" y2="88"/>
  85. <line class="svg-line" x1="490" y1="78" x2="490" y2="88"/>
  86. <text class="svg-tiny" x="40" y="130">boot structures · kernel image · BIOS/UMA · DMA buffers</text>
  87. <text class="svg-tiny" x="40" y="150">The 8 MB reserve protects the shadow buffer; the allocator starts at 0x900000.</text>
  88. </svg>
  89. <figcaption>
  90. <strong>128 MB is a hard ceiling.</strong> The frame bitmap covers
  91. <code>[0, 0x08000000)</code>; addresses above it are not mapped by
  92. the kernel page tables. The VGA shadow reserve is conservative: the
  93. actual buffer is smaller, but the gap keeps a future resolution bump
  94. from colliding with allocated pages.
  95. </figcaption>
  96. </figure>
  97. </section>
  98. <section id="landmarks">
  99. <h2>Address landmarks</h2>
  100. <p>
  101. These addresses are part of the boot contract or the kernel ABI.
  102. Changing one requires updating the assembly, linker scripts, headers
  103. and this table together.
  104. </p>
  105. <div class="table-wrap">
  106. <table>
  107. <thead>
  108. <tr><th>Address</th><th>Purpose</th><th>Written by / notes</th></tr>
  109. </thead>
  110. <tbody>
  111. <tr><td><code>0x500</code></td><td>Boot info, 12 dwords</td><td>VESA stage, UEFI loader or kexec; <code>bootinfo.h</code></td></tr>
  112. <tr><td><code>0x600</code></td><td>Boot framebuffer info</td><td>VESA stage or UEFI loader; <code>vga.h</code></td></tr>
  113. <tr><td><code>0x7C00</code></td><td>Boot-sector load address / UEFI handoff stack top</td><td>BIOS and UEFI trampoline</td></tr>
  114. <tr><td><code>0x7E00</code></td><td>Kernel load address and VESA-stage entry</td><td><code>kernel.ld</code></td></tr>
  115. <tr><td><code>0x800</code></td><td>kexec and UEFI trampoline</td><td>Relocated at runtime; below the kernel</td></tr>
  116. <tr><td><code>0x900</code></td><td>kexec/UEFI flat GDT and GDTR</td><td>Built at runtime; below the kernel</td></tr>
  117. <tr><td><code>0x1000</code></td><td>E820 memory map</td><td>VESA stage, UEFI loader or kexec; parsed by <code>mm.c</code></td></tr>
  118. <tr><td><code>0x3000</code></td><td>UHCI frame list</td><td>1024 × 4 bytes, 4 KB aligned</td></tr>
  119. <tr><td><code>0x4000</code></td><td>ISA DMA buffer</td><td>One sector, used by the floppy driver</td></tr>
  120. <tr><td><code>0x6000</code></td><td>AP real/protected-mode trampoline</td><td>SIPI vector <code>0x06</code>; image ends below <code>0x6C00</code></td></tr>
  121. <tr><td><code>0x6C00</code></td><td>AP trampoline real-mode stack top</td><td>Grows down</td></tr>
  122. <tr><td><code>0x6F00</code></td><td>AP startup info block</td><td>CR3, stack top, C entry, CPU index</td></tr>
  123. <tr><td><code>0x60000</code></td><td>Legacy rootfs staging</td><td><code>0x60000..0x9FBFF</code>, compressed pack only</td></tr>
  124. <tr><td><code>_end</code></td><td>Kernel end</td><td><code>kernel.ld</code> asserts <code>_end ≤ 0x7F000</code></td></tr>
  125. <tr><td><code>0x7FF00</code></td><td>Program API struct</td><td><code>PROGRAM_API_ADDR</code>, <code>run_api.h</code></td></tr>
  126. <tr><td><code>0x80000</code></td><td>Legacy ring-0 program load area</td><td><code>run.c</code>, <code>serial_recv.c</code>, <code>user.ld</code>; max <code>0x6000</code></td></tr>
  127. <tr><td><code>0x900000</code></td><td>Boot pack after relocation; dynamic stacks and kexec staging</td><td>Freed after <code>ramfs_init</code> takes its copy</td></tr>
  128. <tr><td><code>0x100000</code></td><td>VGA shadow buffer base</td><td><code>VGA_SHADOW_BASE</code>; 8 MB reserved</td></tr>
  129. <tr><td><code>0x40000000</code></td><td>Ring-3 ELF link/load address</td><td><code>USER_BASE</code>, 2 MB window up to <code>0x40200000</code></td></tr>
  130. <tr><td><code>0x50000000</code></td><td>Ring-3 window canvas</td><td><code>USER_WIN_BASE</code>, 4 MB slot</td></tr>
  131. <tr><td><code>0x58000000</code></td><td>WM shadow mapping</td><td><code>WM_SHADOW_VA</code>; ring-3 WM only</td></tr>
  132. <tr><td><code>0x59000000</code></td><td>WM content slots</td><td><code>WM_CONTENT_VA_BASE</code> + index × 2 MB (<code>WM_CONTENT_SLOT</code>)</td></tr>
  133. <tr><td><code>0x60000000</code></td><td>Ring-3 anonymous heap</td><td><code>USER_HEAP_BASE</code>, up to 32 MB</td></tr>
  134. </tbody>
  135. </table>
  136. </div>
  137. </section>
  138. <section id="paging">
  139. <h2>Paging &amp; caching</h2>
  140. <p>
  141. <span class="file">paging.c</span> builds an identity map of RAM with
  142. 4 MB pages through <code>CR4.PSE</code>, adds the framebuffer's LFB
  143. range, and enables <code>CR0.PG</code> and <code>CR0.WP</code>. Before
  144. switching, it clears translation controls a BIOS protected-mode call
  145. can leave behind: <code>CR4.PAE/PGE/LA57/PCIDE/SMEP/SMAP</code> and
  146. <code>EFER.LME</code>.
  147. </p>
  148. <ul>
  149. <li><strong>Identity map.</strong> Physical and virtual addresses are equal. Page-directory entries are shared with every process.</li>
  150. <li><strong>Kernel stacks mapped everywhere.</strong> Each process gets a page directory cloned from the kernel identity map, so CR3 can switch while still running on the old kernel stack.</li>
  151. <li><strong>LFB is uncacheable by default.</strong> If <code>mtrr_set_wc()</code> successfully covers the framebuffer, the LFB pages are mapped cacheable so the shadow-buffer flush can write-combine.</li>
  152. <li><strong>No process isolation.</strong> There is one address space with per-process page directories only for mappings; a wild ring-0 or ring-3 pointer can reach any mapped physical page.</li>
  153. </ul>
  154. <p>
  155. There is no cross-CPU TLB shootdown. MMIO mappings for another CPU's
  156. running process rely on not-present TLB entries not being cached;
  157. normal process teardown is unaffected because no two processes share
  158. an address space.
  159. </p>
  160. <div class="note">
  161. <p><strong>NVIDIA ION special case.</strong> On the ION the framebuffer is
  162. stolen DRAM and the VBE LFB is a BAR3 window into it. The BIOS leaves a
  163. 512 MB uncacheable MTRR over the PCIe apertures, so <span class="file">nvidia.c</span>
  164. locates the framebuffer in the stolen VRAM and writes through that
  165. alias once a WC MTRR covers the stolen range. Everything else keeps
  166. the uncacheable LFB.</p>
  167. </div>
  168. </section>
  169. <section id="frame">
  170. <h2>Frame allocator</h2>
  171. <p>
  172. <span class="file">mm.c</span> keeps one bit per 4 KB page for the
  173. whole 128 MB window. The bitmap is not IRQ-safe; callers save and
  174. disable interrupts around allocations. The allocator returns:
  175. </p>
  176. <ul>
  177. <li><code>mm_alloc_page()</code> — one zero-address-ambiguous page (physical address 0 means failure).</li>
  178. <li><code>mm_alloc_pages(n)</code> — <code>n</code> contiguous pages.</li>
  179. <li><code>mm_free_page()</code> / <code>mm_free_range()</code> — release a page or a page-aligned boot reservation.</li>
  180. </ul>
  181. <p>
  182. At boot, <code>mm_init</code> reserves the low 1 MB, the 8 MB VGA
  183. shadow region, any LFB that happens to live below 128 MB, and the
  184. boot-time rootfs pack. The pack reservation is released by
  185. <code>ramfs_init</code> after it copies/inflates the pack into fresh
  186. pages.
  187. </p>
  188. <p>
  189. Fresh pages are <em>not</em> zeroed. Structures whose fields are read
  190. before being written must be cleared explicitly; <code>clear_proc()</code>
  191. in <span class="file">process.c</span> exists because a stale
  192. <code>cr3</code> faulted on real hardware while QEMU's cleared RAM hid
  193. the bug.
  194. </p>
  195. </section>
  196. <section id="heap">
  197. <h2>kmalloc</h2>
  198. <p>
  199. <span class="file">kmalloc.c</span> is a power-of-two heap allocator
  200. on top of the frame allocator. It is intentionally simple: allocation
  201. rounds to a size class, free lists are per class, and large objects
  202. become contiguous page runs. PCBs, command context and
  203. <span class="file">sys_fs.c</span> file loads use it.
  204. </p>
  205. <p>
  206. The kernel heap is not a general-purpose userspace allocator. Ring-3
  207. processes get one anonymous arena through <code>SYS_MEM_ALLOC</code>,
  208. <code>SYS_MEM_GROW</code> and <code>SYS_MEM_FREE</code>.
  209. </p>
  210. </section>
  211. <section id="user">
  212. <h2>Ring-3 memory</h2>
  213. <p>
  214. A ring-3 ELF is linked at <code>0x40000000</code>. The image and its
  215. 8-page user stack live in a 2 MB window ending at
  216. <code>0x40200000</code>. The kernel window server maps a window's
  217. RGB565 back buffer at <code>0x50000000</code>, and the anonymous heap
  218. starts at <code>0x60000000</code>.
  219. </p>
  220. <figure class="figure">
  221. <div class="bars" role="img" aria-label="Ring-3 arena sizes relative to the 32 MB heap cap: image and stack 2 MB, canvas slot 4 MB, heap up to 32 MB.">
  222. <div class="bar-row">
  223. <span class="bar-label">image + stack</span>
  224. <div class="bar-track"><span class="bar-fill blue" style="--w:6.25%"></span></div>
  225. <span class="bar-value">2 MB</span>
  226. </div>
  227. <div class="bar-row">
  228. <span class="bar-label">canvas slot</span>
  229. <div class="bar-track"><span class="bar-fill" style="--w:12.5%"></span></div>
  230. <span class="bar-value">4 MB</span>
  231. </div>
  232. <div class="bar-row">
  233. <span class="bar-label">anonymous heap</span>
  234. <div class="bar-track"><span class="bar-fill green" style="--w:100%"></span></div>
  235. <span class="bar-value">32 MB</span>
  236. </div>
  237. </div>
  238. <figcaption>
  239. <strong>Three separate rings.</strong> The image/stack window is
  240. fixed at link time; the canvas slot is one 4 MB mapping; the heap is
  241. a growable arena. Bar widths are relative to the 32 MB heap cap.
  242. </figcaption>
  243. </figure>
  244. <div class="table-wrap">
  245. <table>
  246. <thead>
  247. <tr><th>Range</th><th>Size</th><th>Contents</th></tr>
  248. </thead>
  249. <tbody>
  250. <tr><td><code>0x40000000</code>–<code>0x40200000</code></td><td>2 MB</td><td>ELF segments, user stack top at <code>0x40200000</code> (8 pages, grows down)</td></tr>
  251. <tr><td><code>0x50000000</code>–<code>0x50400000</code></td><td>4 MB</td><td>One window canvas mapping (RW, per window)</td></tr>
  252. <tr><td><code>0x60000000</code>–<code>0x62000000</code></td><td>up to 32 MB</td><td>Anonymous heap; initial allocation zeroed, growth pages are not</td></tr>
  253. </tbody>
  254. </table>
  255. </div>
  256. <p>
  257. Kernel stacks are 4 KB pages from the frame allocator, with an 8-stack
  258. static fallback pool in BSS. The idle/bootstrap context uses the
  259. in-image 8 KB <code>.stack</code> section. The user stack is separate:
  260. 8 pages just below <code>0x40200000</code>.
  261. </p>
  262. </section>
  263. <section id="pack">
  264. <h2>Rootfs pack lifetime</h2>
  265. <ol>
  266. <li>The boot path places the gzip pack in RAM: the legacy stage in the low gap and then <code>0x900000</code>, UEFI at the top of RAM below 128 MB, kexec in its staging buffer.</li>
  267. <li><code>mm_init</code> reserves <code>[pack_addr, pack_addr + pack_size)</code> so the frame allocator cannot reuse it.</li>
  268. <li><code>ramfs_init</code> takes a private page-allocated copy, validates the <code>NBROOT1</code> header and inflates the pack into it.</li>
  269. <li><code>mm_free_range</code> releases the boot region when it is above the VGA shadow reserve. The private copy lives until <code>ramfs_replace</code> or reboot.</li>
  270. </ol>
  271. <p>
  272. The ramfs holds at most 128 files and 8 MB of decompressed pack. A
  273. larger pack fails to load rather than partially mounting.
  274. </p>
  275. </section>
  276. <section id="limits">
  277. <h2>Limits &amp; leaks</h2>
  278. <ul>
  279. <li><strong>No per-process address spaces.</strong> Paging is identity-mapped; there is no memory protection between processes.</li>
  280. <li><strong>128 MB ceiling.</strong> Physical memory above <code>0x08000000</code> is unmapped. A wild pointer there is an instant <code>#PF</code>.</li>
  281. <li><strong>User page leak on reap.</strong> Killing a ring-3 process frees its kernel stack and page directory but not all user pages mapped into it; a few pages leak per command run.</li>
  282. <li><strong>Heap fragmentation.</strong> Power-of-two size classes make long-running mixed-size allocation wasteful compared with a real allocator.</li>
  283. <li><strong>No TLB shootdown.</strong> Cross-CPU mapping changes rely on not-present entries not being cached.</li>
  284. </ul>
  285. </section>
  286. <section id="sources">
  287. <h2>Source map</h2>
  288. <ul class="filelist">
  289. <li><code>mm.c</code>, <code>mm.h</code> — E820 + bitmap allocator</li>
  290. <li><code>kmalloc.c</code>, <code>kmalloc.h</code> — kernel heap</li>
  291. <li><code>paging.c</code>, <code>paging.h</code> — identity map</li>
  292. <li><code>mtrr.c</code> — write-combining MTRRs</li>
  293. <li><code>process.c</code>, <code>process.h</code> — stacks and user layout</li>
  294. <li><code>ramfs.c</code> — pack copy / inflate</li>
  295. <li><code>efi/main.c</code> — firmware map conversion</li>
  296. <li><code>kernel.ld</code> — low-memory image limits</li>
  297. </ul>
  298. </section>
  299. </main>
  300. <footer class="site-footer">
  301. <div class="inner">
  302. <p><strong>NebOS 0.2.0</strong> · Architecture documentation · <a href="index.html">overview</a></p>
  303. <p>Maintain alongside <code>AGENTS.md</code> and the source files listed on each page.</p>
  304. </div>
  305. </footer>
  306. </body>
  307. </html>