index.html 16 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317
  1. <!doctype html>
  2. <html lang="en">
  3. <head>
  4. <meta charset="utf-8">
  5. <meta name="viewport" content="width=device-width, initial-scale=1">
  6. <title>Overview | NebOS Architecture</title>
  7. <meta name="description" content="Architecture overview of NebOS: a freestanding 32-bit x86 hobby operating system with its own boot chain, kernel, window manager, filesystem and network stack.">
  8. <link rel="icon" href="favicon.svg" type="image/svg+xml">
  9. <link rel="stylesheet" href="style.css">
  10. </head>
  11. <body>
  12. <a class="skip" href="#main">Skip to content</a>
  13. <header class="site-header">
  14. <a class="brand" href="index.html">
  15. <span class="brand-mark" aria-hidden="true"></span>
  16. <span>NebOS <span class="brand-sub">Architecture</span></span>
  17. </a>
  18. <nav class="site-nav" aria-label="Architecture sections">
  19. <a href="index.html" aria-current="page">Overview</a>
  20. <a href="boot.html">Boot</a>
  21. <a href="memory.html">Memory</a>
  22. <a href="cpu.html">CPU</a>
  23. <a href="drivers.html">Drivers</a>
  24. <a href="filesystems.html">Filesystems</a>
  25. <a href="graphics.html">Graphics</a>
  26. <a href="network.html">Network</a>
  27. <a href="userland.html">Userland</a>
  28. <a href="development.html">Development</a>
  29. </nav>
  30. </header>
  31. <main id="main" class="page">
  32. <div class="page-head">
  33. <p class="kicker">NebOS 0.2.0 · Architecture notes</p>
  34. <h1>One small OS, from boot sector to browser.</h1>
  35. <p class="lede">
  36. NebOS is a from-scratch, freestanding 32-bit x86 operating system
  37. written in C and assembly. It targets a first-generation Pentium,
  38. boots from a floppy, a USB disk or x86-64 UEFI firmware, and carries
  39. its own scheduler, filesystems, window manager, TCP/IP stack and
  40. ring-3 command pack.
  41. </p>
  42. <div class="meta">
  43. <span class="pill accent">i586 · 32-bit</span>
  44. <span class="pill">freestanding kernel · ring-3 libc</span>
  45. <span class="pill">BIOS + UEFI</span>
  46. <span class="pill">serial + net hot reload</span>
  47. </div>
  48. </div>
  49. <nav class="toc" aria-label="On this page">
  50. <h2>On this page</h2>
  51. <ul>
  52. <li><a href="#stack">System stack</a></li>
  53. <li><a href="#numbers">At a glance</a></li>
  54. <li><a href="#constraints">Design constraints</a></li>
  55. <li><a href="#reading">Reading path</a></li>
  56. <li><a href="#subsystems">Subsystems</a></li>
  57. <li><a href="#tour">Repository tour</a></li>
  58. </ul>
  59. </nav>
  60. <section id="stack">
  61. <h2>The system stack</h2>
  62. <p>
  63. Almost everything is built in-tree: the boot sectors, the physical
  64. memory manager, the scheduler, the FAT driver, the USB host
  65. controllers, the TCP client, the TLS implementation and the
  66. applications. The only runtime contract with the outside world is
  67. the PC firmware and the x86 instruction set.
  68. </p>
  69. <figure class="figure">
  70. <svg viewBox="0 0 920 474" role="img" aria-labelledby="stack-title stack-desc">
  71. <title id="stack-title">NebOS software stack</title>
  72. <desc id="stack-desc">Five layers: ring-3 commands and GUI apps, the int 0x80 syscall boundary, kernel services, drivers and platform support, and firmware and hardware.</desc>
  73. <rect class="svg-box accent" x="100" y="8" width="720" height="72" rx="9"/>
  74. <text class="svg-text" x="124" y="35">Ring 3 · commands &amp; GUI apps</text>
  75. <text class="svg-sub" x="124" y="55">ls · cat · edit · view · files · process · bootscan · browser · chat · gallery · solitaire</text>
  76. <text class="svg-tiny" x="796" y="35" text-anchor="end">0x40000000</text>
  77. <path class="svg-line" d="M460 80v22"/>
  78. <path class="svg-line" d="M454 96l6 7 6-7"/>
  79. <rect class="svg-box blue" x="100" y="104" width="720" height="72" rx="9"/>
  80. <text class="svg-text" x="124" y="131">Syscall boundary · int 0x80</text>
  81. <text class="svg-sub" x="124" y="151">SYS_* services · one string argument · canvas mapped at 0x50000000</text>
  82. <path class="svg-line" d="M460 176v22"/>
  83. <path class="svg-line" d="M454 192l6 7 6-7"/>
  84. <rect class="svg-box" x="100" y="200" width="720" height="92" rx="9"/>
  85. <text class="svg-text" x="124" y="227">Kernel services</text>
  86. <text class="svg-sub" x="124" y="247">CFS-lite scheduler · VFS · FAT + ramfs · TCP/IP + TLS</text>
  87. <text class="svg-sub" x="124" y="267">window server · file / info / memory services</text>
  88. <path class="svg-line" d="M460 292v22"/>
  89. <path class="svg-line" d="M454 308l6 7 6-7"/>
  90. <rect class="svg-box" x="100" y="316" width="720" height="92" rx="9"/>
  91. <text class="svg-text" x="124" y="343">Drivers &amp; platform</text>
  92. <text class="svg-sub" x="124" y="363">PCI · PATA/SATA · FDC · USB UHCI/OHCI/EHCI/xHCI · NICs</text>
  93. <text class="svg-sub" x="124" y="383">PS/2 + USB HID · serial · ACPI/MADT · thermal · power</text>
  94. <path class="svg-line" d="M460 408v22"/>
  95. <path class="svg-line" d="M454 424l6 7 6-7"/>
  96. <rect class="svg-box green" x="100" y="432" width="720" height="36" rx="9"/>
  97. <text class="svg-text" x="124" y="455">Firmware &amp; hardware · BIOS or UEFI · i586 CPU · VGA/VBE · RAM</text>
  98. <text class="svg-tiny" x="40" y="52" text-anchor="middle">user</text>
  99. <text class="svg-tiny" x="40" y="240" text-anchor="middle">kernel</text>
  100. <text class="svg-tiny" x="40" y="360" text-anchor="middle">drivers</text>
  101. <text class="svg-tiny" x="40" y="454" text-anchor="middle">firmware</text>
  102. </svg>
  103. <figcaption>
  104. <strong>The layering is real, not just conceptual.</strong>
  105. Ring-3 commands enter the kernel only through <code>int 0x80</code>;
  106. the kernel owns the devices and exposes narrow services.
  107. </figcaption>
  108. </figure>
  109. </section>
  110. <section id="numbers">
  111. <h2>At a glance</h2>
  112. <div class="grid">
  113. <div class="card">
  114. <h3>8 CPUs</h3>
  115. <p>BSP plus up to seven APs, discovered through ACPI/MADT and started with INIT-SIPI-SIPI. Without ACPI the machine stays on the uniprocessor path.</p>
  116. </div>
  117. <div class="card">
  118. <h3>64 process slots</h3>
  119. <p>Kernel threads and ring-3 processes share a sparse, stable PCB table. Kernel threads are pinned to the BSP; user work can run on any AP.</p>
  120. </div>
  121. <div class="card">
  122. <h3>8 windows</h3>
  123. <p>Terminal and canvas windows are composited by the window manager, with per-window RGB565 canvases mapped into the owning process.</p>
  124. </div>
  125. <div class="card">
  126. <h3>8 MB rootfs</h3>
  127. <p>The command pack travels as a compressed <code>NBROOT1</code> image and is inflated into a read-only ramfs at boot. Runtime replacement needs no reboot.</p>
  128. </div>
  129. </div>
  130. </section>
  131. <section id="constraints">
  132. <h2>Design constraints</h2>
  133. <p>
  134. The unusual parts of the architecture follow from a few hard rules.
  135. They explain why the code uses fixed low-memory blocks, why the kernel
  136. is linked below <code>0x90000</code>, and why so much care goes into
  137. keeping the kernel image small.
  138. </p>
  139. <ul>
  140. <li><strong>Freestanding C.</strong> The kernel has no libc, <code>malloc</code> or <code>printf</code>; ring-3 commands can link the in-tree <code>libc.a</code> (<code>malloc</code>, strings, stdlib). Formatting and allocation are in-tree either way.</li>
  141. <li><strong>i586 baseline.</strong> Builds use <code>-march=i586 -mno-sse -mno-sse2 -mno-mmx</code>; SSE2 drawing paths are probed at runtime.</li>
  142. <li><strong>Identity-mapped paging.</strong> Physical and virtual addresses are equal. There is no per-process address space or isolation.</li>
  143. <li><strong>1 GB lowmem, 4 GB highmem.</strong> The identity map ends where ring-3 space starts (<code>0x40000000</code>); RAM up to 4 GB backs ring-3 pages through per-CPU <code>kmap</code> slots.</li>
  144. <li><strong>Hardcoded low memory.</strong> Boot info, the E820 map, the framebuffer block and the AP trampoline have fixed addresses that every boot path must honour.</li>
  145. <li><strong>No firmware runtime.</strong> <code>reboot</code> uses the ACPI reset register (with PCH/KBC fallbacks) and <code>poweroff</code> writes ACPI S5 from the FADT/DSDT; UEFI runtime services are not called after <code>ExitBootServices</code>.</li>
  146. </ul>
  147. <div class="warn">
  148. <p><strong>Fresh memory is not zeroed.</strong> <code>mm_alloc_page</code> and <code>kmalloc</code> return whatever RAM contained. Every structure read before it is written must be cleared on allocation; QEMU hides this because its RAM starts clear.</p>
  149. </div>
  150. </section>
  151. <section id="reading">
  152. <h2>Reading path</h2>
  153. <p>If you are new to the tree, follow the machine through a boot:</p>
  154. <ol>
  155. <li><a href="boot.html">Boot</a> — the real-mode stage, the UEFI loader and the <code>0x500</code>/<code>0x600</code> handoff contract.</li>
  156. <li><a href="memory.html">Memory</a> — the fixed low-memory map and the 4 KB frame allocator it feeds.</li>
  157. <li><a href="cpu.html">CPU &amp; scheduling</a> — interrupt setup, the CFS-lite scheduler and the big kernel lock.</li>
  158. <li><a href="drivers.html">Drivers</a> — how storage, USB, input and sensors are discovered.</li>
  159. <li><a href="graphics.html">Graphics</a> — the shadow buffer, compositor and ring-3 canvas model.</li>
  160. </ol>
  161. <p>
  162. The remaining pages cover the filesystem and command-pack pipeline,
  163. the network stack, the ring-3 ABI and the development workflow.
  164. </p>
  165. </section>
  166. <section id="subsystems">
  167. <h2>Subsystem map</h2>
  168. <div class="grid">
  169. <a class="card" href="boot.html">
  170. <h3>Boot &amp; firmware</h3>
  171. <p>BIOS boot sector, VESA stage, MBR, UEFI loader, kexec trampoline and the boot-info block.</p>
  172. <span class="card-tag">boot.html</span>
  173. </a>
  174. <a class="card" href="memory.html">
  175. <h3>Memory &amp; paging</h3>
  176. <p>E820 map, 4 KB frame bitmap, kmalloc, process stacks, the ring-3 arena and libc malloc, and the hardcoded address ladder.</p>
  177. <span class="card-tag">memory.html</span>
  178. </a>
  179. <a class="card" href="cpu.html">
  180. <h3>CPU &amp; scheduling</h3>
  181. <p>GDT/TSS, interrupts, AP bring-up, CFS-lite weights, wait queues, the BKL, idle and power states.</p>
  182. <span class="card-tag">cpu.html</span>
  183. </a>
  184. <a class="card" href="drivers.html">
  185. <h3>Drivers &amp; buses</h3>
  186. <p>PCI enumeration, FDC/PATA/SATA/eMMC, the USB core and four host controllers, HID, the I2C touchpad, serial, board sensors and the ThinkPad/Chrome EC power sources.</p>
  187. <span class="card-tag">drivers.html</span>
  188. </a>
  189. <a class="card" href="filesystems.html">
  190. <h3>Storage &amp; filesystems</h3>
  191. <p>VFS vtable, FAT12/16/32, the ramfs pack pipeline, persistent <code>/data</code>, <code>/share</code>, the <code>disks</code> manager with its mount policies and the WebDAV view of the VFS.</p>
  192. <span class="card-tag">filesystems.html</span>
  193. </a>
  194. <a class="card" href="graphics.html">
  195. <h3>Graphics &amp; UI</h3>
  196. <p>VBE framebuffer, shadow buffer, damage rectangles, compositor, canvas ABI, ngui, the Adwaita UI icons and mouse cursors, the fixed-point TrueType text engine and image codecs.</p>
  197. <span class="card-tag">graphics.html</span>
  198. </a>
  199. <a class="card" href="network.html">
  200. <h3>Networking</h3>
  201. <p>NIC registry, ARP/ICMP/UDP, DHCP, DNS, NTP, TCP client and listener, TLS 1.3/1.2, USB tethering, WiFi, the top-bar control panel, netconsole and netkexec.</p>
  202. <span class="card-tag">network.html</span>
  203. </a>
  204. <a class="card" href="drivers.html#wifi">
  205. <h3>WiFi</h3>
  206. <p>Intel 7265D/3165 (Chromebook, MVM firmware) and 6200/6300 (ThinkPad X201, DVM firmware) on one PCIe transport, plus Realtek RTL8821AU USB sticks; firmware from the command pack, open and WPA2-PSK/CCMP with the handshake in ring 3.</p>
  207. <span class="card-tag">drivers.html#wifi</span>
  208. </a>
  209. <a class="card" href="network.html#chat">
  210. <h3>Chat client</h3>
  211. <p>Native Cloudron Chat app: OIDC username/password/TOTP login without a browser, a uhttp cookie jar, an arena JSON parser and REST polling.</p>
  212. <span class="card-tag">network.html#chat</span>
  213. </a>
  214. <a class="card" href="userland.html">
  215. <h3>Userland</h3>
  216. <p>Ring-3 address space, ELF loading, syscall groups, the optional libc and malloc, PATH dispatch, the command pack and GUI apps.</p>
  217. <span class="card-tag">userland.html</span>
  218. </a>
  219. <a class="card" href="userland.html#libc">
  220. <h3>Ring-3 libc</h3>
  221. <p>Optional static <code>libc.a</code>: boundary-tag malloc/free/realloc, strings, stdlib and heap_reset over the 64 MB SYS_MEM arena.</p>
  222. <span class="card-tag">userland.html#libc</span>
  223. </a>
  224. <a class="card" href="development.html">
  225. <h3>Development</h3>
  226. <p>Toolchain, build images, QEMU targets, serial/network reload, tests and low-level debugging tools.</p>
  227. <span class="card-tag">development.html</span>
  228. </a>
  229. </div>
  230. </section>
  231. <section id="tour">
  232. <h2>Repository tour</h2>
  233. <p>
  234. The tree is split by privilege and lifetime: <span class="file">boot/</span>
  235. holds the boot stages (boot sector, MBR, VESA stage, UEFI loader),
  236. <span class="file">kernel/</span> the ring-0 kernel, and
  237. <span class="file">user/</span> the ring-3 world, with one directory per
  238. program under <span class="file">user/apps/</span> and one per library
  239. under <span class="file">user/lib/</span>. <span class="file">include/</span>
  240. holds the ABI headers every side shares (syscall numbers, the WM ABI,
  241. boot info, the rootfs header), <span class="file">common/</span> the
  242. few freestanding sources linked into both kernel and userland, and
  243. <span class="file">tools/</span> the host-side tools. The top-level
  244. Makefile builds everything; objects go to <span class="file">build/</span>
  245. and the images stay at the top.
  246. </p>
  247. <div class="table-wrap">
  248. <table>
  249. <thead>
  250. <tr><th>Area</th><th>Important files</th></tr>
  251. </thead>
  252. <tbody>
  253. <tr>
  254. <td>Boot chain</td>
  255. <td><span class="file">boot/boot.S</span> <span class="file">boot/mbr.S</span> <span class="file">boot/vesa_stage.S</span> <span class="file">boot/efi/</span> <span class="file">kernel/entry.S</span></td>
  256. </tr>
  257. <tr>
  258. <td>Kernel core</td>
  259. <td><span class="file">kernel/kernel.c</span> <span class="file">kernel/sched.c</span> <span class="file">kernel/process.c</span> <span class="file">kernel/mm.c</span> <span class="file">kernel/paging.c</span> <span class="file">kernel/syscall.c</span> <span class="file">include/syscall.h</span></td>
  260. </tr>
  261. <tr>
  262. <td>Devices</td>
  263. <td><span class="file">kernel/</span>: <span class="file">usb.c</span> <span class="file">uhci.c</span> <span class="file">ohci.c</span> <span class="file">ehci.c</span> <span class="file">xhci.c</span> <span class="file">usbmsc.c</span> <span class="file">ata.c</span> <span class="file">ahci.c</span></td>
  264. </tr>
  265. <tr>
  266. <td>Services</td>
  267. <td><span class="file">kernel/</span>: <span class="file">sys_fs.c</span> <span class="file">sys_net.c</span> <span class="file">gfx.c</span> <span class="file">sys_wm.c</span> <span class="file">sys_info.c</span> <span class="file">install.c</span></td>
  268. </tr>
  269. <tr>
  270. <td>Userland</td>
  271. <td><span class="file">user/apps/&lt;name&gt;/ucmd_&lt;name&gt;.c</span> <span class="file">user/ucmd_lib.h</span> <span class="file">user/lib/libc/</span> <span class="file">user/lib/ngui/</span> <span class="file">user/lib/ufont/</span> <span class="file">user/lib/img/</span> <span class="file">user/lib/tls/</span> <span class="file">user/lib/uhttp/</span> <span class="file">user/lib/ujson/</span> <span class="file">user/apps/browser/</span> <span class="file">user/apps/chat/</span> <span class="file">third_party/quickjs/</span> <span class="file">fonts/</span></td>
  272. </tr>
  273. <tr>
  274. <td>Build &amp; pack</td>
  275. <td><span class="file">Makefile</span> <span class="file">tools/mkpack.c</span> <span class="file">tools/mkrootfs.c</span> <span class="file">kernel/kernel.ld</span> <span class="file">user/ucmd.ld</span></td>
  276. </tr>
  277. <tr>
  278. <td>Documentation</td>
  279. <td><span class="file">AGENTS.md</span> and this <span class="file">docs/</span> folder</td>
  280. </tr>
  281. </tbody>
  282. </table>
  283. </div>
  284. <div class="tip">
  285. <p><strong>Keep both maps current.</strong> Architecture changes belong in <code>AGENTS.md</code> and in the matching page under <code>docs/</code>. The maintenance rule is repeated in <code>AGENTS.md</code>.</p>
  286. </div>
  287. </section>
  288. </main>
  289. <footer class="site-footer">
  290. <div class="inner">
  291. <p><strong>NebOS 0.2.0</strong> · Architecture documentation · <a href="index.html">overview</a></p>
  292. <p>Maintain alongside <code>AGENTS.md</code> and the source files listed on each page.</p>
  293. </div>
  294. </footer>
  295. </body>
  296. </html>