development.html 28 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390
  1. <!doctype html>
  2. <html lang="en">
  3. <head>
  4. <meta charset="utf-8">
  5. <meta name="viewport" content="width=device-width, initial-scale=1">
  6. <title>Development | NebOS Architecture</title>
  7. <meta name="description" content="NebOS development: toolchain and build flags, image targets, QEMU runs, serial and network hot reload, runtime command-pack updates, tests and debugging tools.">
  8. <link rel="icon" href="favicon.svg" type="image/svg+xml">
  9. <link rel="stylesheet" href="style.css">
  10. </head>
  11. <body>
  12. <a class="skip" href="#main">Skip to content</a>
  13. <header class="site-header">
  14. <a class="brand" href="index.html">
  15. <span class="brand-mark" aria-hidden="true"></span>
  16. <span>NebOS <span class="brand-sub">Architecture</span></span>
  17. </a>
  18. <nav class="site-nav" aria-label="Architecture sections">
  19. <a href="index.html">Overview</a>
  20. <a href="boot.html">Boot</a>
  21. <a href="memory.html">Memory</a>
  22. <a href="cpu.html">CPU</a>
  23. <a href="drivers.html">Drivers</a>
  24. <a href="filesystems.html">Filesystems</a>
  25. <a href="graphics.html">Graphics</a>
  26. <a href="network.html">Network</a>
  27. <a href="userland.html">Userland</a>
  28. <a href="development.html" aria-current="page">Development</a>
  29. </nav>
  30. </header>
  31. <main id="main" class="page">
  32. <div class="page-head">
  33. <p class="kicker">Development</p>
  34. <h1>Edit, reload, watch the boot line.</h1>
  35. <p class="lede">
  36. A normal iteration does not reflash anything. The board receives a
  37. new kernel and rootfs over serial or UDP, parks its APs, jumps into
  38. the new image and prints the same version line on COM1. A runtime
  39. command-pack update can skip even the kernel reload.
  40. </p>
  41. <div class="meta">
  42. <span class="pill">gcc · as · ld · objcopy</span>
  43. <span class="pill">QEMU + real hardware</span>
  44. <span class="pill">KXE2 · KXN2 · KXNP</span>
  45. <span class="pill">headless tests</span>
  46. </div>
  47. </div>
  48. <nav class="toc" aria-label="On this page">
  49. <h2>On this page</h2>
  50. <ul>
  51. <li><a href="#toolchain">Toolchain</a></li>
  52. <li><a href="#pipeline">Build pipeline</a></li>
  53. <li><a href="#targets">Make targets</a></li>
  54. <li><a href="#serial">Serial reload</a></li>
  55. <li><a href="#network">Network reload</a></li>
  56. <li><a href="#pack">Runtime pack</a></li>
  57. <li><a href="#tests">Tests</a></li>
  58. <li><a href="#debug">Debugging</a></li>
  59. <li><a href="#conventions">Conventions</a></li>
  60. <li><a href="#docs">Keeping docs current</a></li>
  61. <li><a href="#sources">Source map</a></li>
  62. </ul>
  63. </nav>
  64. <section id="toolchain">
  65. <h2>Toolchain</h2>
  66. <p>
  67. NebOS builds with a normal GNU toolchain for 32-bit x86, plus
  68. <code>mtools</code> for FAT images and QEMU for testing. The UEFI
  69. loader is the only 64-bit component and needs the mingw cross
  70. compiler.
  71. </p>
  72. <ul>
  73. <li><strong>Compiler.</strong> <code>gcc</code> with 32-bit multilib, <code>as</code>, <code>ld</code>, <code>objcopy</code>.</li>
  74. <li><strong>Kernel flags.</strong> <code>-m32 -march=i586 -ffreestanding -fno-pie -nostdlib -fno-stack-protector -mno-sse -mno-sse2 -mno-mmx -O2</code>.</li>
  75. <li><strong>Warnings.</strong> The build uses <code>-Wall -Wextra</code>; a new warning is a bug, not noise.</li>
  76. <li><strong>User commands.</strong> Linked at <code>0x40000000</code> with <code>ucmd.ld</code>, stripped, and packed as <code>bin/&lt;name&gt;</code>.</li>
  77. <li><strong>UEFI.</strong> <code>x86_64-w64-mingw32-gcc</code> builds <code>BOOTX64.EFI</code>; <code>make all</code> does not require it.</li>
  78. <li><strong>Images.</strong> <code>mtools</code> builds <code>root.img</code> and the ESP.</li>
  79. <li><strong>WiFi firmware.</strong> The pack includes <code>firmware/iwlwifi-7265D-29.ucode</code> (Chromebook) and <code>firmware/iwlwifi-6000-4.ucode</code> (X201), decompressed with <code>zstd</code> from the host's linux-firmware (<code>WIFI_FW_7265D=</code> / <code>WIFI_FW_6000=/path/to/file[.zst]</code> override the search). A missing file only prints a note and leaves that card without firmware; the floppy images always use the firmware-less <code>cmds-lite.pack</code>.</li>
  80. </ul>
  81. </section>
  82. <section id="pipeline">
  83. <h2>Build pipeline</h2>
  84. <figure class="figure">
  85. <svg viewBox="0 0 920 360" role="img" aria-labelledby="build-title build-desc">
  86. <title id="build-title">NebOS build pipeline</title>
  87. <desc id="build-desc">Kernel sources become kernel.bin and boot.bin; ring-3 sources become cmds.pack and a gzip rootfs; the UEFI loader embeds kernel, pack and trampoline. The outputs feed floppy, USB and dual-boot images.</desc>
  88. <rect class="svg-box blue" x="20" y="20" width="250" height="64" rx="9"/>
  89. <text class="svg-text" x="40" y="46">kernel sources</text>
  90. <text class="svg-sub" x="40" y="66">*.c · *.S · linker scripts</text>
  91. <rect class="svg-box blue" x="20" y="116" width="250" height="64" rx="9"/>
  92. <text class="svg-text" x="40" y="142">ring-3 sources</text>
  93. <text class="svg-sub" x="40" y="162">ucmd_*.c · ngui · browser</text>
  94. <rect class="svg-box blue" x="20" y="212" width="250" height="64" rx="9"/>
  95. <text class="svg-text" x="40" y="238">UEFI loader</text>
  96. <text class="svg-sub" x="40" y="258">efi/*.c · tramp.S</text>
  97. <path class="svg-line blue" d="M270 52h62"/>
  98. <path class="svg-line blue" d="M326 46l7 6-7 6"/>
  99. <path class="svg-line blue" d="M270 148h62"/>
  100. <path class="svg-line blue" d="M326 142l7 6-7 6"/>
  101. <path class="svg-line blue" d="M270 244h62"/>
  102. <path class="svg-line blue" d="M326 238l7 6-7 6"/>
  103. <rect class="svg-box accent" x="338" y="20" width="230" height="64" rx="9"/>
  104. <text class="svg-text" x="358" y="46">gcc · as · ld · objcopy</text>
  105. <text class="svg-sub" x="358" y="66">kernel.elf → kernel.bin</text>
  106. <rect class="svg-box accent" x="338" y="116" width="230" height="64" rx="9"/>
  107. <text class="svg-text" x="358" y="142">mkpack · gzip · mkrootfs</text>
  108. <text class="svg-sub" x="358" y="162">cmds.pack → NBROOT1</text>
  109. <rect class="svg-box accent" x="338" y="212" width="230" height="64" rx="9"/>
  110. <text class="svg-text" x="358" y="238">mingw gcc</text>
  111. <text class="svg-sub" x="358" y="258">BOOTX64.EFI</text>
  112. <path class="svg-line accent" d="M568 52h62"/>
  113. <path class="svg-line accent" d="M624 46l7 6-7 6"/>
  114. <path class="svg-line accent" d="M568 148h62"/>
  115. <path class="svg-line accent" d="M624 142l7 6-7 6"/>
  116. <path class="svg-line accent" d="M568 244h62"/>
  117. <path class="svg-line accent" d="M624 238l7 6-7 6"/>
  118. <rect class="svg-box green" x="636" y="20" width="264" height="64" rx="9"/>
  119. <text class="svg-text" x="656" y="46">floppy.img · final.bin</text>
  120. <text class="svg-sub" x="656" y="66">boot sector + kernel + rootfs</text>
  121. <rect class="svg-box green" x="636" y="116" width="264" height="64" rx="9"/>
  122. <text class="svg-text" x="656" y="142">usb.img</text>
  123. <text class="svg-sub" x="656" y="162">MBR + boot + 8 MB rootfs</text>
  124. <rect class="svg-box green" x="636" y="212" width="264" height="64" rx="9"/>
  125. <text class="svg-text" x="656" y="238">usb-efi.img · 17 MB</text>
  126. <text class="svg-sub" x="656" y="258">usb.img + 8 MB FAT16 ESP</text>
  127. <line class="svg-line" x1="20" y1="310" x2="900" y2="310"/>
  128. <text class="svg-sub" x="20" y="335">Hot reload skips the image step: make dev / net-dev sends kernel.bin + cmds_pack_gz.bin; cmds-dev / net-cmds sends cmds.pack.</text>
  129. </svg>
  130. <figcaption>
  131. <strong>Two independent products.</strong> The kernel image and
  132. the rootfs container have separate lifetimes: a kernel reload can
  133. keep the running rootfs, and a pack update can keep the kernel.
  134. </figcaption>
  135. </figure>
  136. </section>
  137. <section id="targets">
  138. <h2>Make targets</h2>
  139. <div class="table-wrap">
  140. <table>
  141. <thead>
  142. <tr><th>Target</th><th>What it does</th></tr>
  143. </thead>
  144. <tbody>
  145. <tr><td><code>make all</code></td><td>Build <code>floppy.img</code>, <code>usb.img</code>, <code>final.bin</code> and <code>root.img</code>.</td></tr>
  146. <tr><td><code>make run</code> / <code>run-smp</code></td><td>Boot the floppy in QEMU, optionally with four CPUs (Atom 330 topology).</td></tr>
  147. <tr><td><code>make run-usb</code></td><td>Boot <code>usb.img</code> as an IDE disk.</td></tr>
  148. <tr><td><code>make run-uhci</code> / <code>run-ehci</code> / <code>run-xhci</code></td><td>Exercise the USB backends with a mouse/keyboard; the EHCI target also attaches a USB mass-storage disk, and the installer test covers EHCI and xHCI storage.</td></tr>
  149. <tr><td><code>make run-ehci-asix</code> / <code>run-xhci-asix</code></td><td>Pass through a real ASIX AX88772 dongle.</td></tr>
  150. <tr><td><code>make run-net</code></td><td>Add a PCI NE2000 on QEMU's slirp network with host port 8080 forwarded.</td></tr>
  151. <tr><td><code>make run-rndis</code></td><td>Boot with <code>qemu-xhci</code> and QEMU's full-speed <code>usb-net</code> RNDIS device on slirp; the tethering link gets DHCP from QEMU.</td></tr>
  152. <tr><td><code>make run-ahci</code></td><td>Add an ICH9 AHCI controller and a 512 MB SATA disk.</td></tr>
  153. <tr><td><code>make uefi</code> / <code>run-uefi</code> / <code>test-uefi</code></td><td>Build the dual-boot image, boot it under OVMF, or run the headless serial check.</td></tr>
  154. <tr><td><code>make rootfs</code></td><td>Rebuild <code>cmds.img</code> only.</td></tr>
  155. <tr><td><code>make nowm</code></td><td>Diagnostic kernel with the WM disabled; the boot console stays visible.</td></tr>
  156. <tr><td><code>make dev</code> / <code>kernel-dev</code></td><td>Serial reload of kernel + rootfs, or kernel only (keep the running rootfs).</td></tr>
  157. <tr><td><code>make net-dev</code> / <code>kernel-net-dev</code></td><td>RAM-only network reload of kernel + rootfs, or kernel only.</td></tr>
  158. <tr><td><code>make net-install</code></td><td>Persist kernel + full rootfs to P1/P2 over UDP, keeping <code>/data</code>; <code>NETINSTALL_TARGET=auto</code> selects the internal NebOS disk.</td></tr>
  159. <tr><td><code>make cmds-dev</code> / <code>net-cmds</code></td><td>Replace <code>/ramfs</code> at runtime over serial or UDP.</td></tr>
  160. <tr><td><code>make monitor</code> / <code>netmon</code></td><td>Open the serial console, or listen to the UDP console mirror.</td></tr>
  161. <tr><td><code>make flash</code></td><td>Write the dual-boot image to a USB stick (needs mingw and prompts for the device).</td></tr>
  162. <tr><td><code>make test-usb</code></td><td>Headless QEMU mouse/keyboard regression tests using QMP and screen dumps.</td></tr>
  163. <tr><td><code>make test-install</code></td><td>Boots the dual image with a USB mass-storage source and a blank target, types <code>install ide1</code>, verifies the clone and boots the target in BIOS and UEFI.</td></tr>
  164. <tr><td><code>make test-images</code> / <code>test-images-asan</code></td><td>Host image-decoder and blit tests, with an AddressSanitizer variant.</td></tr>
  165. <tr><td><code>make test-crypto</code></td><td>Host vectors for SHA-256, ChaCha20-Poly1305, X25519 and TLS helpers.</td></tr>
  166. <tr><td><code>make test-wpa</code></td><td>Host vectors for the WPA2 supplicant: SHA-1, RFC 6070 PBKDF2, the 802.11 PSK and PRF examples, FIPS-197 AES, RFC 3394 unwrap, plus a simulated 4-way handshake, group rekey, replay and wrong-passphrase case.</td></tr>
  167. <tr><td><code>make test-wifi-store</code></td><td>Host test for the saved-network file: newest-first ordering, passphrase update, LRU eviction at the 16-entry cap, per-SSID forget, the old one-record format and malformed or duplicate lists.</td></tr>
  168. <tr><td><code>make test-libc</code></td><td>Host tests for the ring-3 malloc, stdio/float formatting, math, 64-bit division helpers and setjmp/longjmp.</td></tr>
  169. <tr><td><code>make test-js</code></td><td>Host tests (ASan/UBSan) for the vendored QuickJS engine through the <code>js_mu</code> stack API: modern syntax (classes, arrows, templates, destructuring, spread, async/await, BigInt, named regex groups), Promise jobs, C functions and <code>new</code>, thrown TypeErrors, userdata hooks and expandos, registry refs and the wall-clock interrupt.</td></tr>
  170. <tr><td><code>make test-font</code> / <code>test-font-asan</code></td><td>Host specimen test for the fixed-point TrueType parser/rasterizer and glyph cache, with an AddressSanitizer/UBSan variant.</td></tr>
  171. <tr><td><code>make test-svg</code></td><td>Host tests (ASan/UBSan) for the inline-SVG rasterizer: exact rect coverage, antialiased edges, relative commands, nonzero vs even-odd, circle area, curves and arcs, transforms, number grammar and strokes; <code>./svg_test out.ppm</code> writes a specimen.</td></tr>
  172. <tr><td><code>make test-touchpad</code></td><td>Host tests for the device-independent contact gesture engine: taps, right-click zones, physical-button latch, two-finger scroll, direction and pointer re-anchoring.</td></tr>
  173. <tr><td><code>make test-chat</code></td><td>Host tests (ASan/UBSan) for the chat client's JSON parser, cookie jar and the OIDC login page/reply/config parsers.</td></tr>
  174. <tr><td><code>make test-chat-e2e</code></td><td>Boots QEMU with an NE2000 on slirp, starts <code>tools/mock_chat.py</code> and drives <code>chat</code> through QMP: native login, send, live poll, read marker, older paging, browse/join, new DM, saved-session resume and sign-out. <code>CHAT_E2E=--totp|--consent|--passkey|--wrong</code> runs a login variant.</td></tr>
  175. <tr><td><code>make compat-serve</code></td><td>Serve the browser compatibility ladder from <code>testdata/browser/compat/</code> on port 8000 for the board or QEMU guest.</td></tr>
  176. <tr><td><code>make fonts</code></td><td>Regenerate the checked-in Arimo/Liberation Mono subsets with <code>pyftsubset</code>.</td></tr>
  177. </tbody>
  178. </table>
  179. </div>
  180. <p>
  181. QEMU guests default to 512 MB, matching <code>MM_MAX_PHYS</code> in
  182. <span class="file">phys.h</span>. Override the size with
  183. <code>QEMU_MEM</code>, for example <code>make run QEMU_MEM=1G</code>;
  184. all run and UEFI targets pass it through.
  185. </p>
  186. </section>
  187. <section id="serial">
  188. <h2>Serial reload</h2>
  189. <p>
  190. COM1 runs at 115200 8N1. The WM idle loop watches the receive ring for
  191. the <code>KXE2</code> magic and auto-spawns the reload, so a plain
  192. <code>make dev</code> is enough. The manual <code>kexec</code> command
  193. is a fallback that waits for the same image.
  194. </p>
  195. <pre><code>[magic "KXE2"][entry u32][kernel_size u32][pack_size u32]
  196. [kernel][rootfs gzip][checksum u8]</code></pre>
  197. <ul>
  198. <li><strong>Full reload.</strong> <code>make dev</code> stages kernel and rootfs in one page-allocated buffer, copies the kernel to <code>0x7E00</code> and publishes the pack range in boot info.</li>
  199. <li><strong>Kernel only.</strong> <code>make kernel-dev</code> sends <code>pack_size == 0</code>; the new kernel keeps the running rootfs.</li>
  200. <li><strong>Pack only.</strong> <code>make cmds-dev</code> uses <code>KXNP</code> and <code>ramfs_replace()</code>, without a kernel jump.</li>
  201. <li><strong>Monitor.</strong> Run <code>make monitor</code> in a second terminal; each reload prints <code>NebOS v0.2.0 booted</code> on COM1.</li>
  202. </ul>
  203. <p>
  204. The serial receive drops the BKL while it waits for bytes, so the WM
  205. and the other CPU's syscalls stay responsive during a reload.
  206. </p>
  207. </section>
  208. <section id="network">
  209. <h2>Network reload</h2>
  210. <p>
  211. For boards without a serial port, the same reload runs over UDP port
  212. 19288 with the <code>KXN2</code> magic. The host tool sends the
  213. kernel and rootfs as one stop-and-wait stream; the board acknowledges
  214. every datagram with the next expected offset and verifies a 32-bit
  215. checksum before jumping.
  216. </p>
  217. <p>
  218. <code>make net-install</code> uses <code>KXI2</code> instead. It
  219. sends the kernel plus a full <code>NBROOT1</code> rootfs image
  220. (<code>rootfs-full.bin</code>) and a 16-byte target name; the board
  221. writes P2, the P1 kernel, patches the exact
  222. <code>kernel_sectors</code>, verifies the result, acks
  223. <code>ACK_INSTALLED</code> and jumps. It never touches P4
  224. (<code>/data</code>) or the MBR, so it is the persistent counterpart
  225. to the RAM-only <code>net-dev</code>. <code>NETINSTALL_TARGET=auto</code>
  226. selects the only non-USB NebOS disk.
  227. </p>
  228. <pre><code>make net-dev # RAM-only kernel + rootfs to 192.168.178.240
  229. make net-dev BOARD_IP=192.168.178.42
  230. make kernel-net-dev # kernel only; keep the rootfs
  231. make net-install # persist kernel + full rootfs, keep /data
  232. make net-install NETINSTALL_TARGET=sata0
  233. make net-cmds # replace /ramfs only</code></pre>
  234. <p>
  235. The board must be idle at a shell prompt. The WM loop answers
  236. ARP/ICMP, detects the <code>KXN2</code>/<code>KXNP</code> header and
  237. pauses the auto-detect while a packet-level command owns the NIC or a
  238. transfer is in progress. Datagrams are at most 1400 bytes.
  239. </p>
  240. <p>
  241. The board address comes from DHCP when a server is present; use
  242. <code>make net-dev BOARD_IP=&lt;leased-address&gt;</code>, or pin an
  243. address with <code>netip</code> on the board.
  244. </p>
  245. </section>
  246. <section id="pack">
  247. <h2>Runtime command pack</h2>
  248. <p>
  249. <code>make cmds-dev</code> sends the raw <code>cmds.pack</code> with
  250. the <code>KXNP</code> magic. The receiver validates the pack before
  251. the swap, so a truncated transfer leaves the running commands
  252. untouched. A good pack is installed by <code>ramfs_replace()</code>
  253. and the old heap copy is freed.
  254. </p>
  255. <ul>
  256. <li><strong>Existing processes are unaffected.</strong> They already loaded their ELF.</li>
  257. <li><strong>The next spawn sees the new files.</strong></li>
  258. <li><strong>RAM only.</strong> A cold boot uses the flashed rootfs; reflash after changing the bundle.</li>
  259. <li><strong>Size.</strong> The raw pack is about 2 MB with the WiFi firmware, so <code>cmds-dev</code> over 115200 serial takes roughly three minutes; prefer <code>net-cmds</code>.</li>
  260. </ul>
  261. </section>
  262. <section id="tests">
  263. <h2>Tests</h2>
  264. <ul>
  265. <li><strong>USB regression.</strong> <code>make test-usb</code> runs QMP cursor and keyboard tests against UHCI, OHCI, EHCI and xHCI, including hotplug.</li>
  266. <li><strong>Installer.</strong> <code>make test-install</code> boots the dual image with its USB mass-storage source and a blank target, types <code>install ide1</code> through QMP, verifies the clone and boots the target in BIOS and (under OVMF) UEFI.</li>
  267. <li><strong>Image library.</strong> <code>make test-images</code> decodes a corpus of PNG/JPEG/GIF/BMP/TGA/PNM/ICO/PAM files and checks blits; the ASan variant catches memory errors.</li>
  268. <li><strong>Crypto.</strong> <code>make test-crypto</code> runs host vectors for the freestanding crypto code.</li>
  269. <li><strong>WPA2.</strong> <code>make test-wpa</code> runs <span class="file">wpa_test.c</span> against <span class="file">wpa_crypto.c</span> and <span class="file">wpa.c</span>. <code>make test-wifi-store</code> runs the saved-network list against an in-memory FAT file: ordering, update, eviction, forget and the legacy/malformed formats. The WiFi driver itself has no QEMU model: test it on the Chromebook over <code>make net-dev</code> with the RNDIS phone or ASIX dongle as the management link and <code>make netmon</code> for the <code>iwl:</code>/<code>wifi:</code> trail.</li>
  270. <li><strong>Ring-3 libc.</strong> <code>make test-libc</code> runs the malloc stress test (64 MB simulated arena with poisoned pages), exact <code>snprintf</code> float/string checks against the host, x87 math values, the 64-bit division helpers and a 32-bit setjmp/longjmp test.</li>
  271. <li><strong>Fonts.</strong> <code>make test-font</code> parses the checked-in Arimo subset on the host, measures Latin-1 text, renders composites (for example <code>Äé</code>) and writes a specimen PPM. The browser must also be exercised across a navigation to verify detach/reattach around <code>heap_reset()</code>.</li>
  272. <li><strong>Touchpad gestures.</strong> <code>make test-touchpad</code> feeds synthetic contact frames to <span class="file">touchpad_gesture.c</span> and checks one/two-finger taps, the corner and physical-button mapping, scroll start/direction, pointer freeze and re-anchoring, and the option switches.</li>
  273. <li><strong>Chat client.</strong> <code>make test-chat</code> covers the pure modules on the host. <code>tools/mock_chat.py</code> is a two-origin mock of Cloudron Chat (port 18080) and the Cloudron OIDC login (18081) with <code>test</code>, <code>totp</code> (code 123456), <code>passkey</code> and <code>consent</code> accounts (password <code>test</code>) and <code>/debug/state</code>, <code>/debug/inject</code> endpoints; run it alone and enter <code>http://10.0.2.2:18080</code> in a QEMU guest's chat, or let <code>make test-chat-e2e</code> drive it. Screenshots land in <code>chat-test-out/</code>.</li>
  274. <li><strong>Browser compatibility.</strong> <code>make compat-serve</code> serves the numbered A/B ladder in <code>testdata/browser/compat/</code>. Open the index on the board (or at <code>http://10.0.2.2:8000/</code> from a QEMU guest with user networking) and compare each pair; the index table records the current status.</li>
  275. <li><strong>UEFI.</strong> <code>make test-uefi</code> boots the dual image under OVMF headless and greps the serial log for the boot line.</li>
  276. <li><strong>Compositor stats.</strong> The ring-3 WM overwrites <code>/share/wmstats</code> every two seconds with <code>frames</code>, total and worst compositing time in milliseconds, and the pixels drawn in the interval. Read it in a terminal or fetch it over WebDAV to measure a target.</li>
  277. </ul>
  278. </section>
  279. <section id="debug">
  280. <h2>Debugging</h2>
  281. <ul>
  282. <li><strong>Serial log.</strong> <code>make monitor</code> shows the boot trace and printf-style driver output. The <code>NebOS v… booted</code> line is written before the framebuffer is ready.</li>
  283. <li><strong>Boot time.</strong> Timestamp COM1 on the host (QEMU <code>-serial stdio</code>): the kernel starts at <code>NebOS v… booted</code>, the desktop at <code>wm: first frame</code>, and the terminal is on screen at <code>wm: first window frame (term) at N ms</code> (kernel uptime). On QEMU the USB image reaches it about 0.65 s after the kernel starts.</li>
  284. <li><strong>Network console.</strong> <code>make netmon</code> listens to UDP 19290 for serial and terminal output; <code>netconsole host &lt;ip&gt;</code> narrows the target.</li>
  285. <li><strong>dbg.</strong> The ring-0 serial monitor accepts <code>r addr len</code>, <code>w addr b0 …</code>, <code>i port</code>, <code>o port val</code>, <code>h</code> and <code>q</code>.</li>
  286. <li><strong>Panic.</strong> A ring-0 fault dumps vector, error code, registers, CRs, EIP bytes and stack candidates to the LFB and COM1. Resolve addresses with <code>nm kernel.elf</code> / <code>addr2line -e kernel.elf</code>.</li>
  287. <li><strong>Profiling.</strong> <code>gfxbench</code> times fill/copy/cell/canvas/flush; <code>top</code>, <code>sched</code>, <code>sensors</code>, <code>powerinfo</code>, <code>power</code> and <code>lspci</code>/<code>lsusb</code> expose live state.</li>
  288. <li><strong>Memory.</strong> <code>peek addr len</code> hex-dumps identity-mapped memory (MMIO too); <code>poke addr val</code> writes one dword and prints the read-back.</li>
  289. </ul>
  290. </section>
  291. <section id="conventions">
  292. <h2>Conventions &amp; gotchas</h2>
  293. <ul>
  294. <li><strong>Freestanding C.</strong> The kernel has no libc, <code>malloc</code> or <code>printf</code>; use <code>io_putstr</code>, <code>io_putchar</code> and <code>io_puthex</code>. Ring-3 commands may link <code>libc.a</code> from <span class="file">libc/</span>; link it explicitly and compile the app with <code>-Ilibc/include</code>.</li>
  295. <li><strong>Fresh memory is not zeroed.</strong> Clear structures whose fields are read before being written. QEMU starts RAM clear and hides the bug.</li>
  296. <li><strong>Keep the memory map in sync.</strong> Hardcoded addresses appear in assembly, C headers and linker scripts; changing one without the others breaks boot or loading.</li>
  297. <li><strong>Ring-3 data segments.</strong> An IRET to ring 3 nulls DS/ES/FS/GS whose DPL is below CPL 3. Kernel stubs must restore selector <code>0x23</code> with <code>IRQ_RETURN_DS</code>. QEMU does not model this; only real hardware fails.</li>
  298. <li><strong>Paging is flat.</strong> Physical equals virtual everywhere; there is no process isolation.</li>
  299. <li><strong>The BKL is not preemptible.</strong> A BKL holder can be in the middle of kernel structures; block points drop it, and <code>bkl_pause()</code> is for long waits.</li>
  300. <li><strong>Kernel size wall.</strong> <code>_end</code> must stay below <code>0x90000</code>; the legacy staging gap also constrains the loaded image and the compressed pack.</li>
  301. </ul>
  302. </section>
  303. <section id="docs">
  304. <h2>Keeping docs current</h2>
  305. <p>
  306. This site is static HTML with one shared stylesheet, no JavaScript and
  307. no build step. It is part of the tree, not generated output. When a
  308. subsystem changes, update the matching page and the overview card in
  309. the same change:
  310. </p>
  311. <div class="table-wrap">
  312. <table>
  313. <thead>
  314. <tr><th>Change</th><th>Update</th></tr>
  315. </thead>
  316. <tbody>
  317. <tr><td>Boot path, boot info, image layout</td><td><a href="boot.html">boot.html</a></td></tr>
  318. <tr><td>Address map, allocator, paging, user arenas</td><td><a href="memory.html">memory.html</a></td></tr>
  319. <tr><td>Scheduler, interrupts, SMP, BKL, power</td><td><a href="cpu.html">cpu.html</a></td></tr>
  320. <tr><td>PCI, storage, USB, input, sensors</td><td><a href="drivers.html">drivers.html</a></td></tr>
  321. <tr><td>VFS, FAT, ramfs, /share, WebDAV</td><td><a href="filesystems.html">filesystems.html</a></td></tr>
  322. <tr><td>Framebuffer, WM, canvas, ngui, images</td><td><a href="graphics.html">graphics.html</a></td></tr>
  323. <tr><td>NIC, protocols, TLS, netconsole</td><td><a href="network.html">network.html</a></td></tr>
  324. <tr><td>Syscalls, shell, pack, apps</td><td><a href="userland.html">userland.html</a></td></tr>
  325. <tr><td>Build targets, reload, tests, debugging</td><td><a href="development.html">development.html</a></td></tr>
  326. </tbody>
  327. </table>
  328. </div>
  329. <p>
  330. <code>AGENTS.md</code> repeats the rule and lists the same page map.
  331. Numbers in charts must come from a source constant or table; if a
  332. constant moves, update the chart caption and the source file that
  333. defines it.
  334. </p>
  335. </section>
  336. <section id="sources">
  337. <h2>Source map</h2>
  338. <ul class="filelist">
  339. <li><code>Makefile</code> — targets and image rules</li>
  340. <li><code>libc/</code>, <code>libc.a</code> — optional ring-3 libc and its host test</li>
  341. <li><code>third_party/quickjs/</code> (see its <code>VENDOR</code>), <code>js_mu.c/h</code>, <code>js_browser.c/h</code> — vendored JavaScript engine, its MuJS-style stack API and the browser bindings; <code>js_test.c</code> host tests</li>
  342. <li><code>ufont.c</code>, <code>ufont_ttf.c</code>, <code>fonts/</code> — UI font engine and subsets; <code>ufont_test.c</code> host test</li>
  343. <li><code>kexec.c</code>, <code>kexec_send.c</code> — serial reload</li>
  344. <li><code>netkexec.c</code>, <code>netkexec_send.c</code> — network reload</li>
  345. <li><code>netlog.c</code>, <code>netmonitor.c</code> — UDP console</li>
  346. <li><code>dbg.c</code> — serial debug monitor</li>
  347. <li><code>panic.c</code> — fatal exception dump</li>
  348. <li><code>kernel.ld</code>, <code>ucmd.ld</code> — link limits</li>
  349. <li><code>tools/</code> — QMP USB and installer tests</li>
  350. <li><code>wpa_test.c</code> — WPA2 supplicant host test (<code>make test-wpa</code>)</li>
  351. <li><code>wifi_store_test.c</code> — saved WiFi list host test (<code>make test-wifi-store</code>)</li>
  352. <li><code>AGENTS.md</code> — developer notes and maintenance rule</li>
  353. </ul>
  354. </section>
  355. </main>
  356. <footer class="site-footer">
  357. <div class="inner">
  358. <p><strong>NebOS 0.2.0</strong> · Architecture documentation · <a href="index.html">overview</a></p>
  359. <p>Maintain alongside <code>AGENTS.md</code> and the source files listed on each page.</p>
  360. </div>
  361. </footer>
  362. </body>
  363. </html>