network.html 33 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531
  1. <!doctype html>
  2. <html lang="en">
  3. <head>
  4. <meta charset="utf-8">
  5. <meta name="viewport" content="width=device-width, initial-scale=1">
  6. <title>Networking | NebOS Architecture</title>
  7. <meta name="description" content="NebOS networking: the netdev registry and NIC drivers, ARP/ICMP/UDP, DHCP, DNS, NTP, the TCP client and listener, TLS 1.3, sys_net ownership, netconsole and netkexec.">
  8. <link rel="icon" href="favicon.svg" type="image/svg+xml">
  9. <link rel="stylesheet" href="style.css">
  10. </head>
  11. <body>
  12. <a class="skip" href="#main">Skip to content</a>
  13. <header class="site-header">
  14. <a class="brand" href="index.html">
  15. <span class="brand-mark" aria-hidden="true"></span>
  16. <span>NebOS <span class="brand-sub">Architecture</span></span>
  17. </a>
  18. <nav class="site-nav" aria-label="Architecture sections">
  19. <a href="index.html">Overview</a>
  20. <a href="boot.html">Boot</a>
  21. <a href="memory.html">Memory</a>
  22. <a href="cpu.html">CPU</a>
  23. <a href="drivers.html">Drivers</a>
  24. <a href="filesystems.html">Filesystems</a>
  25. <a href="graphics.html">Graphics</a>
  26. <a href="network.html" aria-current="page">Network</a>
  27. <a href="userland.html">Userland</a>
  28. <a href="development.html">Development</a>
  29. </nav>
  30. </header>
  31. <main id="main" class="page">
  32. <div class="page-head">
  33. <p class="kicker">Networking</p>
  34. <h1>A small stack with a shared receive pump.</h1>
  35. <p class="lede">
  36. The NIC and the protocol stack live in the kernel; ring-3 commands
  37. drive them through <code>SYS_NET_*</code> and <code>SYS_TCP_*</code>.
  38. Packet-level commands still take the receiver exclusively through
  39. <code>SYS_NET_ACQUIRE</code>, while up to twelve processes can hold a
  40. TCP client connection at the same time. Every stack syscall drains
  41. the NIC through <code>net_pump()</code>, so downloads overlap without
  42. sharing one connection or one receive queue.
  43. </p>
  44. <div class="meta">
  45. <span class="pill">ARP · ICMP · UDP · TCP</span>
  46. <span class="pill">DHCP · DNS · NTP</span>
  47. <span class="pill">TLS 1.3 · X25519</span>
  48. <span class="pill">netconsole + netkexec</span>
  49. </div>
  50. </div>
  51. <nav class="toc" aria-label="On this page">
  52. <h2>On this page</h2>
  53. <ul>
  54. <li><a href="#stack">Protocol stack</a></li>
  55. <li><a href="#netdev">NIC drivers</a></li>
  56. <li><a href="#core">Link &amp; IP layer</a></li>
  57. <li><a href="#dhcp">DHCP, DNS &amp; NTP</a></li>
  58. <li><a href="#tcp">TCP</a></li>
  59. <li><a href="#tls">TLS</a></li>
  60. <li><a href="#abi">sys_net ABI</a></li>
  61. <li><a href="#console">Netconsole &amp; netkexec</a></li>
  62. <li><a href="#http">HTTP clients</a></li>
  63. <li><a href="#chat">Chat client &amp; OIDC login</a></li>
  64. <li><a href="#security">Security posture</a></li>
  65. <li><a href="#sources">Source map</a></li>
  66. </ul>
  67. </nav>
  68. <section id="stack">
  69. <h2>Protocol stack</h2>
  70. <p>
  71. NebOS does not expose a general socket API. A ring-3 program asks the
  72. kernel for info and configuration, then either acquires the receiver
  73. for raw frame paths or opens one TCP client connection. Each
  74. connection has its own receive queue and is owned by one process;
  75. <code>net_pump()</code> dispatches frames to every active client, the
  76. WebDAV listener, a pending DNS query and the ARP cache. HTTP and TLS
  77. are libraries linked into the command, not kernel services.
  78. </p>
  79. <figure class="figure">
  80. <svg viewBox="0 0 920 380" role="img" aria-labelledby="net-title net-desc">
  81. <title id="net-title">NebOS network stack</title>
  82. <desc id="net-desc">Ring-3 curl, browser and websrv use uhttp and TLS, then sys_net syscalls, then TCP/DNS/DHCP/NTP, IPv4/ARP/ICMP, netdev drivers and the NIC.</desc>
  83. <rect class="svg-box accent" x="70" y="18" width="780" height="58" rx="9"/>
  84. <text class="svg-text" x="92" y="43">curl · browser · websrv · ping · dhcp · settime</text>
  85. <text class="svg-sub" x="92" y="63">ring-3 programs</text>
  86. <path class="svg-line" d="M460 76v20"/>
  87. <path class="svg-line" d="M454 90l6 7 6-7"/>
  88. <rect class="svg-box blue" x="70" y="98" width="380" height="58" rx="9"/>
  89. <text class="svg-text" x="92" y="123">uhttp · tls.c</text>
  90. <text class="svg-sub" x="92" y="143">HTTP/1.1 · redirects · chunked · TLS 1.3</text>
  91. <rect class="svg-box" x="470" y="98" width="380" height="58" rx="9"/>
  92. <text class="svg-text" x="492" y="123">kernel services</text>
  93. <text class="svg-sub" x="492" y="143">sys_fs for WebDAV · netlog · netkexec</text>
  94. <path class="svg-line blue" d="M260 156v18"/>
  95. <path class="svg-line blue" d="M254 168l6 7 6-7"/>
  96. <path class="svg-line" d="M660 156v18"/>
  97. <path class="svg-line" d="M654 168l6 7 6-7"/>
  98. <rect class="svg-box green" x="70" y="176" width="780" height="58" rx="9"/>
  99. <text class="svg-text" x="92" y="201">SYS_NET_* · SYS_TCP_* · SYS_DNS_RESOLVE · SYS_DHCP_RUN · SYS_NTP_SYNC</text>
  100. <text class="svg-sub" x="92" y="221">raw owner for packet commands · shared net_pump for TCP/DNS/ARP</text>
  101. <path class="svg-line green" d="M460 234v20"/>
  102. <path class="svg-line green" d="M454 248l6 7 6-7"/>
  103. <rect class="svg-box" x="70" y="256" width="780" height="58" rx="9"/>
  104. <text class="svg-text" x="92" y="281">tcp.c · dns.c · dhcp.c</text>
  105. <text class="svg-sub" x="92" y="301">TCP client table + listener · one pending A query · DISCOVER/OFFER/REQUEST/ACK</text>
  106. <path class="svg-line" d="M460 314v16"/>
  107. <path class="svg-line" d="M454 324l6 7 6-7"/>
  108. <rect class="svg-box blue" x="70" y="332" width="780" height="44" rx="9"/>
  109. <text class="svg-text" x="92" y="359">IPv4 · ARP · ICMP · UDP · netdev registry · NIC driver</text>
  110. </svg>
  111. <figcaption>
  112. <strong>Thin kernel, thick command.</strong> The kernel owns
  113. addressing, ARP, routing and the shared TCP client table. HTTP,
  114. TLS and WebDAV run in ring 3, so a protocol bug kills a command
  115. rather than the kernel.
  116. </figcaption>
  117. </figure>
  118. </section>
  119. <section id="netdev">
  120. <h2>NIC drivers</h2>
  121. <p>
  122. <span class="file">netdev.c</span> keeps a registry of
  123. <code>struct netdev</code> drivers; the first one whose probe succeeds
  124. becomes active. All drivers are polled and share the same transmit,
  125. receive, register-dump and reset interface. <code>netdev</code> with
  126. no argument shows the active driver; <code>netdev rndis</code> or
  127. <code>netdev asix</code> switches to a named one, leaving the current
  128. driver untouched if the probe fails.
  129. </p>
  130. <p>
  131. The top-bar network icon opens <span class="file">ucmd_netmenu.c</span>.
  132. It asks the kernel for every present driver through
  133. <code>SYS_NET_DEVLIST</code> (111), so an RNDIS phone or a USB
  134. Ethernet dongle that was not the boot-time winner still appears.
  135. Selecting one calls <code>SYS_NET_SELECT</code> (112) and then runs
  136. DHCP. The same window lists scanned WiFi networks; selecting a WPA2
  137. network asks for the passphrase and starts the existing
  138. <code>wifi</code> supplicant, which opens the data port and runs DHCP
  139. itself. Mesh/repeater BSSIDs sharing one SSID are collapsed to the
  140. strongest entry (the active BSSID wins when connected), and hidden
  141. SSIDs are omitted because the menu cannot join them.
  142. </p>
  143. <p>
  144. <span class="file">rndis.c</span> is the USB tethering path, not a
  145. WiFi driver: an Android phone (or QEMU's <code>usb-net</code>) is
  146. parsed for an RNDIS control/data interface pair, initialized with
  147. RNDIS INIT/QUERY/SET messages, and then presented as an ordinary
  148. Ethernet NIC. <code>netdev rndis</code> selects it and
  149. <code>dhcp</code> obtains the phone's tethering address. QEMU has no
  150. phone dependency: <code>make run-rndis</code> boots with
  151. <code>qemu-xhci</code> and <code>usb-net</code> on slirp. RNDIS only;
  152. CDC-ECM/NCM, IPv6 and automatic DHCP after a hotplug are not
  153. implemented.
  154. </p>
  155. <p>
  156. <span class="file">wifi.c</span> is the built-in WiFi of the
  157. Chromebook (Intel 7265D/3165) and of the ThinkPad X201 (Intel
  158. 6200/6300), or a Realtek RTL8821AU USB stick when one is plugged
  159. in, see the
  160. <a href="drivers.html#wifi">driver page</a>). It is last in the
  161. registry, so the boot probe only makes it active when nothing
  162. wired is present (a later <code>wifi connect</code> can still select
  163. it over a wired NIC whose cable is unplugged), and it reports
  164. <code>link_up() = 0</code> until it is connected: the boot DHCP
  165. checks that optional hook and skips the 4 s attempt. The
  166. <code>wifi</code> command scans, associates, runs the WPA2-PSK
  167. handshake in ring 3 and then calls <code>SYS_WIFI_AUTHORIZE</code>,
  168. which opens the data port and selects the <code>wifi</code> netdev.
  169. Data frames are converted between Ethernet and non-QoS 802.11 +
  170. LLC/SNAP; EAPOL (ethertype <code>0x888e</code>) never reaches
  171. <code>net_pump()</code> and is queued for the supplicant instead.
  172. The top-bar menu starts the same supplicant through
  173. <code>wifi connectfile &lt;path&gt;</code>: it stages the SSID,
  174. passphrase and BSSID in a small <code>/share/netmenu.wifi</code>
  175. record because a <code>SYS_SPAWN</code> argument is limited to
  176. <code>VFS_PATH_MAX</code> (64) bytes. The helper removes the file
  177. after reading it.
  178. </p>
  179. <p>
  180. A successful <code>wifi connect</code> (from the CLI or the menu) adds
  181. the network to a most-recently-used list of up to
  182. <code>WIFI_CFG_MAX</code> (16) SSID/passphrase records in
  183. <code>/data/WIFI.CFG</code> on the persistent <code>NEBOSDATA</code>
  184. partition. The network menu marks every scanned SSID that is in the
  185. list as <em>saved</em>, joins it with the stored passphrase without
  186. asking again, and its <em>Saved</em> view lists the records with a
  187. <em>Forget</em> button each. The <code>wifi saved</code> command
  188. prints the same list and <code>wifi forget [ssid]</code> removes one
  189. network (or all of them without an SSID).
  190. </p>
  191. <p>
  192. After the boot DHCP attempt and the storage scan, the
  193. <code>bootscan</code> thread spawns <code>wifi autoconnect</code>
  194. when that file exists. It samples the active NIC's
  195. <code>link_up()</code> state: a wired cable that is physically up
  196. keeps the Ethernet path, while no wired link lets WiFi scan and
  197. join the strongest network present whose SSID is in the saved list,
  198. trying the other saved networks found if one fails and rescanning
  199. once after three seconds. An existing WiFi link is left alone. The
  200. file is a plaintext 8.3 FAT file (one list header plus fixed-size
  201. records, most recently used first); no BSSID is stored, so the
  202. strongest AP advertising a saved SSID is used.
  203. </p>
  204. <div class="table-wrap">
  205. <table>
  206. <thead>
  207. <tr><th>Driver</th><th>Hardware</th><th>Transport</th><th>Notes</th></tr>
  208. </thead>
  209. <tbody>
  210. <tr><td><span class="file">e1000e.c</span></td><td>Intel 82540EM, 82577LM, 82578, 82579</td><td>PCI MMIO</td><td>PCI INTx disabled before MMIO; PCH D0; skips the global MAC reset while the link is up; extended RX descriptors only when <code>RFCTL.EXTEN</code> reads back set; the 82540EM (<code>0x100E</code>) is pinned to legacy descriptors, because it has no RFCTL but QEMU stores the register, so EXTEN reads back set.</td></tr>
  211. <tr><td><span class="file">r8168.c</span></td><td>RTL8111/8168</td><td>PCIe MMIO</td><td>C+ mode, 64-entry RX ring, polled.</td></tr>
  212. <tr><td><span class="file">ne2000.c</span></td><td>RTL8029</td><td>ISA / QEMU PCI</td><td>Simple DP8390 path used by <code>make run-net</code>.</td></tr>
  213. <tr><td><span class="file">asix.c</span></td><td>AX88772</td><td>USB bulk</td><td>Runs over the USB core; usable on EHCI/xHCI dongles.</td></tr>
  214. <tr><td><span class="file">rndis.c</span></td><td>Android USB tethering, QEMU <code>usb-net</code></td><td>USB control + bulk</td><td>RNDIS INIT/QUERY/packet filter; presents the phone's tethering link as an Ethernet netdev.</td></tr>
  215. <tr><td><span class="file">wifi.c</span></td><td>Intel 7265D / 3165 (Chromebook), 6200 / 6300 (X201), Realtek RTL8821AU USB stick</td><td>PCIe (firmware) or USB</td><td>Open and WPA2-PSK/CCMP; firmware loaded on first use; link down until <code>wifi connect</code>.</td></tr>
  216. </tbody>
  217. </table>
  218. </div>
  219. <p>
  220. The management NIC used by netlog and netkexec prefers the ASIX dongle
  221. so a direct debug link survives <code>netdev e1000e</code>; otherwise
  222. it falls back to the active driver. <code>netregs</code> dumps registers
  223. and decoded interrupt status; <code>netloop</code> runs a PHY loopback
  224. self-test; <code>netreset</code> re-initialises the active NIC.
  225. </p>
  226. </section>
  227. <section id="core">
  228. <h2>Link &amp; IP layer</h2>
  229. <ul>
  230. <li><strong>Ethernet decode</strong> recognises IPv4, ARP and IPv6 frame types (IPv6 is not implemented).</li>
  231. <li><strong>ARP</strong> resolves a next hop with three tries, one second apart; <code>net_arp_resolve()</code> polls the active NIC and the caller must own it. Cache entries live five minutes and their age slides on every hit, so a DNS or gateway entry does not expire during a long page load. Stack waits drain the receiver in short bursts so a full driver ring can reset and recover.</li>
  232. <li><strong>Routing</strong> sends off-subnet traffic to the gateway when one is known; <code>net_next_hop()</code> is shared by ping, DNS and TCP.</li>
  233. <li><strong>ICMP echo</strong> backs <code>ping</code>, with ARP resolution and round-trip timing.</li>
  234. <li><strong>UDP</strong> has send and parse helpers; DNS, DHCP, NTP and netlog are its users.</li>
  235. <li><strong>Checksums</strong> use the Internet one's-complement sum, with a TCP pseudo-header variant.</li>
  236. </ul>
  237. <p>
  238. The static default address is <code>192.168.178.240</code>. At boot,
  239. when the active NIC reports a physical link, a DHCP client runs with
  240. a 4-second budget (requesting the current address); if no server
  241. answers, the static address stays, and a cable that is not plugged
  242. in skips the wait so the saved WiFi network can be tried. The idle
  243. WM loop answers ARP/ICMP for the board's IP
  244. even when no command is running, so the board is reachable and
  245. MAC-resolvable.
  246. </p>
  247. </section>
  248. <section id="dhcp">
  249. <h2>DHCP, DNS &amp; NTP</h2>
  250. <h3>DHCP</h3>
  251. <p>
  252. <span class="file">dhcp.c</span> is a minimal client:
  253. DISCOVER → OFFER → REQUEST → ACK. It learns the mask, gateway and DNS
  254. server (option 6), and <code>dhcp</code> re-runs it on demand. The
  255. configuration is retained across kexec through the boot-info
  256. <code>BOOTINFO_FLAG_NET</code> fields.
  257. </p>
  258. <h3>DNS</h3>
  259. <p>
  260. <span class="file">dns.c</span> sends one recursive A query over UDP
  261. and handles name compression in the response. One query is in
  262. flight at a time; parallel stack callers wait for it and then hit a
  263. small five-minute A-record cache, so three browser workers asking
  264. for the same host cause one DNS query, not three. <code>netdns</code>
  265. shows or sets the server; <code>host &lt;name&gt;</code> resolves.
  266. Numeric hosts skip DNS entirely.
  267. </p>
  268. <h3>NTP</h3>
  269. <p>
  270. <span class="file">ntp.c</span> is a minimal SNTP (RFC 4330)
  271. client: one 48-byte request to UDP port 123, three one-second
  272. tries, DNS unless the server is a dotted address, and the shared
  273. ARP/router path. An empty name uses <code>pool.ntp.org</code>.
  274. A reply is accepted only when it comes from the
  275. resolved server to the ephemeral source port, is a server/broadcast
  276. reply with a valid stratum, and echoes the request's origin
  277. timestamp; the transmit timestamp is converted from NTP's 1900 epoch
  278. to a UTC <code>struct sys_datetime</code> (with the 2036 era wrap
  279. handled). <code>SYS_NTP_SYNC</code> returns that value to ring 3;
  280. the Date &amp; Time editor then writes it through
  281. <code>SYS_RTC_SET</code>. The service owns the raw receiver for the
  282. duration, so it reports <code>SYS_NTP_EBUSY</code> while another raw
  283. command or a TCP client is active.
  284. </p>
  285. <div class="warn">
  286. <p><strong>NTP is unauthenticated.</strong> The origin-timestamp
  287. check rejects unrelated datagrams but does not authenticate the
  288. server; anyone who can answer for the configured name can set the
  289. board's clock. There is no NTS, no leap-second handling and no
  290. delay/dispersion estimation — the CMOS RTC only resolves seconds.</p>
  291. </div>
  292. </section>
  293. <section id="tcp">
  294. <h2>TCP</h2>
  295. <p>
  296. <span class="file">tcp.c</span> implements a small client table, not a
  297. general socket layer: each process may hold one connect/send/receive/
  298. close connection, and up to <code>TCP_CLIENT_MAX</code> (twelve)
  299. connections can be active. Every connection owns a receive queue
  300. (64 KB from the frame allocator, with 32/16 KB fallbacks), so one
  301. socket's data can never be read or reset by another.
  302. The window update is sent whenever the application drains the queue,
  303. so a full queue cannot deadlock until the peer's zero-window probe.
  304. </p>
  305. <ul>
  306. <li><strong>Demultiplexing.</strong> <code>net_pump()</code> feeds every received frame to <code>tcp_client_input()</code>, which matches the destination IP and port pair and updates that connection's sequence state.</li>
  307. <li><strong>MSS / window.</strong> Our MSS is 1460 bytes and clients advertise up to a 64 KB window (the connection's receive queue; no window scaling). At a 30 ms round trip that is ~2 MB/s per connection instead of the old 4 KB window's ~133 KB/s.</li>
  308. <li><strong>Close.</strong> <code>tcp_client_close()</code> sends FIN and frees the slot immediately instead of waiting for the peer's FIN; every send is synchronous, so no data is outstanding, and a late FIN matches no client and is dropped. <code>uhttp</code> also skips the TLS close_notify when a redirect leaves the origin, so a redirect hop costs no extra round trips.</li>
  309. <li><strong>Retransmit.</strong> A 500 ms retry timer with a small retry count carries the connection through loss; there is no full congestion-control state machine.</li>
  310. <li><strong>Listener.</strong> The server side has one active connection plus three backlog/closing slots. SYN-ACK advertises a zero window while queued; a queued connection may hold 1 KB of early request, and the window opens when <code>tcp_srv_accept()</code> runs.</li>
  311. <li><strong>Bursts.</strong> Responses go out in up to four-segment bursts with a cumulative ACK, rather than one segment per ACK.</li>
  312. <li><strong>No keep-alive.</strong> The WebDAV listener still has one active connection; an idle persistent connection would block the parallel connections Nautilus opens. Client keep-alive is per helper process and does not hold the raw lease.</li>
  313. </ul>
  314. <p>
  315. New SYNs and data are also handed to <code>tcp_srv_input()</code> from
  316. the active connection's receive/send poll loops, not only between
  317. requests, so parallel gvfs clients make progress.
  318. </p>
  319. </section>
  320. <section id="tls">
  321. <h2>TLS 1.3</h2>
  322. <p>
  323. <span class="file">tls.c</span> is a ring-3 client with a deliberately
  324. narrow cipher suite: <code>TLS_CHACHA20_POLY1305_SHA256</code> and
  325. X25519 only. It is stream-oriented over the existing TCP connection
  326. and prints the leaf SHA-256 fingerprint for manual comparison.
  327. </p>
  328. <ul>
  329. <li><strong>Not verified.</strong> Phase A provides encryption without authentication; a MITM is undetected. <code>curl</code> warns, and the browser marks HTTPS as unverified.</li>
  330. <li><strong>Weak DRBG.</strong> <span class="file">tls_rng.c</span> mixes RDTSC jitter, ticks and the MAC; it is not a hardware-backed CSPRNG.</li>
  331. <li><strong>Host tests.</strong> The crypto files are pure freestanding C with host vectors (<code>make test-crypto</code>, <code>tls_test.c</code>).</li>
  332. </ul>
  333. <div class="warn">
  334. <p><strong>Do not treat HTTPS as secure authentication.</strong> The
  335. cipher suite is real, but without certificate verification the
  336. connection authenticates nothing. Use it for confidentiality on a
  337. trusted network, not for secrets against an active attacker.</p>
  338. </div>
  339. </section>
  340. <section id="abi">
  341. <h2>sys_net ABI &amp; ownership</h2>
  342. <p>
  343. <span class="file">sys_net.c</span> exposes NIC info, MAC/IP/mask/
  344. gateway/DNS getters and setters, raw frame TX/RX, decode, reply,
  345. loopback, register dump, reset, speed and MAC overrides. It also
  346. exposes the DNS, DHCP and NTP clients and the TCP/HTTP helpers.
  347. </p>
  348. <ul>
  349. <li><strong>Raw owner.</strong> <code>SYS_NET_ACQUIRE</code> takes the receiver for a packet-level command; the background kexec watcher pauses while one owns it. TCP clients do not take this lease, so they can run alongside each other.</li>
  350. <li><strong>Mode gates.</strong> A raw acquire fails while any TCP client socket is open, and <code>SYS_TCP_CONNECT</code> fails while a raw owner exists. This keeps packet commands and the shared pump from stealing each other's frames.</li>
  351. <li><strong>Reaper safety.</strong> If a raw owner dies, <code>sys_net_reap</code> force-releases the receiver through <code>netdev_release_all()</code>; sockets and a pending DNS query owned by the dead process are closed separately.</li>
  352. <li><strong>Wait semantics.</strong> Stack waits now yield with <code>process_sleep_ms()</code> and re-enable interrupts for driver TX timeouts; the syscall stub enters with interrupts off.</li>
  353. <li><strong>One connection per process.</strong> The connection state lives in the kernel, keyed by the calling pid. <code>SYS_TCP_CONNECT/SEND/RECV/CLOSE</code> keep their ABI, so uhttp and the TLS layer did not need a socket handle.</li>
  354. </ul>
  355. </section>
  356. <section id="console">
  357. <h2>Netconsole &amp; netkexec</h2>
  358. <p>
  359. <span class="file">netlog.c</span> mirrors the serial console and
  360. terminal output to UDP port 19290. The mirror is on by default and
  361. broadcasts; <code>netconsole host &lt;laptop-ip&gt;</code> narrows it,
  362. <code>netconsole off</code>/<code>on</code> toggles it, and
  363. <code>netlog &lt;text&gt;</code> sends an ad-hoc line. The producer
  364. never blocks: a full buffer drops characters and reports it. The
  365. flush pauses while a raw command owns the receiver or a TCP/stack
  366. operation is active, so its broadcast traffic cannot compete with
  367. DNS/ARP replies.
  368. </p>
  369. <p>
  370. <span class="file">netkexec.c</span> receives a new kernel/rootfs over
  371. UDP port 19288 using <code>KXN2</code>, a runtime command pack using
  372. <code>KXNP</code>, and a persistent install using <code>KXI2</code>.
  373. Datagrams are acknowledged with the next expected offset and verified
  374. with a 32-bit checksum. <code>KXN2</code>/<code>KXNP</code> jump or
  375. swap only in RAM. <code>KXI2</code> carries a 16-byte target name;
  376. the board writes P2, then the P1 kernel, patches the exact
  377. <code>kernel_sectors</code>, verifies the disk, sends
  378. <code>ACK_INSTALLED</code>/<code>ACK_INSTALL_FAIL</code>, and jumps.
  379. It never touches P4 (<code>/data</code>) or the MBR. On the host,
  380. <code>make netmon</code>, <code>make net-dev</code> and
  381. <code>make net-install</code> wrap these protocols.
  382. </p>
  383. </section>
  384. <section id="http">
  385. <h2>HTTP clients</h2>
  386. <ul>
  387. <li><strong>curl</strong> — <code>http://</code> or <code>https://</code> GET with DNS, ARP, redirects (up to five hops), chunked decoding, status/headers/body dump and a TLS warning.</li>
  388. <li><strong>uhttp</strong> — shared ring-3 HTTP/1.1 client used by curl, the browser and chat: URL parsing, redirects (up to eight hops; 1 KB paths and 1.5 KB header lines, enough for OIDC URLs), chunked decoding, timeouts and TLS. Requests and bodies of any size are split into pieces the transport accepts (1024-byte TLS records, one peer MSS for plain TCP). Optional per-request fields set the <code>Accept</code> header, extra header lines and a cookie jar; 1xx/204/304 responses are treated as bodyless so keep-alive does not wait for an idle timeout.</li>
  389. <li><strong>uhttp_cookie</strong> — the cookie jar (<span class="file">uhttp_cookie.c</span>, 24 cookies of up to 768 bytes). Every <code>Set-Cookie</code>, including those on redirect responses, is stored; matching cookies are sent on every request and redirect hop. Cookies are host-only (<code>Domain</code> is ignored), <code>Path</code> is a prefix match, <code>Secure</code> cookies only go over https, and <code>Max-Age&lt;=0</code> or a 1970 <code>Expires</code> deletes a cookie. The jar outlives the connection, so a cross-origin redirect that reconnects keeps the session.</li>
  390. <li><strong>browser</strong> — runs network fetches in up to four headless <code>curl __serve</code> workers, each with its own request/body/meta files under <code>/share/bf&lt;session&gt;_&lt;slot&gt;*</code>. The session id comes from the browser pid, so multiple browser windows do not share queue files. Stylesheet and external-script bodies are downloaded in parallel but parsed/executed in document order; one image starts before the script phase, and the remaining images are prefetched in parallel and decoded as jobs complete, so the UI keeps drawing and scrolling. Each worker keeps its own TCP/TLS connection; idle helpers stay warm across navigations (a cross-origin job just drops the old connection, and only busy helpers are killed), and helpers also exit after ten idle seconds or when the window loses focus. A DNS failure is retried once after tearing the helpers down. The helpers' DNS/TLS log lines go to a private tty that the browser drains and discards from every poll loop; a full tty ring blocks its writers, so an undrained one used to hang every helper after about sixteen TLS connections. The status bar reports the page's total, header and first-byte times.</li>
  391. <li><strong>websrv</strong> — the server side, speaking HTTP/1.1 + WebDAV over the TCP listener; see <a href="filesystems.html#webdav">Filesystems → WebDAV</a>.</li>
  392. </ul>
  393. <p>
  394. The browser's helper requests gzip; inflating a multi-megabyte
  395. page in the UI process uses an inflate progress hook that pumps
  396. the UI while decoding.
  397. </p>
  398. </section>
  399. <section id="chat">
  400. <h2>Chat client &amp; OIDC login</h2>
  401. <p>
  402. <code>chat</code> (<span class="file">ucmd_chat.c</span>) is a native
  403. client for Cloudron Chat built on uhttp, the cookie jar, the
  404. <span class="file">ujson.c</span> arena JSON parser (fixed node pool,
  405. escapes, surrogate pairs) and ngui. The web login is a Vue
  406. ES-module SPA the browser cannot run, so
  407. <span class="file">chat_login.c</span> drives the API underneath:
  408. </p>
  409. <ol>
  410. <li><code>GET &lt;server&gt;/auth/login</code> follows the redirect to the Cloudron OIDC provider's <code>/openid/interaction/&lt;uid&gt;</code> page, collecting the PKCE pre-session cookie on the Chat host and the interaction cookies on the provider.</li>
  411. <li>The page's <code>&lt;script id="cloudron-config"&gt;</code> JSON names a relative <code>submitUrl</code> (a plain <code>"submitUrl":"…"</code> scan is the fallback). No config means external SSO: the client offers the cookie screen.</li>
  412. <li><code>POST submitUrl {"username","password","totpToken"}</code> answers 401 with a message, <code>{"twoFactorRequired":true,"totpRequired":true}</code> (the client asks for the code and reposts), <code>totpRequired:false</code> (passkey-only: cookie screen) or <code>{"redirectTo"}</code>.</li>
  413. <li>Following <code>redirectTo</code> ends at <code>/auth/callback</code> on the Chat host, which sets <code>connect.sid</code>. The consent interaction on the way is a bare auto-submit form (no <code>cloudron-config</code>): its <code>&lt;form action="/openid/interaction/&lt;uid&gt;/confirm"&gt;</code> is posted like the web UI does, and an <code>/abort</code> form or page means the account has no access. <code>GET /api/me</code> validates the session.</li>
  414. </ol>
  415. <p>
  416. The password lives only in the form field and the request buffer;
  417. both are wiped after the attempt. The Chat host's cookie header is
  418. saved with the server address in <code>/data/CHAT.CFG</code> when
  419. the persistent <code>/data</code> volume is mounted, or
  420. <code>/share/chat.cfg</code> (RAM-only) otherwise; another path can
  421. be given as the argument or on the login screen, and a session left
  422. in the old <code>/share</code> file is copied over once. The
  423. config is re-saved when the server rolls the cookie. The saved
  424. cookie is a bearer credential. Passkey and external-SSO accounts
  425. paste a <code>connect.sid</code> cookie from a signed-in browser
  426. instead.
  427. </p>
  428. <p>
  429. Message image attachments, link-preview images and profile
  430. pictures are fetched through the same uhttp connection and
  431. cookie jar, one per main-loop pass, and decoded by the in-tree
  432. image library (PNG, JPEG, GIF first frame, BMP, TGA, PNM, ICO
  433. and PAM). Message images are scaled to at most 480x320 and
  434. cached in a 16-entry, 6 MB LRU; avatars are 32x32 with a
  435. 64-entry cache and a circular alpha mask, and fall back to
  436. initials until they load or when the account has no picture.
  437. The fetch is capped at 4 MB and the decoder at 4 MP / 12 MB
  438. of scratch.
  439. </p>
  440. <p>
  441. There is no WebSocket: the open conversation is polled with
  442. <code>GET /api/channels/:id/messages?after=&lt;newest&gt;</code> every
  443. 2 s (5 s while the window is unfocused) and the directory
  444. (<code>/api/channels</code>, <code>/api/dms</code>) every 12 s (20 s),
  445. and polling waits 1 s after keyboard or pointer input. Errors back
  446. off 2/5/10/30 s. Sends carry a <code>clientId</code> so a retried
  447. POST cannot duplicate a message. An open TCP socket pauses the
  448. board's netkexec watcher, so the client only keeps its keep-alive
  449. connection while its window is focused and the user was active in
  450. the last 30 s; otherwise it closes it after each poll. Other
  451. clients see the NebOS user as offline (presence is WebSocket-based).
  452. </p>
  453. </section>
  454. <section id="security">
  455. <h2>Security posture</h2>
  456. <div class="table-wrap">
  457. <table>
  458. <thead>
  459. <tr><th>Surface</th><th>State</th><th>Consequence</th></tr>
  460. </thead>
  461. <tbody>
  462. <tr><td>TLS certificates</td><td>Not verified</td><td>Encryption without authentication; MITM is undetected.</td></tr>
  463. <tr><td>TLS DRBG</td><td>Weak (RDTSC/ticks/MAC)</td><td>Not suitable for high-value secrets.</td></tr>
  464. <tr><td>WebDAV</td><td>No authentication</td><td>Anyone on the LAN can read mounts and write <code>/share</code>.</td></tr>
  465. <tr><td>Chat session</td><td>Cookie saved in <code>/data/CHAT.CFG</code>, or <code>/share/chat.cfg</code> without a <code>/data</code> mount</td><td>A bearer credential on disk while <code>/data</code> is mounted, otherwise RAM until sign-out; the password is sent over unverified TLS and never stored.</td></tr>
  466. <tr><td>WiFi WPA2</td><td>PSK/CCMP, handshake in ring 3</td><td>The SNonce comes from the weak TLS DRBG. The PTK/replay are kept in the kernel (<code>SYS_WIFI_SET_PTK</code>/<code>SYS_WIFI_GET_PTK</code>) and fetched by the <code>wifi __keep</code> helper by connection ID for group rekeys; no WPA3/PMF.</td></tr>
  467. <tr><td>Saved WiFi credentials</td><td>Plaintext <code>/data/WIFI.CFG</code></td><td>Up to 16 SSID/passphrase pairs are stored in an 8.3 FAT file (most recently used first) so <code>wifi autoconnect</code> can join any known network at boot; anyone who can read the disk can read the passphrases. <code>wifi forget [ssid]</code> and the menu's <em>Saved</em> view remove records.</td></tr>
  468. <tr><td>NTP</td><td>Unauthenticated SNTP</td><td>Anyone who can answer for the server name can set the clock.</td></tr>
  469. <tr><td>NIC ownership</td><td>Raw lease for packet commands; shared TCP client table</td><td>Raw commands are mutually exclusive; up to twelve processes can hold one TCP connection each. A killed owner is force-released.</td></tr>
  470. <tr><td>Kernel/user isolation</td><td>None (identity map)</td><td>A ring-3 bug can reach any mapped kernel page.</td></tr>
  471. </tbody>
  472. </table>
  473. </div>
  474. </section>
  475. <section id="sources">
  476. <h2>Source map</h2>
  477. <ul class="filelist">
  478. <li><code>netdev.c</code>, <code>netdev.h</code> — registry</li>
  479. <li><code>e1000e.c</code>, <code>r8168.c</code>, <code>ne2000.c</code>, <code>asix.c</code>, <code>rndis.c</code></li>
  480. <li><code>wifi.c</code>, <code>wifi_fw.c</code>, <code>iwl_mvm.c</code>, <code>iwl_dvm.c</code>, <code>iwl_trans.c</code>, <code>rtl8821au.c</code> — Intel 7265D/3165 and 6200/6300 WiFi, Realtek RTL8821AU USB WiFi</li>
  481. <li><code>wpa.c</code>, <code>wpa_crypto.c</code>, <code>ucmd_wifi.c</code> — WPA2-PSK supplicant and the <code>wifi</code> command (<code>make test-wpa</code>)</li>
  482. <li><code>ucmd_netmenu.c</code>, <code>wifi_cfg.h</code>, <code>wifi_store.c/h</code> — top-bar network menu, staged WiFi connect request and the saved-network list</li>
  483. <li><code>net.c</code>, <code>net.h</code> — ARP cache/ICMP/UDP/IP</li>
  484. <li><code>netcore.c</code>, <code>netcore.h</code> — shared receive pump and IRQ helpers</li>
  485. <li><code>dhcp.c</code>, <code>dns.c</code>, <code>ntp.c</code></li>
  486. <li><code>tcp.c</code>, <code>tcp.h</code> — client table + listener</li>
  487. <li><code>tls.c</code>, <code>tls_*.c</code> — TLS 1.3</li>
  488. <li><code>sys_net.c</code> — ring-3 service</li>
  489. <li><code>netlog.c</code>, <code>netkexec.c</code></li>
  490. <li><code>uhttp.c</code>, <code>ucmd_curl.c</code>, <code>ucmd_websrv.c</code></li>
  491. <li><code>uhttp_cookie.c</code>, <code>ujson.c</code> — cookie jar, JSON parser</li>
  492. <li><code>chat_login.c</code>, <code>ucmd_chat.c</code> — OIDC login driver, chat client</li>
  493. <li><code>tools/mock_chat.py</code>, <code>tools/qmp_chat_test.py</code> — mock server, QEMU end-to-end test</li>
  494. </ul>
  495. </section>
  496. </main>
  497. <footer class="site-footer">
  498. <div class="inner">
  499. <p><strong>NebOS 0.2.0</strong> · Architecture documentation · <a href="index.html">overview</a></p>
  500. <p>Maintain alongside <code>AGENTS.md</code> and the source files listed on each page.</p>
  501. </div>
  502. </footer>
  503. </body>
  504. </html>