network.html 19 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375
  1. <!doctype html>
  2. <html lang="en">
  3. <head>
  4. <meta charset="utf-8">
  5. <meta name="viewport" content="width=device-width, initial-scale=1">
  6. <title>Networking | NebOS Architecture</title>
  7. <meta name="description" content="NebOS networking: the netdev registry and NIC drivers, ARP/ICMP/UDP, DHCP, DNS, NTP, the TCP client and listener, TLS 1.3, sys_net ownership, netconsole and netkexec.">
  8. <link rel="icon" href="favicon.svg" type="image/svg+xml">
  9. <link rel="stylesheet" href="style.css">
  10. </head>
  11. <body>
  12. <a class="skip" href="#main">Skip to content</a>
  13. <header class="site-header">
  14. <a class="brand" href="index.html">
  15. <span class="brand-mark" aria-hidden="true"></span>
  16. <span>NebOS <span class="brand-sub">Architecture</span></span>
  17. </a>
  18. <nav class="site-nav" aria-label="Architecture sections">
  19. <a href="index.html">Overview</a>
  20. <a href="boot.html">Boot</a>
  21. <a href="memory.html">Memory</a>
  22. <a href="cpu.html">CPU</a>
  23. <a href="drivers.html">Drivers</a>
  24. <a href="filesystems.html">Filesystems</a>
  25. <a href="graphics.html">Graphics</a>
  26. <a href="network.html" aria-current="page">Network</a>
  27. <a href="userland.html">Userland</a>
  28. <a href="development.html">Development</a>
  29. </nav>
  30. </header>
  31. <main id="main" class="page">
  32. <div class="page-head">
  33. <p class="kicker">Networking</p>
  34. <h1>A small stack with a shared receive pump.</h1>
  35. <p class="lede">
  36. The NIC and the protocol stack live in the kernel; ring-3 commands
  37. drive them through <code>SYS_NET_*</code> and <code>SYS_TCP_*</code>.
  38. Packet-level commands still take the receiver exclusively through
  39. <code>SYS_NET_ACQUIRE</code>, while up to six processes can hold a
  40. TCP client connection at the same time. Every stack syscall drains
  41. the NIC through <code>net_pump()</code>, so downloads overlap without
  42. sharing one connection or one receive queue.
  43. </p>
  44. <div class="meta">
  45. <span class="pill">ARP · ICMP · UDP · TCP</span>
  46. <span class="pill">DHCP · DNS · NTP</span>
  47. <span class="pill">TLS 1.3 · X25519</span>
  48. <span class="pill">netconsole + netkexec</span>
  49. </div>
  50. </div>
  51. <nav class="toc" aria-label="On this page">
  52. <h2>On this page</h2>
  53. <ul>
  54. <li><a href="#stack">Protocol stack</a></li>
  55. <li><a href="#netdev">NIC drivers</a></li>
  56. <li><a href="#core">Link &amp; IP layer</a></li>
  57. <li><a href="#dhcp">DHCP, DNS &amp; NTP</a></li>
  58. <li><a href="#tcp">TCP</a></li>
  59. <li><a href="#tls">TLS</a></li>
  60. <li><a href="#abi">sys_net ABI</a></li>
  61. <li><a href="#console">Netconsole &amp; netkexec</a></li>
  62. <li><a href="#http">HTTP clients</a></li>
  63. <li><a href="#security">Security posture</a></li>
  64. <li><a href="#sources">Source map</a></li>
  65. </ul>
  66. </nav>
  67. <section id="stack">
  68. <h2>Protocol stack</h2>
  69. <p>
  70. NebOS does not expose a general socket API. A ring-3 program asks the
  71. kernel for info and configuration, then either acquires the receiver
  72. for raw frame paths or opens one TCP client connection. Each
  73. connection has its own receive queue and is owned by one process;
  74. <code>net_pump()</code> dispatches frames to every active client, the
  75. WebDAV listener, a pending DNS query and the ARP cache. HTTP and TLS
  76. are libraries linked into the command, not kernel services.
  77. </p>
  78. <figure class="figure">
  79. <svg viewBox="0 0 920 380" role="img" aria-labelledby="net-title net-desc">
  80. <title id="net-title">NebOS network stack</title>
  81. <desc id="net-desc">Ring-3 curl, browser and websrv use uhttp and TLS, then sys_net syscalls, then TCP/DNS/DHCP/NTP, IPv4/ARP/ICMP, netdev drivers and the NIC.</desc>
  82. <rect class="svg-box accent" x="70" y="18" width="780" height="58" rx="9"/>
  83. <text class="svg-text" x="92" y="43">curl · browser · websrv · ping · dhcp · settime</text>
  84. <text class="svg-sub" x="92" y="63">ring-3 programs</text>
  85. <path class="svg-line" d="M460 76v20"/>
  86. <path class="svg-line" d="M454 90l6 7 6-7"/>
  87. <rect class="svg-box blue" x="70" y="98" width="380" height="58" rx="9"/>
  88. <text class="svg-text" x="92" y="123">uhttp · tls.c</text>
  89. <text class="svg-sub" x="92" y="143">HTTP/1.1 · redirects · chunked · TLS 1.3</text>
  90. <rect class="svg-box" x="470" y="98" width="380" height="58" rx="9"/>
  91. <text class="svg-text" x="492" y="123">kernel services</text>
  92. <text class="svg-sub" x="492" y="143">sys_fs for WebDAV · netlog · netkexec</text>
  93. <path class="svg-line blue" d="M260 156v18"/>
  94. <path class="svg-line blue" d="M254 168l6 7 6-7"/>
  95. <path class="svg-line" d="M660 156v18"/>
  96. <path class="svg-line" d="M654 168l6 7 6-7"/>
  97. <rect class="svg-box green" x="70" y="176" width="780" height="58" rx="9"/>
  98. <text class="svg-text" x="92" y="201">SYS_NET_* · SYS_TCP_* · SYS_DNS_RESOLVE · SYS_DHCP_RUN · SYS_NTP_SYNC</text>
  99. <text class="svg-sub" x="92" y="221">raw owner for packet commands · shared net_pump for TCP/DNS/ARP</text>
  100. <path class="svg-line green" d="M460 234v20"/>
  101. <path class="svg-line green" d="M454 248l6 7 6-7"/>
  102. <rect class="svg-box" x="70" y="256" width="780" height="58" rx="9"/>
  103. <text class="svg-text" x="92" y="281">tcp.c · dns.c · dhcp.c</text>
  104. <text class="svg-sub" x="92" y="301">TCP client table + listener · one pending A query · DISCOVER/OFFER/REQUEST/ACK</text>
  105. <path class="svg-line" d="M460 314v16"/>
  106. <path class="svg-line" d="M454 324l6 7 6-7"/>
  107. <rect class="svg-box blue" x="70" y="332" width="780" height="44" rx="9"/>
  108. <text class="svg-text" x="92" y="359">IPv4 · ARP · ICMP · UDP · netdev registry · NIC driver</text>
  109. </svg>
  110. <figcaption>
  111. <strong>Thin kernel, thick command.</strong> The kernel owns
  112. addressing, ARP, routing and the shared TCP client table. HTTP,
  113. TLS and WebDAV run in ring 3, so a protocol bug kills a command
  114. rather than the kernel.
  115. </figcaption>
  116. </figure>
  117. </section>
  118. <section id="netdev">
  119. <h2>NIC drivers</h2>
  120. <p>
  121. <span class="file">netdev.c</span> keeps a registry of
  122. <code>struct netdev</code> drivers; the first one whose probe succeeds
  123. becomes active. All drivers are polled and share the same transmit,
  124. receive, register-dump and reset interface. <code>netdev</code> with
  125. no argument shows the active driver; <code>netdev asix</code> switches
  126. to a named one, leaving the current driver untouched if the probe
  127. fails.
  128. </p>
  129. <div class="table-wrap">
  130. <table>
  131. <thead>
  132. <tr><th>Driver</th><th>Hardware</th><th>Transport</th><th>Notes</th></tr>
  133. </thead>
  134. <tbody>
  135. <tr><td><span class="file">e1000e.c</span></td><td>Intel 82540EM, 82577LM, 82578, 82579</td><td>PCI MMIO</td><td>PCI INTx disabled before MMIO; PCH D0; skips the global MAC reset while the link is up.</td></tr>
  136. <tr><td><span class="file">r8168.c</span></td><td>RTL8111/8168</td><td>PCIe MMIO</td><td>C+ mode, 64-entry RX ring, polled.</td></tr>
  137. <tr><td><span class="file">ne2000.c</span></td><td>RTL8029</td><td>ISA / QEMU PCI</td><td>Simple DP8390 path used by <code>make run-net</code>.</td></tr>
  138. <tr><td><span class="file">asix.c</span></td><td>AX88772</td><td>USB bulk</td><td>Runs over the USB core; usable on EHCI/xHCI dongles.</td></tr>
  139. </tbody>
  140. </table>
  141. </div>
  142. <p>
  143. The management NIC used by netlog and netkexec prefers the ASIX dongle
  144. so a direct debug link survives <code>netdev e1000e</code>; otherwise
  145. it falls back to the active driver. <code>netregs</code> dumps registers
  146. and decoded interrupt status; <code>netloop</code> runs a PHY loopback
  147. self-test; <code>netreset</code> re-initialises the active NIC.
  148. </p>
  149. </section>
  150. <section id="core">
  151. <h2>Link &amp; IP layer</h2>
  152. <ul>
  153. <li><strong>Ethernet decode</strong> recognises IPv4, ARP and IPv6 frame types (IPv6 is not implemented).</li>
  154. <li><strong>ARP</strong> resolves a next hop with three tries, one second apart; <code>net_arp_resolve()</code> polls the active NIC and the caller must own it.</li>
  155. <li><strong>Routing</strong> sends off-subnet traffic to the gateway when one is known; <code>net_next_hop()</code> is shared by ping, DNS and TCP.</li>
  156. <li><strong>ICMP echo</strong> backs <code>ping</code>, with ARP resolution and round-trip timing.</li>
  157. <li><strong>UDP</strong> has send and parse helpers; DNS, DHCP, NTP and netlog are its users.</li>
  158. <li><strong>Checksums</strong> use the Internet one's-complement sum, with a TCP pseudo-header variant.</li>
  159. </ul>
  160. <p>
  161. The static default address is <code>192.168.178.240</code>. At boot,
  162. with a NIC present, a DHCP client runs with a 4-second budget
  163. (requesting the current address); if no server answers, the static
  164. address stays. The idle WM loop answers ARP/ICMP for the board's IP
  165. even when no command is running, so the board is reachable and
  166. MAC-resolvable.
  167. </p>
  168. </section>
  169. <section id="dhcp">
  170. <h2>DHCP, DNS &amp; NTP</h2>
  171. <h3>DHCP</h3>
  172. <p>
  173. <span class="file">dhcp.c</span> is a minimal client:
  174. DISCOVER → OFFER → REQUEST → ACK. It learns the mask, gateway and DNS
  175. server (option 6), and <code>dhcp</code> re-runs it on demand. The
  176. configuration is retained across kexec through the boot-info
  177. <code>BOOTINFO_FLAG_NET</code> fields.
  178. </p>
  179. <h3>DNS</h3>
  180. <p>
  181. <span class="file">dns.c</span> sends one recursive A query over UDP
  182. and handles name compression in the response. <code>netdns</code>
  183. shows or sets the server; <code>host &lt;name&gt;</code> resolves.
  184. Numeric hosts skip DNS entirely.
  185. </p>
  186. <h3>NTP</h3>
  187. <p>
  188. <span class="file">ntp.c</span> is a minimal SNTP (RFC 4330)
  189. client: one 48-byte request to UDP port 123, three one-second
  190. tries, DNS unless the server is a dotted address, and the shared
  191. ARP/router path. An empty name uses <code>pool.ntp.org</code>.
  192. A reply is accepted only when it comes from the
  193. resolved server to the ephemeral source port, is a server/broadcast
  194. reply with a valid stratum, and echoes the request's origin
  195. timestamp; the transmit timestamp is converted from NTP's 1900 epoch
  196. to a UTC <code>struct sys_datetime</code> (with the 2036 era wrap
  197. handled). <code>SYS_NTP_SYNC</code> returns that value to ring 3;
  198. the Date &amp; Time editor then writes it through
  199. <code>SYS_RTC_SET</code>. The service owns the receiver for the
  200. duration, so it reports <code>SYS_NTP_EBUSY</code> while another
  201. network command owns the NIC.
  202. </p>
  203. <div class="warn">
  204. <p><strong>NTP is unauthenticated.</strong> The origin-timestamp
  205. check rejects unrelated datagrams but does not authenticate the
  206. server; anyone who can answer for the configured name can set the
  207. board's clock. There is no NTS, no leap-second handling and no
  208. delay/dispersion estimation — the CMOS RTC only resolves seconds.</p>
  209. </div>
  210. </section>
  211. <section id="tcp">
  212. <h2>TCP</h2>
  213. <p>
  214. <span class="file">tcp.c</span> implements a small client table, not a
  215. general socket layer: each process may hold one connect/send/receive/
  216. close connection, and up to <code>TCP_CLIENT_MAX</code> (six)
  217. connections can be active. Every connection owns a 4 KB receive
  218. queue, so one socket's data can never be read or reset by another.
  219. The window update is sent whenever the application drains the queue,
  220. so a full queue cannot deadlock until the peer's zero-window probe.
  221. </p>
  222. <ul>
  223. <li><strong>Demultiplexing.</strong> <code>net_pump()</code> feeds every received frame to <code>tcp_client_input()</code>, which matches the destination IP and port pair and updates that connection's sequence state.</li>
  224. <li><strong>MSS / window.</strong> Our MSS is 1460 bytes and the advertised window is 4 KB.</li>
  225. <li><strong>Retransmit.</strong> A 500 ms retry timer with a small retry count carries the connection through loss; there is no full congestion-control state machine.</li>
  226. <li><strong>Listener.</strong> The server side has one active connection plus three backlog/closing slots. SYN-ACK advertises a zero window while queued; a queued connection may hold 1 KB of early request, and the window opens when <code>tcp_srv_accept()</code> runs.</li>
  227. <li><strong>Bursts.</strong> Responses go out in up to four-segment bursts with a cumulative ACK, rather than one segment per ACK.</li>
  228. <li><strong>No keep-alive.</strong> The WebDAV listener still has one active connection; an idle persistent connection would block the parallel connections Nautilus opens. Client keep-alive is per helper process and does not hold the raw lease.</li>
  229. </ul>
  230. <p>
  231. New SYNs and data are also handed to <code>tcp_srv_input()</code> from
  232. the active connection's receive/send poll loops, not only between
  233. requests, so parallel gvfs clients make progress.
  234. </p>
  235. </section>
  236. <section id="tls">
  237. <h2>TLS 1.3</h2>
  238. <p>
  239. <span class="file">tls.c</span> is a ring-3 client with a deliberately
  240. narrow cipher suite: <code>TLS_CHACHA20_POLY1305_SHA256</code> and
  241. X25519 only. It is stream-oriented over the existing TCP connection
  242. and prints the leaf SHA-256 fingerprint for manual comparison.
  243. </p>
  244. <ul>
  245. <li><strong>Not verified.</strong> Phase A provides encryption without authentication; a MITM is undetected. <code>curl</code> warns, and the browser marks HTTPS as unverified.</li>
  246. <li><strong>Weak DRBG.</strong> <span class="file">tls_rng.c</span> mixes RDTSC jitter, ticks and the MAC; it is not a hardware-backed CSPRNG.</li>
  247. <li><strong>Host tests.</strong> The crypto files are pure freestanding C with host vectors (<code>make test-crypto</code>, <code>tls_test.c</code>).</li>
  248. </ul>
  249. <div class="warn">
  250. <p><strong>Do not treat HTTPS as secure authentication.</strong> The
  251. cipher suite is real, but without certificate verification the
  252. connection authenticates nothing. Use it for confidentiality on a
  253. trusted network, not for secrets against an active attacker.</p>
  254. </div>
  255. </section>
  256. <section id="abi">
  257. <h2>sys_net ABI &amp; ownership</h2>
  258. <p>
  259. <span class="file">sys_net.c</span> exposes NIC info, MAC/IP/mask/
  260. gateway/DNS getters and setters, raw frame TX/RX, decode, reply,
  261. loopback, register dump, reset, speed and MAC overrides. It also
  262. exposes the DNS, DHCP and NTP clients and the TCP/HTTP helpers.
  263. </p>
  264. <ul>
  265. <li><strong>Raw owner.</strong> <code>SYS_NET_ACQUIRE</code> takes the receiver for a packet-level command; the background kexec watcher pauses while one owns it. TCP clients do not take this lease, so they can run alongside each other.</li>
  266. <li><strong>Mode gates.</strong> A raw acquire fails while any TCP client socket is open, and <code>SYS_TCP_CONNECT</code> fails while a raw owner exists. This keeps packet commands and the shared pump from stealing each other's frames.</li>
  267. <li><strong>Reaper safety.</strong> If a raw owner dies, <code>sys_net_reap</code> force-releases the receiver through <code>netdev_release_all()</code>; sockets and a pending DNS query owned by the dead process are closed separately.</li>
  268. <li><strong>Wait semantics.</strong> Stack waits now yield with <code>process_sleep_ms()</code> and re-enable interrupts for driver TX timeouts; the syscall stub enters with interrupts off.</li>
  269. <li><strong>One connection per process.</strong> The connection state lives in the kernel, keyed by the calling pid. <code>SYS_TCP_CONNECT/SEND/RECV/CLOSE</code> keep their ABI, so uhttp and the TLS layer did not need a socket handle.</li>
  270. </ul>
  271. </section>
  272. <section id="console">
  273. <h2>Netconsole &amp; netkexec</h2>
  274. <p>
  275. <span class="file">netlog.c</span> mirrors the serial console and
  276. terminal output to UDP port 19290. The mirror is on by default and
  277. broadcasts; <code>netconsole host &lt;laptop-ip&gt;</code> narrows it,
  278. <code>netconsole off</code>/<code>on</code> toggles it, and
  279. <code>netlog &lt;text&gt;</code> sends an ad-hoc line. The producer
  280. never blocks: a full buffer drops characters and reports it. The
  281. flush pauses while a raw command owns the receiver or a TCP/stack
  282. operation is active, so its broadcast traffic cannot compete with
  283. DNS/ARP replies.
  284. </p>
  285. <p>
  286. <span class="file">netkexec.c</span> receives a new kernel/rootfs over
  287. UDP port 19288 using <code>KXN2</code>, and a runtime command pack
  288. using <code>KXNP</code>. Datagrams are acknowledged with the next
  289. expected offset and verified with a 32-bit checksum before the jump
  290. or pack swap. On the host, <code>make netmon</code> and
  291. <code>make net-dev</code> wrap these protocols.
  292. </p>
  293. </section>
  294. <section id="http">
  295. <h2>HTTP clients</h2>
  296. <ul>
  297. <li><strong>curl</strong> — <code>http://</code> or <code>https://</code> GET with DNS, ARP, redirects (up to five hops), chunked decoding, status/headers/body dump and a TLS warning.</li>
  298. <li><strong>uhttp</strong> — shared ring-3 HTTP/1.1 client used by curl and the browser: URL parsing, redirects, chunked decoding, timeouts and TLS.</li>
  299. <li><strong>browser</strong> — runs network fetches in up to three headless <code>curl __serve</code> workers, each with its own request/body/meta files under <code>/share</code>. Images are prefetched in parallel and decoded as jobs complete, so the UI keeps drawing and scrolling. Each worker keeps its own TCP/TLS connection.</li>
  300. <li><strong>websrv</strong> — the server side, speaking HTTP/1.1 + WebDAV over the TCP listener; see <a href="filesystems.html#webdav">Filesystems → WebDAV</a>.</li>
  301. </ul>
  302. <p>
  303. The browser's helper requests identity encoding because inflating
  304. multi-megabyte pages in the UI process stalled. If a server still
  305. sends gzip, the browser's inflate hook pumps the UI while decoding.
  306. </p>
  307. </section>
  308. <section id="security">
  309. <h2>Security posture</h2>
  310. <div class="table-wrap">
  311. <table>
  312. <thead>
  313. <tr><th>Surface</th><th>State</th><th>Consequence</th></tr>
  314. </thead>
  315. <tbody>
  316. <tr><td>TLS certificates</td><td>Not verified</td><td>Encryption without authentication; MITM is undetected.</td></tr>
  317. <tr><td>TLS DRBG</td><td>Weak (RDTSC/ticks/MAC)</td><td>Not suitable for high-value secrets.</td></tr>
  318. <tr><td>WebDAV</td><td>No authentication</td><td>Anyone on the LAN can read mounts and write <code>/share</code>.</td></tr>
  319. <tr><td>NTP</td><td>Unauthenticated SNTP</td><td>Anyone who can answer for the server name can set the clock.</td></tr>
  320. <tr><td>NIC ownership</td><td>Raw lease for packet commands; shared TCP client table</td><td>Raw commands are mutually exclusive; up to six processes can hold one TCP connection each. A killed owner is force-released.</td></tr>
  321. <tr><td>Kernel/user isolation</td><td>None (identity map)</td><td>A ring-3 bug can reach any mapped kernel page.</td></tr>
  322. </tbody>
  323. </table>
  324. </div>
  325. </section>
  326. <section id="sources">
  327. <h2>Source map</h2>
  328. <ul class="filelist">
  329. <li><code>netdev.c</code>, <code>netdev.h</code> — registry</li>
  330. <li><code>e1000e.c</code>, <code>r8168.c</code>, <code>ne2000.c</code>, <code>asix.c</code></li>
  331. <li><code>net.c</code>, <code>net.h</code> — ARP cache/ICMP/UDP/IP</li>
  332. <li><code>netcore.c</code>, <code>netcore.h</code> — shared receive pump and IRQ helpers</li>
  333. <li><code>dhcp.c</code>, <code>dns.c</code>, <code>ntp.c</code></li>
  334. <li><code>tcp.c</code>, <code>tcp.h</code> — client table + listener</li>
  335. <li><code>tls.c</code>, <code>tls_*.c</code> — TLS 1.3</li>
  336. <li><code>sys_net.c</code> — ring-3 service</li>
  337. <li><code>netlog.c</code>, <code>netkexec.c</code></li>
  338. <li><code>uhttp.c</code>, <code>ucmd_curl.c</code>, <code>ucmd_websrv.c</code></li>
  339. </ul>
  340. </section>
  341. </main>
  342. <footer class="site-footer">
  343. <div class="inner">
  344. <p><strong>NebOS 0.2.0</strong> · Architecture documentation · <a href="index.html">overview</a></p>
  345. <p>Maintain alongside <code>AGENTS.md</code> and the source files listed on each page.</p>
  346. </div>
  347. </footer>
  348. </body>
  349. </html>