filesystems.html 18 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400
  1. <!doctype html>
  2. <html lang="en">
  3. <head>
  4. <meta charset="utf-8">
  5. <meta name="viewport" content="width=device-width, initial-scale=1">
  6. <title>Storage &amp; Filesystems | NebOS Architecture</title>
  7. <meta name="description" content="NebOS filesystems: the VFS fs_ops layer, FAT12/16/32, the command-pack ramfs pipeline, the writable /share filesystem and the WebDAV view of the VFS.">
  8. <link rel="icon" href="favicon.svg" type="image/svg+xml">
  9. <link rel="stylesheet" href="style.css">
  10. </head>
  11. <body>
  12. <a class="skip" href="#main">Skip to content</a>
  13. <header class="site-header">
  14. <a class="brand" href="index.html">
  15. <span class="brand-mark" aria-hidden="true"></span>
  16. <span>NebOS <span class="brand-sub">Architecture</span></span>
  17. </a>
  18. <nav class="site-nav" aria-label="Architecture sections">
  19. <a href="index.html">Overview</a>
  20. <a href="boot.html">Boot</a>
  21. <a href="memory.html">Memory</a>
  22. <a href="cpu.html">CPU</a>
  23. <a href="drivers.html">Drivers</a>
  24. <a href="filesystems.html" aria-current="page">Filesystems</a>
  25. <a href="graphics.html">Graphics</a>
  26. <a href="network.html">Network</a>
  27. <a href="userland.html">Userland</a>
  28. <a href="development.html">Development</a>
  29. </nav>
  30. </header>
  31. <main id="main" class="page">
  32. <div class="page-head">
  33. <p class="kicker">Storage &amp; filesystems</p>
  34. <h1>One VFS, three very different backends.</h1>
  35. <p class="lede">
  36. The kernel keeps filesystem policy thin: a mount table and an
  37. <code>fs_ops</code> vtable. FAT provides persistent read/write storage,
  38. the command-pack ramfs is a read-only system image, and
  39. <code>/share</code> is a writable RAM filesystem for network uploads.
  40. </p>
  41. <div class="meta">
  42. <span class="pill">FAT12/16/32</span>
  43. <span class="pill">read-only ramfs</span>
  44. <span class="pill">/share · 16 MB</span>
  45. <span class="pill">WebDAV</span>
  46. </div>
  47. </div>
  48. <nav class="toc" aria-label="On this page">
  49. <h2>On this page</h2>
  50. <ul>
  51. <li><a href="#vfs">VFS layer</a></li>
  52. <li><a href="#fat">FAT</a></li>
  53. <li><a href="#ramfs">ramfs &amp; the command pack</a></li>
  54. <li><a href="#share">/share</a></li>
  55. <li><a href="#sysfs">File service ABI</a></li>
  56. <li><a href="#webdav">WebDAV</a></li>
  57. <li><a href="#capacity">Capacity chart</a></li>
  58. <li><a href="#limits">Limits</a></li>
  59. <li><a href="#sources">Source map</a></li>
  60. </ul>
  61. </nav>
  62. <section id="vfs">
  63. <h2>VFS layer</h2>
  64. <p>
  65. <span class="file">vfs.c</span> mounts every detected FAT device under
  66. <code>/</code>, keeps the two memory filesystems available throughout
  67. the boot scan, and dispatches file operations through
  68. <code>struct fs_ops</code>. The shell, the file service and the
  69. WebDAV server never call <code>fat_*</code> directly.
  70. </p>
  71. <figure class="figure">
  72. <svg viewBox="0 0 920 330" role="img" aria-labelledby="vfs-title vfs-desc">
  73. <title id="vfs-title">VFS and filesystem backends</title>
  74. <desc id="vfs-desc">The shell, sys_fs and WebDAV use the VFS mount table, which dispatches to FAT, ramfs or share through fs_ops. FAT sits on the block-device registry.</desc>
  75. <rect class="svg-box blue" x="30" y="30" width="200" height="60" rx="9"/>
  76. <text class="svg-text" x="50" y="56">shell / commands</text>
  77. <text class="svg-sub" x="50" y="76">ls, cat, save, files</text>
  78. <rect class="svg-box blue" x="255" y="30" width="200" height="60" rx="9"/>
  79. <text class="svg-text" x="275" y="56">sys_fs · WebDAV</text>
  80. <text class="svg-sub" x="275" y="76">ring-3 file service</text>
  81. <rect class="svg-box accent" x="485" y="30" width="200" height="60" rx="9"/>
  82. <text class="svg-text" x="505" y="56">vfs.c</text>
  83. <text class="svg-sub" x="505" y="76">mount table · fs_ops</text>
  84. <path class="svg-line blue" d="M230 60h18"/>
  85. <path class="svg-line blue" d="M240 54l7 6-7 6"/>
  86. <path class="svg-line blue" d="M455 60h23"/>
  87. <path class="svg-line blue" d="M470 54l7 6-7 6"/>
  88. <rect class="svg-box green" x="712" y="30" width="180" height="60" rx="9"/>
  89. <text class="svg-text" x="732" y="56">directory cache</text>
  90. <text class="svg-sub" x="732" y="76">last 64 FAT entries</text>
  91. <path class="svg-line accent" d="M685 60h20"/>
  92. <path class="svg-line accent" d="M697 54l7 6-7 6"/>
  93. <rect class="svg-box" x="30" y="150" width="200" height="64" rx="9"/>
  94. <text class="svg-text" x="50" y="176">FAT12/16/32</text>
  95. <text class="svg-sub" x="50" y="196">read + write · 8.3 names</text>
  96. <rect class="svg-box" x="255" y="150" width="200" height="64" rx="9"/>
  97. <text class="svg-text" x="275" y="176">ramfs</text>
  98. <text class="svg-sub" x="275" y="196">read-only · command pack</text>
  99. <rect class="svg-box" x="485" y="150" width="200" height="64" rx="9"/>
  100. <text class="svg-text" x="505" y="176">/share</text>
  101. <text class="svg-sub" x="505" y="196">writable · RAM only</text>
  102. <rect class="svg-box" x="30" y="252" width="200" height="58" rx="9"/>
  103. <text class="svg-text" x="50" y="277">block devices</text>
  104. <text class="svg-sub" x="50" y="297">floppy · ide · sata</text>
  105. <path class="svg-line" d="M585 90v28"/>
  106. <path class="svg-line" d="M130 118h455"/>
  107. <path class="svg-line" d="M130 118v24"/>
  108. <path class="svg-line" d="M355 118v24"/>
  109. <path class="svg-line" d="M585 118v24"/>
  110. <path class="svg-line" d="M124 143l6 7 6-7"/>
  111. <path class="svg-line" d="M349 143l6 7 6-7"/>
  112. <path class="svg-line" d="M579 143l6 7 6-7"/>
  113. <path class="svg-line" d="M130 214v26"/>
  114. <path class="svg-line" d="M124 240l6 7 6-7"/>
  115. </svg>
  116. <figcaption>
  117. <strong>Same operations, different storage.</strong> Each backend
  118. implements read, load/release, stat, write, unlink, mkdir, rmdir and
  119. index-based directory listing. Paths passed to the backend are
  120. relative to the mount; the VFS has already normalised them.
  121. </figcaption>
  122. </figure>
  123. <p>
  124. <code>vfs_resolve()</code> drops <code>.</code> components, collapses
  125. and trims slashes, and resolves <code>..</code> against the mount.
  126. The shell's working directory is per-process and inherited across
  127. commands, so relative paths keep working after <code>cd</code>.
  128. </p>
  129. </section>
  130. <section id="fat">
  131. <h2>FAT</h2>
  132. <p>
  133. <span class="file">fat.c</span> supports FAT12, FAT16 and FAT32 on
  134. any registered block device. It reads the BPB, validates the media
  135. byte and mounts partitions as separate entries such as
  136. <code>ide0p1</code> or <code>sata0p2</code>.
  137. </p>
  138. <ul>
  139. <li><strong>Read/write.</strong> Read, write, delete, mkdir and rmdir are implemented; FAT32 allocation uses an FSInfo-seeded free-cluster hint.</li>
  140. <li><strong>8.3 names.</strong> The on-disk name is the canonical one. The shell and <code>files</code> display the space-padded form; a long name cannot be created.</li>
  141. <li><strong>Directory cache.</strong> The last FAT directory's first 64 entries are cached in <code>vfs.c</code>, so WebDAV's index sequence is one walk instead of N. Any write invalidates the cache.</li>
  142. <li><strong>Serialization.</strong> A FAT operation sets <code>sys_fs_busy</code>; the scheduler does not switch processes during it because the driver has static state and the floppy needs IRQ 6.</li>
  143. </ul>
  144. <p>
  145. <span class="file">fatdetect.c</span> and <span class="file">fatformat.c</span>
  146. back the <code>format</code> command. A disk can be formatted in
  147. binary units (<code>format ide0 100mb</code>, <code>1gb</code>) or
  148. whole-disk.
  149. </p>
  150. <div class="warn">
  151. <p><strong>FAT32 limits.</strong> The FSInfo free-count is not updated,
  152. and a completely full FAT32 directory is not grown: new cluster
  153. allocation for directories is not implemented.</p>
  154. </div>
  155. </section>
  156. <section id="ramfs">
  157. <h2>ramfs &amp; the command pack</h2>
  158. <p>
  159. The system root is a compressed command pack, not a disk image. Every
  160. ring-3 command is a freestanding ELF linked at <code>0x40000000</code>,
  161. packed as <code>bin/&lt;name&gt;</code>, gzipped and wrapped in a
  162. one-sector <code>NBROOT1</code> header. At boot the path loads the
  163. pack into RAM and <span class="file">ramfs.c</span> inflates it into a
  164. private page-allocated copy mounted read-only at <code>/ramfs</code>.
  165. </p>
  166. <figure class="figure">
  167. <svg viewBox="0 0 920 312" role="img" aria-labelledby="pack-title pack-desc">
  168. <title id="pack-title">Command pack build and boot pipeline</title>
  169. <desc id="pack-desc">ucmd ELFs are packed by mkpack, gzipped, wrapped by mkrootfs, loaded by the boot path into RAM, then inflated by ramfs_init and mounted at /ramfs.</desc>
  170. <rect class="svg-box" x="20" y="26" width="150" height="64" rx="9"/>
  171. <text class="svg-text" x="40" y="52">ucmd_*.c</text>
  172. <text class="svg-sub" x="40" y="72">42 UCMDS + 3 app ELFs</text>
  173. <rect class="svg-box" x="196" y="26" width="150" height="64" rx="9"/>
  174. <text class="svg-text" x="216" y="52">mkpack</text>
  175. <text class="svg-sub" x="216" y="72">NBPAK1 · bin/name</text>
  176. <rect class="svg-box" x="372" y="26" width="150" height="64" rx="9"/>
  177. <text class="svg-text" x="392" y="52">gzip -12</text>
  178. <text class="svg-sub" x="392" y="72">cmds_pack_gz.bin</text>
  179. <rect class="svg-box" x="548" y="26" width="150" height="64" rx="9"/>
  180. <text class="svg-text" x="568" y="52">mkrootfs</text>
  181. <text class="svg-sub" x="568" y="72">NBROOT1 header</text>
  182. <rect class="svg-box green" x="724" y="26" width="174" height="64" rx="9"/>
  183. <text class="svg-text" x="744" y="52">boot path</text>
  184. <text class="svg-sub" x="744" y="72">pack_addr / pack_size</text>
  185. <path class="svg-line" d="M170 58h16"/>
  186. <path class="svg-line" d="M180 52l7 6-7 6"/>
  187. <path class="svg-line" d="M346 58h16"/>
  188. <path class="svg-line" d="M356 52l7 6-7 6"/>
  189. <path class="svg-line" d="M522 58h16"/>
  190. <path class="svg-line" d="M532 52l7 6-7 6"/>
  191. <path class="svg-line green" d="M698 58h16"/>
  192. <path class="svg-line green" d="M708 52l7 6-7 6"/>
  193. <rect class="svg-box" x="390" y="150" width="170" height="64" rx="9"/>
  194. <text class="svg-text" x="410" y="176">ramfs_init</text>
  195. <text class="svg-sub" x="410" y="196">inflate + validate</text>
  196. <rect class="svg-box accent" x="620" y="150" width="200" height="64" rx="9"/>
  197. <text class="svg-text" x="640" y="176">/ramfs</text>
  198. <text class="svg-sub" x="640" y="196">read-only · bin/command</text>
  199. <path class="svg-line green" d="M810 90v40h-335v20"/>
  200. <path class="svg-line green" d="M469 142l6 7 6-7"/>
  201. <path class="svg-line" d="M560 182h52"/>
  202. <path class="svg-line" d="M606 176l7 6-7 6"/>
  203. <text class="svg-tiny" x="20" y="272">Runtime replacement: KXNP packs replace /ramfs in place.</text>
  204. <text class="svg-tiny" x="20" y="290">Existing processes keep their loaded ELFs.</text>
  205. </svg>
  206. <figcaption>
  207. <strong>The pack is the root filesystem.</strong> The kernel no
  208. longer embeds it, so command growth does not move <code>_end</code>.
  209. The legacy boot path still caps the compressed pack at about 255 KB;
  210. UEFI and kexec can carry more.
  211. </figcaption>
  212. </figure>
  213. <div class="table-wrap">
  214. <table>
  215. <thead>
  216. <tr><th>Container</th><th>Format</th><th>Purpose</th></tr>
  217. </thead>
  218. <tbody>
  219. <tr><td><code>cmds.pack</code></td><td><code>NBPAK1</code></td><td>Raw command archive: <code>bin/&lt;name&gt;</code> entries.</td></tr>
  220. <tr><td><code>cmds_pack_gz.bin</code></td><td>gzip</td><td>Compressed pack; <code>libdeflate-gzip -12</code> when available.</td></tr>
  221. <tr><td><code>rootfs.bin</code></td><td><code>NBROOT1</code> + gzip</td><td>Floppy append (header + pack).</td></tr>
  222. <tr><td><code>cmds.img</code></td><td><code>NBROOT1</code> padded to 8 MB</td><td>USB rootfs partition image.</td></tr>
  223. <tr><td><code>/share</code></td><td>RAM entries</td><td>Writable upload target; not part of the pack.</td></tr>
  224. </tbody>
  225. </table>
  226. </div>
  227. <p>
  228. The pack build is part of the normal Makefile. Add a
  229. <code>ucmd_&lt;name&gt;.c</code> to <code>UCMDS</code> and the pack,
  230. the mount and the PATH listing pick it up automatically. Runtime
  231. replacement uses <code>make cmds-dev</code> or
  232. <code>make net-cmds</code> and validates the pack before swapping it.
  233. </p>
  234. </section>
  235. <section id="share">
  236. <h2>/share</h2>
  237. <p>
  238. <span class="file">share.c</span> is a writable, page-backed in-memory
  239. filesystem for network uploads. It matters on boards with no writable
  240. disk, or when the only writable medium is a floppy. Entries are full
  241. relative paths plus explicit directory entries, so empty directories
  242. work; names are plain strings up to 63 bytes.
  243. </p>
  244. <ul>
  245. <li><strong>Limits.</strong> 64 entries and 16 MB total, allocated as pages and freed on replace/delete.</li>
  246. <li><strong>Volatile.</strong> A reboot or kexec starts empty.</li>
  247. <li><strong>Mount point.</strong> <code>/share</code>, listed by <code>ls /</code> and visible in <code>files</code>.</li>
  248. <li><strong>Writable by anyone on the LAN.</strong> The WebDAV server has no authentication.</li>
  249. </ul>
  250. </section>
  251. <section id="sysfs">
  252. <h2>File service ABI</h2>
  253. <p>
  254. Ring-3 commands use <code>SYS_READ_FILE</code>,
  255. <code>SYS_WRITE_FILE</code>, <code>SYS_LIST_DIR</code>,
  256. <code>SYS_LIST_MOUNTS</code>, <code>SYS_UNLINK</code>,
  257. <code>SYS_MKDIR</code>, <code>SYS_RMDIR</code>,
  258. <code>SYS_FS_STAT</code> and <code>SYS_GETCWD</code>.
  259. <span class="file">sys_fs.c</span> validates the path, resolves it
  260. against the caller's cwd, calls the VFS and maps backend errors to
  261. the stable <code>SYS_E*</code> values.
  262. </p>
  263. <p>
  264. The file service reads whole files, not streams. The shell loads an
  265. ELF through <code>vfs_load_file</code> and spawns it; FAT ELFs are
  266. loaded through <code>kmalloc</code> under <code>sys_fs_enter/exit</code>,
  267. since the floppy needs IRQ 6.
  268. </p>
  269. <p>
  270. Directory entries carry a 64-byte display name plus <code>raw_83</code>,
  271. so FAT's space-padded names and the string-named memory filesystems
  272. share one ABI.
  273. </p>
  274. </section>
  275. <section id="webdav">
  276. <h2>WebDAV</h2>
  277. <p>
  278. <span class="file">websrv</span> is a ring-3 HTTP/1.1 + WebDAV server
  279. built on the kernel TCP listener. It serves the VFS:
  280. <code>/</code> lists mounts, <code>/ramfs</code> is read-only, FAT
  281. mounts accept only names that round-trip to 8.3, and
  282. <code>/share</code> is the writable target.
  283. </p>
  284. <ul>
  285. <li><strong>Concurrency.</strong> One active connection plus a small backlog (three backlog/closing slots in the current TCP server). Parallel clients queue with a zero window until the active request finishes.</li>
  286. <li><strong>Bodies.</strong> Whole-file buffering, 4 MB normally and 1 MB when the heap cannot provide the larger buffer. A larger PUT gets 413.</li>
  287. <li><strong>Methods.</strong> GET, PUT, DELETE, MKCOL, PROPFIND and MOVE to the extent described; directory <code>MOVE</code> is 501 and <code>PROPPATCH</code> is answered 200 without persisting.</li>
  288. <li><strong>No auth.</strong> This is a trusted-LAN service; <code>/share</code> is writable by anyone who can reach it.</li>
  289. </ul>
  290. <p>
  291. The WebDAV server and the packet-level <code>ping</code>/<code>curl</code>/
  292. <code>netrecv</code> commands are mutually exclusive: both need the
  293. single NIC receiver. Starting <code>websrv</code> while another
  294. command owns the receiver fails.
  295. </p>
  296. </section>
  297. <section id="capacity">
  298. <h2>Capacity chart</h2>
  299. <figure class="figure">
  300. <div class="bars" role="img" aria-label="Capacity limits: /share 16 MB, ramfs decompressed pack 8 MB, USB rootfs image 8 MB, legacy compressed pack about 255 KB. Bar widths are relative to 16 MB.">
  301. <div class="bar-row">
  302. <span class="bar-label">/share</span>
  303. <div class="bar-track"><span class="bar-fill green" style="--w:100%"></span></div>
  304. <span class="bar-value">16 MB</span>
  305. </div>
  306. <div class="bar-row">
  307. <span class="bar-label">ramfs (inflated)</span>
  308. <div class="bar-track"><span class="bar-fill" style="--w:50%"></span></div>
  309. <span class="bar-value">8 MB</span>
  310. </div>
  311. <div class="bar-row">
  312. <span class="bar-label">rootfs image</span>
  313. <div class="bar-track"><span class="bar-fill blue" style="--w:50%"></span></div>
  314. <span class="bar-value">8 MB</span>
  315. </div>
  316. <div class="bar-row">
  317. <span class="bar-label">legacy pack</span>
  318. <div class="bar-track"><span class="bar-fill" style="--w:1.56%"></span></div>
  319. <span class="bar-value">~255 KB</span>
  320. </div>
  321. </div>
  322. <figcaption>
  323. <strong>Different limits for different paths.</strong> The legacy
  324. boot stage's compressed-pack cap is much smaller than the 8 MB
  325. ramfs cap; UEFI and kexec avoid the low-gap bottleneck.
  326. </figcaption>
  327. </figure>
  328. </section>
  329. <section id="limits">
  330. <h2>Limits</h2>
  331. <ul>
  332. <li><strong>Whole-file service.</strong> Commands read a complete file into memory; there is no stream API for general files.</li>
  333. <li><strong>FAT names.</strong> No long filename creation or lookup. WebDAV refuses names that cannot be represented exactly in 8.3.</li>
  334. <li><strong>ramfs is read-only.</strong> Writes return <code>SYS_EROFS</code>; use <code>/share</code> for mutable data.</li>
  335. <li><strong>Pack volatile at runtime.</strong> Runtime pack replacement is RAM only; a cold boot uses the flashed rootfs.</li>
  336. <li><strong>Directory listing cache.</strong> Only the first 64 entries of the last FAT directory are cached; a very large directory still costs a walk.</li>
  337. </ul>
  338. </section>
  339. <section id="sources">
  340. <h2>Source map</h2>
  341. <ul class="filelist">
  342. <li><code>vfs.c</code>, <code>vfs.h</code> — mounts and dispatch</li>
  343. <li><code>fat.c</code>, <code>fat.h</code> — FAT12/16/32</li>
  344. <li><code>ramfs.c</code>, <code>ramfs.h</code> — pack inflate</li>
  345. <li><code>share.c</code>, <code>share.h</code> — /share</li>
  346. <li><code>sys_fs.c</code> — ring-3 file service</li>
  347. <li><code>mkpack.c</code>, <code>mkrootfs.c</code>, <code>rootfs.h</code></li>
  348. <li><code>inflate.c</code> — DEFLATE/gzip</li>
  349. <li><code>ucmd_websrv.c</code> — HTTP/WebDAV</li>
  350. <li><code>fatdetect.c</code>, <code>fatformat.c</code> — format</li>
  351. </ul>
  352. </section>
  353. </main>
  354. <footer class="site-footer">
  355. <div class="inner">
  356. <p><strong>NebOS 0.2.0</strong> · Architecture documentation · <a href="index.html">overview</a></p>
  357. <p>Maintain alongside <code>AGENTS.md</code> and the source files listed on each page.</p>
  358. </div>
  359. </footer>
  360. </body>
  361. </html>