Parcourir la source

rootfs: raise the compressed pack cap to 4 MB and reloads to 8 MB

PACK_MAX (vesa_stage.S), CMDS_GZ_MAX (Makefile) and the embedded-pack
check in the UEFI loader now allow a 4 MB gzip pack; the legacy copy
ends at 0xD00000, inside the 16 MB no-E820 fallback.  KEXEC_MAX,
PAYLOAD_MAX and netkexec_send take up to 8 MB, the ramfs cap, so the
4.35 MB raw pack of cmds-dev/net-cmds is accepted again.

Tested in QEMU: a 4.15 MB pack boots on the legacy and UEFI paths and
over net-dev, the raw pack goes through net-cmds, and make test-install
passes.
Johannes Zellner il y a 1 jour
Parent
commit
ccb4fdc5ba
10 fichiers modifiés avec 48 ajouts et 31 suppressions
  1. 27 20
      AGENTS.md
  2. 4 1
      Makefile
  3. 1 1
      boot/efi/main.c
  4. 5 1
      boot/vesa_stage.S
  5. 4 3
      docs/boot.html
  6. 1 1
      docs/development.html
  7. 1 1
      docs/memory.html
  8. 3 1
      kernel/kexec.h
  9. 1 1
      kernel/payload.c
  10. 1 1
      tools/netkexec_send.c

+ 27 - 20
AGENTS.md

@@ -130,7 +130,7 @@ make clean
 - Build flags (in `Makefile`): `-m32 -march=i586 -ffreestanding -fno-pie -nostdlib
   -fno-stack-protector -mno-sse -mno-sse2 -mno-mmx -O2`.  The ring-3 WM and
   viewer are per-object `-Os` with function sections, like the other size-
-  sensitive GUIs, because the 2 MB gzip rootfs pack is close to its cap.
+  sensitive GUIs, to keep the gzip rootfs pack small (its cap is 4 MB).
 - `make browser-host` is a development-only harness, not part of the images: it
   builds `browser.c`, ngui, the image/TLS stack and QuickJS as a 32-bit Linux
   program against the same `SYS_*` ABI (`host/syscall.c`, an SDL3 window,
@@ -228,7 +228,8 @@ and `make net-dev`/`kernel-net-dev` reload over the network.  `bootlog save
    12-dword `struct boot_info` (magic/checksum/rsdp/flags/pack range) to
    `0x500`, enters protected mode and jumps to `kernel_entry` (`entry.S`,
    re-points ESP to the current stack top, then `kernel_main`).  The pack
-   never occupies the low gap, so the cap is `PACK_MAX` (2 MB) and the
+   never occupies the low gap, so the cap is `PACK_MAX` (4 MB, copied to
+   `0x900000..0xD00000`) and the
    kernel BSS/boot stack may extend past `0x60000`.  The stage prints
    `Loading rootfs [d=<drive> lba=<hex> e=<0|1> m=<C|L>]...`, then a dot
    after the header and one per 16 KB chunk, so a BIOS hang is localised.
@@ -293,7 +294,7 @@ a Pentium 1 (and a Pentium II board without ACPI) on the UP path.
 | `0x60000` | No longer a rootfs staging area: the legacy stage copies each 16 KB chunk straight to `0x900000`.  The kernel BSS/boot stack may use this range |
 | `_end`    | Kernel end (linker symbol; `kernel.ld` keeps `_end <= 0x90000` so the top of the low-memory gap and the EBDA stay untouched).  BSS and the boot stack may extend past `0x60000` |
 | (in-image) | Bootstrap/protected-mode stack: 8 KB `.stack` NOLOAD section, `__boot_stack_top`/`__boot_stack_bottom` (travels with the kernel) |
-| `0x900000`| Legacy boot-time rootfs pack written directly by `vesa_stage.S` (reserved by `mm_init` until `ramfs_init` takes its copy, then freed); `mm_init` places the frame bitmap in the first RAM after it (`0xAF3000` with today's pack); process stacks and kexec staging also start here |
+| `0x900000`| Legacy boot-time rootfs pack written directly by `vesa_stage.S` (reserved by `mm_init` until `ramfs_init` takes its copy, then freed); `mm_init` places the frame bitmap in the first RAM after it (`0xB02000` with today's 2.1 MB pack, at most `0xD00000`); process stacks and kexec staging also start here |
 | `0x100000`| VGA shadow buffer (`VGA_SHADOW_BASE`, `vga.h`) |
 
 The legacy stage uses `0x3000..0x6FFF` as a 16 KB-chunk rootfs staging
@@ -2474,8 +2475,9 @@ The commands live in `cmds.pack` (`bin/<name>` entries, built by `mkpack`),
 compressed with `libdeflate-gzip -12` when it is installed (falling back to
 `gzip -9`) to `cmds_pack_gz.bin` and wrapped by `mkrootfs` with a
 one-sector `NBROOT1` header (magic, version, exact `pack_bytes`, checksum,
-gzip flag).  The build rejects a pack above the `PACK_MAX` cap (2 MB,
-`CMDS_GZ_MAX` in the Makefile, `PACK_MAX` in `vesa_stage.S`);
+gzip flag).  The build rejects a pack above the `PACK_MAX` cap (4 MB,
+`CMDS_GZ_MAX` in the Makefile, `PACK_MAX` in `vesa_stage.S`, the embedded
+cap in `boot/efi/main.c`; all three must match);
 `libdeflate` buys several KB over `gzip -9` and is still preferred.  The boot
 path loads the gz pack into RAM — the legacy stage copies it directly to
 `0x900000` with its protected-mode `rep movsb`, the UEFI loader into a
@@ -2483,7 +2485,10 @@ loader-data buffer below 512 MB, kexec via its staging buffer — and
 `ramfs_init()`
 inflates it into page-allocated memory and mounts it read-only at `/ramfs`.
 The kernel image no longer contains it, so the pack no longer moves `_end`;
-the high-memory legacy path supports up to 2 MB (known issue 8).  The UI
+the high-memory legacy path supports up to 4 MB (known issue 8).  Hot
+reloads stage kernel + gzip pack, or a raw runtime pack (`cmds-dev`/
+`net-cmds`, up to the 8 MB ramfs cap), in one buffer of at most `KEXEC_MAX`
+(8 MB, `kexec.h`; `PAYLOAD_MAX` in `payload.c` and `netkexec_send.c` match).  The UI
 font subsets live in the pack under `fonts/`, the WiFi firmware under
 `firmware/`, the mouse cursors at `cursors/adwaita24.cur` (full pack only),
 the UI icons at `icons/adwaita16.icn` (both packs) and the flat kernel at `install/kernel.bin` (full pack only;
@@ -2991,19 +2996,21 @@ and the WM falls back to polling (`io_keyboard_irq_masked()`).
    load (the USB rootfs partition is 8 MB, so this only bites if the cap is
    raised without a bigger partition).  The legacy stage copies the
    compressed pack directly to `0x900000` with a protected-mode
-   `rep movsb`, so the cap is `PACK_MAX` 2 MB (raised from 1 MB for the WiFi
-   firmware).  The browser and
+   `rep movsb`, so the cap is `PACK_MAX` 4 MB (raised from 1 MB for the WiFi
+   firmware, then from 2 MB when the browser grew; it ends at `0xD00000`,
+   inside the 16 MB no-E820 fallback).  A 4.15 MB pack was boot-tested on
+   the legacy and UEFI paths and over `net-dev`.  The browser and
    its ring-3 dependencies are compiled with function sections and linked
    with `--gc-sections`; the current full pack with the UI fonts, both
    WiFi firmwares (7265D 1,036,312 bytes, ~442 KiB gzip; 6000 454,608 bytes,
    ~204 KiB gzip), the RTL8821A firmware and tables (31,898 + 11,668
-   bytes) and `install/kernel.bin` is ~2.00 MiB gzip (~4 KB under the
-   cap), inflating to ~4.13 MB, which adds about
+   bytes) and `install/kernel.bin` is ~2.0 MiB gzip (~2 MB under the
+   cap), inflating to ~4.35 MB, which adds about
    half a second to the boot; the floppy lite pack is ~641 KiB gzip and
    deliberately omits the firmware, `install/kernel.bin` and the browser
    (`BROWSER` on `root.img` is the floppy copy).  `libdeflate-gzip -12` is still preferred.  Over
-   115200 serial a raw `cmds-dev` pack (4.1 MiB) now takes about four
-   minutes; use `net-cmds`.
+   115200 serial a raw `cmds-dev` pack (4.3 MiB) now takes about four
+   minutes; use `net-cmds` (~45 s in QEMU).
 9. SMP is **ACPI/MADT only**: no Intel MP-table fallback, so a Pentium II SMP
     board that exposes only MP tables stays UP (deliberate; there is no test
     hardware).
@@ -3132,15 +3139,8 @@ and the WM falls back to polling (`io_keyboard_irq_masked()`).
     background layers, `background-attachment`, 3D/perspective transforms
     and transformed text/descendants (`scale`/`rotate` paint the element box
     and inline images; layout geometry is unchanged), border-radius clipping
-    of descendant content, gradients/box-shadow/transitions (a gradient
+    of descendant content, gradients/box-shadow (a gradient
     background falls back to its last solid colour), or CSS `color-mix`.
-    A non-uniform `background-size` uses the width scale for both axes.
-    CSS custom properties (`var()`: one document-wide table, expanded when
-    a sheet is parsed; definitions on `:root`/`html`/`body`/`*`/`:host`
-    win, a component-level definition applies only while no other rule
-    gives the name a different value, otherwise `var()` uses its fallback
-    -- Font Awesome's competing `--fa-width` values made every icon 100%
-    wide; rules touching such names are re-expanded per element against
     Transitions animate only `opacity` (painted only on absolute/fixed
     boxes; in-flow translucency paints opaque) and `translate()`; colours,
     sizes, `scale`/`rotate`, `@keyframes` animations, per-property
@@ -3148,6 +3148,13 @@ and the WM falls back to polling (`io_keyboard_irq_masked()`).
     `transitionstart` are not supported, and every animation frame is a
     full page re-render (~25 ms on the host, several times that on the
     ION, so slides are choppy there).
+    A non-uniform `background-size` uses the width scale for both axes.
+    CSS custom properties (`var()`: one document-wide table, expanded when
+    a sheet is parsed; definitions on `:root`/`html`/`body`/`*`/`:host`
+    win, a component-level definition applies only while no other rule
+    gives the name a different value, otherwise `var()` uses its fallback
+    -- Font Awesome's competing `--fa-width` values made every icon 100%
+    wide; rules touching such names are re-expanded per element against
     that element's own matched rules, but values are not inherited from
     ancestors' rules), percentage heights (against the containing block's definite
     height, `lay.cb_h`, with the viewport for `html`/`body`; used as a

+ 4 - 1
Makefile

@@ -919,7 +919,10 @@ cmds-lite.pack: $(PACK_DEPS)
 # floppy images use the firmware-less lite pack (rootfs.bin).
 CMDS_GZ     = $(shell command -v libdeflate-gzip >/dev/null 2>&1 && \
 		     echo "libdeflate-gzip -12 -c" || echo "gzip -9 -n -c")
-CMDS_GZ_MAX = 2097152
+# The compressed rootfs cap: PACK_MAX in boot/vesa_stage.S (the legacy stage
+# copies the pack to 0x900000..0xD00000) and the embedded cap in
+# boot/efi/main.c must match.
+CMDS_GZ_MAX = 4194304
 cmds_lite_gz.bin: cmds-lite.pack
 	$(CMDS_GZ) $< > $@
 

+ 1 - 1
boot/efi/main.c

@@ -698,7 +698,7 @@ EFI_STATUS efi_main(EFI_HANDLE image, EFI_SYSTEM_TABLE *st)
 	embed_psize = (UINT32)(pack_image_end - pack_image_start);
 	if (embed_ksize == 0 || embed_ksize > KERNEL_MAX)
 		fail_before(st, "kernel image size invalid");
-	if (embed_psize == 0 || embed_psize > 0x400000u)
+	if (embed_psize == 0 || embed_psize > 0x400000u) /* PACK_MAX */
 		fail_before(st, "rootfs image size invalid");
 	kernel_size = embed_ksize;
 	pack_size = embed_psize;

+ 5 - 1
boot/vesa_stage.S

@@ -28,7 +28,11 @@
 
 /* Rootfs/boot-info constants (must match rootfs.h / bootinfo.h). */
 .equ ROOTFS_VERSION,   1
-.equ PACK_MAX,         0x200000      /* 2 MB compressed rootfs cap */
+.equ PACK_MAX,         0x400000      /* 4 MB compressed rootfs cap; also
+                                       * CMDS_GZ_MAX (Makefile) and the
+                                       * embedded cap in boot/efi/main.c.
+                                       * 0x900000..0xD00000 stays below the
+                                       * 16 MB no-E820 fallback. */
 .equ PACK_HIGH_ADDR,   0x900000      /* above the 8 MB VGA-shadow reserve */
 .equ BOOT_INFO_MAGIC,  0x4E425349
 .equ BOOT_INFO_VERSION, 2

+ 4 - 3
docs/boot.html

@@ -176,8 +176,9 @@
 		</ol>
 		<p>
 			The pack no longer occupies the old <code>0x60000..0x9FBFF</code>
-			low gap, so the legacy cap is <code>PACK_MAX</code> (2 MB, raised
-			from 1 MB for the WiFi firmware) and the
+			low gap, so the legacy cap is <code>PACK_MAX</code> (4 MB, raised
+			from 1 MB for the WiFi firmware and from 2 MB as the browser grew; the
+			copy ends at <code>0xD00000</code>) and the
 			kernel BSS/boot stack may use the gap. The kernel image only has to
 			stay below <code>0x90000</code> so the top of low memory and the
 			EBDA remain untouched; <span class="file">kernel.ld</span> enforces
@@ -444,7 +445,7 @@
 		<h2>Failure modes to know</h2>
 		<ul>
 			<li><strong>No VBE mode.</strong> The legacy stage cannot produce a framebuffer; the kernel may start headless or the boot stops in the stage.</li>
-			<li><strong>Pack larger than the cap.</strong> The legacy BIOS path caps the compressed pack at 2 MB <code>PACK_MAX</code> (<code>CMDS_GZ_MAX</code> in the Makefile); UEFI and kexec can carry larger packs up to the 8 MB ramfs cap. The WiFi firmware, the bundled install kernel and the UI assets make the full pack about 1.98 MB, close to the cap, and inflating it adds roughly half a second to the boot.</li>
+			<li><strong>Pack larger than the cap.</strong> The compressed pack is capped at 4 MB: <code>PACK_MAX</code> in <span class="file">vesa_stage.S</span>, <code>CMDS_GZ_MAX</code> in the Makefile and the embedded-pack check in <span class="file">boot/efi/main.c</span> must agree. Hot reloads stage kernel + pack, or a raw runtime pack, in at most <code>KEXEC_MAX</code> (8 MB, <span class="file">kexec.h</span>), matching the 8 MB ramfs cap. The full pack is about 2.0 MB today, and inflating it adds roughly half a second to the boot.</li>
 			<li><strong>PIT not ticking.</strong> <code>kernel_main</code> prints <code>*** PIT NOT TICKING ***</code> and halts; no process scheduling is possible.</li>
 			<li><strong>UEFI Secure Boot.</strong> The unsigned <code>BOOTX64.EFI</code> is rejected before any NebOS code runs.</li>
 			<li><strong>Stale boot info.</strong> The checksum makes an invalid <code>0x500</code> block read as absent rather than as a bogus pack or RSDP.</li>

+ 1 - 1
docs/development.html

@@ -330,7 +330,7 @@ make net-cmds                   # replace /ramfs only</code></pre>
 			<li><strong>Existing processes are unaffected.</strong> They already loaded their ELF.</li>
 			<li><strong>The next spawn sees the new files.</strong></li>
 			<li><strong>RAM only.</strong> A cold boot uses the flashed rootfs; reflash after changing the bundle.</li>
-			<li><strong>Size.</strong> The raw pack is about 2 MB with the WiFi firmware, so <code>cmds-dev</code> over 115200 serial takes roughly three minutes; prefer <code>net-cmds</code>.</li>
+			<li><strong>Size.</strong> The raw pack is about 4.3 MB with the WiFi firmware, so <code>cmds-dev</code> over 115200 serial takes roughly four minutes; prefer <code>net-cmds</code> (about 45 s in QEMU). Reloads carry up to <code>KEXEC_MAX</code> (8 MB).</li>
 		</ul>
 	</section>
 

+ 1 - 1
docs/memory.html

@@ -319,7 +319,7 @@
 	<section id="pack">
 		<h2>Rootfs pack lifetime</h2>
 		<ol>
-			<li>The compressed pack is at most 2 MB (<code>PACK_MAX</code>), so the legacy copy ends below <code>0xB00000</code>; inflated it is about 4.1 MB with the WiFi firmware (the 8 MB <code>RAMFS_MAX_DECOMPRESSED</code> cap applies).</li>
+			<li>The compressed pack is at most 4 MB (<code>PACK_MAX</code>), so the legacy copy ends below <code>0xD00000</code>, still inside the 16 MB no-E820 fallback; inflated it is about 4.35 MB with the WiFi firmware (the 8 MB <code>RAMFS_MAX_DECOMPRESSED</code> cap applies).</li>
 			<li>The boot path places the gzip pack in RAM: the legacy stage copies it directly to <code>0x900000</code> with a protected-mode <code>rep movsb</code>, UEFI into a loader-data buffer below 1 GB, kexec in its staging buffer.</li>
 			<li><code>mm_init</code> reserves <code>[pack_addr, pack_addr + pack_size)</code> so the frame allocator cannot reuse it.</li>
 			<li><code>ramfs_init</code> takes a private page-allocated copy, validates the <code>NBROOT1</code> header and inflates the pack into it.</li>

+ 3 - 1
kernel/kexec.h

@@ -9,7 +9,9 @@ void program_kexec_main(void);
  * fixed region below the VGA window, which forced the low-memory map to move
  * whenever the kernel grew. */
 #define KEXEC_LOAD  0x7E00
-#define KEXEC_MAX   0x400000
+/* Kernel + gzip rootfs (PACK_MAX is 4 MB) or a raw runtime pack, which
+ * ramfs caps at RAMFS_MAX_DECOMPRESSED (8 MB). */
+#define KEXEC_MAX   0x800000
 
 /* Allocate the staging buffer for an image of `size` bytes (contiguous,
  * page-aligned, identity-mapped).  Sets kexec_stage; returns 1 on success. */

+ 1 - 1
kernel/payload.c

@@ -21,7 +21,7 @@
 #include "vfs.h"
 
 #define PAYLOAD_MAGIC 0x504E584B /* "KXNP" little-endian */
-#define PAYLOAD_MAX   0x400000   /* sanity cap, like KEXEC_MAX */
+#define PAYLOAD_MAX   0x800000   /* raw pack; RAMFS_MAX_DECOMPRESSED, KEXEC_MAX */
 
 int payload_pending(void)
 {

+ 1 - 1
tools/netkexec_send.c

@@ -177,7 +177,7 @@ int main(int argc, char **argv)
 	if (pack_path)
 		pdata = read_file(pack_path, &psize);
 	size = ksize + psize;
-	if (size <= 0 || size > 0x400000) {   /* matches kexec.h KEXEC_MAX */
+	if (size <= 0 || size > 0x800000) {   /* matches kexec.h KEXEC_MAX */
 		fprintf(stderr, "netkexec_send: bad total size %ld\n", size);
 		return 1;
 	}