Explorar o código

Use the libc heap in websrv and term

The last two raw SYS_MEM_* users now allocate through libc.a: websrv's 4 MB transfer buffer and the terminal's scrollback rows come from malloc, and both commands link libc.a with --gc-sections.  Remove the u_mem_alloc/u_mem_grow/u_mem_free wrappers from ucmd_lib.h so application code can no longer create a second owner of the arena; the syscalls remain the libc backend.

Update the ownership comments in sys_mem.h and syscall.h, AGENTS.md and the userland/memory pages.  The pack is 248428 bytes compressed, 12691 bytes under the cap.

Verified with make test-libc, a clean make all, a QEMU boot showing the terminal, and websrv reporting its 4096 KB transfer buffer.
Johannes Zellner hai 1 semana
pai
achega
309b00a544
Modificáronse 9 ficheiros con 53 adicións e 63 borrados
  1. 16 15
      AGENTS.md
  2. 9 4
      Makefile
  3. 8 8
      docs/memory.html
  4. 4 4
      docs/userland.html
  5. 3 3
      sys_mem.h
  6. 3 3
      syscall.h
  7. 0 20
      ucmd_lib.h
  8. 6 4
      ucmd_term.c
  9. 4 2
      ucmd_websrv.c

+ 16 - 15
AGENTS.md

@@ -567,9 +567,10 @@ it to `0x7E00`.  The kernel region itself only has one wall (`_end` below
   `USER_HEAP_BASE` (`0x60000000`), capped at 32 MB; the initial allocation is
   zeroed but pages appended by `SYS_MEM_GROW` are not (decoders initialize
   what they use), which keeps large image arenas fast to map.  The reaper
-  frees it with the rest of the process and `top` counts its pages.  Commands
-  that link `libc.a` own this arena through `libc/malloc.c`; the raw syscalls
-  and libc malloc must never be mixed in one process.  The canvas mapping at
+  frees it with the rest of the process and `top` counts its pages.  Ring-3
+  commands use `libc/malloc.c` (malloc/free/heap_reset), which owns this
+  arena; the `SYS_MEM_*` calls are the libc backend (`libc/ucmd_mem.c`) and
+  are not called by application code.  The canvas mapping at
   `USER_GFX_BASE` (`0x50000000`) and the WM mappings are separate.
 - **Ring-3 libc**: `libc/include/` (stddef/stdint/stdarg/stdlib/string/ctype/
   assert/ucmd_heap), `libc/malloc.c`, `libc/string.c`, `libc/strdup.c`,
@@ -580,8 +581,8 @@ it to `0x7E00`.  The kernel region itself only has one wall (`_end` below
   grows in page-rounded steps with 64 KB slack up to the 32 MB arena cap.
   `heap_reset()` (`ucmd_heap.h`) discards every allocation and unmaps the
   arena for page-scoped apps such as the browser.  There are no threads, so
-  there is no lock; each process uses either this heap or the raw
-  `SYS_MEM_*` calls, never both.  The archive plus
+  there is no lock; application code uses this heap, and the libc backend is
+  the only caller of `SYS_MEM_*`.  The archive plus
   `-ffunction-sections`/`--gc-sections` keeps unused code out of the legacy
   pack.  Host test: `make test-libc` (`libc/test_malloc.c`).
 - **Ring-3 commands**: `ucmd_*.c`, `ucmd_lib.h`, `ucmd.ld`, `mkpack.c`,
@@ -1124,9 +1125,9 @@ and the WM falls back to polling (`io_keyboard_irq_masked()`).
 ## Conventions & gotchas
 
 - **Freestanding C** — the kernel has no libc, no `malloc`, no `printf`; use
-  `io_putstr` / `io_putchar` / `io_puthex` (`io.h`).  Ring-3 commands may
-  link the in-tree `libc.a` (`malloc`, strings, stdlib); a command either uses
-  libc malloc or the raw `SYS_MEM_*` arena, never both.
+  `io_putstr` / `io_putchar` / `io_puthex` (`io.h`).  Ring-3 commands link
+  the in-tree `libc.a` when they need allocation or standard string/stdlib
+  functions; the `SYS_MEM_*` syscalls are internal to the libc backend.
 - **Paging is on but flat** — identity-mapped 4 MB pages (`paging.c`), so
   physical == virtual everywhere. Still no per-process address spaces or process
   isolation: everything runs ring-0 and shares the address space.
@@ -1184,7 +1185,7 @@ and the WM falls back to polling (`io_keyboard_irq_masked()`).
    legacy path; a larger pack needs the UEFI/kexec path or a chunked
    high-memory loader.  The browser and its ring-3 dependencies are compiled
    with function sections and linked with `--gc-sections`; the current pack
-   is ~242 KiB (247493 bytes), leaving ~13 KB under the cap.  Command growth
+   is ~243 KiB (248428 bytes), leaving ~12 KB under the cap.  Command growth
    still needs care because `libdeflate-gzip -12` is already required (plain
    `gzip -9` is over the cap).
 9. SMP is **ACPI/MADT only**: no Intel MP-table fallback, so a Pentium II SMP
@@ -1317,9 +1318,9 @@ and the WM falls back to polling (`io_keyboard_irq_masked()`).
     pointer into the middle of a block or a stale pointer is silently
     ignored by `free()`.  The allocator is first-fit with immediate
     coalescing, so a pathological small/large lifetime mix can still
-    fragment; the browser and viewer use it for a handful of large,
-    page-scoped buffers.  A program must not mix `malloc` with raw
-    `u_mem_alloc`/`u_mem_grow`/`u_mem_free`, and near-32 MB requests fail
-    because the block headers are charged to the same cap.  The `libc.a`
-    members are only pulled into commands that reference them; the legacy
-    pack still needs every added byte checked against `CMDS_GZ_MAX`.
+    fragment; the terminal, browser, viewer, websrv and curl use it for a
+    handful of large buffers.  Commands allocate through `libc.a`; the raw
+    `SYS_MEM_*` calls are backend-only.  Near-32 MB requests fail because the
+    block headers are charged to the same cap.  The `libc.a` members are only
+    pulled into commands that reference them; the legacy pack still needs
+    every added byte checked against `CMDS_GZ_MAX`.

+ 9 - 4
Makefile

@@ -561,6 +561,11 @@ ucmd_%.elf: ucmd_%.o ucmd.ld
 	$(LD) -T ucmd.ld -z max-page-size=0x100 -nostdlib -m elf_i386 -o $@ $<
 	$(OBJCOPY) --strip-all $@ $@
 
+# websrv buffers its whole PUT/GET body on the libc heap.
+ucmd_websrv.elf: ucmd_websrv.o ucmd.ld libc.a
+	$(LD) -T ucmd.ld -z max-page-size=0x100 --gc-sections -nostdlib -m elf_i386 -o $@ $(filter-out ucmd.ld,$^)
+	$(OBJCOPY) --strip-all $@ $@
+
 # edit draws text itself, so it links the kernel's 8x8 font into its ELF.
 ucmd_edit.o: ucmd_edit.c ucmd_lib.h syscall.h font_8x16.h
 	$(CC) $(CFLAGS) -o $@ $<
@@ -580,11 +585,11 @@ ucmd_wm.elf: ucmd_wm.o uwm_lib.o font_8x16.o ucmd.ld
 	$(LD) -T ucmd.ld -z max-page-size=0x100 -nostdlib -m elf_i386 -o $@ $(filter %.o,$^)
 	$(OBJCOPY) --strip-all $@ $@
 
-ucmd_term.o: ucmd_term.c ucmd_lib.h syscall.h wm_abi.h uwm_lib.h io.h vfs.h
-	$(CC) $(CFLAGS) -o $@ $<
+ucmd_term.o: ucmd_term.c ucmd_lib.h syscall.h wm_abi.h uwm_lib.h io.h vfs.h $(LIBC_HEADERS)
+	$(CC) $(CFLAGS) -Ilibc/include -o $@ $<
 
-ucmd_term.elf: ucmd_term.o uwm_lib.o font_8x16.o ucmd.ld
-	$(LD) -T ucmd.ld -z max-page-size=0x100 -nostdlib -m elf_i386 -o $@ $(filter %.o,$^)
+ucmd_term.elf: ucmd_term.o uwm_lib.o font_8x16.o ucmd.ld libc.a
+	$(LD) -T ucmd.ld -z max-page-size=0x100 --gc-sections -nostdlib -m elf_i386 -o $@ $(filter-out ucmd.ld,$^)
 	$(OBJCOPY) --strip-all $@ $@
 
 # ngui: the userland immediate-mode GUI toolkit and its apps.

+ 8 - 8
docs/memory.html

@@ -216,11 +216,11 @@
 		<p>
 			The kernel heap is separate from userspace. Ring-3 processes get one
 			anonymous arena through <code>SYS_MEM_ALLOC</code>,
-			<code>SYS_MEM_GROW</code> and <code>SYS_MEM_FREE</code>; commands
-			that link <span class="file">libc.a</span> manage it with
-			<code>malloc</code>/<code>free</code> from
-			<span class="file">libc/malloc.c</span>, while small commands may
-			still use the raw syscalls. A process must not mix the two.
+			<code>SYS_MEM_GROW</code> and <code>SYS_MEM_FREE</code>;
+			commands manage it with <code>malloc</code>/<code>free</code>
+			from <span class="file">libc/malloc.c</span>. The syscalls are the
+			backend in <span class="file">libc/ucmd_mem.c</span> and are not
+			called by application code.
 		</p>
 	</section>
 
@@ -272,8 +272,8 @@
 			</table>
 		</div>
 		<p>
-			Commands that link <span class="file">libc.a</span> treat that arena
-			as a real heap: <span class="file">libc/malloc.c</span> maps 64 KB on
+			Ring-3 commands treat that arena as a real heap:
+			<span class="file">libc/malloc.c</span> maps 64 KB on
 			first use, grows in page-rounded steps with 64 KB slack, and uses
 			boundary-tag blocks with a first-fit free list and immediate
 			coalescing. <code>heap_reset()</code> discards everything and unmaps
@@ -310,7 +310,7 @@
 			<li><strong>No per-process address spaces.</strong> Paging is identity-mapped; there is no memory protection between processes.</li>
 			<li><strong>128 MB ceiling.</strong> Physical memory above <code>0x08000000</code> is unmapped. A wild pointer there is an instant <code>#PF</code>.</li>
 			<li><strong>User page leak on reap.</strong> Killing a ring-3 process frees its kernel stack and page directory but not all user pages mapped into it; a few pages leak per command run.</li>
-			<li><strong>Heap fragmentation.</strong> Power-of-two size classes make long-running mixed-size allocation wasteful compared with a real allocator.</li>
+			<li><strong>Kernel heap fragmentation.</strong> <code>kmalloc</code>'s power-of-two size classes make long-running mixed-size kernel allocation wasteful; ring-3 commands use the separate first-fit libc heap described above.</li>
 			<li><strong>No TLB shootdown.</strong> Cross-CPU mapping changes rely on not-present entries not being cached.</li>
 		</ul>
 	</section>

+ 4 - 4
docs/userland.html

@@ -110,10 +110,10 @@
 			<li><strong>heap_reset.</strong> Discards every allocation and
 			unmaps the whole arena. Page-scoped programs such as the browser
 			use it on navigation instead of tracking individual frees.</li>
-			<li><strong>Ownership.</strong> A command either uses libc
-			<code>malloc</code> or the raw
-			<code>SYS_MEM_ALLOC/GROW/FREE</code> arena, never both. The raw
-			calls remain for small commands that allocate one buffer and exit.</li>
+			<li><strong>Ownership.</strong> Commands allocate through libc
+			<code>malloc</code>; the <code>SYS_MEM_ALLOC/GROW/FREE</code>
+			syscalls are the backend in <span class="file">libc/ucmd_mem.c</span>
+			and are not called by application code.</li>
 			<li><strong>Limits.</strong> No threads or locking, no
 			<code>free()</code>-to-page unmap, and requests near 32 MB fail
 			because headers are charged against the same arena. Console

+ 3 - 3
sys_mem.h

@@ -5,9 +5,9 @@
  * USER_HEAP_BASE into the caller's page directory.  The reaper frees it with
  * the rest of the process.
  *
- * Commands that link the ring-3 libc own this arena through libc/malloc.c
- * (malloc/free/heap_reset); the raw syscalls are for commands that do not
- * link libc.  The two must not be mixed in one process. */
+ * Ring-3 commands use the libc heap (libc/malloc.c: malloc/free/heap_reset),
+ * which owns this arena.  The SYS_MEM_* syscalls are the libc backend
+ * (libc/ucmd_mem.c) and are not called directly by application code. */
 #ifndef SYS_MEM_H
 #define SYS_MEM_H
 

+ 3 - 3
syscall.h

@@ -102,9 +102,9 @@
 
 /* Per-process anonymous heap (see sys_mem.c).  One growable arena at
  * USER_HEAP_BASE (0x60000000), at most USER_HEAP_MAX (32 MB), freed when the
- * process exits.  SYS_MEM_ALLOC returns the base address, or <0.  Commands
- * that link libc.a use malloc/free (libc/malloc.c) and leave these syscalls
- * to the libc backend. */
+ * process exits.  SYS_MEM_ALLOC returns the base address, or <0.  Ring-3
+ * commands use malloc/free (libc/malloc.c); these syscalls are the libc
+ * backend (libc/ucmd_mem.c) and are not called by application code. */
 #define SYS_MEM_ALLOC      88  /* (unsigned bytes) -> VA or <0 */
 #define SYS_MEM_GROW       89  /* (unsigned total_bytes) -> 0/-1; new pages not zeroed */
 #define SYS_MEM_FREE       90  /* () -> 0/-1 */

+ 0 - 20
ucmd_lib.h

@@ -395,26 +395,6 @@ static inline int u_power_info(struct sys_powerinfo *out, unsigned int flags)
 	return u_syscall2(SYS_POWER_INFO, (int)out, (int)flags);
 }
 
-/* Raw per-process anonymous heap.  Commands that link libc.a should use
- * malloc/free instead; never mix these raw calls with the libc heap in one
- * process, because both own the same SYS_MEM_* arena. */
-static inline void *u_mem_alloc(unsigned int bytes)
-{
-	int r = u_syscall1(SYS_MEM_ALLOC, (int)bytes);
-
-	return r < 0 ? 0 : (void *)r;
-}
-
-static inline int u_mem_grow(unsigned int bytes)
-{
-	return u_syscall1(SYS_MEM_GROW, (int)bytes);
-}
-
-static inline int u_mem_free(void)
-{
-	return u_syscall0(SYS_MEM_FREE);
-}
-
 /* Path metadata (size, directory flag, read-only mount). */
 static inline int u_stat(const char *path, struct sys_stat *out)
 {

+ 6 - 4
ucmd_term.c

@@ -8,6 +8,8 @@
  * flag (the line continued because it did not fit), which lets a window
  * resize re-flow the whole log at the new width instead of clearing it. */
 
+#include <stdlib.h>
+
 #include "ucmd_lib.h"
 #include "uwm_lib.h"
 #include "io.h"
@@ -25,8 +27,8 @@
 #define SB_W 8
 #define SB_BOTTOM 16
 
-/* Total row storage (screen + scrollback).  Allocated from the per-process
- * heap; the fallback keeps a screen-only terminal working if that fails. */
+/* Total row storage (screen + scrollback).  Allocated from the libc heap;
+ * the fallback keeps a screen-only terminal working if that fails. */
 #define SCROLL_ROWS_PREF 512
 #define SCROLL_ROWS_MIN  128
 
@@ -1381,11 +1383,11 @@ static void term_alloc_rows(void)
 
 	row_cap = SCROLL_ROWS_PREF;
 	bytes = 2 * row_cap * (unsigned int)sizeof(struct trow);
-	row_mem = (struct trow *)u_mem_alloc(bytes);
+	row_mem = (struct trow *)malloc(bytes);
 	if (!row_mem) {
 		row_cap = SCROLL_ROWS_MIN;
 		bytes = 2 * row_cap * (unsigned int)sizeof(struct trow);
-		row_mem = (struct trow *)u_mem_alloc(bytes);
+		row_mem = (struct trow *)malloc(bytes);
 	}
 	if (row_mem) {
 		row_scratch = row_mem + row_cap;

+ 4 - 2
ucmd_websrv.c

@@ -16,6 +16,8 @@
  * because the VFS has no open/read/write-at interface.
  */
 
+#include <stdlib.h>
+
 #include "ucmd_lib.h"
 
 #define REQ_MAX       4096u
@@ -1657,10 +1659,10 @@ void _start(const char *arg)
 	}
 	u_syscall1(SYS_NET_GET_IP, (int)ip);
 
-	body = u_mem_alloc(BODY_MAX);
+	body = (unsigned char *)malloc(BODY_MAX);
 	body_max = BODY_MAX;
 	if (!body) {
-		body = u_mem_alloc(BODY_FALLBACK);
+		body = (unsigned char *)malloc(BODY_FALLBACK);
 		body_max = BODY_FALLBACK;
 	}
 	if (!body) {