Преглед изворни кода

Replace round-robin with a CFS scheduler

- Stable, separately allocated PCBs in a sparse pointer table; idle is no
  longer a process slot and a kworker handles housekeeping and reaping.
- Per-CPU current/GDT/TSS scaffolding via %gs, with this_cpu()/current_proc()
  replacing the global current_pid; CR3 and TSS.esp0 switch in C.
- CFS-lite policy: weights 1..16 (WM 16, terminals 12, kworker 1, commands 8),
  64-bit vruntime, 10 ms slices, a run queue, a sorted sleep list and wait
  queues with timed waits.
- Immediate sleep/block/exit through the int 0x81 reschedule gate; zombie
  reaping moved from the PIT handler to the kworker.
- preempt_count replaces the sys_fs_busy and vga_present_busy scheduler gates.
- Event-driven ring-3 terminal input (SYS_WIN_KEY_WAIT) instead of 10 ms
  polling; the kernel terminal fallback uses the same wait queue.
- sched/renice and the spin CPU-load command; ps/top show weight and CPU,
  top shows idle time, and SYS_SCHEDINFO exposes scheduler counters.
- AGENTS.md updated for the scheduler, PCBs and per-process page directories.

Only the BSP is started; ACPI/APIC/AP bring-up and cross-CPU locking remain
future work.
Johannes Zellner пре 2 недеља
родитељ
комит
f9512a7471
38 измењених фајлова са 1832 додато и 570 уклоњено
  1. 22 18
      AGENTS.md
  2. 9 6
      Makefile
  3. 4 2
      io.c
  4. 13 6
      irqret.inc
  5. 57 21
      kernel.c
  6. 31 4
      kmalloc.c
  7. 2 2
      netdev.c
  8. 4 1
      paging.c
  9. 2 1
      paging.h
  10. 8 7
      panic.c
  11. 432 275
      process.c
  12. 45 10
      process.h
  13. 630 0
      sched.c
  14. 98 0
      sched.h
  15. 52 58
      switch.S
  16. 5 7
      sys_fs.c
  17. 2 6
      sys_fs.h
  18. 9 9
      sys_gfx.c
  19. 1 1
      sys_net.c
  20. 6 1
      sys_wm.c
  21. 1 0
      sys_wm.h
  22. 27 10
      syscall.c
  23. 18 1
      syscall.h
  24. 5 3
      syscall_stub.S
  25. 69 7
      terminal.c
  26. 8 1
      terminal.h
  27. 3 0
      timer.S
  28. 60 32
      tss.c
  29. 10 6
      tss.h
  30. 4 3
      tty.c
  31. 10 0
      ucmd_lib.h
  32. 36 0
      ucmd_spin.c
  33. 12 2
      ucmd_term.c
  34. 80 47
      ucmd_top.c
  35. 2 5
      vga.c
  36. 0 4
      vga.h
  37. 53 14
      wm.c
  38. 2 0
      wm.h

+ 22 - 18
AGENTS.md

@@ -62,25 +62,29 @@ below the run area), so adding code no longer moves any address.
 
 ## Key files
 
-- **Multitasking**: `process.c/h`, `switch.S` — ring-0 preemptive round-robin,
-  dynamically-sized process table (kmalloc'd, grows on demand), 20 ms timeslice,
-  `ps`/`kill`. Context switch happens in the PIT stub. Each process carries a
-  `struct text_screen *screen` output target (child processes inherit it, so
-  command output lands in the owning terminal). Exited processes are reaped and
-  their stacks freed by the scheduler. The PIT stub also credits every tick to
-  the running process (`cpu_ticks`/`start_ticks`; `process_procinfo()`, exposed
-  as `SYS_PROCINFO`), which the ring-3 `top` command turns into a live CPU%.
+- **Multitasking**: `sched.c/h`, `process.c/h`, `tss.c/h`, `switch.S` — CFS-lite
+  weighted fair scheduling (weight 1..16, 64-bit vruntime, 10 ms slice; WM 16,
+  terminals 12, kworker 1, commands 8), a sorted sleep list and wait queues
+  (no polling scans), stable per-PCB objects in a sparse `PROC_MAX` pointer
+  table, per-CPU `struct cpu`/GDT/TSS with `%gs`-based `this_cpu()`, and a
+  per-CPU `current` (no global `current_pid`).  Idle is not a process slot; a
+  normal `kworker` process does the housekeeping and the zombie reaper.  Ring 0
+  is preemptible at tick boundaries unless `preempt_count` is set (FAT and the
+  shadow flush), and voluntary blocks/yields go through the `int 0x81` gate.
+  `SYS_PROCINFO` exposes weight/state/CPU/ticks for the ring-3 `top` command,
+  and `sched`/`renice` plus `SYS_SCHEDINFO` expose the scheduler counters.
 - **Memory**: `mm.c/h` — INT 15h E820 detection (real-mode in `vesa_stage.S`,
   map at `0x1000`) + a bitmap page-frame allocator (4 KB pages). Reserves low
   1 MB and an 8 MB VGA-shadow region. `kmalloc.c/h` — a power-of-two heap
-  allocator on top of the frame allocator (used for the process table). Process
-  stacks (≥ `0x900000`) come from the frame allocator, with an 8-stack fallback
-  pool (a static BSS array; no fixed address).
+  allocator on top of the frame allocator (used for PCBs and command context).
+  Process stacks (≥ `0x900000`) come from the frame allocator, with an 8-stack
+  fallback pool (a static BSS array; no fixed address).
 - **Paging**: `paging.c/h` — identity-maps RAM (4 MB pages via CR4.PSE) + the
-  LFB (uncacheable) and enables CR0.PG/WP. A single shared page directory for
-  now (no per-process address spaces yet). The kexec trampoline
-  (`kexec_tramp.S`) clears CR0.PG before the copy so it can overwrite the old
-  kernel's page tables safely.
+  LFB (uncacheable) and enables CR0.PG/WP. Each process gets a page directory
+  cloned from the kernel identity map (kernel stacks are mapped everywhere, so
+  CR3 can switch on the old stack); `paging_current_cr3()` reads CR3. The kexec
+  trampoline (`kexec_tramp.S`) clears CR0.PG before the copy so it can overwrite
+  the old kernel's page tables safely.
 - **Window manager**: `wm.c/h` — slot 0 runs `wm_run()`; fixed-size draggable
   terminal windows (64×32) with a system bar (banner + purple "+ New" button +
   clock + rainbow). Click-to-focus, drag-by-title-bar, close (×). The "+ New"
@@ -293,8 +297,8 @@ below the run area), so adding code no longer moves any address.
 `cd [path]`, `ls [path]`, `cat <path>`, `save <path> <text>`, `rm <path>`,
 `mkdir <path>`, `rmdir <path>`, `hello`, `reboot`, `poweroff`, `run <name>|<file>`, `space`,
 `prog`,
-`display`, `cpuinfo`, `mouseinfo`, `gfxbench`, `lsusb`, `lspci [id]`, `peek <addr> [len]`, `poke <addr> <val>`, `storage`, `meminfo`, `top`, `format <device>`,
-`ps`, `kill <pid>`, `help`, `clear`, `serial test|recv`, `kexec`, `dbg`.
+`display`, `cpuinfo`, `mouseinfo`, `gfxbench`, `lsusb`, `lspci [id]`, `peek <addr> [len]`, `poke <addr> <val>`, `storage`, `meminfo`, `top`, `sched`, `format <device>`,
+`ps`, `renice <pid> <weight>`, `kill <pid>`, `spin [ms]`, `help`, `clear`, `serial test|recv`, `kexec`, `dbg`.
 
 Network: `netinfo`, `netrecv`, `netsend`, `netip [a.b.c.d]`, `dhcp` (re-run the
 DHCP client), `pong` (responds to ARP/ICMP for our IP for 30 s),
@@ -323,7 +327,7 @@ changed with `cd`; paths may be absolute (`/floppy1/…`) or relative (`.`, `..`
 nested). `/` is the read-only VFS root listing the mounted filesystems (the
 filesystem commands operate inside a mount, e.g. `cat /floppy1/HELLO.TXT`).
 
-`cpuinfo`, `ls`, `cat`, `save`, `rm`, `mkdir`, `rmdir`, `meminfo`, `top`,
+`cpuinfo`, `ls`, `cat`, `save`, `rm`, `mkdir`, `rmdir`, `meminfo`, `top`, `spin`,
 `clock`, `colors`, `mouseinfo`, `storage`, `netinfo`, `netip`, `netdns`, `host`, `dhcp`, `ping`,
 `pong`, `netsend`, `netrecv`, `netlog`, `netconsole`, `netregs`, `netloop`,
 `netreset`, `curl` and `hello` are files in `/ramfs/bin` (packed by the

+ 9 - 6
Makefile

@@ -20,7 +20,7 @@ all: floppy.img usb.img final.bin root.img
 boot.o: boot.S
 	$(AS) $(ASFLAGS) -o $@ $<
 
-kernel.o: kernel.c idle.h io.h mouse.h panic.h process.h terminal.h timer.h serial.h usbhid.h version.h vga.h
+kernel.o: kernel.c idle.h io.h mouse.h panic.h process.h sched.h terminal.h timer.h serial.h tss.h usbhid.h version.h vga.h
 	$(CC) $(CFLAGS) -o $@ $<
 
 screen.o: screen.c screen.h
@@ -230,7 +230,10 @@ switch.o: switch.S irqret.inc
 entry.o: entry.S
 	$(AS) $(ASFLAGS) -o $@ $<
 
-process.o: process.c process.h io.h kmalloc.h mm.h serial.h idle.h timer.h
+sched.o: sched.c sched.h process.h paging.h sys_fs.h timer.h tss.h vga.h
+	$(CC) $(CFLAGS) -o $@ $<
+
+process.o: process.c process.h sched.h io.h kmalloc.h mm.h serial.h idle.h timer.h
 	$(CC) $(CFLAGS) -o $@ $<
 
 mm.o: mm.c mm.h
@@ -245,7 +248,7 @@ paging.o: paging.c paging.h mm.h mtrr.h vga.h serial.h
 elf.o: elf.c elf.h mm.h paging.h
 	$(CC) $(CFLAGS) -o $@ $<
 
-tss.o: tss.c tss.h
+tss.o: tss.c tss.h sched.h
 	$(CC) $(CFLAGS) -o $@ $<
 
 syscall.o: syscall.c syscall.h io.h process.h timer.h sys_wm.h wm_abi.h
@@ -263,7 +266,7 @@ sys_net.o: sys_net.c sys_net.h syscall.h dhcp.h dns.h net.h netdev.h netlog.h
 syscall_stub.o: syscall_stub.S irqret.inc
 	$(AS) $(ASFLAGS) -o $@ $<
 
-kernel.elf: vesa_stage.o entry.o kernel.o timer.o switch.o process.o mm.o kmalloc.o paging.o elf.o tss.o syscall.o sys_wm.o syscall_stub.o screen.o io.o tty.o rtc.o terminal.o wm.o reboot.o poweroff.o disk.o fat.o run.o serial.o serial_ring_buf.o serial_loop.o serial_recv.o bin_recv.o vga.o blit.o cpu.o vga_copy.o gfxbench.o gpu.o ati.o cirrus.o nvidia.o nv50.o radeon.o mtrr.o font_8x8.o fbtext.o gfx.o sys_gfx.o display.o mouse.o uhci.o ohci.o usb.o usbhid.o lsusb.o panic.o blkdev.o ata.o ahci.o fatdetect.o fatformat.o format.o vfs.o sys_fs.o sys_info.o sys_net.o idle.o kexec.o kexec_tramp.o dbg.o lspci.o peek.o ne2000.o r8168.o netdev.o net.o netkexec.o netlog.o dhcp.o dns.o tcp.o prog_embed.o space_embed.o cmds_pack.o ramfs.o payload.o
+kernel.elf: vesa_stage.o entry.o kernel.o timer.o switch.o sched.o process.o mm.o kmalloc.o paging.o elf.o tss.o syscall.o sys_wm.o syscall_stub.o screen.o io.o tty.o rtc.o terminal.o wm.o reboot.o poweroff.o disk.o fat.o run.o serial.o serial_ring_buf.o serial_loop.o serial_recv.o bin_recv.o vga.o blit.o cpu.o vga_copy.o gfxbench.o gpu.o ati.o cirrus.o nvidia.o nv50.o radeon.o mtrr.o font_8x8.o fbtext.o gfx.o sys_gfx.o display.o mouse.o uhci.o ohci.o usb.o usbhid.o lsusb.o panic.o blkdev.o ata.o ahci.o fatdetect.o fatformat.o format.o vfs.o sys_fs.o sys_info.o sys_net.o idle.o kexec.o kexec_tramp.o dbg.o lspci.o peek.o ne2000.o r8168.o netdev.o net.o netkexec.o netlog.o dhcp.o dns.o tcp.o prog_embed.o space_embed.o cmds_pack.o ramfs.o payload.o
 	$(LD) $(KERNEL_LDFLAGS) -o $@ $^
 
 MBR_LDFLAGS = -T mbr.ld -nostdlib -m elf_i386
@@ -349,7 +352,7 @@ prog: prog.bin
 # ucmd_<name>.c to UCMDS; the pack is rebuilt automatically.
 UCMDS = ucmd_cpuinfo ucmd_ls ucmd_cat ucmd_save ucmd_rm ucmd_mkdir ucmd_rmdir \
         ucmd_meminfo ucmd_mouseinfo ucmd_storage ucmd_top \
-        ucmd_clock ucmd_colors ucmd_edit ucmd_wm ucmd_term \
+        ucmd_clock ucmd_colors ucmd_edit ucmd_wm ucmd_term ucmd_spin \
         ucmd_netinfo ucmd_netip ucmd_netdns ucmd_host ucmd_dhcp \
         ucmd_netsend ucmd_netrecv ucmd_ping ucmd_pong ucmd_netlog \
         ucmd_netconsole ucmd_netregs ucmd_netloop ucmd_netreset ucmd_curl
@@ -624,4 +627,4 @@ net-dev: kernel.bin netkexec_send
 	./netkexec_send $(BOARD_IP) $(NETKEXEC_PORT) 0x$$entry kernel.bin
 
 clean:
-	rm -f boot.o vesa_stage.o entry.o kernel.o timer.o switch.o process.o mm.o kmalloc.o paging.o elf.o tss.o syscall.o sys_wm.o syscall_stub.o screen.o io.o tty.o rtc.o terminal.o wm.o hello.o reboot.o poweroff.o fat.o run.o disk.o serial.o serial_ring_buf.o serial_loop.o serial_recv.o bin_recv.o vga.o blit.o cpu.o vga_copy.o gfxbench.o gpu.o ati.o cirrus.o nvidia.o nv50.o radeon.o mtrr.o font_8x8.o fbtext.o gfx.o sys_gfx.o display.o mouse.o uhci.o ohci.o usb.o usbhid.o lsusb.o panic.o blkdev.o ata.o ahci.o fatdetect.o fatformat.o format.o vfs.o sys_fs.o sys_info.o idle.o kexec.o kexec_tramp.o dbg.o lspci.o peek.o ne2000.o r8168.o netdev.o net.o netkexec.o netlog.o dhcp.o dns.o tcp.o mbr.o boot.elf kernel.elf mbr.elf boot.bin mbr.bin kernel.bin boot.bin.tmp boot.bin.patched image.bin floppy.img usb.img final.bin root.img prog.o prog.elf prog.bin space.o space.elf space.bin hello.elf hello_embed.o prog_embed.o space_embed.o cmds.pack cmds_pack.o ramfs.o payload.o mkpack uwm_lib.o *.boot.tmp $(addsuffix .o,$(UCMDS)) $(addsuffix .elf,$(UCMDS)) $(TLSLIB_OBJS) tls_test bootstrap.o bootstrap.elf bootstrap.bin send_serial kexec_send netkexec_send netmonitor
+	rm -f boot.o vesa_stage.o entry.o kernel.o timer.o switch.o sched.o process.o mm.o kmalloc.o paging.o elf.o tss.o syscall.o sys_wm.o syscall_stub.o screen.o io.o tty.o rtc.o terminal.o wm.o hello.o reboot.o poweroff.o fat.o run.o disk.o serial.o serial_ring_buf.o serial_loop.o serial_recv.o bin_recv.o vga.o blit.o cpu.o vga_copy.o gfxbench.o gpu.o ati.o cirrus.o nvidia.o nv50.o radeon.o mtrr.o font_8x8.o fbtext.o gfx.o sys_gfx.o display.o mouse.o uhci.o ohci.o usb.o usbhid.o lsusb.o panic.o blkdev.o ata.o ahci.o fatdetect.o fatformat.o format.o vfs.o sys_fs.o sys_info.o idle.o kexec.o kexec_tramp.o dbg.o lspci.o peek.o ne2000.o r8168.o netdev.o net.o netkexec.o netlog.o dhcp.o dns.o tcp.o mbr.o boot.elf kernel.elf mbr.elf boot.bin mbr.bin kernel.bin boot.bin.tmp boot.bin.patched image.bin floppy.img usb.img final.bin root.img prog.o prog.elf prog.bin space.o space.elf space.bin hello.elf hello_embed.o prog_embed.o space_embed.o cmds.pack cmds_pack.o ramfs.o payload.o mkpack uwm_lib.o *.boot.tmp $(addsuffix .o,$(UCMDS)) $(addsuffix .elf,$(UCMDS)) $(TLSLIB_OBJS) tls_test bootstrap.o bootstrap.elf bootstrap.bin send_serial kexec_send netkexec_send netmonitor

+ 4 - 2
io.c

@@ -58,7 +58,8 @@ static struct text_screen io_fallback;
 
 static struct text_screen *io_current_screen(void)
 {
-	struct text_screen *s = procs[current_pid].screen;
+	struct process *p = current_proc();
+	struct text_screen *s = p ? p->screen : 0;
 	return s ? s : &io_fallback;
 }
 
@@ -163,7 +164,8 @@ void io_init(void)
  * the old cell-screen path until the ring-3 terminal is running. */
 static int io_tty(void)
 {
-	int id = procs[current_pid].tty;
+	struct process *p = current_proc();
+	int id = p ? p->tty : 0;
 
 	return tty_valid(id) ? id : 0;
 }

+ 13 - 6
irqret.inc

@@ -1,7 +1,7 @@
 /* IRQ_RETURN_DS <cs_offset>
  *
  * Load DS/ES/FS/GS for the context we are about to IRET to, in a stub that
- * switched them to the kernel data selector (0x10) to run C code.
+ * switched them to the kernel selectors to run C code.
  *
  * Why: an IRET to a less privileged level nulls DS/ES/FS/GS whose DPL is
  * lower than the new CPL.  Coming back to ring 3 with 0x10 (DPL 0) therefore
@@ -9,10 +9,12 @@
  * access (even a simple stack-buffer store through a pointer).  QEMU does not
  * model this nulling, so it only shows up on real hardware.
  *
- * So: ring-3 contexts get the user data selector (0x23), ring-0 contexts keep
- * 0x10.  <cs_offset> is the byte offset of the saved CS in the current frame
- * (pusha is 32 bytes, then the CPU's EIP/CS/EFLAGS: offset 36).  Clobbers %cx,
- * which the caller restores via popa.
+ * Ring-3 contexts get the user data selector (0x23).  Ring-0 contexts get
+ * 0x10 for DS/ES/FS and 0x28 for GS: GS is the per-CPU data selector whose
+ * base points at this CPU's struct cpu, and this_cpu()/current_proc() read
+ * through it.  <cs_offset> is the byte offset of the saved CS in the current
+ * frame (pusha is 32 bytes, then the CPU's EIP/CS/EFLAGS: offset 36).
+ * Clobbers %cx, which the caller restores via popa.
  */
 .macro IRQ_RETURN_DS cs_offset
 	movw  \cs_offset(%esp), %cx
@@ -20,12 +22,17 @@
 	cmpw  $3, %cx
 	jne   1f
 	movw  $0x23, %cx
+	movw  %cx, %ds
+	movw  %cx, %es
+	movw  %cx, %fs
+	movw  %cx, %gs
 	jmp   2f
 1:
 	movw  $0x10, %cx
-2:
 	movw  %cx, %ds
 	movw  %cx, %es
 	movw  %cx, %fs
+	movw  $0x28, %cx
 	movw  %cx, %gs
+2:
 .endm

+ 57 - 21
kernel.c

@@ -49,6 +49,7 @@ extern void pit_irq_stub(void);
 extern void kbd_irq_stub(void);
 extern void fdc_irq_stub(void);
 extern void default_irq_stub(void);
+extern void resched_stub(void);
 extern void syscall_stub(void);
 extern unsigned int exc_stub_table[];
 
@@ -148,6 +149,8 @@ static void timer_init(void)
 
 	/* Syscall gate (ring-3 can int 0x80). */
 	set_idt_gate_user(0x80, (unsigned int)syscall_stub);
+	/* Kernel-only voluntary reschedule gate (int 0x81). */
+	set_idt_gate(0x81, (unsigned int)resched_stub);
 
 	idtr.limit = (unsigned short)(sizeof(idt) - 1);
 	idtr.base  = (unsigned int)idt;
@@ -162,23 +165,43 @@ static void timer_init(void)
 
 extern unsigned char __bss_start[], __bss_end[];
 
-/* The idle process: keeps the parts of the system that must not depend on
- * the window manager running.  Input routing, cursor blink, kexec recovery,
- * netlog flushing and gfx-session cleanup live here; the WM process only
- * handles window state and drawing. */
+static int kworker_pid = -1;
+
+/* Parts of the system that must keep running independently of the WM:
+ * input routing, cursor blink, kexec recovery, netlog flushing, gfx-session
+ * cleanup and the shadow-buffer flush. */
+static void system_housekeeping(void)
+{
+	process_reap_dead();
+	mouse_poll_update();
+	wm_pump_keyboard();
+	wm_blink_cursor();
+	netlog_flush();
+	wm_handle_kexec();
+	wm_server_supervise();
+	if (wm_server_can_reap())
+		sys_gfx_collect();
+	if (!wm_server_frame_active())
+		vga_fb_flush_budget(VGA_FLUSH_BUDGET_BYTES);
+}
+
+static void kworker_main(void)
+{
+	for (;;) {
+		system_housekeeping();
+		process_sleep_ms(5);
+	}
+}
+
+/* Idle is not a normal runnable task: schedule() selects it only when no
+ * READY process exists.  The kworker does the housekeeping; if it dies (or
+ * could not be created), idle takes over as a degraded fallback. */
 static void kernel_idle_loop(void)
 {
 	for (;;) {
-		mouse_poll_update();
-		wm_pump_keyboard();
-		wm_blink_cursor();
-		netlog_flush();
-		wm_handle_kexec();
-		wm_server_supervise();
-		if (wm_server_can_reap())
-			sys_gfx_collect();
-		if (!wm_server_frame_active())
-			vga_fb_flush_budget(VGA_FLUSH_BUDGET_BYTES);
+		if (kworker_pid <= 0 ||
+		    !process_is_alive((unsigned int)kworker_pid))
+			system_housekeeping();
 		cpu_idle();
 	}
 }
@@ -196,6 +219,12 @@ void kernel_main(void)
 	 * early boot output until the WM loop starts flushing it. */
 	netlog_init();
 	serial_puts(SERIAL_COM1, "\n" OS_NAME " v" OS_VERSION " booted\n");
+
+	/* GDT/TSS and the per-CPU GS base must exist before anything that can
+	 * use current_proc()/this_cpu().  The BSS is already cleared and the
+	 * GDT/TSS live at identity-mapped addresses. */
+	gdt_tss_init(0);
+
 	mm_init();
 	mm_dump();
 	kmalloc_init();
@@ -203,12 +232,10 @@ void kernel_main(void)
 	io_init();
 	idle_init();
 
-	/* process_init + gdt_tss_init must run BEFORE timer_init: once the PIT is
-	 * started (sti in timer_init) the context-switch stub runs on every tick
-	 * and reads procs[].cr3 / procs[].kstack_top / the TSS, so those have to be
-	 * valid first (otherwise the first switch loads a garbage CR3 and faults). */
+	/* process_init must run before timer_init: once the PIT is started (sti
+	 * in timer_init) the context-switch stub runs on every tick and reads
+	 * the current process, its CR3 and the TSS, so those must be valid. */
 	process_init();
-	gdt_tss_init();
 	timer_init();
 	mouse_init();
 	/* USB HID backend (mouse + keyboard): controller bring-up + first
@@ -239,9 +266,18 @@ void kernel_main(void)
 	/* With a NIC present, try DHCP; on failure the static default stays. */
 	if (netdev_active())
 		dhcp_run(3000);
-	/* Slot 0 is the idle loop; the window server is initialized before the
-	 * ring-3 WM is spawned from /ramfs/bin/wm. */
+	/* The idle process is the kernel_main context; the ring-3 WM is spawned
+	 * from /ramfs/bin/wm.  Housekeeping runs in its own process so it cannot
+	 * be starved by a busy system. */
 	wm_server_init();
 	wm_server_start();
+	kworker_pid = process_create("kworker", kworker_main);
+	if (kworker_pid < 0) {
+		serial_puts(SERIAL_COM1, "kworker: cannot create, using idle fallback\n");
+		kworker_pid = -1;
+	} else {
+		process_set_weight((unsigned int)kworker_pid, 1);
+		process_set_affinity((unsigned int)kworker_pid, 1);
+	}
 	kernel_idle_loop();
 }

+ 31 - 4
kmalloc.c

@@ -15,6 +15,21 @@ union km_block {
 
 static union km_block *free_lists[KMALLOC_CLASSES];
 
+/* The free lists are shared by every process; the PIT must not switch in the
+ * middle of a list update.  mm_alloc_page()/mm_free_page() have their own
+ * nested IRQ gates. */
+static inline unsigned int irq_save_disable(void)
+{
+	unsigned int flags;
+	__asm__ volatile ("pushfl; popl %0; cli" : "=r"(flags));
+	return flags;
+}
+
+static inline void irq_restore(unsigned int flags)
+{
+	__asm__ volatile ("pushl %0; popfl" : : "r"(flags));
+}
+
 static inline unsigned int round_up_pow2(unsigned int n)
 {
 	n--;
@@ -46,7 +61,7 @@ void kmalloc_init(void)
 
 void *kmalloc(unsigned int size)
 {
-	unsigned int total, cls, off;
+	unsigned int total, cls, off, flags;
 	union km_block *b;
 	void *page;
 
@@ -58,13 +73,18 @@ void *kmalloc(unsigned int size)
 	if (total < KMALLOC_MIN)
 		total = KMALLOC_MIN;
 
+	flags = irq_save_disable();
+
 	/* Large allocation: whole contiguous pages. */
 	if (total > PAGE_SIZE) {
 		unsigned int pages = (total + PAGE_SIZE - 1) / PAGE_SIZE;
 		page = (void *)mm_alloc_pages(pages);
-		if (!page)
+		if (!page) {
+			irq_restore(flags);
 			return 0;
+		}
 		((union km_block *)page)->h.size = total;
+		irq_restore(flags);
 		return (void *)((unsigned int)page + sizeof(unsigned int));
 	}
 
@@ -73,8 +93,10 @@ void *kmalloc(unsigned int size)
 	if (!b) {
 		/* Refill this size class from a fresh page. */
 		page = (void *)mm_alloc_page();
-		if (!page)
+		if (!page) {
+			irq_restore(flags);
 			return 0;
+		}
 		for (off = 0; off + total <= PAGE_SIZE; off += total) {
 			union km_block *blk =
 				(union km_block *)((unsigned int)page + off);
@@ -86,13 +108,14 @@ void *kmalloc(unsigned int size)
 
 	free_lists[cls] = b->next;
 	b->h.size = total;
+	irq_restore(flags);
 	return (void *)((unsigned int)b + sizeof(unsigned int));
 }
 
 void kfree(void *ptr)
 {
 	union km_block *b;
-	unsigned int size, cls;
+	unsigned int size, cls, flags;
 
 	if (!ptr)
 		return;
@@ -100,16 +123,20 @@ void kfree(void *ptr)
 	b = (union km_block *)((unsigned int)ptr - sizeof(unsigned int));
 	size = b->h.size;
 
+	flags = irq_save_disable();
+
 	if (size > PAGE_SIZE) {
 		unsigned int pages = (size + PAGE_SIZE - 1) / PAGE_SIZE;
 		unsigned int addr = (unsigned int)b;
 		unsigned int i;
 		for (i = 0; i < pages; i++)
 			mm_free_page(addr + i * PAGE_SIZE);
+		irq_restore(flags);
 		return;
 	}
 
 	cls = size_class_of(size);
 	b->next = free_lists[cls];
 	free_lists[cls] = b;
+	irq_restore(flags);
 }

+ 2 - 2
netdev.c

@@ -28,8 +28,8 @@ unsigned char *netdev_rx_scratch(void)
 
 void netdev_acquire(void)
 {
-	if (busy == 0 && procs)
-		owner_pid = procs[current_pid].pid;
+	if (busy == 0 && current_proc())
+		owner_pid = current_proc()->pid;
 	busy++;
 }
 

+ 4 - 1
paging.c

@@ -153,7 +153,10 @@ void paging_init(void)
 
 unsigned int paging_current_cr3(void)
 {
-	return (unsigned int)page_dir;
+	unsigned int cr3;
+
+	__asm__ volatile ("movl %%cr3, %0" : "=r"(cr3));
+	return cr3 & 0xFFFFF000u;
 }
 
 unsigned int paging_new_directory(void)

+ 2 - 1
paging.h

@@ -13,7 +13,8 @@ int paging_lfb_wc(void);
  * is whatever the MTRRs say (write-combining when we added an entry). */
 void paging_map_ram(unsigned int base, unsigned int size);
 
-/* Physical address of the bootstrap (current) page directory. */
+/* Physical address of the page directory currently loaded in CR3.  On SMP this
+ * is per-CPU; on one CPU it is the running process's directory. */
 unsigned int paging_current_cr3(void);
 
 /* Allocate a fresh page directory that clones the kernel identity map and

+ 8 - 7
panic.c

@@ -165,6 +165,7 @@ void panic_report(const struct exc_state *st, const char *name)
 	unsigned char code[16];
 	unsigned int cand_val[12], cand_off[12];
 	unsigned int cand_n = 0;
+	struct process *cur = current_proc();
 	int stack_ok, user = (st->cs & 3) == 3;
 
 	__asm__ volatile ("mov %%cr0, %0" : "=r"(cr0));
@@ -281,11 +282,11 @@ void panic_report(const struct exc_state *st, const char *name)
 	/* Current process and uptime. */
 	i = 0;
 	i = app_str(buf, i, "pid ");
-	i = app_dec(buf, i, current_pid);
-	if (procs) {
+	i = app_dec(buf, i, cur ? cur->pid : 0);
+	if (cur) {
 		i = app_str(buf, i, " (");
-		for (n = 0; n < PROC_NAME_LEN && procs[current_pid].name[n]; n++)
-			buf[i++] = procs[current_pid].name[n];
+		for (n = 0; n < PROC_NAME_LEN && cur->name[n]; n++)
+			buf[i++] = cur->name[n];
 		buf[i++] = ')';
 	}
 	i = app_str(buf, i, "  ticks ");
@@ -294,12 +295,12 @@ void panic_report(const struct exc_state *st, const char *name)
 	emit(buf);
 
 	/* Kernel stack bounds: an ESP below the base means stack overflow. */
-	if (procs) {
+	if (cur) {
 		i = 0;
 		i = app_str(buf, i, "kstack base ");
-		i = app_ptr(buf, i, procs[current_pid].stack_base);
+		i = app_ptr(buf, i, cur->stack_base);
 		i = app_str(buf, i, "  top ");
-		i = app_ptr(buf, i, procs[current_pid].kstack_top);
+		i = app_ptr(buf, i, cur->kstack_top);
 		buf[i] = '\0';
 		emit(buf);
 	}

+ 432 - 275
process.c

@@ -1,4 +1,9 @@
-/* Process management: create, exit, round-robin schedule, ps, kill. */
+/* Process management: create, exit, reap, ps, kill.
+ *
+ * The scheduler itself lives in sched.c.  This file owns the stable PCB
+ * objects and the lifecycle around them: allocation, initial stack frames,
+ * address-space cleanup, and the user-facing helpers.
+ */
 
 #include "process.h"
 #include "elf.h"
@@ -17,23 +22,17 @@
 #include "vga.h"
 #include "wm_abi.h"
 
-struct process *procs;               /* dynamically-sized process table */
-volatile unsigned int current_pid;
+struct process *procs[PROC_MAX];
 
-static unsigned int proc_cap;            /* current capacity of procs[] */
+static struct process idle_proc;
 static unsigned int next_pid_counter = 1; /* monotonic, wraps */
 static unsigned int fallback_stack_next;  /* 0..PROC_FALLBACK_STACK_MAX */
+static struct process *zombie_head, *zombie_tail;
+static struct wait_queue exit_wq;
 
 /* Top of the in-image bootstrap stack (see kernel.ld). */
 extern char __boot_stack_top[];
 
-#define INITIAL_PROC_CAP 16
-
-/* switch.S addresses the table entries with an immediate PCB_SIZE/PCB_CPU_TICKS
- * and cannot see this struct; keep the two in sync. */
-_Static_assert(sizeof(struct process) == 124,
-	       "struct process layout changed: update PCB_SIZE/PCB_CPU_TICKS in switch.S");
-
 /* ---- interrupt flag save/restore (for short critical sections) ---- */
 static inline unsigned int irq_save_disable(void)
 {
@@ -72,9 +71,8 @@ static unsigned int stack_alloc(int *owned)
 }
 
 /* Clear one control block.  kmalloc memory is not zeroed and QEMU hides that
- * (guest RAM starts cleared), so a fresh slot would otherwise carry garbage in
- * cr3/pid/state; schedule() and process_map_mmio_all() trust those fields and
- * a garbage cr3 made the boot-time MMIO mapping fault on real hardware. */
+ * (guest RAM starts cleared), so a fresh PCB would otherwise carry garbage in
+ * cr3/pid/state; the scheduler and MMIO mapping code trust those fields. */
 static void clear_proc(struct process *p)
 {
 	unsigned char *b = (unsigned char *)p;
@@ -92,77 +90,113 @@ static void name_copy(char *dst, const char *src)
 	dst[i] = '\0';
 }
 
-static void inherit_cwd(struct process *p)
+static void cwd_copy(struct process *p, const char *cwd)
 {
 	unsigned int i;
 
-	for (i = 0; i + 1 < sizeof(p->cwd) && procs[current_pid].cwd[i]; i++)
-		p->cwd[i] = procs[current_pid].cwd[i];
+	for (i = 0; cwd && cwd[i] && i + 1 < sizeof(p->cwd); i++)
+		p->cwd[i] = cwd[i];
 	p->cwd[i] = '\0';
 }
 
-/* Double the process table.  Called with interrupts disabled. */
-static int grow_proc_table(void)
-{
-	unsigned int new_cap = proc_cap * 2;
-	struct process *np =
-		(struct process *)kmalloc(new_cap * sizeof(struct process));
-	unsigned int i;
-
-	if (!np)
-		return 0;
-	for (i = 0; i < proc_cap; i++)
-		np[i] = procs[i];
-	for (i = proc_cap; i < new_cap; i++)
-		clear_proc(&np[i]);
-	kfree(procs);
-	procs = np;
-	proc_cap = new_cap;
-	return 1;
-}
-
-/* Find a free process slot (1..proc_cap-1), growing the table if needed.
- * Slot 0 is the kernel idle process and is never returned.  Called with interrupts
- * disabled.  Returns the slot index, or -1 on failure. */
+/* Find a free slot in the stable pointer table.  Called with interrupts
+ * disabled.  Returns the slot index, or -1 when the table is full. */
 static int alloc_slot(void)
 {
 	unsigned int slot;
-	for (slot = 1; slot < proc_cap; slot++)
-		if (procs[slot].state == PROC_UNUSED)
-			return (int)slot;
-	if (!grow_proc_table())
-		return -1;
-	for (slot = 1; slot < proc_cap; slot++)
-		if (procs[slot].state == PROC_UNUSED)
+
+	for (slot = 0; slot < PROC_MAX; slot++)
+		if (!procs[slot])
 			return (int)slot;
 	return -1;
 }
 
+/* Allocate and clear a PCB, then reserve a table slot (but do not publish it
+ * yet).  Called with interrupts disabled. */
+static struct process *proc_alloc(int *slot_out)
+{
+	struct process *p = (struct process *)kmalloc(sizeof(*p));
+	int slot;
+
+	if (!p)
+		return 0;
+	clear_proc(p);
+	slot = alloc_slot();
+	if (slot < 0) {
+		kfree(p);
+		return 0;
+	}
+	*slot_out = slot;
+	return p;
+}
+
+/* Fill in the common fields and publish the process in its slot.  The caller
+ * has already built the initial kernel stack frame.  Called with interrupts
+ * disabled and with parent = current_proc() (possibly idle). */
+static void proc_commit(struct process *p, int slot, const char *name,
+			unsigned int esp, int owned, unsigned int stack_base,
+			unsigned int cr3, unsigned int kstack_top)
+{
+	struct process *parent = current_proc();
+
+	p->esp = esp;
+	p->pid = next_pid_counter++;
+	p->slot = (unsigned int)slot;
+	p->stack_base = owned ? stack_base : 0;
+	p->cr3 = cr3;
+	p->kstack_top = kstack_top;
+	p->cpu_ticks = 0;
+	p->start_ticks = get_ticks_ms();
+	p->on_cpu = PROC_NO_CPU;
+	p->last_cpu = PROC_NO_CPU;
+	p->affinity = 0xFFFFFFFFu;
+	p->weight = 0;
+	p->stride = 0;
+	p->vruntime = 0;
+	p->slice_start = 0;
+	p->screen = parent ? parent->screen : (void *)0;
+	p->tty = parent ? parent->tty : 0;
+	if (parent)
+		cwd_copy(p, parent->cwd);
+	else
+		p->cwd[0] = '\0';
+	name_copy(p->name, name);
+	procs[slot] = p;
+	sched_add_ready(p);
+}
+
 void process_init(void)
 {
 	unsigned int i;
 
-	procs = (struct process *)kmalloc(INITIAL_PROC_CAP * sizeof(struct process));
-	if (!procs) {
-		serial_puts(SERIAL_COM1, "process_init: out of memory\n");
-		for (;;)
-			__asm__ volatile ("cli; hlt");
-	}
-	proc_cap = INITIAL_PROC_CAP;
-	for (i = 0; i < proc_cap; i++)
-		clear_proc(&procs[i]);
-
-	procs[0].state      = PROC_RUNNING;
-	procs[0].pid        = 0;
-	procs[0].esp        = 0;
-	procs[0].screen     = (void *)0;
-	procs[0].stack_base = 0;  /* bootstrap stack, never freed */
-	procs[0].cr3        = paging_current_cr3();
-	procs[0].kstack_top = (unsigned int)__boot_stack_top;
-	procs[0].cpu_ticks  = 0;
-	procs[0].start_ticks = 0;  /* the WM "starts" at boot */
-	name_copy(procs[0].name, "idle");
-	current_pid = 0;
+	sched_init();
+	zombie_head = zombie_tail = 0;
+	wait_queue_init(&exit_wq);
+	for (i = 0; i < PROC_MAX; i++)
+		procs[i] = 0;
+
+	clear_proc(&idle_proc);
+	idle_proc.state      = PROC_RUNNING;
+	idle_proc.pid        = 0;
+	idle_proc.slot       = PROC_MAX;
+	idle_proc.esp        = 0;
+	idle_proc.screen     = (void *)0;
+	idle_proc.stack_base = 0;  /* bootstrap stack, never freed */
+	idle_proc.cr3        = paging_current_cr3();
+	idle_proc.kstack_top = (unsigned int)__boot_stack_top;
+	idle_proc.cpu_ticks  = 0;
+	idle_proc.start_ticks = 0;
+	idle_proc.on_cpu     = 0;
+	idle_proc.last_cpu   = 0;
+	idle_proc.affinity   = 1;
+	idle_proc.weight     = 0;
+	idle_proc.stride     = 0;
+	idle_proc.vruntime   = 0;
+	idle_proc.on_runq    = 0;
+	name_copy(idle_proc.name, "idle");
+
+	cpus[0].idle = &idle_proc;
+	cpus[0].current = &idle_proc;
 }
 
 /*
@@ -185,20 +219,22 @@ void process_init(void)
 static int setup_process(const char *name, unsigned int entry,
 			 int has_arg, const char *arg)
 {
-	int slot, owned;
+	int slot, owned, pid;
 	unsigned int base, top, *sp, arg_ptr = 0, cr3;
 	unsigned int flags;
+	struct process *p;
 
 	flags = irq_save_disable();
 
-	slot = alloc_slot();
-	if (slot < 0) {
+	p = proc_alloc(&slot);
+	if (!p) {
 		irq_restore(flags);
 		return -1;
 	}
 
 	cr3 = paging_new_directory();
 	if (!cr3) {
+		kfree(p);
 		irq_restore(flags);
 		return -1;
 	}
@@ -206,6 +242,7 @@ static int setup_process(const char *name, unsigned int entry,
 	base = stack_alloc(&owned);
 	if (!base) {
 		mm_free_page(cr3);
+		kfree(p);
 		irq_restore(flags);
 		return -1;
 	}
@@ -243,23 +280,10 @@ static int setup_process(const char *name, unsigned int entry,
 	*--sp = 0;              /* ESI */
 	*--sp = 0;              /* EDI */
 
-	procs[slot].esp        = (unsigned int)sp;
-	procs[slot].state      = PROC_READY;
-	procs[slot].pid        = next_pid_counter++;
-	procs[slot].stack_base = owned ? base : 0;
-	procs[slot].cr3        = cr3;
-	procs[slot].kstack_top = top;
-	procs[slot].cpu_ticks  = 0;
-	procs[slot].start_ticks = get_ticks_ms();
-	name_copy(procs[slot].name, name);
-	/* Child inherits the parent's output screen/tty (command output lands
-	   in the terminal that spawned it). */
-	procs[slot].screen = procs[current_pid].screen;
-	procs[slot].tty = procs[current_pid].tty;
-	inherit_cwd(&procs[slot]);
-
+	proc_commit(p, slot, name, (unsigned int)sp, owned, base, cr3, top);
+	pid = (int)p->pid;
 	irq_restore(flags);
-	return (int)procs[slot].pid;
+	return pid;
 }
 
 int process_create(const char *name, void (*entry)(void))
@@ -276,20 +300,22 @@ int process_create_arg(const char *name, void (*entry)(const char *),
 /* Create a process whose entry takes a raw pointer argument (no string copy). */
 int process_create_ptr(const char *name, void (*entry)(void *), void *arg)
 {
-	int slot, owned;
+	int slot, owned, pid;
 	unsigned int base, top, *sp, cr3;
 	unsigned int flags;
+	struct process *p;
 
 	flags = irq_save_disable();
 
-	slot = alloc_slot();
-	if (slot < 0) {
+	p = proc_alloc(&slot);
+	if (!p) {
 		irq_restore(flags);
 		return -1;
 	}
 
 	cr3 = paging_new_directory();
 	if (!cr3) {
+		kfree(p);
 		irq_restore(flags);
 		return -1;
 	}
@@ -297,6 +323,7 @@ int process_create_ptr(const char *name, void (*entry)(void *), void *arg)
 	base = stack_alloc(&owned);
 	if (!base) {
 		mm_free_page(cr3);
+		kfree(p);
 		irq_restore(flags);
 		return -1;
 	}
@@ -311,41 +338,33 @@ int process_create_ptr(const char *name, void (*entry)(void *), void *arg)
 	*--sp = 0; *--sp = 0; *--sp = 0; *--sp = 0;  /* EAX ECX EDX EBX */
 	*--sp = 0; *--sp = 0; *--sp = 0; *--sp = 0;  /* ESP EBP ESI EDI */
 
-	procs[slot].esp        = (unsigned int)sp;
-	procs[slot].state      = PROC_READY;
-	procs[slot].pid        = next_pid_counter++;
-	procs[slot].stack_base = owned ? base : 0;
-	procs[slot].cr3        = cr3;
-	procs[slot].kstack_top = top;
-	procs[slot].cpu_ticks  = 0;
-	procs[slot].start_ticks = get_ticks_ms();
-	name_copy(procs[slot].name, name);
-	procs[slot].screen = procs[current_pid].screen;
-	procs[slot].tty = procs[current_pid].tty;
-	inherit_cwd(&procs[slot]);
+	proc_commit(p, slot, name, (unsigned int)sp, owned, base, cr3, top);
+	pid = (int)p->pid;
 	irq_restore(flags);
-	return (int)procs[slot].pid;
+	return pid;
 }
 
 /* Load an ELF image into a fresh address space and run it as a ring-3 process. */
 int process_create_user_tty(const char *name, const void *image, unsigned int size,
 			    const char *arg, const char *cwd, unsigned int tty)
 {
-	int slot, owned;
+	int slot, owned, pid;
 	unsigned int base, top, *sp, cr3, entry;
 	unsigned int flags;
+	struct process *p;
 	int i;
 
 	flags = irq_save_disable();
 
-	slot = alloc_slot();
-	if (slot < 0) {
+	p = proc_alloc(&slot);
+	if (!p) {
 		irq_restore(flags);
 		return -1;
 	}
 
 	cr3 = paging_new_directory();
 	if (!cr3) {
+		kfree(p);
 		irq_restore(flags);
 		return -1;
 	}
@@ -353,6 +372,7 @@ int process_create_user_tty(const char *name, const void *image, unsigned int si
 	entry = elf_load(cr3, image, size);
 	if (!entry) {
 		mm_free_page(cr3);
+		kfree(p);
 		irq_restore(flags);
 		return -2;   /* not a loadable ELF */
 	}
@@ -371,6 +391,7 @@ int process_create_user_tty(const char *name, const void *image, unsigned int si
 			    !paging_map_page(cr3, va, phys,
 					     PTE_USER | PTE_WRITABLE | PTE_PRESENT)) {
 				mm_free_page(cr3);
+				kfree(p);
 				irq_restore(flags);
 				return -1;
 			}
@@ -397,6 +418,7 @@ int process_create_user_tty(const char *name, const void *image, unsigned int si
 			va = (va_top - (len + 1)) & ~15u;
 			if (va - 8 < USER_STACK_TOP - USER_STACK_PAGES * 0x1000) {
 				mm_free_page(cr3);
+				kfree(p);
 				irq_restore(flags);
 				return -1;   /* argument does not fit the stack */
 			}
@@ -411,6 +433,7 @@ int process_create_user_tty(const char *name, const void *image, unsigned int si
 		base = stack_alloc(&owned);
 		if (!base) {
 			mm_free_page(cr3);
+			kfree(p);
 			irq_restore(flags);
 			return -1;
 		}
@@ -425,27 +448,15 @@ int process_create_user_tty(const char *name, const void *image, unsigned int si
 		*--sp = 0; *--sp = 0; *--sp = 0; *--sp = 0;  /* ESP EBP ESI EDI */
 	}
 
-	procs[slot].esp        = (unsigned int)sp;
-	procs[slot].state      = PROC_READY;
-	procs[slot].pid        = next_pid_counter++;
-	procs[slot].stack_base = owned ? base : 0;
-	procs[slot].cr3        = cr3;
-	procs[slot].kstack_top = top;
-	procs[slot].cpu_ticks  = 0;
-	procs[slot].start_ticks = get_ticks_ms();
-	name_copy(procs[slot].name, name);
-	procs[slot].screen = procs[current_pid].screen;
-	procs[slot].tty = tty ? (int)tty : procs[current_pid].tty;
-	{
-		unsigned int k;
-
-		for (k = 0; cwd && cwd[k] && k + 1 < sizeof(procs[slot].cwd); k++)
-			procs[slot].cwd[k] = cwd[k];
-		procs[slot].cwd[k] = '\0';
-	}
+	proc_commit(p, slot, name, (unsigned int)sp, owned, base, cr3, top);
+	if (cwd)
+		cwd_copy(p, cwd);
+	if (tty)
+		p->tty = (int)tty;
+	pid = (int)p->pid;
 
 	irq_restore(flags);
-	return (int)procs[slot].pid;
+	return pid;
 }
 
 int process_create_user(const char *name, const void *image, unsigned int size,
@@ -458,133 +469,140 @@ int process_create_user(const char *name, const void *image, unsigned int size,
    at startup to bind itself to its own screen). */
 void process_set_screen(struct text_screen *screen)
 {
-	procs[current_pid].screen = screen;
+	struct process *p = current_proc();
+
+	if (p)
+		p->screen = screen;
 }
 
 /* Render target override for the running process (0 = the screen).  Lives in
  * the PCB so a preempted layer draw can never leak into another process. */
 struct surface *process_render_target(void)
 {
-	if (!procs || current_pid >= proc_cap)
-		return 0;
-	return procs[current_pid].target;
+	struct process *p = current_proc();
+
+	return p ? p->target : 0;
 }
 
 void process_set_render_target(struct surface *s)
 {
-	if (!procs || current_pid >= proc_cap)
-		return;
-	procs[current_pid].target = s;
+	struct process *p = current_proc();
+
+	if (p)
+		p->target = s;
 }
 
 void process_map_mmio_all(unsigned int phys_base, unsigned int size)
 {
-	unsigned int i, kernel_cr3 = paging_current_cr3();
+	unsigned int i, flags;
 
+	/* Map the kernel directory (the one new process directories clone) and
+	 * every live process copy.  paging_current_cr3() is the *current* process
+	 * directory, which may be one of these; mapping it twice is harmless.
+	 * The reaper frees PCBs, so hold off the PIT while iterating. */
+	flags = irq_save_disable();
 	paging_map_mmio(phys_base, size);
-	for (i = 0; i < proc_cap; i++) {
-		if (procs[i].state == PROC_UNUSED || procs[i].state == PROC_DEAD)
+	for (i = 0; i < PROC_MAX; i++) {
+		struct process *p = procs[i];
+
+		if (!p || p->state == PROC_DEAD)
 			continue;
-		if (procs[i].cr3 && procs[i].cr3 != kernel_cr3)
-			paging_map_mmio_cr3(procs[i].cr3, phys_base, size);
+		if (p->cr3)
+			paging_map_mmio_cr3(p->cr3, phys_base, size);
 	}
+	irq_restore(flags);
 }
 
-void process_exit(void)
+/* Unpublish a dead process and hand it to the kworker.  Called with interrupts
+ * disabled (from schedule(), process_kill(), or process_exit()'s path). */
+void process_zombie_add(struct process *p)
 {
-	procs[current_pid].state = PROC_DEAD;
-	__asm__ volatile ("sti");  /* re-enable IRQs so the PIT can switch us out */
-	for (;;)
-		__asm__ volatile ("hlt");
+	if (!p || p->zombie_queued)
+		return;
+	p->zombie_queued = 1;
+	p->on_cpu = PROC_NO_CPU;
+	if (p->slot < PROC_MAX)
+		procs[p->slot] = 0;
+	p->zombie_next = 0;
+	if (zombie_tail)
+		zombie_tail->zombie_next = p;
+	else
+		zombie_head = p;
+	zombie_tail = p;
 }
 
-/*
- * Round-robin scheduler.  Called from the PIT IRQ stub with interrupts
- * disabled.  Picks the next runnable process and updates current_pid.
- * The caller (switch.S) takes care of actually switching ESP.
- */
-void schedule(void)
-{
-	unsigned int i, next;
-
-	/* Reap dead processes, freeing their stacks.  Skip the current slot: we
-	 * may be running on its stack right now (it just called process_exit),
-	 * so it is reaped on a later schedule() from another process's context. */
-	for (i = 0; i < proc_cap; i++) {
-		if (procs[i].state == PROC_DEAD && i != current_pid) {
-			sys_net_reap(procs[i].pid);   /* release a held NIC */
-			sys_gfx_reap(procs[i].pid);   /* flag a gfx session dead */
-			tty_reap(procs[i].pid);       /* drop its terminal queue */
-			if (procs[i].stack_base)
-				mm_free_page(procs[i].stack_base);
-			if (procs[i].cr3 &&
-			    procs[i].cr3 != paging_current_cr3()) {
-				/* User code/stack pages belong to the process.
-				 * The gfx canvas mapping is only a window into
-				 * pages owned by the session: release the PTEs
-				 * and page tables, not the pages. */
-				paging_release_user_range(procs[i].cr3,
-					USER_BASE, USER_STACK_TOP - USER_BASE, 1);
-				paging_release_user_range(procs[i].cr3,
-					USER_GFX_BASE, USER_GFX_RANGE, 0);
-				/* WM surface mappings: page tables only; the
-				 * shadow/terminal/canvas pages belong to the
-				 * window server. */
-				paging_release_user_range(procs[i].cr3,
-					WM_SHADOW_VA, 0x00800000, 0);
-				paging_release_user_range(procs[i].cr3,
-					WM_CONTENT_VA_BASE,
-					WM_CONTENT_SLOT * WM_MAX_WINDOWS, 0);
-				mm_free_page(procs[i].cr3);
-			}
-			procs[i].stack_base = 0;
-			procs[i].cr3 = 0;
-			procs[i].state = PROC_UNUSED;
-		}
-	}
+unsigned int process_zombie_count(void)
+{
+	unsigned int flags, n = 0;
+	struct process *p;
 
-	/* A file syscall or a shadow-buffer flush is in progress: stay on the
-	 * current process.  The FAT driver has static state, and the flush walks
-	 * and trims a shared damage list (and must not be abandoned halfway
-	 * through by a context switch).  Interrupts stay enabled, so IRQs are
-	 * still serviced. */
-	if (sys_fs_busy || vga_present_busy()) {
-		if (procs[current_pid].state == PROC_READY)
-			procs[current_pid].state = PROC_RUNNING;
-		return;
-	}
+	flags = irq_save_disable();
+	for (p = zombie_head; p; p = p->zombie_next)
+		n++;
+	irq_restore(flags);
+	return n;
+}
 
-	/* Wake sleeping processes whose deadline has passed. */
-	for (i = 0; i < proc_cap; i++) {
-		if (procs[i].state == PROC_SLEEPING &&
-		    (int)(procs[i].wake_ms - get_ticks_ms()) <= 0)
-			procs[i].state = PROC_READY;
-	}
+/* Free switched-out dead processes.  Runs in the kworker (or the idle
+ * fallback), not in the PIT handler.  Preemption is disabled while the
+ * resources are torn down: sys_gfx_reap()/tty_reap()/the page-table walk are
+ * not re-entrant, matching the old IRQ-context reaper. */
+void process_reap_dead(void)
+{
+	unsigned int cur_cr3 = paging_current_cr3();
+	unsigned int flags;
+	struct process *p, *list;
 
-	/* Mark current as READY (it was RUNNING). */
-	if (procs[current_pid].state == PROC_RUNNING)
-		procs[current_pid].state = PROC_READY;
-
-	/* Find next runnable slot. */
-	next = current_pid;
-	for (i = 0; i < proc_cap; i++) {
-		next = (next + 1) % proc_cap;
-		if (procs[next].state == PROC_READY) {
-			procs[next].state = PROC_RUNNING;
-			current_pid = next;
-			return;
+	preempt_disable();
+	flags = irq_save_disable();
+	list = zombie_head;
+	zombie_head = zombie_tail = 0;
+	irq_restore(flags);
+
+	for (p = list; p; ) {
+		struct process *next = p->zombie_next;
+
+		sys_net_reap(p->pid);   /* release a held NIC */
+		sys_gfx_reap(p->pid);   /* flag a gfx session dead */
+		tty_reap(p->pid);       /* drop its terminal queue */
+		if (p->stack_base)
+			mm_free_page(p->stack_base);
+		if (p->cr3 && p->cr3 != cur_cr3) {
+			/* User code/stack pages belong to the process.  The gfx
+			 * canvas mapping is only a window into pages owned by
+			 * the session: release the PTEs and page tables, not
+			 * the pages. */
+			paging_release_user_range(p->cr3,
+				USER_BASE, USER_STACK_TOP - USER_BASE, 1);
+			paging_release_user_range(p->cr3,
+				USER_GFX_BASE, USER_GFX_RANGE, 0);
+			/* WM surface mappings: page tables only; the shadow/
+			 * terminal/canvas pages belong to the window server. */
+			paging_release_user_range(p->cr3,
+				WM_SHADOW_VA, 0x00800000, 0);
+			paging_release_user_range(p->cr3,
+				WM_CONTENT_VA_BASE,
+				WM_CONTENT_SLOT * WM_MAX_WINDOWS, 0);
+			mm_free_page(p->cr3);
 		}
+		p->stack_base = 0;
+		p->cr3 = 0;
+		kfree(p);
+		p = next;
 	}
+	preempt_enable();
+}
 
-	/* Nothing else runnable -- stay on current if still alive. */
-	if (procs[current_pid].state == PROC_READY) {
-		procs[current_pid].state = PROC_RUNNING;
-		return;
-	}
+void process_exit(void)
+{
+	struct process *p = current_proc();
 
-	/* Everything dead: fall back to slot 0 (kernel idle). */
-	procs[0].state = PROC_RUNNING;
-	current_pid = 0;
+	if (p)
+		p->state = PROC_DEAD;
+	wait_wake_all(&exit_wq);
+	sched_yield();          /* switch away immediately via int 0x81 */
+	for (;;)
+		__asm__ volatile ("hlt");  /* the scheduler never resumes us */
 }
 
 /* ---- user-facing helpers ---- */
@@ -592,9 +610,11 @@ void schedule(void)
 static const char *state_str(unsigned int s)
 {
 	switch (s) {
-	case PROC_RUNNING: return "run  ";
-	case PROC_READY:   return "ready";
-	default:           return "?    ";
+	case PROC_RUNNING:  return "run  ";
+	case PROC_READY:    return "ready";
+	case PROC_SLEEPING: return "sleep";
+	case PROC_BLOCKED:  return "block";
+	default:            return "?    ";
 	}
 }
 
@@ -616,17 +636,43 @@ static void put_dec(unsigned int n)
 
 void process_ps(void)
 {
-	unsigned int i;
-	io_putstr("PID  STATE  NAME\n");
-	for (i = 0; i < proc_cap; i++) {
-		if (procs[i].state == PROC_UNUSED || procs[i].state == PROC_DEAD)
+	unsigned int i, j;
+	io_putstr("PID  STATE  W CPU NAME\n");
+	for (i = 0; i < PROC_MAX; i++) {
+		struct process *p;
+		unsigned int pid, state, weight, cpu, flags;
+		char name[PROC_NAME_LEN];
+
+		/* The reaper frees PCBs; snapshot this slot with the PIT off and
+		 * print the copy outside the critical section. */
+		flags = irq_save_disable();
+		p = procs[i];
+		if (!p || p->state == PROC_DEAD) {
+			irq_restore(flags);
 			continue;
+		}
+		pid = p->pid;
+		state = p->state;
+		weight = p->weight;
+		cpu = p->on_cpu;
+		for (j = 0; j < sizeof(name) - 1 && p->name[j]; j++)
+			name[j] = p->name[j];
+		name[j] = '\0';
+		irq_restore(flags);
+
 		io_putchar(' ');
-		put_dec(procs[i].pid);
+		put_dec(pid);
 		io_putstr("  ");
-		io_putstr(state_str(procs[i].state));
+		io_putstr(state_str(state));
 		io_putstr("  ");
-		io_putstr(procs[i].name);
+		put_dec(weight);
+		io_putstr("   ");
+		if (cpu != PROC_NO_CPU)
+			put_dec(cpu);
+		else
+			io_putchar('-');
+		io_putstr("   ");
+		io_putstr(name);
 		io_putchar('\n');
 	}
 }
@@ -634,8 +680,7 @@ void process_ps(void)
 /* Count the 4 KB user pages mapped between USER_BASE and USER_STACK_TOP.
  * A read-only page-directory walk (paging keeps everything identity-mapped,
  * so the tables are directly addressable).  Used by `top` for the memory
- * hint; ring-3 processes currently leak these pages on reap (known issue),
- * but ex-processes are gone from the table by then anyway. */
+ * hint. */
 static unsigned int count_user_pages(unsigned int cr3)
 {
 	const unsigned int *pd;
@@ -665,91 +710,203 @@ static unsigned int count_user_pages(unsigned int cr3)
  * live processes in slot order, so callers iterate 0, 1, ... until 0. */
 int process_procinfo(unsigned int index, struct sys_procinfo *out)
 {
-	unsigned int i, seen = 0, life;
+	unsigned int i, seen = 0, life, flags;
 
 	if (!out)
 		return 0;
-	for (i = 0; i < proc_cap; i++) {
-		if (procs[i].state == PROC_UNUSED || procs[i].state == PROC_DEAD)
+	flags = irq_save_disable();
+	for (i = 0; i < PROC_MAX; i++) {
+		struct process *p = procs[i];
+
+		if (!p || p->state == PROC_DEAD)
 			continue;
 		if (seen++ != index)
 			continue;
 
-		life = get_ticks_ms() - procs[i].start_ticks;
-		out->pid        = procs[i].pid;
-		out->state      = procs[i].state;
-		out->cpu_ticks  = procs[i].cpu_ticks;
+		life = get_ticks_ms() - p->start_ticks;
+		out->pid        = p->pid;
+		out->state      = p->state;
+		out->cpu_ticks  = p->cpu_ticks;
 		out->life_ticks = life ? life : 1;
 		/* Kernel stack page + page directory + user pages (the WM shares
 		 * the kernel directory; counting one is close enough). */
-		out->pages      = 2 + count_user_pages(procs[i].cr3);
-		name_copy(out->name, procs[i].name);
+		out->pages      = 2 + count_user_pages(p->cr3);
+		out->weight     = p->weight;
+		out->cpu        = p->on_cpu;
+		name_copy(out->name, p->name);
+		irq_restore(flags);
 		return 1;
 	}
+	irq_restore(flags);
 	return 0;
 }
 
 int process_kill(unsigned int pid)
 {
-	unsigned int i;
+	struct process *target = 0;
+	unsigned int i, flags;
+
 	if (pid == 0)
 		return -1; /* cannot kill the kernel idle process */
-	for (i = 1; i < proc_cap; i++) {
-		if (procs[i].state != PROC_UNUSED && procs[i].state != PROC_DEAD &&
-		    procs[i].pid == pid) {
-			procs[i].state = PROC_DEAD;
+	flags = irq_save_disable();
+	for (i = 0; i < PROC_MAX; i++) {
+		struct process *p = procs[i];
+
+		if (p && p->state != PROC_DEAD && p->pid == pid) {
+			target = p;
+			break;
+		}
+	}
+	if (!target) {
+		irq_restore(flags);
+		return -1;
+	}
+	if (target == current_proc()) {
+		target->state = PROC_DEAD;
+		wait_wake_all(&exit_wq);
+		irq_restore(flags);
+		sched_yield();     /* never resumes */
+		return 0;
+	}
+	sched_unlink(target);
+	target->state = PROC_DEAD;
+	process_zombie_add(target);
+	wait_wake_all(&exit_wq);
+	irq_restore(flags);
+	return 0;
+}
+
+int process_set_weight(unsigned int pid, unsigned int weight)
+{
+	unsigned int i, flags;
+
+	if (weight < 1)
+		weight = 1;
+	if (weight > SCHED_WEIGHT_MAX)
+		weight = SCHED_WEIGHT_MAX;
+
+	flags = irq_save_disable();
+	for (i = 0; i < PROC_MAX; i++) {
+		struct process *p = procs[i];
+
+		if (p && p->state != PROC_DEAD && p->pid == pid) {
+			sched_set_weight_locked(p, weight);
+			irq_restore(flags);
 			return 0;
 		}
 	}
+	irq_restore(flags);
 	return -1;
 }
 
-/* Return 1 if a process with `pid` is still alive (READY/RUNNING/SLEEPING). */
-int process_is_alive(unsigned int pid)
+int process_set_affinity(unsigned int pid, unsigned int mask)
+{
+	unsigned int i, flags;
+
+	if (!mask)
+		return -1;
+	flags = irq_save_disable();
+	for (i = 0; i < PROC_MAX; i++) {
+		struct process *p = procs[i];
+
+		if (p && p->state != PROC_DEAD && p->pid == pid) {
+			p->affinity = mask;
+			irq_restore(flags);
+			return 0;
+		}
+	}
+	irq_restore(flags);
+	return -1;
+}
+
+/* Interrupts off. */
+static int process_alive_locked(unsigned int pid)
 {
 	unsigned int i;
-	for (i = 0; i < proc_cap; i++)
-		if (procs[i].pid == pid &&
-		    (procs[i].state == PROC_READY ||
-		     procs[i].state == PROC_RUNNING ||
-		     procs[i].state == PROC_SLEEPING))
+
+	if (pid == 0)
+		return 1;   /* idle is not in the table but always exists */
+	for (i = 0; i < PROC_MAX; i++) {
+		struct process *p = procs[i];
+
+		if (p && p->pid == pid &&
+		    (p->state == PROC_READY ||
+		     p->state == PROC_RUNNING ||
+		     p->state == PROC_SLEEPING ||
+		     p->state == PROC_BLOCKED))
 			return 1;
+	}
 	return 0;
 }
 
+/* Return 1 if a process with `pid` is still alive. */
+int process_is_alive(unsigned int pid)
+{
+	unsigned int flags;
+	int alive;
+
+	flags = irq_save_disable();
+	alive = process_alive_locked(pid);
+	irq_restore(flags);
+	return alive;
+}
+
 int process_is_sleeping(unsigned int pid)
 {
-	unsigned int i;
-	for (i = 0; i < proc_cap; i++)
-		if (procs[i].pid == pid)
-			return procs[i].state == PROC_SLEEPING;
-	return 0;
+	unsigned int i, flags;
+	int sleeping = 0;
+
+	if (pid == 0)
+		return 0;
+	flags = irq_save_disable();
+	for (i = 0; i < PROC_MAX; i++) {
+		struct process *p = procs[i];
+
+		if (p && p->pid == pid) {
+			sleeping = (p->state == PROC_SLEEPING);
+			break;
+		}
+	}
+	irq_restore(flags);
+	return sleeping;
 }
 
 void process_set_tty(unsigned int pid, int tty)
 {
-	unsigned int i;
-	for (i = 0; i < proc_cap; i++)
-		if (procs[i].pid == pid && procs[i].state != PROC_UNUSED) {
-			procs[i].tty = tty;
-			return;
+	unsigned int i, flags;
+
+	flags = irq_save_disable();
+	for (i = 0; i < PROC_MAX; i++) {
+		struct process *p = procs[i];
+
+		if (p && p->pid == pid && p->state != PROC_DEAD) {
+			p->tty = tty;
+			break;
 		}
+	}
+	irq_restore(flags);
 }
 
-/* Block the current process for `ms` milliseconds.  Marks it sleeping and
- * yields to the scheduler; it is woken when its deadline passes. */
+/* Block the current process for `ms` milliseconds.  The sleep list wakes it
+ * from the PIT; no polling loop is needed. */
 void process_sleep_ms(unsigned int ms)
 {
-	procs[current_pid].wake_ms = get_ticks_ms() + ms;
-	procs[current_pid].state = PROC_SLEEPING;
-	__asm__ volatile ("sti");  /* re-enable IRQs so the PIT can switch us out */
-	while (procs[current_pid].state == PROC_SLEEPING)
-		cpu_idle();
+	sched_sleep_ms(ms);
 }
 
-/* Wait (yield) until the process with `pid` has exited. */
+/* Wait until the process with `pid` has exited.  The exit path wakes the
+ * queue, so this is event-driven rather than a 10 ms poll. */
 void process_wait(unsigned int pid)
 {
-	while (process_is_alive(pid))
-		process_sleep_ms(10);
+	for (;;) {
+		unsigned int flags = irq_save_disable();
+
+		if (!process_alive_locked(pid)) {
+			irq_restore(flags);
+			return;
+		}
+		sched_block_on_locked(&exit_wq);
+		irq_restore(flags);
+		sched_yield();
+	}
 }

+ 45 - 10
process.h

@@ -1,7 +1,9 @@
-/* Preemptive round-robin multitasking for i586, ring-0 only. */
+/* Process objects and lifecycle.  Scheduling itself lives in sched.c. */
 #ifndef PROCESS_H
 #define PROCESS_H
 
+#include "sched.h"
+
 #define PROC_STACK_SIZE 0x1000  /* 4 KB per process (= 1 page) */
 
 /* Process stacks are allocated from the frame allocator (mm.c), i.e. above the
@@ -9,11 +11,19 @@
  * pool in BSS so the system still boots with fewer processes. */
 #define PROC_FALLBACK_STACK_MAX  8
 
+/* Stable process table: slots hold pointers to individually allocated PCBs, so
+ * a process pointer never moves.  Keep the table small; PCBs are heap objects. */
+#define PROC_MAX 64
+
 #define PROC_UNUSED  0
 #define PROC_READY   1
 #define PROC_RUNNING 2
 #define PROC_DEAD    3
 #define PROC_SLEEPING 4
+#define PROC_BLOCKED 5
+
+/* on_cpu when the process is not currently running anywhere. */
+#define PROC_NO_CPU 0xFFFFFFFFu
 
 #define TIMESLICE_TICKS 20 /* context switch every 20 ms (PIT @ 1 kHz) */
 
@@ -25,9 +35,10 @@ struct sys_procinfo; /* forward decl (see syscall.h; `top` snapshot) */
 struct surface;      /* forward decl (see vga.h; render target override) */
 
 struct process {
-	unsigned int esp;
+	unsigned int esp;            /* offset 0: switch.S saves it here */
 	unsigned int state;
 	unsigned int pid;
+	unsigned int slot;           /* index in procs[], PROC_MAX for idle */
 	char         name[PROC_NAME_LEN];
 	struct text_screen *screen;  /* output target for io_putchar & co. */
 	unsigned int stack_base;     /* page to free on exit, or 0 (bootstrap/fallback) */
@@ -35,17 +46,33 @@ struct process {
 	unsigned int kstack_top;     /* ring-0 stack top (for TSS esp0) */
 	unsigned int wake_ms;        /* tick deadline when PROC_SLEEPING */
 	char cwd[PROC_CWD_MAX];      /* working directory (ring-3 commands) */
-	/* CPU accounting for `top`.  switch.S credits the running process on
-	 * every PIT tick, so the layout after cwd must match PCB_CPU_TICKS /
-	 * PCB_SIZE there (see the _Static_assert in process.c). */
+	/* CPU accounting for `top`. */
 	unsigned int cpu_ticks;      /* ticks spent as the running process */
 	unsigned int start_ticks;    /* get_ticks_ms() at creation */
 	/* Render target override: 0 = the screen.  Set while the window manager
 	 * draws into a layer, so another process's drawing can never land in
-	 * it.  Kept before tty so the PIT hot path offsets stay put. */
+	 * it. */
 	struct surface *target;
 	/* TTY output id inherited by children (0 = none, use the old screen). */
 	int tty;
+	/* Per-CPU scaffolding: these are maintained even with one CPU so the
+	 * scheduler can be extended without reshaping the PCB. */
+	unsigned int on_cpu;         /* PROC_NO_CPU when not running */
+	unsigned int last_cpu;
+	unsigned int affinity;       /* CPU bitmask; all CPUs by default */
+	/* CFS-lite scheduling entity.  vruntime/stride are 64-bit; no 64-bit
+	 * division happens on the tick path (stride is computed in set_weight). */
+	unsigned int weight;         /* 1..SCHED_WEIGHT_MAX */
+	unsigned int on_runq;        /* linked in the ready queue */
+	unsigned long long vruntime;
+	unsigned long long stride;
+	unsigned int slice_start;    /* tick when the current slice began */
+	struct process *run_next, *run_prev;
+	struct process *sleep_next, *sleep_prev;
+	struct process *wait_next, *wait_prev;
+	struct wait_queue *waiting_on;
+	struct process *zombie_next;    /* reaped by the kworker */
+	unsigned int zombie_queued;
 };
 
 /* User-mode virtual address layout (ring-3 processes). */
@@ -58,9 +85,8 @@ struct process {
 #define USER_GFX_BASE    0x50000000
 #define USER_GFX_RANGE   0x00400000   /* one 4 MB slot */
 
-/* Dynamically-sized process table (grown on demand; see process.c). */
-extern struct process *procs;
-extern volatile unsigned int current_pid;
+/* Stable, sparse process table (see process.c). */
+extern struct process *procs[PROC_MAX];
 
 void process_init(void);
 int  process_create(const char *name, void (*entry)(void));
@@ -89,7 +115,12 @@ void process_set_render_target(struct surface *s);
  * would only be visible to the kernel bootstrap context. */
 void process_map_mmio_all(unsigned int phys_base, unsigned int size);
 void process_exit(void);
-void schedule(void);
+/* Called by the kworker: free switched-out dead processes. */
+void process_reap_dead(void);
+unsigned int process_zombie_count(void);
+/* Called from schedule()/kill with interrupts disabled: unpublish a dead PCB
+ * and put it on the kworker's zombie list. */
+void process_zombie_add(struct process *p);
 void process_ps(void);
 
 /* Fill *out with a snapshot of the index-th live process (slot order) for
@@ -102,6 +133,10 @@ int  process_is_alive(unsigned int pid);
 int  process_is_sleeping(unsigned int pid);
 void process_set_tty(unsigned int pid, int tty);
 void process_wait(unsigned int pid);
+/* Adjust the CFS weight of a live process (1..SCHED_WEIGHT_MAX). */
+int  process_set_weight(unsigned int pid, unsigned int weight);
+/* Restrict a live process to a CPU bitmask (0 is ignored). */
+int  process_set_affinity(unsigned int pid, unsigned int mask);
 
 /* Block the current process for `ms` milliseconds (yields the CPU). */
 void process_sleep_ms(unsigned int ms);

+ 630 - 0
sched.c

@@ -0,0 +1,630 @@
+/* CFS-lite scheduler core: per-CPU state, a weighted run queue, a sorted sleep
+ * list and the timer entry.
+ *
+ * Weight is an integer 1..SCHED_WEIGHT_MAX.  Each ready process has a 64-bit
+ * virtual runtime advanced by (elapsed ticks * stride) while it runs, where
+ * stride = SCHED_WEIGHT_SCALE / weight.  The smallest vruntime gets the CPU,
+ * so the long-term share is proportional to the weight.  A 10 ms slice bounds
+ * latency; a wake-up gets clamped to the current minimum vruntime so an
+ * interactive process is scheduled promptly without accumulating unbounded
+ * credit.
+ *
+ * The run queue is a single global list for now.  That is also the shape an
+ * SMP version wants: one queue protected by a lock, with per-CPU current.
+ */
+
+#include "sched.h"
+#include "io.h"
+#include "paging.h"
+#include "process.h"
+#include "syscall.h"
+#include "timer.h"
+#include "tss.h"
+
+/* 2^20 / weight, so weight 8 advances 131072 per tick and weight 16 half
+ * that.  The product fits comfortably in the 64-bit vruntime. */
+#define SCHED_WEIGHT_SCALE (1u << 20)
+
+struct cpu cpus[MAX_CPUS];
+unsigned int cpu_count;
+unsigned int cpu_gs_ready;
+
+static struct process *runq_head, *runq_tail;
+static struct process *sleep_head, *sleep_tail;
+static unsigned int wakeups;
+
+/* switch.S reads these two facts directly: current at %gs:0 and esp at
+ * offset 0 of the process.  Keep them true. */
+_Static_assert(__builtin_offsetof(struct cpu, current) == 0,
+	       "switch.S reads current from %gs:0");
+_Static_assert(__builtin_offsetof(struct process, esp) == 0,
+	       "switch.S saves the interrupted esp at offset 0");
+
+static inline unsigned int irq_save_disable(void)
+{
+	unsigned int flags;
+	__asm__ volatile ("pushfl; popl %0; cli" : "=r"(flags));
+	return flags;
+}
+
+static inline void irq_restore(unsigned int flags)
+{
+	__asm__ volatile ("pushl %0; popfl" : : "r"(flags));
+}
+
+static inline unsigned long long weight_stride(unsigned int weight)
+{
+	if (weight < 1)
+		weight = 1;
+	if (weight > SCHED_WEIGHT_MAX)
+		weight = SCHED_WEIGHT_MAX;
+	return (unsigned long long)(SCHED_WEIGHT_SCALE / weight);
+}
+
+/* ---- run queue (interrupts off) ---- */
+
+static void runq_add_locked(struct process *p)
+{
+	if (p->on_runq)
+		return;
+	p->on_runq = 1;
+	p->run_prev = runq_tail;
+	p->run_next = 0;
+	if (runq_tail)
+		runq_tail->run_next = p;
+	else
+		runq_head = p;
+	runq_tail = p;
+}
+
+static void runq_del_locked(struct process *p)
+{
+	if (!p->on_runq)
+		return;
+	if (p->run_prev)
+		p->run_prev->run_next = p->run_next;
+	else
+		runq_head = p->run_next;
+	if (p->run_next)
+		p->run_next->run_prev = p->run_prev;
+	else
+		runq_tail = p->run_prev;
+	p->on_runq = 0;
+	p->run_next = p->run_prev = 0;
+}
+
+static void sleepq_add_locked(struct process *p)
+{
+	struct process *it = sleep_head, *prev = 0;
+
+	while (it && (int)(it->wake_ms - p->wake_ms) <= 0) {
+		prev = it;
+		it = it->sleep_next;
+	}
+	p->sleep_prev = prev;
+	p->sleep_next = it;
+	if (prev)
+		prev->sleep_next = p;
+	else
+		sleep_head = p;
+	if (it)
+		it->sleep_prev = p;
+	else
+		sleep_tail = p;
+}
+
+static void sleepq_del_locked(struct process *p)
+{
+	if (p->sleep_prev)
+		p->sleep_prev->sleep_next = p->sleep_next;
+	else if (sleep_head == p)
+		sleep_head = p->sleep_next;
+	if (p->sleep_next)
+		p->sleep_next->sleep_prev = p->sleep_prev;
+	else if (sleep_tail == p)
+		sleep_tail = p->sleep_prev;
+	p->sleep_next = p->sleep_prev = 0;
+}
+
+void wait_queue_init(struct wait_queue *wq)
+{
+	wq->head = wq->tail = 0;
+}
+
+static void waitq_add_locked(struct wait_queue *wq, struct process *p)
+{
+	p->waiting_on = wq;
+	p->wait_next = 0;
+	p->wait_prev = wq->tail;
+	if (wq->tail)
+		wq->tail->wait_next = p;
+	else
+		wq->head = p;
+	wq->tail = p;
+}
+
+static void waitq_del_locked(struct process *p)
+{
+	struct wait_queue *wq = p->waiting_on;
+
+	if (!wq)
+		return;
+	if (p->wait_prev)
+		p->wait_prev->wait_next = p->wait_next;
+	else
+		wq->head = p->wait_next;
+	if (p->wait_next)
+		p->wait_next->wait_prev = p->wait_prev;
+	else
+		wq->tail = p->wait_prev;
+	p->wait_next = p->wait_prev = 0;
+	p->waiting_on = 0;
+}
+
+/* Current minimum vruntime among READY processes.  If nothing is ready, use
+ * the running process's vruntime so a freshly woken process does not receive
+ * an enormous credit relative to a CPU-bound one. */
+static unsigned long long min_vruntime_locked(void)
+{
+	struct process *p;
+	unsigned long long min = 0;
+	int found = 0;
+
+	for (p = runq_head; p; p = p->run_next) {
+		if (!found || p->vruntime < min) {
+			min = p->vruntime;
+			found = 1;
+		}
+	}
+	if (found)
+		return min;
+	p = current_proc();
+	return (p && p != this_cpu()->idle) ? p->vruntime : 0;
+}
+
+static void account_locked(struct process *p)
+{
+	unsigned int elapsed;
+
+	if (!p || p == this_cpu()->idle || p->state != PROC_RUNNING)
+		return;
+	elapsed = get_ticks_ms() - p->slice_start;
+	if (elapsed) {
+		p->vruntime += (unsigned long long)elapsed * p->stride;
+		p->slice_start = get_ticks_ms();
+	}
+}
+
+/* Clamp a woken process to the current minimum and make it runnable. */
+static void wake_locked(struct process *p)
+{
+	struct process *cur = current_proc();
+	struct cpu *c = this_cpu();
+
+	if (!p || p->state == PROC_DEAD || p->state == PROC_UNUSED)
+		return;
+	if (p->waiting_on)
+		waitq_del_locked(p);
+	if (p->state == PROC_SLEEPING)
+		sleepq_del_locked(p);
+	p->state = PROC_READY;
+	if (p->vruntime < min_vruntime_locked())
+		p->vruntime = min_vruntime_locked();
+	runq_add_locked(p);
+	wakeups++;
+	if (cur && (cur == c->idle ||
+		    p->weight >= cur->weight))
+		c->need_resched = 1;
+}
+
+void sched_add_ready(struct process *p)
+{
+	if (!p->weight) {
+		p->weight = SCHED_WEIGHT_DEFAULT;
+		p->stride = weight_stride(p->weight);
+		p->vruntime = min_vruntime_locked();
+	}
+	p->state = PROC_READY;
+	runq_add_locked(p);
+}
+
+void sched_unlink(struct process *p)
+{
+	runq_del_locked(p);
+	if (p->waiting_on)
+		waitq_del_locked(p);
+	if (p->state == PROC_SLEEPING)
+		sleepq_del_locked(p);
+}
+
+void sched_set_weight_locked(struct process *p, unsigned int weight)
+{
+	if (!p)
+		return;
+	if (weight < 1)
+		weight = 1;
+	if (weight > SCHED_WEIGHT_MAX)
+		weight = SCHED_WEIGHT_MAX;
+	if (p->state == PROC_RUNNING)
+		account_locked(p);
+	p->weight = weight;
+	p->stride = weight_stride(weight);
+}
+
+/* Add the current process to a wait queue.  The caller holds interrupts off
+ * and must restore the flags and call sched_yield() itself. */
+void sched_block_on_locked(struct wait_queue *wq)
+{
+	struct process *p = current_proc();
+
+	if (!p)
+		return;
+	runq_del_locked(p);
+	waitq_add_locked(wq, p);
+	p->state = PROC_BLOCKED;
+	p->on_runq = 0;
+}
+
+void sched_block_on(struct wait_queue *wq)
+{
+	unsigned int flags;
+
+	if (this_cpu()->preempt_count)
+		return;
+	flags = irq_save_disable();
+	sched_block_on_locked(wq);
+	irq_restore(flags);
+	sched_yield();
+}
+
+/* Block on a wait queue with a deadline.  Either a waiter wake or the sleep
+ * list expiry removes the process from both queues. */
+void sched_block_timeout(struct wait_queue *wq, unsigned int ms)
+{
+	struct process *p = current_proc();
+	unsigned int flags;
+
+	if (!p)
+		return;
+	if (ms == 0) {
+		sched_block_on(wq);
+		return;
+	}
+	if (this_cpu()->preempt_count)
+		return;
+	flags = irq_save_disable();
+	p->wake_ms = get_ticks_ms() + ms;
+	p->state = PROC_SLEEPING;
+	runq_del_locked(p);
+	waitq_add_locked(wq, p);
+	sleepq_add_locked(p);
+	irq_restore(flags);
+	sched_yield();
+}
+
+static void wake_one_locked(struct wait_queue *wq)
+{
+	struct process *p = wq->head;
+
+	if (!p)
+		return;
+	waitq_del_locked(p);
+	if (p->state == PROC_BLOCKED || p->state == PROC_SLEEPING)
+		wake_locked(p);
+}
+
+void wait_wake_one(struct wait_queue *wq)
+{
+	unsigned int flags = irq_save_disable();
+
+	wake_one_locked(wq);
+	irq_restore(flags);
+}
+
+void wait_wake_all(struct wait_queue *wq)
+{
+	unsigned int flags = irq_save_disable();
+
+	while (wq->head)
+		wake_one_locked(wq);
+	irq_restore(flags);
+}
+
+void sched_init(void)
+{
+	unsigned int i;
+
+	cpu_count = 1;
+	runq_head = runq_tail = 0;
+	sleep_head = sleep_tail = 0;
+	wakeups = 0;
+
+	for (i = 0; i < MAX_CPUS; i++) {
+		cpus[i].current = 0;
+		cpus[i].idle = 0;
+		cpus[i].id = i;
+		cpus[i].lapic_id = 0;
+		cpus[i].need_resched = 0;
+		cpus[i].preempt_count = 0;
+		cpus[i].idle_ticks = 0;
+		cpus[i].switches = 0;
+	}
+}
+
+static void sched_switch_to(struct process *next)
+{
+	struct cpu *c = this_cpu();
+	unsigned int cr3 = paging_current_cr3();
+
+	if (next->cr3 && next->cr3 != cr3) {
+		__asm__ volatile ("movl %0, %%cr3"
+				  :
+				  : "r"(next->cr3)
+				  : "memory");
+	}
+	if (c->tss && next->kstack_top)
+		c->tss->esp0 = next->kstack_top;
+}
+
+/* Pick the READY process with the smallest vruntime that may run on this
+ * CPU.  Ties go to the lower pid for determinism. */
+static struct process *pick_locked(void)
+{
+	struct process *p, *best = 0;
+	unsigned int mask = 1u << this_cpu()->id;
+
+	for (p = runq_head; p; p = p->run_next) {
+		if (!(p->affinity & mask))
+			continue;
+		if (!best || p->vruntime < best->vruntime ||
+		    (p->vruntime == best->vruntime && p->pid < best->pid))
+			best = p;
+	}
+	return best;
+}
+
+static void wake_sleepers_locked(void)
+{
+	unsigned int now = get_ticks_ms();
+
+	while (sleep_head && (int)(sleep_head->wake_ms - now) <= 0) {
+		struct process *p = sleep_head;
+
+		sleepq_del_locked(p);
+		p->state = PROC_READY;
+		p->wake_ms = 0;
+		wake_locked(p);
+	}
+}
+
+/*
+ * CFS-lite schedule.  Called from the PIT stub (after ESP has been saved) or
+ * from the int 0x81 gate, always with interrupts disabled.
+ */
+void schedule(void)
+{
+	struct cpu *c = this_cpu();
+	struct process *cur = c->current;
+	struct process *next;
+
+	if (cur && cur != c->idle) {
+		if (cur->state == PROC_RUNNING) {
+			account_locked(cur);
+			cur->state = PROC_READY;
+			runq_add_locked(cur);
+		} else if (cur->state == PROC_READY) {
+			/* Woken between setting BLOCKED/SLEEPING and entering
+			 * the gate: already on the run queue. */
+		} else if (cur->state == PROC_DEAD) {
+			/* Keep the PCB alive until the kworker can free it, but
+			 * unpublish it immediately. */
+			process_zombie_add(cur);
+		}
+	}
+
+	next = pick_locked();
+	if (!next) {
+		next = c->idle;
+		if (!next) {
+			for (;;)
+				__asm__ volatile ("cli; hlt");
+		}
+	}
+
+	if (next != cur) {
+		if (cur)
+			cur->on_cpu = PROC_NO_CPU;
+		c->switches++;
+	}
+	runq_del_locked(next);
+	next->state = PROC_RUNNING;
+	next->on_cpu = c->id;
+	next->last_cpu = c->id;
+	next->slice_start = get_ticks_ms();
+	c->current = next;
+	c->need_resched = 0;
+	sched_switch_to(next);
+}
+
+void sched_yield(void)
+{
+	struct cpu *c = this_cpu();
+
+	/* Blocking helpers must not be used inside a preempt-disabled section.
+	 * If one is, record the request instead of switching with the count
+	 * still owned by the old process. */
+	if (c->preempt_count) {
+		c->need_resched = 1;
+		return;
+	}
+	__asm__ volatile ("int $0x81" ::: "memory");
+}
+
+void sched_sleep_ms(unsigned int ms)
+{
+	struct process *p = current_proc();
+	unsigned int flags;
+
+	if (!p)
+		return;
+	if (ms == 0) {
+		sched_yield();
+		return;
+	}
+	if (this_cpu()->preempt_count)
+		return;   /* sleeping inside a non-preemptible section is invalid */
+	flags = irq_save_disable();
+	p->wake_ms = get_ticks_ms() + ms;
+	p->state = PROC_SLEEPING;
+	runq_del_locked(p);
+	sleepq_add_locked(p);
+	irq_restore(flags);
+	sched_yield();
+}
+
+/* Called from the PIT after ticks has been incremented.  Returns 1 when the
+ * assembly should call schedule(). */
+int sched_tick_bsp(void)
+{
+	struct cpu *c = this_cpu();
+	struct process *p = c->current;
+
+	if (p) {
+		p->cpu_ticks++;
+		if (p == c->idle)
+			c->idle_ticks++;
+	}
+
+	wake_sleepers_locked();
+
+	if (c->preempt_count) {
+		if (p && p != c->idle && p->state == PROC_RUNNING &&
+		    (unsigned int)(get_ticks_ms() - p->slice_start) >=
+		    SCHED_SLICE_TICKS)
+			c->need_resched = 1;
+		return 0;
+	}
+
+	if (!p)
+		return 0;
+	if (p == c->idle) {
+		if (runq_head) {
+			c->need_resched = 1;
+			return 1;
+		}
+		return 0;
+	}
+	if (p->state != PROC_RUNNING) {
+		c->need_resched = 1;
+		return 1;
+	}
+	if ((unsigned int)(get_ticks_ms() - p->slice_start) >= SCHED_SLICE_TICKS)
+		c->need_resched = 1;
+	return c->need_resched ? 1 : 0;
+}
+
+void preempt_disable(void)
+{
+	this_cpu()->preempt_count++;
+}
+
+void preempt_enable(void)
+{
+	struct cpu *c = this_cpu();
+
+	if (c->preempt_count)
+		c->preempt_count--;
+}
+
+int sched_snapshot(struct sys_schedinfo *out)
+{
+	unsigned int flags, i;
+	struct process *p;
+
+	if (!out)
+		return -1;
+	flags = irq_save_disable();
+	out->ncpus = cpu_count;
+	out->runnable = 0;
+	for (p = runq_head; p; p = p->run_next)
+		out->runnable++;
+	out->sleeping = 0;
+	for (p = sleep_head; p; p = p->sleep_next)
+		out->sleeping++;
+	out->blocked = 0;
+	for (i = 0; i < PROC_MAX; i++) {
+		p = procs[i];
+		if (p && p->state == PROC_BLOCKED)
+			out->blocked++;
+	}
+	out->zombies = process_zombie_count();
+	out->switches = cpus[0].switches;
+	out->wakeups = wakeups;
+	out->idle_ticks = cpus[0].idle_ticks;
+	out->slice_ticks = SCHED_SLICE_TICKS;
+	irq_restore(flags);
+	return 0;
+}
+
+static void sched_put_dec(unsigned int n)
+{
+	char buf[12];
+	unsigned int i = sizeof(buf) - 1;
+
+	buf[i] = '\0';
+	if (n == 0) {
+		io_putchar('0');
+		return;
+	}
+	while (n && i > 0) {
+		buf[--i] = (char)('0' + (n % 10));
+		n /= 10;
+	}
+	io_putstr(buf + i);
+}
+
+void sched_print(void)
+{
+	struct sys_schedinfo s;
+	unsigned int uptime, idle_pm, up_s, idle_s, idle_frac;
+
+	if (sched_snapshot(&s) != 0)
+		return;
+	uptime = get_ticks_ms();
+
+	/* Avoid 64-bit division: scale to seconds first.  ticks wraps at ~49
+	 * days, so idle_s * 1000 stays within 32 bits for the useful range. */
+	up_s = uptime / 1000u;
+	if (up_s == 0) {
+		idle_pm = (s.idle_ticks >= uptime) ? 1000 : 0;
+	} else {
+		idle_s = s.idle_ticks / 1000u;
+		idle_frac = s.idle_ticks % 1000u;
+		idle_pm = (idle_s * 1000u) / up_s;
+		idle_pm += (idle_frac * 1000u / up_s) / 1000u;
+		if (idle_pm > 1000u)
+			idle_pm = 1000u;
+	}
+
+	io_putstr("sched: cfs, ");
+	sched_put_dec(s.ncpus);
+	io_putstr(" cpu, ");
+	sched_put_dec(s.slice_ticks);
+	io_putstr(" ms slice\nrunq ");
+	sched_put_dec(s.runnable);
+	io_putstr("  sleep ");
+	sched_put_dec(s.sleeping);
+	io_putstr("  block ");
+	sched_put_dec(s.blocked);
+	io_putstr("  zombie ");
+	sched_put_dec(s.zombies);
+	io_putstr("\nswitches ");
+	sched_put_dec(s.switches);
+	io_putstr("  wakeups ");
+	sched_put_dec(s.wakeups);
+	io_putstr("  idle ");
+	sched_put_dec(idle_pm / 10);
+	io_putchar('.');
+	io_putchar((char)('0' + idle_pm % 10));
+	io_putstr("%\n");
+}

+ 98 - 0
sched.h

@@ -0,0 +1,98 @@
+/* Scheduler core: per-CPU state, the running process, and the tick entry.
+ *
+ * Only CPU 0 exists today, but everything is shaped for more CPUs later:
+ * current is per-CPU, reached through a per-CPU GS segment, and the timer
+ * path goes through sched_tick_cpu-style code rather than a global pid index.
+ */
+#ifndef SCHED_H
+#define SCHED_H
+
+#define MAX_CPUS 8
+
+#define SCHED_WEIGHT_DEFAULT 8
+#define SCHED_WEIGHT_MAX     16
+#define SCHED_SLICE_TICKS    10   /* PIT ticks (1 kHz) per timeslice */
+
+struct process;
+struct tss;
+struct sys_schedinfo;
+
+struct wait_queue {
+	struct process *head;
+	struct process *tail;
+};
+
+struct cpu {
+	struct process *current;   /* offset 0, read by switch.S (%gs:0) */
+	struct process *idle;
+	struct tss     *tss;       /* this CPU's TSS; schedule() updates esp0 */
+	struct cpu     *self;      /* this_cpu() reads it through %gs */
+	unsigned int    id;
+	unsigned int    lapic_id;
+	unsigned int    need_resched;
+	unsigned int    preempt_count;
+	unsigned int    idle_ticks;
+	unsigned int    switches;
+};
+
+extern struct cpu cpus[MAX_CPUS];
+extern unsigned int cpu_count;
+extern unsigned int cpu_gs_ready;
+
+/* The GS descriptor points at this CPU's struct cpu.  Before the GDT has
+ * been loaded (early boot, or a fault before gdt_tss_init), fall back to
+ * the BSP entry so panic/console code can still run safely. */
+static inline struct cpu *this_cpu(void)
+{
+	struct cpu *c;
+
+	if (!cpu_gs_ready)
+		return &cpus[0];
+	__asm__ volatile ("movl %%gs:%c1, %0"
+			  : "=r"(c)
+			  : "i"(__builtin_offsetof(struct cpu, self)));
+	return c;
+}
+
+static inline struct process *current_proc(void)
+{
+	return this_cpu()->current;
+}
+
+void sched_init(void);
+void schedule(void);
+int  sched_tick_bsp(void);
+
+/* Voluntary reschedule through the int 0x81 gate.  Must not be called with
+ * preempt_count set; blocking helpers below handle that. */
+void sched_yield(void);
+
+/* Block the current process until its deadline, or add it to a ready queue.
+ * Both are the entry points process.c uses. */
+void sched_sleep_ms(unsigned int ms);
+void sched_add_ready(struct process *p);
+void sched_unlink(struct process *p);
+
+/* Wait queues.  wait_wake_* may be called from any process context; the
+ * block helpers handle interrupts themselves. */
+void wait_queue_init(struct wait_queue *wq);
+void sched_block_on(struct wait_queue *wq);
+void sched_block_on_locked(struct wait_queue *wq);
+void sched_block_timeout(struct wait_queue *wq, unsigned int ms);
+void wait_wake_one(struct wait_queue *wq);
+void wait_wake_all(struct wait_queue *wq);
+
+/* Use with interrupts disabled.  sched_set_weight_locked accounts the current
+ * slice with the old weight before changing the stride. */
+void sched_set_weight_locked(struct process *p, unsigned int weight);
+
+/* Per-CPU non-preemptible sections (the FAT driver and the shadow flush).
+ * The PIT still increments ticks and wakes sleepers, but does not switch. */
+void preempt_disable(void);
+void preempt_enable(void);
+
+/* Snapshot for SYS_SCHEDINFO and the `sched` command. */
+int  sched_snapshot(struct sys_schedinfo *out);
+void sched_print(void);
+
+#endif

+ 52 - 58
switch.S

@@ -1,50 +1,41 @@
 /*
  * Context-switch PIT IRQ stub.
  *
- * On entry the CPU has already pushed EFLAGS, CS, EIP onto the
- * current process's stack.  We save the rest, bump the tick counter,
- * and -- when the timeslice expires -- call schedule() and swap ESP
- * to the next process.
+ * On entry the CPU has already pushed EFLAGS, CS, EIP onto the current
+ * process's stack.  We save the rest, bump the tick counter, and -- when
+ * sched_tick_bsp() says the timeslice expired -- call schedule() and swap
+ * ESP to the next process.
  *
- * Shadow buffer flush: damage-driven, done by the main loop (byte budget).
- * Main loop checks dirty rows and copies only what changed.
+ * The only process facts this file knows are:
+ *   - %gs:0 is struct cpu.current
+ *   - offset 0 of struct process is esp (saved/restored here)
+ * CR3 and TSS.esp0 are switched by schedule() in C.
  *
- * Serial poll: every tick, check UART for received data and push
- * to ring buffer so no bytes are lost during preemption.
+ * Serial poll: every tick, check UART for received data and push to the ring
+ * buffer so no bytes are lost during preemption.
  */
 .code32
 .text
 
 .extern ticks
-.extern current_pid
-.extern procs
 .extern schedule
+.extern sched_tick_bsp
 .extern serial_poll
-.extern tss
 
 .include "irqret.inc"
 
-/* struct process offsets (must match process.h / process.c) */
-.equ PCB_ESP,   0
-.equ PCB_CR3,   32
-.equ PCB_KSTACK_TOP, 36
-.equ PCB_CPU_TICKS, 108  /* after cwd[64] */
-.equ PCB_SIZE,  124     /* sizeof(struct process): 44 + cwd[64] + 2*4 + target + tty */
-
-.equ TIMESLICE, 20
-
 .globl pit_irq_stub
 pit_irq_stub:
 	pusha
 
-	/* Load the kernel data segment.  The PIT can fire while a ring-3 process
-	 * is running, and in that case the CPU has switched CS/SS but left DS
-	 * null (IRET reset it when it returned to ring-3); the data accesses
-	 * below (ticks, counters, procs, tss) go through DS and would #GP. */
+	/* Load the kernel selectors.  The PIT can fire while a ring-3 process
+	 * is running; DS would then be null after the IRET that entered ring 3,
+	 * and the accesses below would #GP.  GS is the per-CPU data selector. */
 	movw  $0x10, %ax
 	movw  %ax, %ds
 	movw  %ax, %es
 	movw  %ax, %fs
+	movw  $0x28, %ax
 	movw  %ax, %gs
 
 	/* ticks++ and send EOI -- always, even when we don't switch. */
@@ -52,51 +43,54 @@ pit_irq_stub:
 	movb  $0x20, %al
 	outb  %al, $0x20
 
-	/* Account this tick to the process that was interrupted (for `top`). */
-	movl  current_pid, %eax
-	imull $PCB_SIZE, %eax
-	addl  procs, %eax
-	incl  PCB_CPU_TICKS(%eax)
-
 	/* Poll UART for received serial data (prevents FIFO overflow). */
 	call  serial_poll
 
-	/* Decrement timeslice counter; switch only when it reaches 0. */
-	decl  slice_counter
-	jnz   .no_switch
+	/* Account the tick and decide whether the timeslice expired. */
+	call  sched_tick_bsp
+	testl %eax, %eax
+	jz    .no_switch
 
-	/* Reset counter for next timeslice. */
-	movl  $TIMESLICE, slice_counter
+	/* Save the interrupted ESP into current->esp. */
+	movl  %gs:0, %eax
+	movl  %esp, (%eax)
 
-	/* Save current ESP into procs[current_pid].esp (procs is a pointer). */
-	movl  current_pid, %eax
-	imull $PCB_SIZE, %eax
-	addl  procs, %eax
-	movl  %esp, PCB_ESP(%eax)
-
-	/* Let the C scheduler pick the next process. */
+	/* Let the C scheduler pick the next process and switch CR3/TSS. */
 	call  schedule
 
-	/* Load new ESP, CR3, and TSS esp0 from procs[current_pid]. */
-	movl  current_pid, %eax
-	imull $PCB_SIZE, %eax
-	addl  procs, %eax
-	movl  PCB_ESP(%eax), %esp
-	movl  PCB_CR3(%eax), %ebx
-	movl  %ebx, %cr3
-	movl  PCB_KSTACK_TOP(%eax), %ebx
-	movl  %ebx, tss+4           /* TSS.esp0 = this process's ring-0 stack top */
+	/* Load the new process's ESP. */
+	movl  %gs:0, %eax
+	movl  (%eax), %esp
 
 .no_switch:
-	/* Both paths (return to the interrupted process, or to one just switched
-	 * in) must leave a usable data segment selector for the context being
-	 * resumed; ring-3 needs 0x23 (see irqret.inc). */
+	/* Both paths must leave a usable data selector for the context being
+	 * resumed; ring-3 needs 0x23, ring-0 uses 0x10 + GS 0x28. */
 	IRQ_RETURN_DS 36
 
 	popa
 	iret
 
-.data
-.align 4
-slice_counter:
-	.long TIMESLICE
+/*
+ * Voluntary reschedule gate: int 0x81.  Entered from ring 0 only, with the
+ * CPU having pushed EFLAGS/CS/EIP exactly like the PIT path.  No EOI and no
+ * tick accounting; just save ESP, pick a new process and restore its frame.
+ */
+.globl resched_stub
+resched_stub:
+	pusha
+	movw  $0x10, %ax
+	movw  %ax, %ds
+	movw  %ax, %es
+	movw  %ax, %fs
+	movw  $0x28, %ax
+	movw  %ax, %gs
+
+	movl  %gs:0, %eax
+	movl  %esp, (%eax)
+	call  schedule
+	movl  %gs:0, %eax
+	movl  (%eax), %esp
+
+	IRQ_RETURN_DS 36
+	popa
+	iret

+ 5 - 7
sys_fs.c

@@ -5,8 +5,8 @@
  *   - the FAT driver keeps state in static buffers, so no other process may
  *     run its code concurrently;
  *   - the floppy driver waits for IRQ6, so interrupts must stay enabled.
- * So instead of a lock we set `sys_fs_busy` for the duration: the scheduler
- * (process.c) will not switch away from the running process while it is set.
+ * So instead of a lock we disable preemption for the duration: the scheduler
+ * will not switch away from the running process while the count is set.
  * That keeps each operation atomic (and immune to Ctrl+C killing the process
  * mid-write) at the cost of a brief stall during disk I/O.  Buffers point
  * into the caller's address space, which is also the active one during the
@@ -16,18 +16,16 @@
 #include "process.h"
 #include "vfs.h"
 
-volatile int sys_fs_busy;
-
 void sys_fs_enter(void)
 {
-	sys_fs_busy = 1;
+	preempt_disable();
 	__asm__ volatile ("sti");   /* the disk may need its IRQ */
 }
 
 void sys_fs_exit(void)
 {
 	__asm__ volatile ("cli");
-	sys_fs_busy = 0;
+	preempt_enable();
 }
 
 static int sys_err(int vfs_err)
@@ -53,7 +51,7 @@ static int fs_resolve(const char *path, struct vfs_mount **m,
 	/* Relative paths resolve against the calling process's cwd (inherited
 	 * from the terminal that spawned it); "" means the cwd itself. */
 	if (path[0] != '/') {
-		if (vfs_join_path(procs[current_pid].cwd, path, abs, sizeof(abs)) != 0)
+		if (vfs_join_path(current_proc()->cwd, path, abs, sizeof(abs)) != 0)
 			return SYS_EINVAL;
 		path = abs;
 	}

+ 2 - 6
sys_fs.h

@@ -4,12 +4,8 @@
 
 #include "syscall.h"
 
-/* Set while a FAT operation is in progress: the scheduler must not switch
- * away from the running process (see sys_fs.c). */
-extern volatile int sys_fs_busy;
-
-/* Enter/leave the fs-busy section (used by other disk-touching services,
- * e.g. the storage info refresh). */
+/* Enter/leave the fs non-preemptible section (used by other disk-touching
+ * services, e.g. the storage info refresh).  See sys_fs.c. */
 void sys_fs_enter(void);
 void sys_fs_exit(void);
 

+ 9 - 9
sys_gfx.c

@@ -89,9 +89,9 @@ int sys_gfx_begin(struct sys_gfx_info *info)
 
 	if (!info)
 		return -1;
-	if (by_pid(procs[current_pid].pid))
+	if (by_pid(current_proc()->pid))
 		return -1;                       /* one session per process */
-	t = wm_term_for_screen(procs[current_pid].screen);
+	t = wm_term_for_screen(current_proc()->screen);
 	if (!t)
 		return -1;                       /* no window (not spawned by a terminal) */
 	if (by_screen(&t->screen))
@@ -120,12 +120,12 @@ int sys_gfx_begin(struct sys_gfx_info *info)
 		s->back[i] = 0;
 
 	for (i = 0; i < pages; i++) {
-		if (!paging_map_page(procs[current_pid].cr3,
+		if (!paging_map_page(current_proc()->cr3,
 				     GFX_USER_BASE + i * PAGE_SIZE,
 				     (unsigned int)s->back + i * PAGE_SIZE,
 				     PTE_USER | PTE_WRITABLE | PTE_PRESENT)) {
 			while (i--)
-				paging_unmap_page(procs[current_pid].cr3,
+				paging_unmap_page(current_proc()->cr3,
 						  GFX_USER_BASE + i * PAGE_SIZE);
 			free_back(s);
 			gfx_free(&s->c);
@@ -136,8 +136,8 @@ int sys_gfx_begin(struct sys_gfx_info *info)
 
 	s->used = 1;
 	s->dead = 0;
-	s->pid = procs[current_pid].pid;
-	s->cr3 = procs[current_pid].cr3;
+	s->pid = current_proc()->pid;
+	s->cr3 = current_proc()->cr3;
 	s->term = t;
 	s->key_head = s->key_tail = 0;
 
@@ -151,7 +151,7 @@ int sys_gfx_begin(struct sys_gfx_info *info)
 
 int sys_gfx_present(void)
 {
-	struct gfx_session *s = by_pid(procs[current_pid].pid);
+	struct gfx_session *s = by_pid(current_proc()->pid);
 	const unsigned short *src;   /* the caller's own view of its canvas */
 	unsigned int n, i;
 
@@ -177,7 +177,7 @@ int sys_gfx_present(void)
 
 int sys_gfx_end(void)
 {
-	struct gfx_session *s = by_pid(procs[current_pid].pid);
+	struct gfx_session *s = by_pid(current_proc()->pid);
 
 	if (!s)
 		return -1;
@@ -191,7 +191,7 @@ int sys_gfx_end(void)
 
 int sys_gfx_key_poll(void)
 {
-	struct gfx_session *s = by_pid(procs[current_pid].pid);
+	struct gfx_session *s = by_pid(current_proc()->pid);
 	int k;
 
 	if (!s)

+ 1 - 1
sys_net.c

@@ -40,7 +40,7 @@ static void net_exit(void)
 
 static unsigned int caller_pid(void)
 {
-	return procs[current_pid].pid;
+	return current_proc()->pid;
 }
 
 /* Called from the scheduler's reap loop for every dead process. */

+ 6 - 1
sys_wm.c

@@ -9,7 +9,7 @@
 
 static int user_ok(const void *p, unsigned int size, int write)
 {
-	return paging_user_range_ok(procs[current_pid].cr3,
+	return paging_user_range_ok(current_proc()->cr3,
 				    (unsigned int)p, size, write);
 }
 
@@ -168,3 +168,8 @@ int sys_win_close(void)
 {
 	return wm_server_win_close();
 }
+
+int sys_win_key_wait(unsigned int timeout_ms)
+{
+	return wm_server_key_wait(timeout_ms);
+}

+ 1 - 0
sys_wm.h

@@ -20,5 +20,6 @@ int sys_wm_sync(void);
 int sys_win_create(const struct win_create *in, struct win_info *out);
 int sys_win_present(void);
 int sys_win_close(void);
+int sys_win_key_wait(unsigned int timeout_ms);
 
 #endif

+ 27 - 10
syscall.c

@@ -21,7 +21,7 @@ static int copy_user_string(char *dst, unsigned int max, const char *src)
 	unsigned int i;
 
 	if (!src || !max ||
-	    !paging_user_range_ok(procs[current_pid].cr3, (unsigned int)src,
+	    !paging_user_range_ok(current_proc()->cr3, (unsigned int)src,
 				  max, 0))
 		return -1;
 	for (i = 0; i + 1 < max && src[i]; i++)
@@ -145,7 +145,7 @@ int syscall_dispatch(int num, int a1, int a2, int a3)
 	case SYS_NET_CONFIGURED:
 		return sys_net_configured();
 	case SYS_GETCWD: {
-		const char *cwd = procs[current_pid].cwd;
+		const char *cwd = current_proc()->cwd;
 		char *out = (char *)a1;
 		unsigned int i = 0, max = (unsigned int)a2;
 
@@ -190,7 +190,7 @@ int syscall_dispatch(int num, int a1, int a2, int a3)
 		unsigned int i;
 
 		if ((unsigned int)a2 < sizeof(buf) ||
-		    !paging_user_range_ok(procs[current_pid].cr3,
+		    !paging_user_range_ok(current_proc()->cr3,
 					  (unsigned int)out, sizeof(buf), 1))
 			return -1;
 		rtc_read_datetime(buf);
@@ -232,6 +232,13 @@ int syscall_dispatch(int num, int a1, int a2, int a3)
 		return sys_win_present();
 	case SYS_WIN_CLOSE:
 		return sys_win_close();
+	case SYS_WIN_KEY_WAIT:
+		return sys_win_key_wait((unsigned int)a1);
+	case SYS_SCHEDINFO:
+		if (!paging_user_range_ok(current_proc()->cr3, (unsigned int)a1,
+					  sizeof(struct sys_schedinfo), 1))
+			return -1;
+		return sched_snapshot((struct sys_schedinfo *)a1);
 	case SYS_TTY_CREATE:
 		return tty_create();
 	case SYS_TTY_READ: {
@@ -240,7 +247,7 @@ int syscall_dispatch(int num, int a1, int a2, int a3)
 		unsigned int max = (unsigned int)a3;
 
 		if (!tty_valid(id) || !buf || !max ||
-		    !paging_user_range_ok(procs[current_pid].cr3,
+		    !paging_user_range_ok(current_proc()->cr3,
 					  (unsigned int)buf, max, 1))
 			return -1;
 		return tty_read(id, buf, max);
@@ -250,9 +257,10 @@ int syscall_dispatch(int num, int a1, int a2, int a3)
 		struct vfs_file f;
 		struct vfs_mount *m = 0;
 		char path[VFS_PATH_MAX], arg[VFS_PATH_MAX], cwd[VFS_PATH_MAX];
+		const char *name;
 		int pid;
 
-		if (!paging_user_range_ok(procs[current_pid].cr3,
+		if (!paging_user_range_ok(current_proc()->cr3,
 					  (unsigned int)a1, sizeof(in), 0))
 			return -1;
 		{
@@ -271,7 +279,16 @@ int syscall_dispatch(int num, int a1, int a2, int a3)
 			return -1;
 		if (vfs_load_file(path, &f, &m) != 0)
 			return -1;
-		pid = process_create_user_tty(path, f.data, f.size, arg, cwd,
+		name = path;
+		{
+			const char *q = path;
+			while (*q) {
+				if (*q == '/')
+					name = q + 1;
+				q++;
+			}
+		}
+		pid = process_create_user_tty(name, f.data, f.size, arg, cwd,
 					      in.tty);
 		vfs_release_file(m, &f);
 		return pid;
@@ -297,16 +314,16 @@ int syscall_dispatch(int num, int a1, int a2, int a3)
 
 		if (copy_user_string(arg, sizeof(arg), (const char *)a1) != 0)
 			return -1;
-		if (vfs_join_path(procs[current_pid].cwd, arg[0] ? arg : ".",
+		if (vfs_join_path(current_proc()->cwd, arg[0] ? arg : ".",
 				  abs, sizeof(abs)) != 0)
 			return -1;
 		if (vfs_stat(abs, &size, &is_dir) != 0 || !is_dir)
 			return -1;
 		{
 			unsigned int i;
-			for (i = 0; abs[i] && i + 1 < sizeof(procs[current_pid].cwd); i++)
-				procs[current_pid].cwd[i] = abs[i];
-			procs[current_pid].cwd[i] = '\0';
+			for (i = 0; abs[i] && i + 1 < sizeof(current_proc()->cwd); i++)
+				current_proc()->cwd[i] = abs[i];
+			current_proc()->cwd[i] = '\0';
 		}
 		return 0;
 	}

+ 18 - 1
syscall.h

@@ -112,6 +112,8 @@
 #define SYS_ALIVE          77  /* (pid) -> 1/0 */
 #define SYS_KILL           78  /* (pid) -> 0/-1 */
 #define SYS_CHDIR          79  /* (path) -> 0/-1 */
+#define SYS_WIN_KEY_WAIT   80  /* (timeout_ms) -> key / 0 timeout / -1 */
+#define SYS_SCHEDINFO      81  /* (struct sys_schedinfo *out) -> 0/-1 */
 
 struct sys_spawn {
 	const char *path;
@@ -168,13 +170,28 @@ struct sys_meminfo {
  * are PIT ticks (1 kHz), so lifecycle percentages can be derived. */
 struct sys_procinfo {
 	unsigned int pid;
-	unsigned int state;       /* PROC_*: 1 ready, 2 running, 4 sleeping */
+	unsigned int state;       /* PROC_*: 1 ready, 2 running, 4 sleep, 5 block */
 	unsigned int cpu_ticks;   /* ticks spent as the running process */
 	unsigned int life_ticks;  /* ticks since creation, >= 1 */
 	unsigned int pages;       /* resident 4 KB pages (stack + dir + user) */
+	unsigned int weight;      /* CFS weight, 1..SCHED_WEIGHT_MAX */
+	unsigned int cpu;         /* CPU id, or 0xFFFFFFFF when not running */
 	char name[12];
 };
 
+/* Scheduler counters for `top` and the `sched` command. */
+struct sys_schedinfo {
+	unsigned int ncpus;
+	unsigned int runnable;
+	unsigned int sleeping;
+	unsigned int blocked;
+	unsigned int zombies;
+	unsigned int switches;
+	unsigned int wakeups;
+	unsigned int idle_ticks;
+	unsigned int slice_ticks;
+};
+
 /* Window canvas handed to a ring-3 graphics command (see sys_gfx.c). */
 struct sys_gfx_info {
 	unsigned int width;    /* canvas (= window content) size in pixels */

+ 5 - 3
syscall_stub.S

@@ -16,14 +16,16 @@ syscall_stub:
 	 * re-enable interrupts themselves. */
 	cli
 
-	/* Load the kernel data segment into DS/ES/FS/GS.  int 0x80 only switches
-	 * CS and SS; IRET reset DS (etc.) to null when it returned to ring-3, so
-	 * DS is null here and the C dispatcher's data accesses would #GP. */
+	/* Load the kernel selectors: DS/ES/FS = 0x10, GS = 0x28 (the per-CPU
+	 * data selector used by this_cpu()).  int 0x80 only switches CS and SS;
+	 * IRET reset DS (etc.) to null when it returned to ring-3, so DS is null
+	 * here and the C dispatcher's data accesses would #GP. */
 	pushl %eax
 	movw  $0x10, %ax
 	movw  %ax, %ds
 	movw  %ax, %es
 	movw  %ax, %fs
+	movw  $0x28, %ax
 	movw  %ax, %gs
 	popl  %eax
 

+ 69 - 7
terminal.c

@@ -25,6 +25,18 @@ extern void program_lspci_main(const char *filter);
 extern void program_peek_main(const char *arg);
 extern void program_poke_main(const char *arg);
 
+static inline unsigned int term_irq_save_disable(void)
+{
+	unsigned int flags;
+	__asm__ volatile ("pushfl; popl %0; cli" : "=r"(flags));
+	return flags;
+}
+
+static inline void term_irq_restore(unsigned int flags)
+{
+	__asm__ volatile ("pushl %0; popfl" : : "r"(flags));
+}
+
 /* Print the prompt (with the current directory) and set line_start. */
 static void term_print_prompt(struct term *t, int first)
 {
@@ -80,6 +92,8 @@ static int cmd_kexec(struct term *t, const char *args);
 static int cmd_kill(struct term *t, const char *args);
 static int cmd_poweroff(struct term *t, const char *args);
 static int cmd_ps(struct term *t, const char *args);
+static int cmd_renice(struct term *t, const char *args);
+static int cmd_sched(struct term *t, const char *args);
 static int cmd_reboot(struct term *t, const char *args);
 static int cmd_run(struct term *t, const char *args);
 static int cmd_serial(struct term *t, const char *args);
@@ -104,6 +118,8 @@ static const struct kcmd cmds[] = {
 	{ "peek",       "peek <addr> [len]",             CK_SPAWN_ARG, 0, 0, program_peek_main, 0, 0 },
 	{ "poke",       "poke <addr> <val>",             CK_SPAWN_ARG, CMD_REQARG, 0, program_poke_main, 0, 0 },
 	{ "ps",         "ps",                            CK_CUSTOM, 0, 0, 0, 0, cmd_ps },
+	{ "sched",      "sched",                         CK_CUSTOM, 0, 0, 0, 0, cmd_sched },
+	{ "renice",     "renice <pid> <weight>",         CK_CUSTOM, CMD_REQARG, 0, 0, 0, cmd_renice },
 	{ "kill",       "kill <pid>",                    CK_CUSTOM, 0, 0, 0, 0, cmd_kill },
 	{ "kexec",      "kexec",                         CK_CUSTOM, 0, 0, 0, 0, cmd_kexec },
 	{ "dbg",        "dbg",                           CK_CUSTOM, 0, 0, 0, 0, cmd_dbg },
@@ -464,6 +480,46 @@ static int cmd_ps(struct term *t, const char *args)
 	return 0;
 }
 
+static int cmd_sched(struct term *t, const char *args)
+{
+	(void)t;
+	(void)args;
+	sched_print();
+	return 0;
+}
+
+static int cmd_renice(struct term *t, const char *args)
+{
+	unsigned int pid = 0, weight = 0;
+	int have_pid = 0, have_weight = 0;
+
+	(void)t;
+	while (*args >= '0' && *args <= '9') {
+		pid = pid * 10 + (unsigned int)(*args - '0');
+		args++;
+		have_pid = 1;
+	}
+	while (*args == ' ')
+		args++;
+	while (*args >= '0' && *args <= '9') {
+		weight = weight * 10 + (unsigned int)(*args - '0');
+		args++;
+		have_weight = 1;
+	}
+	while (*args == ' ')
+		args++;
+	if (!have_pid || !have_weight || *args ||
+	    weight < 1 || weight > SCHED_WEIGHT_MAX) {
+		io_putstr("renice: usage renice <pid> <1..16>\n");
+		return 0;
+	}
+	if (process_set_weight(pid, weight) < 0)
+		io_putstr("renice: no such process\n");
+	else
+		io_putstr("weight changed\n");
+	return 0;
+}
+
 static int cmd_reboot(struct term *t, const char *args)
 {
 	(void)t;
@@ -1191,9 +1247,18 @@ void term_run(struct term *t)
 			continue;
 		}
 		{
+			unsigned int flags = term_irq_save_disable();
 			int k = term_key_pop(t);
-			if (k) {
-				if (k == IO_KEY_CTRL_C) {
+
+			if (!k) {
+				sched_block_on_locked(&t->key_wq);
+				term_irq_restore(flags);
+				sched_yield();
+				continue;
+			}
+			term_irq_restore(flags);
+
+			if (k == IO_KEY_CTRL_C) {
 					/* Ctrl+C at the prompt: discard the line. */
 					io_putstr("^C\n");
 					t->line_len = 0;
@@ -1286,12 +1351,9 @@ void term_run(struct term *t)
 					redraw_line_from(t->line, t->line_len, t->line_pos, t->line_start, t->line_pos - 1);
 				}
 			}
-		} else {
-			process_sleep_ms(10);
 		}
 	}
 }
-}
 
 struct kcmd_ctx {
 	char name[16];
@@ -1332,8 +1394,8 @@ static void kcmd_entry(void *arg)
 					p[i] = 0;
 				t->fg_pid = -1;
 				for (i = 0; i + 1 < sizeof(t->cwd) &&
-					    procs[current_pid].cwd[i]; i++)
-					t->cwd[i] = procs[current_pid].cwd[i];
+					    current_proc()->cwd[i]; i++)
+					t->cwd[i] = current_proc()->cwd[i];
 				t->cwd[i] = '\0';
 				{
 					const char *def = "/ramfs/bin";

+ 8 - 1
terminal.h

@@ -2,6 +2,7 @@
 #ifndef TERMINAL_H
 #define TERMINAL_H
 
+#include "sched.h"
 #include "screen.h"
 #include "vfs.h"
 
@@ -23,10 +24,11 @@ struct term {
 	int fg_pid;                 /* foreground child pid, or -1 when none */
 	char cwd[VFS_PATH_MAX];     /* absolute current working directory */
 	char path[VFS_PATH_MAX];    /* command search path (colon separated) */
-	/* Per-terminal key queue: WM (slot 0) pushes, terminal process pops. */
+	/* Per-terminal key queue: the kworker pushes, the terminal pops. */
 	int keys[KEY_BUF_SIZE];
 	unsigned int key_head;
 	unsigned int key_tail;
+	struct wait_queue key_wq;
 };
 
 /* Entry point for a terminal process (arg = struct term *). */
@@ -57,4 +59,9 @@ static inline int term_key_pop(struct term *t)
 	return k;
 }
 
+static inline void term_key_wake(struct term *t)
+{
+	wait_wake_one(&t->key_wq);
+}
+
 #endif

+ 3 - 0
timer.S

@@ -14,6 +14,7 @@ kbd_irq_stub:
 	movw  %ax, %ds
 	movw  %ax, %es
 	movw  %ax, %fs
+	movw  $0x28, %ax
 	movw  %ax, %gs
 	call  kbd_irq
 	IRQ_RETURN_DS 36
@@ -27,6 +28,7 @@ fdc_irq_stub:
 	movw  %ax, %ds
 	movw  %ax, %es
 	movw  %ax, %fs
+	movw  $0x28, %ax
 	movw  %ax, %gs
 	call  fdc_irq
 	IRQ_RETURN_DS 36
@@ -102,6 +104,7 @@ exc_common:
 	movw  %ax, %ds
 	movw  %ax, %es
 	movw  %ax, %fs
+	movw  $0x28, %ax
 	movw  %ax, %gs
 	popl  %eax
 

+ 60 - 32
tss.c

@@ -1,12 +1,13 @@
-/* TSS + flat GDT (kernel code/data, user code/data, TSS). */
+/* TSS + per-CPU flat GDT (see tss.h for the selector layout). */
 
 #include "tss.h"
+#include "sched.h"
 
 /* Top of the in-image bootstrap stack (see kernel.ld): used until the first
  * context switch replaces TSS.esp0 with the process's own kernel stack. */
 extern char __boot_stack_top[];
 
-struct tss tss;
+#define GDT_ENTRIES (6 + MAX_CPUS)
 
 struct gdt_entry {
 	unsigned short limit_low;
@@ -22,46 +23,73 @@ struct gdt_ptr {
 	unsigned int   base;
 } __attribute__((packed));
 
-static struct gdt_entry gdt[6];
-static struct gdt_ptr   gdtp;
+static struct tss tss[MAX_CPUS];
+static struct gdt_entry gdt[MAX_CPUS][GDT_ENTRIES]
+	__attribute__((aligned(8)));
 
-static void set_entry(int i, unsigned int base, unsigned int limit,
-		      unsigned char access, unsigned char flags)
+static void set_entry(struct gdt_entry *g, int i, unsigned int base,
+		      unsigned int limit, unsigned char access,
+		      unsigned char flags)
 {
-	gdt[i].limit_low  = (unsigned short)(limit & 0xFFFF);
-	gdt[i].base_low   = (unsigned short)(base & 0xFFFF);
-	gdt[i].base_mid   = (unsigned char)((base >> 16) & 0xFF);
-	gdt[i].access     = access;
-	gdt[i].limit_high_flags =
+	g[i].limit_low  = (unsigned short)(limit & 0xFFFF);
+	g[i].base_low   = (unsigned short)(base & 0xFFFF);
+	g[i].base_mid   = (unsigned char)((base >> 16) & 0xFF);
+	g[i].access     = access;
+	g[i].limit_high_flags =
 		(unsigned char)(((limit >> 16) & 0x0F) | (flags & 0xF0));
-	gdt[i].base_high  = (unsigned char)((base >> 24) & 0xFF);
+	g[i].base_high  = (unsigned char)((base >> 24) & 0xFF);
 }
 
-void gdt_tss_init(void)
+void gdt_tss_init(unsigned int cpu)
 {
+	struct cpu *c;
+	struct gdt_entry *g;
+	struct gdt_ptr gp;
+	unsigned short sel;
 	unsigned int i;
 
-	for (i = 0; i < sizeof(gdt); i++)
-		((unsigned char *)gdt)[i] = 0;
-	for (i = 0; i < sizeof(tss); i++)
-		((unsigned char *)&tss)[i] = 0;
+	if (cpu >= MAX_CPUS)
+		return;
+
+	c = &cpus[cpu];
+	g = gdt[cpu];
+
+	for (i = 0; i < sizeof(gdt[cpu]); i++)
+		((unsigned char *)g)[i] = 0;
+	for (i = 0; i < sizeof(tss[cpu]); i++)
+		((unsigned char *)&tss[cpu])[i] = 0;
+
+	c->self = c;
+	c->tss = &tss[cpu];
+	c->id = cpu;
 
 	/* 0x00 null */
 	/* 0x08 kernel code, 0x10 kernel data */
-	set_entry(1, 0, 0xFFFFF, 0x9A, 0xCF);
-	set_entry(2, 0, 0xFFFFF, 0x92, 0xCF);
+	set_entry(g, 1, 0, 0xFFFFF, 0x9A, 0xCF);
+	set_entry(g, 2, 0, 0xFFFFF, 0x92, 0xCF);
 	/* 0x18 user code, 0x20 user data (DPL=3) */
-	set_entry(3, 0, 0xFFFFF, 0xFA, 0xCF);
-	set_entry(4, 0, 0xFFFFF, 0xF2, 0xCF);
-	/* 0x28 TSS */
-	set_entry(5, (unsigned int)&tss, sizeof(tss) - 1, 0x89, 0x00);
-
-	gdtp.limit = (unsigned short)(sizeof(gdt) - 1);
-	gdtp.base  = (unsigned int)gdt;
-	__asm__ volatile ("lgdt %0" : : "m"(gdtp));
-
-	tss.ss0  = 0x10;    /* kernel data segment */
-	tss.esp0 = (unsigned int)__boot_stack_top;
-	tss.iomap_base = 0xFFFF;  /* no I/O permission bitmap (deny ring-3 I/O) */
-	__asm__ volatile ("ltr %0" : : "r"((unsigned short)0x28));
+	set_entry(g, 3, 0, 0xFFFFF, 0xFA, 0xCF);
+	set_entry(g, 4, 0, 0xFFFFF, 0xF2, 0xCF);
+	/* 0x28 per-CPU data: GS base = this CPU's state */
+	set_entry(g, 5, (unsigned int)c, 0xFFFFF, 0x92, 0xCF);
+	/* 0x30 + 8*cpu: one 32-bit TSS per CPU */
+	for (i = 0; i < MAX_CPUS; i++)
+		set_entry(g, 6 + i, (unsigned int)&tss[i],
+			  sizeof(tss[i]) - 1, 0x89, 0x00);
+
+	tss[cpu].ss0 = 0x10;    /* kernel data segment */
+	tss[cpu].esp0 = (unsigned int)__boot_stack_top;
+	tss[cpu].iomap_base = 0xFFFF;  /* no I/O permission bitmap */
+
+	gp.limit = (unsigned short)(sizeof(gdt[cpu]) - 1);
+	gp.base  = (unsigned int)&g[0];
+	__asm__ volatile ("lgdt %0" : : "m"(gp));
+
+	sel = (unsigned short)(0x30 + 8 * cpu);
+	__asm__ volatile ("ltr %0" : : "r"(sel));
+
+	sel = 0x28;
+	__asm__ volatile ("movw %0, %%gs" : : "r"(sel));
+
+	cpu_gs_ready = 1;
 }

+ 10 - 6
tss.h

@@ -1,4 +1,9 @@
-/* TSS + flat GDT (kernel/user segments + TSS descriptor). */
+/* TSS + per-CPU flat GDT.
+ *
+ * Layout: 0x00 null, 0x08 kernel code, 0x10 kernel data, 0x18 user code,
+ * 0x20 user data, 0x28 per-CPU data (GS base), 0x30 + 8*cpu TSS.
+ * Every CPU gets its own GDT because the GS descriptor base differs.
+ */
 #ifndef TSS_H
 #define TSS_H
 
@@ -14,10 +19,9 @@ struct tss {
 	unsigned short trap, iomap_base;
 } __attribute__((packed));
 
-extern struct tss tss;
-
-/* Build the 6-entry GDT (null / kernel code / kernel data / user code / user
- * data / TSS), load it, and load the TSS.  Call once from kernel_main. */
-void gdt_tss_init(void);
+/* Build/load the GDT and TSS for one CPU and point GS at cpus[cpu].
+ * CPU 0 is initialized from kernel_main before paging; AP bring-up will
+ * call the same function with its own index later. */
+void gdt_tss_init(unsigned int cpu);
 
 #endif

+ 4 - 3
tty.c

@@ -62,18 +62,19 @@ int tty_valid(int id)
 
 int tty_create(void)
 {
+	struct process *p = current_proc();
 	unsigned int i;
 
-	if (!tty_init())
+	if (!p || !tty_init())
 		return -1;
 	for (i = 0; i < TTY_MAX; i++) {
 		struct tty *t = tty_slot(i);
 
 		if (!t->used) {
 			t->used = 1;
-			t->owner = procs[current_pid].pid;
+			t->owner = p->pid;
 			t->head = t->tail = 0;
-			procs[current_pid].tty = (int)(i + 1);
+			p->tty = (int)(i + 1);
 			return (int)(i + 1);
 		}
 	}

+ 10 - 0
ucmd_lib.h

@@ -285,6 +285,11 @@ static inline int u_win_close(void)
 	return u_syscall0(SYS_WIN_CLOSE);
 }
 
+static inline int u_win_key_wait(unsigned int timeout_ms)
+{
+	return u_syscall1(SYS_WIN_KEY_WAIT, (int)timeout_ms);
+}
+
 static inline int u_tty_create(void)
 {
 	return u_syscall0(SYS_TTY_CREATE);
@@ -347,4 +352,9 @@ static inline int u_key_poll(void)
 	return u_syscall0(SYS_KEY_POLL);
 }
 
+static inline int u_schedinfo(struct sys_schedinfo *out)
+{
+	return u_syscall1(SYS_SCHEDINFO, (int)out);
+}
+
 #endif

+ 36 - 0
ucmd_spin.c

@@ -0,0 +1,36 @@
+/* spin: CPU-bound test process for scheduler/priority checks.
+ *
+ * Usage: spin [ms]   (default 5000 ms)
+ */
+#include "ucmd_lib.h"
+
+static unsigned int parse_ms(const char *s)
+{
+	unsigned int n = 0;
+
+	while (*s >= '0' && *s <= '9') {
+		n = n * 10 + (unsigned int)(*s - '0');
+		s++;
+	}
+	return n;
+}
+
+void _start(const char *arg)
+{
+	unsigned int ms = parse_ms(arg);
+	unsigned int t0, i;
+	unsigned long loops = 0;
+
+	if (ms == 0)
+		ms = 5000;
+	t0 = u_ticks_ms();
+	while (u_ticks_ms() - t0 < ms) {
+		for (i = 0; i < 10000; i++)
+			__asm__ volatile ("" ::: "memory");
+		loops++;
+	}
+	u_puts("spin: done (");
+	u_putdec(loops);
+	u_puts(" loops)\n");
+	u_exit(0);
+}

+ 12 - 2
ucmd_term.c

@@ -423,15 +423,23 @@ static void run_line(void)
 		struct sys_procinfo pi;
 		unsigned int i = 0;
 
-		term_puts("PID  STATE  NAME\n");
+		term_puts("PID  STATE  W CPU NAME\n");
 		while (u_syscall2(SYS_PROCINFO, (int)i, (int)&pi) == 1) {
 			print_num(pi.pid);
 			term_puts("  ");
 			if (pi.state == 1) term_puts("ready");
 			else if (pi.state == 2) term_puts("run  ");
 			else if (pi.state == 4) term_puts("sleep");
+			else if (pi.state == 5) term_puts("block");
 			else term_puts("?    ");
 			term_puts("  ");
+			print_num(pi.weight);
+			term_puts("   ");
+			if (pi.cpu != 0xFFFFFFFFu)
+				print_num(pi.cpu);
+			else
+				term_putc('-');
+			term_puts("   ");
 			term_puts(pi.name);
 			term_putc('\n');
 			i++;
@@ -630,7 +638,9 @@ void _start(const char *arg)
 					handle_key(k);
 			}
 		} else {
-			int k = u_key_poll();
+			int k = u_win_key_wait(100);
+			if (k < 0)
+				u_exit(1);
 			while (k) {
 				handle_key(k);
 				got = 1;

+ 80 - 47
ucmd_top.c

@@ -2,9 +2,9 @@
  *
  * The kernel credits every PIT tick to the process that was running it
  * (SYS_PROCINFO exposes the counters), so each frame samples the counters
- * twice, one second apart: CPU% is the share of ticks in that window.  The
- * WM slot ("wm") runs the idle loop, so on an idle system it owns nearly
- * every tick -- the footer says as much.
+ * twice, one second apart: CPU% is the share of ticks in that window.  Idle
+ * is not a process slot anymore, so an idle system simply shows little CPU
+ * against the live processes.
  *
  * The frame is repainted in place.  It has a fixed shape (23 rows x 63
  * columns + newlines), so the number of characters written per frame is a
@@ -21,11 +21,13 @@
 #define FRAME_COLS  63            /* terminal screens are 64x32 (kernel screen.h) */
 #define COL_PID     4
 #define COL_STATE   7
+#define COL_W       3
 #define COL_CPU     6
 #define COL_TIME    6
 #define COL_MEM     6
-/* five single-space separators between the six columns */
-#define NAME_W      (FRAME_COLS - COL_PID - COL_STATE - COL_CPU - COL_TIME - COL_MEM - 5)
+/* six single-space separators between the seven columns */
+#define NAME_W      (FRAME_COLS - COL_PID - COL_STATE - COL_W - COL_CPU - \
+		     COL_TIME - COL_MEM - 6)
 #define PRINT_ROWS  20            /* process rows (header + footer excluded) */
 #define MAX_SAMPLE  32            /* processes captured per sample */
 #define FRAME_ROWS  (3 + PRINT_ROWS)
@@ -38,11 +40,14 @@ struct sample {
 };
 
 static struct sample before[MAX_SAMPLE];
-static unsigned int nbefore;
+static struct sample after[MAX_SAMPLE];
+static unsigned int nbefore, nafter;
+static struct sys_procinfo snaps[MAX_SAMPLE];
 
 struct prow {
 	unsigned int pid;
 	unsigned int state;
+	unsigned int weight;
 	unsigned int cpu_pm;     /* CPU per mille over the window */
 	unsigned int cpu_ticks;  /* total ticks spent running */
 	unsigned int pages;      /* resident pages */
@@ -53,6 +58,7 @@ static struct prow rows[PRINT_ROWS];
 static unsigned int nrows, ntotal;
 static unsigned int wall;                 /* sample window, ms */
 static unsigned int uptime_s;
+static unsigned int idle_pm;              /* idle per mille over the window */
 static unsigned int free_kb, total_kb;
 static unsigned int line_chars;           /* chars written into the current row */
 
@@ -154,6 +160,8 @@ static const char *state_name(unsigned int s)
 		return "ready";
 	if (s == 4)
 		return "sleep";
+	if (s == 5)
+		return "block";
 	return "?";
 }
 
@@ -169,42 +177,58 @@ static void end_line(int last)
 	}
 }
 
-/* Take the "after" sample and turn it into rows, CPU% = ticks during the
- * window (a process that appeared during the window shows 0%). */
-static void gather(void)
+/* Snapshot the live process list once.  Both the "before" and "after"
+ * samples go through this helper; no process-info syscall happens after the
+ * window is closed, so CPU% and idle% refer to the same interval. */
+static unsigned int sample_procs(struct sample *dst, struct sys_procinfo *snap)
 {
 	struct sys_procinfo p;
-	unsigned int i, j;
+	unsigned int i, n = 0;
 
-	ntotal = 0;
-	nrows = 0;
-	for (i = 0; ; i++) {
+	for (i = 0; i < MAX_SAMPLE; i++) {
 		if (u_syscall2(SYS_PROCINFO, (int)i, (int)&p) != 1)
 			break;
-		if (ntotal < PRINT_ROWS) {
-			struct prow *r = &rows[ntotal];
-			unsigned int delta = 0, pm = 0;
-
-			for (j = 0; j < nbefore; j++)
-				if (before[j].pid == p.pid) {
-					delta = p.cpu_ticks - before[j].cpu_ticks;
-					break;
-				}
-			if (j < nbefore) {
-				pm = delta * 1000u / wall;
-				if (pm > 1000u)
-					pm = 1000u;
+		dst[n].pid = p.pid;
+		dst[n].cpu_ticks = p.cpu_ticks;
+		if (snap)
+			snap[n] = p;
+		n++;
+	}
+	return n;
+}
+
+/* Turn the snapshots into rows, CPU% = ticks during the window (a process
+ * that appeared during the window shows 0%). */
+static void gather(void)
+{
+	unsigned int i, j;
+
+	ntotal = nafter;
+	nrows = 0;
+	for (i = 0; i < nafter && i < PRINT_ROWS; i++) {
+		const struct sys_procinfo *p = &snaps[i];
+		struct prow *r = &rows[i];
+		unsigned int delta = 0, pm = 0;
+
+		for (j = 0; j < nbefore; j++)
+			if (before[j].pid == p->pid) {
+				delta = p->cpu_ticks - before[j].cpu_ticks;
+				break;
 			}
-			r->pid = p.pid;
-			r->state = p.state;
-			r->cpu_pm = pm;
-			r->cpu_ticks = p.cpu_ticks;
-			r->pages = p.pages;
-			for (j = 0; j < sizeof(r->name); j++)
-				r->name[j] = p.name[j];
-			nrows++;
+		if (j < nbefore) {
+			pm = delta * 1000u / wall;
+			if (pm > 1000u)
+				pm = 1000u;
 		}
-		ntotal++;
+		r->pid = p->pid;
+		r->state = p->state;
+		r->weight = p->weight;
+		r->cpu_pm = pm;
+		r->cpu_ticks = p->cpu_ticks;
+		r->pages = p->pages;
+		for (j = 0; j < sizeof(r->name); j++)
+			r->name[j] = p->name[j];
+		nrows++;
 	}
 }
 
@@ -232,6 +256,8 @@ static void paint(void)
 	putc_(' ');
 	str_w("STATE", COL_STATE);
 	putc_(' ');
+	str_wr("W", COL_W);
+	putc_(' ');
 	str_wr("CPU%", COL_CPU);
 	putc_(' ');
 	str_wr("TIME", COL_TIME);
@@ -249,6 +275,8 @@ static void paint(void)
 			putc_(' ');
 			str_w(state_name(r->state), COL_STATE);
 			putc_(' ');
+			dec_w(r->weight, COL_W);
+			putc_(' ');
 			cpu_w(r->cpu_pm);
 			putc_(' ');
 			dec_w(r->cpu_ticks / 1000, COL_TIME - 1);
@@ -267,7 +295,11 @@ static void paint(void)
 		dec_(ntotal - nrows);
 		puts_(" more  ");
 	}
-	puts_("Ctrl+C to quit - wm = idle loop");
+	puts_("Ctrl+C to quit  idle ");
+	dec_(idle_pm / 10);
+	putc_('.');
+	dec_(idle_pm % 10);
+	putc_('%');
 	end_line(1);
 }
 
@@ -280,8 +312,8 @@ static void frame(void)
 void _start(const char *arg)
 {
 	struct sys_meminfo m;
-	struct sys_procinfo p;
-	unsigned int i, t0, t1;
+	struct sys_schedinfo s0, s1;
+	unsigned int t0, t1;
 
 	(void)arg;
 
@@ -290,21 +322,22 @@ void _start(const char *arg)
 	free_kb = m.free_pages * (m.page_size / 1024);
 
 	for (;;) {
-		/* Sample before the wait, repaint after it. */
-		nbefore = 0;
-		for (i = 0; i < MAX_SAMPLE; i++) {
-			if (u_syscall2(SYS_PROCINFO, (int)i, (int)&p) != 1)
-				break;
-			before[nbefore].pid = p.pid;
-			before[nbefore].cpu_ticks = p.cpu_ticks;
-			nbefore++;
-		}
+		/* Timestamp first: the sampling itself can be preempted, and the
+		 * per-process counters and idle ticks must refer to the same
+		 * window or the two percentages will not add up. */
+		nbefore = sample_procs(before, 0);
 		t0 = u_ticks_ms();
+		u_schedinfo(&s0);
 		u_sleep_ms(1000);
+		nafter = sample_procs(after, snaps);
 		t1 = u_ticks_ms();
+		u_schedinfo(&s1);
 		wall = t1 - t0;
 		if (!wall)
 			wall = 1;
+		idle_pm = (s1.idle_ticks - s0.idle_ticks) * 1000u / wall;
+		if (idle_pm > 1000u)
+			idle_pm = 1000u;
 		uptime_s = t1 / 1000;
 
 		u_puts("\x1b[2J\x1b[H");

+ 2 - 5
vga.c

@@ -647,11 +647,6 @@ static unsigned int shadow_copy_damage(unsigned int max_bytes)
 /* Bytes copied by the presenter so far (see vga.h). */
 unsigned int vga_present_copied;
 
-int vga_present_busy(void)
-{
-	return present_busy;
-}
-
 /* Copy up to `max_bytes` of the pending damage to VRAM.  Called by the main
  * loop on every iteration: small updates (a cursor move is ~1 KB) reach the
  * screen within one loop instead of waiting for a fixed 30 Hz tick, while a
@@ -673,6 +668,7 @@ void vga_present_shadow(unsigned int max_bytes)
 		return;
 	}
 	present_busy = 1;
+	preempt_disable();
 	irq_restore(flags);
 	vga_present_copied += shadow_copy_damage(max_bytes);
 	flags = irq_save_disable();
@@ -681,6 +677,7 @@ void vga_present_shadow(unsigned int max_bytes)
 		damage_deferred = 0;
 		damage_add_locked(0, 0, vga_fb_w, vga_fb_h);
 	}
+	preempt_enable();
 	irq_restore(flags);
 }
 

+ 0 - 4
vga.h

@@ -100,10 +100,6 @@ void vga_fb_flush(void);
  * within one loop, large ones are chunked so they cannot stall input. */
 void vga_fb_flush_budget(unsigned int max_bytes);
 
-/* 1 while the presenter is copying damage to VRAM.  The scheduler must not
- * switch away from that context: the flush trims a shared damage list. */
-int vga_present_busy(void);
-
 /* Low-level row helpers from vga_copy.S (the blit backend uses the fill). */
 void vga_fb_copy_row(void *dst, const void *src, unsigned int dwords);
 void vga_fb_copy_row_nt(void *dst, const void *src, unsigned int dwords);

+ 53 - 14
wm.c

@@ -148,7 +148,7 @@ int wm_content_size(struct term *t, unsigned int *w, unsigned int *h)
 
 int wm_server_is_wm(void)
 {
-	return wm_attached && procs[current_pid].pid == wm_pid;
+	return wm_attached && current_proc()->pid == wm_pid;
 }
 
 int wm_server_frame_active(void)
@@ -259,8 +259,8 @@ int wm_server_win_create(const struct win_create *in, struct win_info *out)
 
 	if (!in || !out)
 		return -1;
-	if (procs[current_pid].pid == 0 ||
-	    wm_window_for_pid(procs[current_pid].pid) >= 0)
+	if (current_proc()->pid == 0 ||
+	    wm_window_for_pid(current_proc()->pid) >= 0)
 		return -1;
 	cw = in->w;
 	ch = in->h;
@@ -287,7 +287,7 @@ int wm_server_win_create(const struct win_create *in, struct win_info *out)
 		back[j] = 0;
 
 	flags = irq_save_disable();
-	base = procs[current_pid].cr3;
+	base = current_proc()->cr3;
 	for (j = 0; j < pages; j++) {
 		if (!paging_map_page(base, USER_GFX_BASE + j * PAGE_SIZE,
 				     (unsigned int)back + j * PAGE_SIZE,
@@ -307,7 +307,7 @@ int wm_server_win_create(const struct win_create *in, struct win_info *out)
 	w->used = 1;
 	w->canvas_window = 1;
 	w->term = 0;
-	w->pid = procs[current_pid].pid;
+	w->pid = current_proc()->pid;
 	w->owner_cr3 = base;
 	w->back = back;
 	w->back_pages = pages;
@@ -328,6 +328,7 @@ int wm_server_win_create(const struct win_create *in, struct win_info *out)
 	w->mapped_pages = 0;
 	w->mapped_src = 0;
 	w->key_head = w->key_tail = 0;
+	wait_queue_init(&w->key_wq);
 	for (j = 0; j < sizeof(w->title) - 1 && in->title[j]; j++)
 		w->title[j] = in->title[j];
 	w->title[j] = '\0';
@@ -346,7 +347,7 @@ int wm_server_win_create(const struct win_create *in, struct win_info *out)
 
 int wm_server_win_present(void)
 {
-	int idx = wm_window_for_pid(procs[current_pid].pid);
+	int idx = wm_window_for_pid(current_proc()->pid);
 	struct window *w;
 	const unsigned short *src;
 	unsigned int n, i;
@@ -368,7 +369,7 @@ int wm_server_win_present(void)
 
 int wm_server_win_close(void)
 {
-	int idx = wm_window_for_pid(procs[current_pid].pid);
+	int idx = wm_window_for_pid(current_proc()->pid);
 
 	if (idx < 0 || !wm_server_can_reap())
 		return -1;
@@ -385,7 +386,7 @@ int wm_server_win_close(void)
 
 int wm_server_key_poll(void)
 {
-	int idx = wm_window_for_pid(procs[current_pid].pid);
+	int idx = wm_window_for_pid(current_proc()->pid);
 	struct window *w;
 	int k;
 
@@ -399,6 +400,29 @@ int wm_server_key_poll(void)
 	return k;
 }
 
+/* Block until a key arrives for the caller's window, or `timeout_ms` passes
+ * (0 = wait forever).  Returns the key, 0 on timeout/no key, -1 if the caller
+ * has no window. */
+int wm_server_key_wait(unsigned int timeout_ms)
+{
+	int idx = wm_window_for_pid(current_proc()->pid);
+	struct window *w;
+	int k;
+
+	if (idx < 0)
+		return -1;
+	w = &windows[idx];
+
+	k = wm_server_key_poll();
+	if (k)
+		return k;
+	if (timeout_ms)
+		sched_block_timeout(&w->key_wq, timeout_ms);
+	else
+		sched_block_on(&w->key_wq);
+	return wm_server_key_poll();
+}
+
 void wm_content_changed(struct text_screen *screen)
 {
 	unsigned int i;
@@ -416,11 +440,11 @@ int wm_server_attach(struct wm_screen_info *out)
 	unsigned int pages, i, bytes;
 
 	if (!out || !wm_expected_pid ||
-	    procs[current_pid].pid != wm_expected_pid || wm_attached) {
+	    current_proc()->pid != wm_expected_pid || wm_attached) {
 		serial_puts(SERIAL_COM1, "wm: attach rejected (expected ");
 		serial_puthex(SERIAL_COM1, wm_expected_pid);
 		serial_puts(SERIAL_COM1, " current ");
-		serial_puthex(SERIAL_COM1, procs[current_pid].pid);
+		serial_puthex(SERIAL_COM1, current_proc()->pid);
 		serial_puts(SERIAL_COM1, ")\n");
 		return -1;
 	}
@@ -434,7 +458,7 @@ int wm_server_attach(struct wm_screen_info *out)
 
 	bytes = s->pitch * s->h;
 	pages = (bytes + PAGE_SIZE - 1) / PAGE_SIZE;
-	wm_cr3 = procs[current_pid].cr3;
+	wm_cr3 = current_proc()->cr3;
 	for (i = 0; i < pages; i++) {
 		unsigned int va = WM_SHADOW_VA + i * PAGE_SIZE;
 
@@ -456,7 +480,7 @@ int wm_server_attach(struct wm_screen_info *out)
 		windows[i].mapped_src = 0;
 	}
 
-	wm_pid = procs[current_pid].pid;
+	wm_pid = current_proc()->pid;
 	wm_attached = 1;
 	wm_frame_active = 0;
 	wm_last_seen = get_ticks_ms();
@@ -803,6 +827,7 @@ void wm_server_start(void)
 		irq_restore(flags);
 		vfs_release_file(m, &f);
 		if (pid > 0) {
+			process_set_weight((unsigned int)pid, 16);
 			serial_puts(SERIAL_COM1, "wm: ring-3 WM started\n");
 			return;
 		}
@@ -817,8 +842,10 @@ void wm_server_start(void)
 	if (pid > 0)
 		wm_expected_pid = (unsigned int)pid;
 	irq_restore(flags);
-	if (pid > 0)
+	if (pid > 0) {
+		process_set_weight((unsigned int)pid, 16);
 		serial_puts(SERIAL_COM1, "wm: kernel WM fallback\n");
+	}
 }
 
 void wm_server_supervise(void)
@@ -931,6 +958,7 @@ int wm_spawn_terminal(void)
 			irq_restore(flags);
 			vfs_release_file(m, &f);
 			if (pid > 0) {
+				process_set_weight((unsigned int)pid, 12);
 				wm_request_redraw();
 				return 0;
 			}
@@ -950,6 +978,7 @@ int wm_spawn_terminal(void)
 	screen_init(&t->screen);
 	t->screen.cursor_visible = 1;
 	t->key_head = t->key_tail = 0;
+	wait_queue_init(&t->key_wq);
 	t->history_count = 0;
 	t->history_next = 0;
 	t->cwd[0] = '/';
@@ -965,6 +994,7 @@ int wm_spawn_terminal(void)
 	w->back_pages = 0;
 	w->owner_cr3 = 0;
 	w->key_head = w->key_tail = 0;
+	wait_queue_init(&w->key_wq);
 	w->pid = 0;
 	w->focused = 0;
 	w->pw = SCREEN_COLS * 8 + 2;
@@ -992,6 +1022,7 @@ int wm_spawn_terminal(void)
 		w->used = 0;
 		return -1;
 	}
+	process_set_weight((unsigned int)pid, 12);
 	w->pid = (unsigned int)pid;
 
 	for (j = 0; j < MAX_WINDOWS; j++)
@@ -1027,6 +1058,11 @@ static void wm_close_window(unsigned int idx)
 {
 	struct window *w = &windows[idx];
 
+	/* A terminal/key reader blocked on this window must not sleep through
+	 * the close; process_kill() also removes it from the queue. */
+	wait_wake_all(&w->key_wq);
+	if (w->term)
+		wait_wake_all(&w->term->key_wq);
 	if (w->term) {
 		sys_gfx_kill_for_screen(&w->term->screen); /* stop a demo */
 		if (w->pid)
@@ -1252,8 +1288,10 @@ void wm_pump_keyboard(void)
 			 * Ctrl+C always reaches the terminal so its foreground
 			 * kill keeps working. */
 			if (k == IO_KEY_CTRL_C ||
-			    !sys_gfx_push_key(&fw->term->screen, k))
+			    !sys_gfx_push_key(&fw->term->screen, k)) {
 				term_key_push(fw->term, k);
+				term_key_wake(fw->term);
+			}
 		} else if (fw->canvas_window) {
 			unsigned int next = (fw->key_head + 1) % 16;
 
@@ -1261,6 +1299,7 @@ void wm_pump_keyboard(void)
 				fw->keys[fw->key_head] = k;
 				fw->key_head = next;
 			}
+			wait_wake_one(&fw->key_wq);
 		}
 	}
 }

+ 2 - 0
wm.h

@@ -24,6 +24,7 @@ struct window {
 	unsigned int owner_cr3;
 	int keys[16];
 	unsigned int key_head, key_tail;
+	struct wait_queue key_wq;
 	/* Ring-3 WM content mapping/bookkeeping. */
 	unsigned int content_gen; /* bumped when the content kind/session changes */
 	unsigned int mapped_kind;
@@ -63,6 +64,7 @@ int  wm_server_win_create(const struct win_create *in, struct win_info *out);
 int  wm_server_win_present(void);
 int  wm_server_win_close(void);
 int  wm_server_key_poll(void);
+int  wm_server_key_wait(unsigned int timeout_ms);
 void wm_content_changed(struct text_screen *screen);
 void wm_server_unmap_screen(struct text_screen *screen);
 void wm_server_unmap_canvas(struct text_screen *screen, const void *canvas);