Selaa lähdekoodia

Keep the WM running while lsusb and storage wait on hardware

The USB host-controller completion and port-reset loops, the ASIX/RNDIS/
usbmsc probe delays and the SDHCI command/CMD1 polls busy-waited with the
BKL held, so every other syscall (the WM included) stalled for the whole
scan.  The HCD poll loops now call usb_poll_relax(), which yields once a
wait passes 2 ms when the caller owns the bus for a sleepable scan, and
the probe delays go through usb_delay_ms().  A nestable usb_bus_claim()
replaces the scan-only flag; usbmsc claims the bus per command, so the
storage rescan can drop the BKL mid-transfer safely.  SDHCI waits use
vfs_scan_poll()/vfs_scan_relax().
Johannes Zellner 6 päivää sitten
vanhempi
sitoutus
70a4a22f0f
13 muutettua tiedostoa jossa 206 lisäystä ja 65 poistoa
  1. 26 10
      AGENTS.md
  2. 3 3
      asix.c
  3. 20 10
      docs/drivers.html
  4. 1 1
      docs/filesystems.html
  5. 2 0
      ehci.c
  6. 2 0
      ohci.c
  7. 3 4
      rndis.c
  8. 9 1
      sdhci.c
  9. 1 0
      uhci.c
  10. 67 21
      usb.c
  11. 18 6
      usb.h
  12. 48 9
      usbmsc.c
  13. 6 0
      xhci.c

+ 26 - 10
AGENTS.md

@@ -610,11 +610,14 @@ it to `0x7E00`.  The kernel region itself only has one wall (`_end` below
   first; their operations (and SYS_SPAWN of a RAM path,
   first; their operations (and SYS_SPAWN of a RAM path,
   `sys_fs_enter_path()`) do not wait for `vfs_scan_busy`
   `sys_fs_enter_path()`) do not wait for `vfs_scan_busy`
   (`vfs_mount_is_ram()`), only disk paths do.  The scan runs preemptible;
   (`vfs_mount_is_ram()`), only disk paths do.  The scan runs preemptible;
-  the floppy/ATA/AHCI/SDHCI wait loops call `vfs_scan_poll(t0)` /
+  the floppy/ATA/AHCI/SDHCI wait loops (including the SDHCI command,
+  clock/reset and CMD1 power-up polls) call `vfs_scan_poll(t0)` /
   `vfs_scan_relax(ms)`, which sleep (dropping the BKL) only for the scan's
   `vfs_scan_relax(ms)`, which sleep (dropping the BKL) only for the scan's
   owning process, with interrupts on and preemption enabled, and only once
   owning process, with interrupts on and preemption enabled, and only once
   a wait passes 2 ms; at boot in the idle context and in normal I/O they
   a wait passes 2 ms; at boot in the idle context and in normal I/O they
-  are no-ops.  So `storage` no longer freezes the WM.  `disk_probe()` skips
+  are no-ops.  A USB disk read by the rescan claims the USB bus
+  (`usb_bus_claim()` in `usbmsc_read/write/flush`), so its HCD waits yield
+  as well (see the USB bullet).  So `storage` no longer freezes the WM.  `disk_probe()` skips
   a floppy whose CMOS drive type (register `0x10`) is 0.
   a floppy whose CMOS drive type (register `0x10`) is 0.
 - **Installer**: `install.c/h` — the privileged `install <target> [source|self]`
 - **Installer**: `install.c/h` — the privileged `install <target> [source|self]`
   command clones the live NebOS image or writes the running image.  A clone
   command clones the live NebOS image or writes the running image.  A clone
@@ -980,14 +983,27 @@ it to `0x7E00`.  The kernel region itself only has one wall (`_end` below
   and USB3 root ports; halt/HCH reset sequence and BIOS/OS handoff)
   and USB3 root ports; halt/HCH reset sequence and BIOS/OS handoff)
   implement it.
   implement it.
   `lsusb.c` — the `lsusb` command (lists every controller and its tree).
   `lsusb.c` — the `lsusb` command (lists every controller and its tree).
-  Every HCD delay goes through `usb_delay_ms()`: inside a scan
-  (`usb_scan`/`usb_hotplug_poll`) owned by a sleepable process — `lsusb`,
-  the kworker's HID rescan/hotplug enumeration — it sleeps and drops the
-  BKL, so port resets no longer stall the system; elsewhere it busy-waits.
-  While such a scan sleeps, `usb_scan_foreign()` makes the `usb.c`
-  wrappers refuse other contexts (control/bulk transfers fail, interrupt
-  polls return 0, `usb_port_connected` reports 1) and `usbhid_poll` skips
-  its tick; USB NICs, the RTL8821AU and USB disks see a transient error.
+  A scan (`usb_scan`/`usb_hotplug_poll`) and every `usbmsc` command claim
+  the controllers for the calling process (`usb_bus_claim()`/
+  `usb_bus_release()`, nesting).  Every HCD delay, and the ASIX/RNDIS/
+  usbmsc probe delays, go through `usb_delay_ms()`, and every HCD
+  completion/port-reset poll loop calls `usb_poll_relax(t0)` outside its
+  spinlock.  When the claim belongs to a sleepable scan — `lsusb`, the
+  kworker's HID rescan/hotplug enumeration, or the `storage` rescan's
+  `vfs_mount_all()` reading a USB disk — the delay sleeps and the poll
+  yields once a wait passes 2 ms, both dropping the BKL; elsewhere
+  (pid 0 at boot, preemption off, interrupts-off syscalls, normal file I/O,
+  the installer) they busy-wait as before.  The poll yields rather than
+  sleeping 1 ms because a sleep on an idle CPU wakes on the 100 Hz idle
+  tick, which made each slow transfer cost ~10 ms.  So `lsusb` no longer
+  freezes the WM (in QEMU the longest BKL hold dropped from ~105 ms in
+  xHCI `port_reset` to under the 50 ms sampling threshold; on hardware
+  the ASIX probe alone used to spin ~450 ms).  While another context owns
+  the bus, `usb_scan_foreign()` makes the `usb.c` wrappers refuse access
+  (control/bulk transfers fail, interrupt polls return 0,
+  `usb_port_connected` reports 1), `usb_hotplug_poll` skips its pass and
+  `usbhid_poll` skips its tick; USB NICs, the RTL8821AU and USB disks see
+  a transient error.
   `usbmsc.c/h` — USB mass-storage class driver (BOT/SCSI) exposed as
   `usbmsc.c/h` — USB mass-storage class driver (BOT/SCSI) exposed as
   `usb0..`; its device probe is registered next to ASIX and `blk_scan()`
   `usb0..`; its device probe is registered next to ASIX and `blk_scan()`
   registers the devices in the block registry.
   registers the devices in the block registry.

+ 3 - 3
asix.c

@@ -96,11 +96,11 @@ static unsigned int rx_pos, rx_end;
 
 
 
 
 
 
+/* Sleeps inside a sleepable bus scan (the probe runs there); busy-waits
+ * in the netdev paths, which may run with interrupts off. */
 static void delay_ms(unsigned int ms)
 static void delay_ms(unsigned int ms)
 {
 {
-	unsigned int t0 = get_ticks_ms();
-	while (get_ticks_ms() - t0 < ms)
-		;
+	usb_delay_ms(ms);
 }
 }
 
 
 static int read_cmd(unsigned char cmd, unsigned short value,
 static int read_cmd(unsigned char cmd, unsigned short value,

+ 20 - 10
docs/drivers.html

@@ -298,17 +298,27 @@
 			recursively. This is what makes the X201's internal PCH hubs work.
 			recursively. This is what makes the X201's internal PCH hubs work.
 			Hotplug watches root ports; a device behind a hub is only found when a
 			Hotplug watches root ports; a device behind a hub is only found when a
 			full scan runs (<code>lsusb</code> also bumps the USB generation so
 			full scan runs (<code>lsusb</code> also bumps the USB generation so
-			<code>usbhid</code> re-attaches).  Every HCD delay goes through
-			<code>usb_delay_ms()</code>: inside a scan owned by a sleepable
-			process (<code>lsusb</code>, the kworker's HID rescan/hotplug) it
-			sleeps and drops the BKL, so port resets and settle times no longer
-			freeze the system.  While such a scan sleeps, the
+			<code>usbhid</code> re-attaches).  A scan and every mass-storage
+			command claim the controllers for the calling process
+			(<code>usb_bus_claim()</code>/<code>usb_bus_release()</code>, nesting).
+			Every HCD delay and the ASIX/RNDIS/usbmsc probe delays go through
+			<code>usb_delay_ms()</code>, and every HCD completion and port-reset
+			poll loop calls <code>usb_poll_relax(t0)</code> outside its spinlock.
+			When the claim belongs to a sleepable scan (<code>lsusb</code>, the
+			kworker's HID rescan/hotplug, or the <code>storage</code> rescan
+			reading a USB disk), the delay sleeps and the poll yields once a wait
+			passes 2&nbsp;ms, both dropping the BKL, so port resets, NAKing
+			devices and settle times no longer freeze the WM; everywhere else
+			they busy-wait as before.  The poll yields instead of sleeping
+			because a 1&nbsp;ms sleep on an idle CPU wakes on the 100&nbsp;Hz idle
+			tick.  While another context owns the bus, the
 			<span class="file">usb.c</span> wrappers refuse controller access
 			<span class="file">usb.c</span> wrappers refuse controller access
-			from any other context (<code>usb_scan_foreign()</code>: control and
-			bulk transfers fail, interrupt polls return no data, ports read as
-			connected) and <code>usbhid_poll</code> skips its tick.  A USB NIC,
-			the RTL8821AU or a USB disk therefore sees a transient error during a
-			bus scan; the scan re-enumerates those devices anyway.
+			(<code>usb_scan_foreign()</code>: control and bulk transfers fail,
+			interrupt polls return no data, ports read as connected),
+			<code>usb_hotplug_poll</code> skips its pass and
+			<code>usbhid_poll</code> skips its tick.  A USB NIC, the RTL8821AU or
+			a USB disk therefore sees a transient error during a bus scan; the
+			scan re-enumerates those devices anyway.
 		</p>
 		</p>
 
 
 		<h3>Mass storage</h3>
 		<h3>Mass storage</h3>

+ 1 - 1
docs/filesystems.html

@@ -165,7 +165,7 @@
 			<li><strong>Read/write.</strong> Read, write, delete, mkdir and rmdir are implemented; FAT32 allocation uses an FSInfo-seeded free-cluster hint.</li>
 			<li><strong>Read/write.</strong> Read, write, delete, mkdir and rmdir are implemented; FAT32 allocation uses an FSInfo-seeded free-cluster hint.</li>
 			<li><strong>8.3 names.</strong> The on-disk name is the canonical one. The shell and <code>files</code> display the space-padded form; a long name cannot be created.</li>
 			<li><strong>8.3 names.</strong> The on-disk name is the canonical one. The shell and <code>files</code> display the space-padded form; a long name cannot be created.</li>
 			<li><strong>Directory cache.</strong> The last FAT directory's first 64 entries are cached in <code>vfs.c</code>, so WebDAV's index sequence is one walk instead of N. Any write invalidates the cache.</li>
 			<li><strong>Directory cache.</strong> The last FAT directory's first 64 entries are cached in <code>vfs.c</code>, so WebDAV's index sequence is one walk instead of N. Any write invalidates the cache.</li>
-			<li><strong>Serialization.</strong> A file operation runs in a non-preemptible section because the FAT driver has static state and the floppy needs IRQ 6; <code>vfs_scan_busy</code>/<code>format_busy</code> make it sleep first while a mount scan or a format owns the disks. A format releases the BKL for its raw-sector loop, and only disk-backed mounts wait for it, so <code>/ramfs</code> and <code>/share</code> stay usable.  A rescan (the <code>bootscan</code> thread or the <code>storage</code> command) re-adds <code>/ramfs</code>/<code>/share</code> first and is skipped by their operations too (<code>vfs_mount_is_ram</code>); it runs preemptible, and the floppy/ATA/AHCI/SDHCI wait loops call <code>vfs_scan_poll</code>/<code>vfs_scan_relax</code>, which sleep (dropping the BKL) once a wait passes 2 ms and only for the scan's owning process, so <code>storage</code> no longer freezes the WM.  The floppy probe is skipped for a drive whose CMOS type (register <code>0x10</code>) is 0.</li>
+			<li><strong>Serialization.</strong> A file operation runs in a non-preemptible section because the FAT driver has static state and the floppy needs IRQ 6; <code>vfs_scan_busy</code>/<code>format_busy</code> make it sleep first while a mount scan or a format owns the disks. A format releases the BKL for its raw-sector loop, and only disk-backed mounts wait for it, so <code>/ramfs</code> and <code>/share</code> stay usable.  A rescan (the <code>bootscan</code> thread or the <code>storage</code> command) re-adds <code>/ramfs</code>/<code>/share</code> first and is skipped by their operations too (<code>vfs_mount_is_ram</code>); it runs preemptible, and the floppy/ATA/AHCI/SDHCI wait loops (including the SDHCI command and CMD1 power-up polls) call <code>vfs_scan_poll</code>/<code>vfs_scan_relax</code>, which sleep (dropping the BKL) once a wait passes 2 ms and only for the scan's owning process.  A USB disk read by the rescan claims the USB bus (<code>usb_bus_claim()</code>), so its host-controller waits yield too; <code>storage</code> no longer freezes the WM.  The floppy probe is skipped for a drive whose CMOS type (register <code>0x10</code>) is 0.</li>
 		</ul>
 		</ul>
 		<p>
 		<p>
 			<span class="file">fatdetect.c</span> and <span class="file">fatformat.c</span>
 			<span class="file">fatdetect.c</span> and <span class="file">fatformat.c</span>

+ 2 - 0
ehci.c

@@ -412,6 +412,7 @@ static int qtd_wait(struct ehci_qtd *qtd, struct ehci_qtd *early,
 			return -1;
 			return -1;
 		if (get_ticks_ms() - t0 >= ms)
 		if (get_ticks_ms() - t0 >= ms)
 			return -1;
 			return -1;
+		usb_poll_relax(t0);
 		__asm__ volatile ("pause");
 		__asm__ volatile ("pause");
 	}
 	}
 	return 0;
 	return 0;
@@ -959,6 +960,7 @@ static int ehci_port_reset_once(unsigned int ctrl, unsigned int port)
 	while (!(rd32(port_reg(c, port)) & PORT_PE)) {
 	while (!(rd32(port_reg(c, port)) & PORT_PE)) {
 		if (get_ticks_ms() - t0 > 200)
 		if (get_ticks_ms() - t0 > 200)
 			break;
 			break;
+		usb_poll_relax(t0);
 	}
 	}
 	ps = rd32(port_reg(c, port));
 	ps = rd32(port_reg(c, port));
 
 

+ 2 - 0
ohci.c

@@ -421,6 +421,7 @@ static int ohci_port_reset(unsigned int ctrl, unsigned int port)
 			break;
 			break;
 		if (get_ticks_ms() - t0 > 200)
 		if (get_ticks_ms() - t0 > 200)
 			return -1;
 			return -1;
+		usb_poll_relax(t0);
 	}
 	}
 	delay_ms(10);
 	delay_ms(10);
 
 
@@ -526,6 +527,7 @@ static int ohci_transfer(struct ohci_ctrl *c, unsigned int addr,
 			wr(c, R_CTRL_HEAD_ED, 0);
 			wr(c, R_CTRL_HEAD_ED, 0);
 			return -1;
 			return -1;
 		}
 		}
+		usb_poll_relax(t0);
 	}
 	}
 	ctrl_ed.head = (unsigned int)&ctrl_dummy;
 	ctrl_ed.head = (unsigned int)&ctrl_dummy;
 	wr(c, R_CTRL_HEAD_ED, 0);
 	wr(c, R_CTRL_HEAD_ED, 0);

+ 3 - 4
rndis.c

@@ -223,12 +223,11 @@ static void wr32le(unsigned char *p, unsigned int v)
 	p[3] = (unsigned char)((v >> 24) & 0xFF);
 	p[3] = (unsigned char)((v >> 24) & 0xFF);
 }
 }
 
 
+/* Sleeps inside a sleepable bus scan (the probe runs there); busy-waits
+ * in the netdev paths, which may run with interrupts off. */
 static void rndis_delay(unsigned int ms)
 static void rndis_delay(unsigned int ms)
 {
 {
-	unsigned int t0 = get_ticks_ms();
-
-	while (get_ticks_ms() - t0 < ms)
-		;
+	usb_delay_ms(ms);
 }
 }
 
 
 static void put_mac_serial(void)
 static void put_mac_serial(void)

+ 9 - 1
sdhci.c

@@ -242,6 +242,7 @@ static int sdhci_cmd(unsigned int cmd, unsigned int arg, unsigned int flags,
 			log_fail("command timeout", cmd, sts);
 			log_fail("command timeout", cmd, sts);
 			return -1;
 			return -1;
 		}
 		}
+		vfs_scan_poll(t0);
 	}
 	}
 	wr32(SDHCI_INT_STATUS, SDHCI_INT_CMD_COMPLETE);
 	wr32(SDHCI_INT_STATUS, SDHCI_INT_CMD_COMPLETE);
 
 
@@ -264,6 +265,7 @@ static int sdhci_cmd(unsigned int cmd, unsigned int arg, unsigned int flags,
 				log_fail("transfer timeout", cmd, sts);
 				log_fail("transfer timeout", cmd, sts);
 				return -1;
 				return -1;
 			}
 			}
+			vfs_scan_poll(t0);
 		}
 		}
 		wr32(SDHCI_INT_STATUS, 0xFFFFFFFFu);
 		wr32(SDHCI_INT_STATUS, 0xFFFFFFFFu);
 	}
 	}
@@ -285,6 +287,7 @@ static int sdhci_clock_div(unsigned int div)
 	while (!(rd16(SDHCI_CLOCK_CONTROL) & SDHCI_CLOCK_INT_STABLE)) {
 	while (!(rd16(SDHCI_CLOCK_CONTROL) & SDHCI_CLOCK_INT_STABLE)) {
 		if (get_ticks_ms() - t0 > 100)
 		if (get_ticks_ms() - t0 > 100)
 			return -1;
 			return -1;
+		vfs_scan_poll(t0);
 	}
 	}
 	wr16(SDHCI_CLOCK_CONTROL,
 	wr16(SDHCI_CLOCK_CONTROL,
 	     (unsigned short)((div << 8) | SDHCI_CLOCK_INT_EN |
 	     (unsigned short)((div << 8) | SDHCI_CLOCK_INT_EN |
@@ -302,6 +305,7 @@ static int sdhci_reset_host(void)
 	while (rd8(SDHCI_SOFTWARE_RESET) & SDHCI_RESET_ALL) {
 	while (rd8(SDHCI_SOFTWARE_RESET) & SDHCI_RESET_ALL) {
 		if (get_ticks_ms() - t0 > 100)
 		if (get_ticks_ms() - t0 > 100)
 			return -1;
 			return -1;
+		vfs_scan_poll(t0);
 	}
 	}
 	return 0;
 	return 0;
 }
 }
@@ -486,7 +490,11 @@ static int sdhci_init(void)
 			serial_puts(SERIAL_COM1, "\n");
 			serial_puts(SERIAL_COM1, "\n");
 			return -1;
 			return -1;
 		}
 		}
-		tiny_delay(1);
+		/* The card can take up to ~1 s to leave its power-up busy
+		 * state: sleep between polls during the storage rescan
+		 * (tiny_delay(1) alone never reaches vfs_scan_poll's 2 ms). */
+		if (!vfs_scan_relax(1))
+			tiny_delay(1);
 	}
 	}
 	serial_puts(SERIAL_COM1, "sdhci: ocr 0x");
 	serial_puts(SERIAL_COM1, "sdhci: ocr 0x");
 	serial_puthex(SERIAL_COM1, ocr);
 	serial_puthex(SERIAL_COM1, ocr);

+ 1 - 0
uhci.c

@@ -416,6 +416,7 @@ static int control_transfer(unsigned int dev_addr, unsigned int mps,
 			qh.element = 0x1u;
 			qh.element = 0x1u;
 			return -1;
 			return -1;
 		}
 		}
+		usb_poll_relax(t0);
 	}
 	}
 
 
 	qh.element = 0x1u;
 	qh.element = 0x1u;

+ 67 - 21
usb.c

@@ -8,6 +8,7 @@
 #include "timer.h"
 #include "timer.h"
 #include "usb.h"
 #include "usb.h"
 #include "usb_hcd.h"
 #include "usb_hcd.h"
+#include "vfs.h"
 
 
 /* One global controller index space.  Each entry maps to one instance of
 /* One global controller index space.  Each entry maps to one instance of
    one controller type; a machine with EHCI + UHCI companions gets several.
    one controller type; a machine with EHCI + UHCI companions gets several.
@@ -111,8 +112,12 @@ static usb_probe_fn current_probe;
 static usb_device_probe_fn device_probes[USB_MAX_DEVICE_PROBES];
 static usb_device_probe_fn device_probes[USB_MAX_DEVICE_PROBES];
 static unsigned int n_device_probes;
 static unsigned int n_device_probes;
 static int print_devices;
 static int print_devices;
-static int scan_busy;
-static unsigned int scan_owner_pid;     /* pid that set scan_busy */
+static int scan_busy;                   /* a usb_scan/hotplug pass runs */
+/* Bus ownership: a scan, or a mass-storage command issued by the storage
+ * rescan, claims the controllers for one process.  While the owner sleeps
+ * in a wait loop, the transfer wrappers refuse every other context. */
+static unsigned int bus_depth;
+static unsigned int bus_owner_pid;
 
 
 static unsigned int cur_pid(void)
 static unsigned int cur_pid(void)
 {
 {
@@ -121,27 +126,53 @@ static unsigned int cur_pid(void)
 	return p ? p->pid : 0;
 	return p ? p->pid : 0;
 }
 }
 
 
-static void scan_begin(void)
+int usb_bus_claim(void)
 {
 {
-	scan_busy = 1;
-	scan_owner_pid = cur_pid();
+	unsigned int pid = cur_pid();
+
+	if (bus_depth && pid != bus_owner_pid)
+		return -1;
+	if (!bus_depth++)
+		bus_owner_pid = pid;
+	return 0;
+}
+
+void usb_bus_release(void)
+{
+	if (bus_depth && !--bus_depth)
+		bus_owner_pid = 0;
 }
 }
 
 
 int usb_scan_foreign(void)
 int usb_scan_foreign(void)
 {
 {
-	return scan_busy && cur_pid() != scan_owner_pid;
+	return bus_depth && cur_pid() != bus_owner_pid;
 }
 }
 
 
-void usb_delay_ms(unsigned int ms)
+/* 1 when the caller may sleep inside a USB wait: it owns the bus for a scan
+ * (lsusb, the kworker's HID rescan/hotplug) or for a command issued by the
+ * storage rescan, and it is not the idle/boot context (pid 0), not inside a
+ * non-preemptible section and not in an interrupts-off syscall (SYS_WIFI
+ * runs with IF clear).  Other callers (normal file I/O, the installer, NIC
+ * paths) keep busy-waiting exactly as before. */
+static int usb_can_sleep(void)
 {
 {
-	unsigned int t0, flags;
+	unsigned int flags, pid = cur_pid();
 
 
+	if (!bus_depth || !bus_owner_pid || pid != bus_owner_pid)
+		return 0;
+	if (!scan_busy && !(vfs_scan_busy && vfs_scan_owner_pid == pid))
+		return 0;
+	if (this_cpu()->preempt_count)
+		return 0;
 	__asm__ volatile ("pushfl; popl %0" : "=r"(flags));
 	__asm__ volatile ("pushfl; popl %0" : "=r"(flags));
-	/* Only the scan owner sleeps, never the idle/boot context (pid 0),
-	 * a non-preemptible section or an interrupts-off syscall (SYS_WIFI
-	 * runs with IF clear). */
-	if (scan_busy && scan_owner_pid && cur_pid() == scan_owner_pid &&
-	    !this_cpu()->preempt_count && (flags & 0x200u)) {
+	return (flags & 0x200u) != 0;
+}
+
+void usb_delay_ms(unsigned int ms)
+{
+	unsigned int t0;
+
+	if (usb_can_sleep()) {
 		process_sleep_ms(ms ? ms : 1);
 		process_sleep_ms(ms ? ms : 1);
 		return;
 		return;
 	}
 	}
@@ -150,6 +181,17 @@ void usb_delay_ms(unsigned int ms)
 		;
 		;
 }
 }
 
 
+/* A yield, not a sleep: it hands the BKL to a waiting process (the WM, a
+ * terminal) and returns at once when nothing else is runnable, so a
+ * completion that lands a few ms later is still seen promptly.  A 1 ms
+ * sleep would wake on the 100 Hz idle tick and made every slow transfer
+ * cost ~10 ms. */
+void usb_poll_relax(unsigned int t0)
+{
+	if (get_ticks_ms() - t0 >= 2 && usb_can_sleep())
+		sched_yield();
+}
+
 static unsigned char dev_desc[18];
 static unsigned char dev_desc[18];
 static unsigned char desc8[64];
 static unsigned char desc8[64];
 static unsigned char hub_desc[16];
 static unsigned char hub_desc[16];
@@ -795,15 +837,16 @@ int usb_scan(usb_probe_fn probe)
 {
 {
 	unsigned int c, p;
 	unsigned int c, p;
 
 
-	if (scan_busy)
+	if (scan_busy || usb_bus_claim() != 0)
 		return -2;
 		return -2;
-	/* Claim the bus before the first controller bring-up: the boot scan
-	 * runs while the WM and the first terminal are already up, and their
-	 * USB calls must see the scan (usb_scan_foreign) rather than half-
-	 * initialized controllers. */
-	scan_begin();
+	/* The bus is claimed before the first controller bring-up: the boot
+	 * scan runs while the WM and the first terminal are already up, and
+	 * their USB calls must see the scan (usb_scan_foreign) rather than
+	 * half-initialized controllers. */
+	scan_busy = 1;
 	if (usb_hcd_init() != 0) {
 	if (usb_hcd_init() != 0) {
 		scan_busy = 0;
 		scan_busy = 0;
+		usb_bus_release();
 		return -1;
 		return -1;
 	}
 	}
 	generation++;
 	generation++;
@@ -862,6 +905,7 @@ int usb_scan(usb_probe_fn probe)
 	current_probe = 0;
 	current_probe = 0;
 	print_devices = 0;
 	print_devices = 0;
 	scan_busy = 0;
 	scan_busy = 0;
+	usb_bus_release();
 	return 0;
 	return 0;
 }
 }
 
 
@@ -871,7 +915,7 @@ int usb_hotplug_poll(usb_probe_fn probe)
 	unsigned int now, c, p;
 	unsigned int now, c, p;
 	int n = 0;
 	int n = 0;
 
 
-	if (!n_usb_ctrls || scan_busy)
+	if (!n_usb_ctrls || scan_busy || bus_depth)
 		return 0;
 		return 0;
 
 
 	now = get_ticks_ms();
 	now = get_ticks_ms();
@@ -879,7 +923,8 @@ int usb_hotplug_poll(usb_probe_fn probe)
 		return 0;
 		return 0;
 	last = now;
 	last = now;
 
 
-	scan_begin();
+	usb_bus_claim();
+	scan_busy = 1;
 	current_probe = probe;
 	current_probe = probe;
 	for (c = 0; c < n_usb_ctrls; c++) {
 	for (c = 0; c < n_usb_ctrls; c++) {
 		const struct usb_hcd_ops *ops = usb_ctrls[c].ops;
 		const struct usb_hcd_ops *ops = usb_ctrls[c].ops;
@@ -899,6 +944,7 @@ int usb_hotplug_poll(usb_probe_fn probe)
 	}
 	}
 	current_probe = 0;
 	current_probe = 0;
 	scan_busy = 0;
 	scan_busy = 0;
+	usb_bus_release();
 	return n;
 	return n;
 }
 }
 
 

+ 18 - 6
usb.h

@@ -23,13 +23,25 @@ const char *usb_hcd_name_ctrl(unsigned int ctrl);
 int usb_scan(usb_probe_fn probe);
 int usb_scan(usb_probe_fn probe);
 
 
 /* Delay used by the USB core and every HCD.  Inside a bus scan owned by a
 /* Delay used by the USB core and every HCD.  Inside a bus scan owned by a
- * sleepable process (lsusb, the kworker's HID rescan) it sleeps, which
- * drops the BKL so the WM and other processes keep running through port
- * resets and settle times; otherwise it busy-waits as before. */
+ * sleepable process (lsusb, the kworker's HID rescan), or a command the
+ * storage rescan issues with the bus claimed, it sleeps, which drops the
+ * BKL so the WM and other processes keep running through port resets and
+ * settle times; otherwise it busy-waits as before. */
 void usb_delay_ms(unsigned int ms);
 void usb_delay_ms(unsigned int ms);
-/* 1 while another context's scan owns the controllers (it is sleeping in
- * usb_delay_ms).  The transfer wrappers then refuse access; pollers such as
- * usbhid skip their tick. */
+/* Call from every HCD completion-poll loop (outside its spinlock) with the
+ * loop's start time: once the wait has run 2 ms, a sleepable bus owner
+ * yields (dropping the BKL for any waiter) instead of spinning with it
+ * held.  A no-op otherwise. */
+void usb_poll_relax(unsigned int t0);
+/* Claim the controllers for the calling process (nests).  Returns -1 when
+ * another context owns them.  Every USB transaction that may sleep in a
+ * wait loop must run inside a claim, so no other context can touch the
+ * controller while the owner sleeps. */
+int usb_bus_claim(void);
+void usb_bus_release(void);
+/* 1 while another context owns the controllers (a scan, or a claimed
+ * mass-storage command, possibly sleeping).  The transfer wrappers then
+ * refuse access; pollers such as usbhid skip their tick. */
 int usb_scan_foreign(void);
 int usb_scan_foreign(void);
 
 
 /* Cheap periodic watcher: enumerates devices on root ports whose connect
 /* Cheap periodic watcher: enumerates devices on root ports whose connect

+ 48 - 9
usbmsc.c

@@ -78,12 +78,11 @@ static void put32be(unsigned char *p, unsigned int v)
 	p[3] = (unsigned char)(v & 0xFF);
 	p[3] = (unsigned char)(v & 0xFF);
 }
 }
 
 
+/* Sleeps inside a bus scan or a claimed storage-rescan command (see
+ * usb_delay_ms); busy-waits everywhere else. */
 static void msc_delay(unsigned int ms)
 static void msc_delay(unsigned int ms)
 {
 {
-	unsigned int t0 = get_ticks_ms();
-
-	while (get_ticks_ms() - t0 < ms)
-		;
+	usb_delay_ms(ms);
 }
 }
 
 
 /* Bulk IN with a deadline.  Transfers larger than the host controller's
 /* Bulk IN with a deadline.  Transfers larger than the host controller's
@@ -111,6 +110,8 @@ static int msc_bulk_in(struct usb_msc *m, unsigned char *buf, unsigned int len)
 			t0 = get_ticks_ms();
 			t0 = get_ticks_ms();
 		} else if (get_ticks_ms() - t0 > MSC_TIMEOUT_MS) {
 		} else if (get_ticks_ms() - t0 > MSC_TIMEOUT_MS) {
 			return -1;
 			return -1;
+		} else {
+			usb_poll_relax(t0);
 		}
 		}
 	}
 	}
 	return 0;
 	return 0;
@@ -492,8 +493,8 @@ static int msc_get(unsigned int unit, struct usb_msc **out)
 	return 0;
 	return 0;
 }
 }
 
 
-int usbmsc_read(unsigned int unit, unsigned int lba, void *buf,
-		unsigned int count)
+static int msc_read_claimed(unsigned int unit, unsigned int lba, void *buf,
+			    unsigned int count)
 {
 {
 	struct usb_msc *m;
 	struct usb_msc *m;
 	unsigned char *p = (unsigned char *)buf;
 	unsigned char *p = (unsigned char *)buf;
@@ -544,8 +545,8 @@ int usbmsc_read(unsigned int unit, unsigned int lba, void *buf,
 	return 0;
 	return 0;
 }
 }
 
 
-int usbmsc_write(unsigned int unit, unsigned int lba, const void *buf,
-		 unsigned int count)
+static int msc_write_claimed(unsigned int unit, unsigned int lba,
+			     const void *buf, unsigned int count)
 {
 {
 	struct usb_msc *m;
 	struct usb_msc *m;
 	const unsigned char *p = (const unsigned char *)buf;
 	const unsigned char *p = (const unsigned char *)buf;
@@ -569,7 +570,7 @@ int usbmsc_write(unsigned int unit, unsigned int lba, const void *buf,
 	return 0;
 	return 0;
 }
 }
 
 
-int usbmsc_flush(unsigned int unit)
+static int msc_flush_claimed(unsigned int unit)
 {
 {
 	struct usb_msc *m;
 	struct usb_msc *m;
 
 
@@ -582,3 +583,41 @@ int usbmsc_flush(unsigned int unit)
 	}
 	}
 	return 0;
 	return 0;
 }
 }
+
+/* The block-layer entry points claim the bus for the whole command, so the
+ * storage rescan may sleep in the HCD wait loops (dropping the BKL) while
+ * no other context can start a transfer on the same controller. */
+int usbmsc_read(unsigned int unit, unsigned int lba, void *buf,
+		unsigned int count)
+{
+	int r;
+
+	if (usb_bus_claim() != 0)
+		return -1;
+	r = msc_read_claimed(unit, lba, buf, count);
+	usb_bus_release();
+	return r;
+}
+
+int usbmsc_write(unsigned int unit, unsigned int lba, const void *buf,
+		 unsigned int count)
+{
+	int r;
+
+	if (usb_bus_claim() != 0)
+		return -1;
+	r = msc_write_claimed(unit, lba, buf, count);
+	usb_bus_release();
+	return r;
+}
+
+int usbmsc_flush(unsigned int unit)
+{
+	int r;
+
+	if (usb_bus_claim() != 0)
+		return -1;
+	r = msc_flush_claimed(unit);
+	usb_bus_release();
+	return r;
+}

+ 6 - 0
xhci.c

@@ -508,6 +508,7 @@ static int xhci_cmd(unsigned int p0, unsigned int p1, unsigned int control,
 		spin_unlock_irqrestore(&xhci_lock, flags);
 		spin_unlock_irqrestore(&xhci_lock, flags);
 		if (rc)
 		if (rc)
 			break;
 			break;
+		usb_poll_relax(t0);
 		__asm__ volatile ("pause");
 		__asm__ volatile ("pause");
 	}
 	}
 
 
@@ -922,6 +923,7 @@ static int control_transfer(unsigned int ctrl, unsigned int addr,
 		spin_unlock_irqrestore(&xhci_lock, flags);
 		spin_unlock_irqrestore(&xhci_lock, flags);
 		if (rc)
 		if (rc)
 			break;
 			break;
+		usb_poll_relax(t0);
 		__asm__ volatile ("pause");
 		__asm__ volatile ("pause");
 	}
 	}
 
 
@@ -1216,6 +1218,7 @@ static int xhci_bulk_out(unsigned int ctrl, unsigned int addr, unsigned int ep,
 		spin_unlock_irqrestore(&xhci_lock, flags);
 		spin_unlock_irqrestore(&xhci_lock, flags);
 		if (done)
 		if (done)
 			break;
 			break;
+		usb_poll_relax(t0);
 		__asm__ volatile ("pause");
 		__asm__ volatile ("pause");
 	}
 	}
 
 
@@ -1264,6 +1267,7 @@ static int port_reset(unsigned int ctrl, unsigned int port)
 			ps = mmio_rd(op(XHCI_OP_PORTSC) + port * 0x10);
 			ps = mmio_rd(op(XHCI_OP_PORTSC) + port * 0x10);
 			if (!(ps & PORTSC_PED))
 			if (!(ps & PORTSC_PED))
 				break;
 				break;
+			usb_poll_relax(t0);
 		}
 		}
 	}
 	}
 
 
@@ -1283,6 +1287,7 @@ static int port_reset(unsigned int ctrl, unsigned int port)
 		ps = mmio_rd(op(XHCI_OP_PORTSC) + port * 0x10);
 		ps = mmio_rd(op(XHCI_OP_PORTSC) + port * 0x10);
 		if (ps & PORTSC_PRC)
 		if (ps & PORTSC_PRC)
 			break;
 			break;
+		usb_poll_relax(t0);
 	}
 	}
 	if (!(ps & PORTSC_PRC)) {
 	if (!(ps & PORTSC_PRC)) {
 		dbg_hex("xhci: port reset timeout ", port);
 		dbg_hex("xhci: port reset timeout ", port);
@@ -1296,6 +1301,7 @@ static int port_reset(unsigned int ctrl, unsigned int port)
 		ps = mmio_rd(op(XHCI_OP_PORTSC) + port * 0x10);
 		ps = mmio_rd(op(XHCI_OP_PORTSC) + port * 0x10);
 		if (ps & PORTSC_PED)
 		if (ps & PORTSC_PED)
 			break;
 			break;
+		usb_poll_relax(t0);
 	}
 	}
 	if (!(ps & PORTSC_PED)) {
 	if (!(ps & PORTSC_PED)) {
 		dbg_hex("xhci: port not enabled ", port);
 		dbg_hex("xhci: port not enabled ", port);