graphics.html 21 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409
  1. <!doctype html>
  2. <html lang="en">
  3. <head>
  4. <meta charset="utf-8">
  5. <meta name="viewport" content="width=device-width, initial-scale=1">
  6. <title>Graphics &amp; UI | NebOS Architecture</title>
  7. <meta name="description" content="NebOS graphics architecture: VBE framebuffer, shadow buffer, damage rectangles, blit vtable, window manager, canvas ABI, ngui toolkit and image library.">
  8. <link rel="icon" href="favicon.svg" type="image/svg+xml">
  9. <link rel="stylesheet" href="style.css">
  10. </head>
  11. <body>
  12. <a class="skip" href="#main">Skip to content</a>
  13. <header class="site-header">
  14. <a class="brand" href="index.html">
  15. <span class="brand-mark" aria-hidden="true"></span>
  16. <span>NebOS <span class="brand-sub">Architecture</span></span>
  17. </a>
  18. <nav class="site-nav" aria-label="Architecture sections">
  19. <a href="index.html">Overview</a>
  20. <a href="boot.html">Boot</a>
  21. <a href="memory.html">Memory</a>
  22. <a href="cpu.html">CPU</a>
  23. <a href="drivers.html">Drivers</a>
  24. <a href="filesystems.html">Filesystems</a>
  25. <a href="graphics.html" aria-current="page">Graphics</a>
  26. <a href="network.html">Network</a>
  27. <a href="userland.html">Userland</a>
  28. <a href="development.html">Development</a>
  29. </nav>
  30. </header>
  31. <main id="main" class="page">
  32. <div class="page-head">
  33. <p class="kicker">Graphics &amp; UI</p>
  34. <h1>Draw into a shadow, publish by damage.</h1>
  35. <p class="lede">
  36. The display path is intentionally simple: drawing lands in a RAM
  37. shadow buffer, damage rectangles describe what changed, and a
  38. budgeted flush copies those rectangles to VRAM. Windows add a
  39. compositor and a mapped-canvas ABI on top; a small immediate-mode
  40. toolkit supplies the widgets.
  41. </p>
  42. <div class="meta">
  43. <span class="pill">VBE LFB</span>
  44. <span class="pill">1 MiB flush budget</span>
  45. <span class="pill">8 windows</span>
  46. <span class="pill">RGB565 canvases</span>
  47. </div>
  48. </div>
  49. <nav class="toc" aria-label="On this page">
  50. <h2>On this page</h2>
  51. <ul>
  52. <li><a href="#framebuffer">Framebuffer</a></li>
  53. <li><a href="#shadow">Shadow &amp; damage</a></li>
  54. <li><a href="#blit">Blit backends</a></li>
  55. <li><a href="#wm">Window manager</a></li>
  56. <li><a href="#canvas">Canvas ABI</a></li>
  57. <li><a href="#ngui">ngui toolkit</a></li>
  58. <li><a href="#apps">Applications</a></li>
  59. <li><a href="#images">Image library</a></li>
  60. <li><a href="#gpu">GPU quirks</a></li>
  61. <li><a href="#sources">Source map</a></li>
  62. </ul>
  63. </nav>
  64. <section id="framebuffer">
  65. <h2>Framebuffer</h2>
  66. <p>
  67. The boot path writes a <code>struct vga_boot_fb_info</code> to
  68. <code>0x600</code>: LFB base, pitch, width, height, bits per pixel
  69. and an RGBX flag for GOP's byte order. The legacy VESA stage chooses
  70. a mode with at least 16 bpp, a linear framebuffer, it must fit the
  71. card's VRAM, and width at most 1280 pixels. The UEFI loader imposes
  72. its own GOP constraints.
  73. </p>
  74. <p>
  75. <span class="file">vga.c</span> owns the visible surface. On a
  76. normal boot the render target is a RAM shadow buffer at
  77. <code>0x100000</code>, not the LFB; the 8 MB reserve leaves room for
  78. future modes. A surface carries width, height, pitch, format and a
  79. pixel-pointer, so the same drawing code can target a window layer or
  80. a canvas.
  81. </p>
  82. </section>
  83. <section id="shadow">
  84. <h2>Shadow buffer &amp; damage</h2>
  85. <p>
  86. Drawing never writes VRAM directly. Each operation records a damage
  87. rectangle; overlapping or touching rectangles merge, so a keystroke's
  88. cells collapse into one small rect while unrelated damage stays
  89. apart. The main loop calls <code>vga_present_shadow()</code> on every
  90. iteration with a byte budget of <code>VGA_FLUSH_BUDGET_BYTES</code>
  91. (1 MiB by default).
  92. </p>
  93. <ul>
  94. <li><strong>Small rects copy whole.</strong> A window repaint within the budget never tears.</li>
  95. <li><strong>Large rects are chunked.</strong> A full-screen redraw continues across loop iterations; newer damage is queued behind it.</li>
  96. <li><strong>No fixed frame rate.</strong> The loop flushes whenever there is pending damage, so a cursor move reaches the screen within one loop.</li>
  97. <li><strong>Re-entry guard.</strong> <code>vga_present_shadow()</code> refuses to run while another process is mid-flush; a full-screen copy is long enough to be preempted.</li>
  98. <li><strong>Window drags flush immediately.</strong> <code>wm_drag_shadow</code> copies each drag step as one consistent frame.</li>
  99. </ul>
  100. <p>
  101. When the target is not the screen — a window layer or an app canvas —
  102. <code>vga_target_damage()</code> is a no-op. The render target
  103. override is stored in the PCB, so a preempted draw cannot leak into
  104. another process's window.
  105. </p>
  106. </section>
  107. <section id="blit">
  108. <h2>Blit backends</h2>
  109. <p>
  110. <span class="file">blit.c</span> is the pixel-operation vtable:
  111. fill, copy, blend, canvas, present and flip. Software is the only
  112. backend today, but the WM calls the vtable rather than a fixed
  113. function, so an accelerator can replace individual operations
  114. without touching window logic.
  115. </p>
  116. <p>
  117. The software fill uses an SSE2 row store when the CPU supports it.
  118. Rows that do not fit the cache use streaming stores
  119. (<code>movntdq</code>, one <code>sfence</code> per fill), which
  120. avoids read-for-ownership traffic that makes large fills DRAM-bound
  121. on the Atom. SSE/SSE2 are never assumed: the kernel is built with
  122. <code>-mno-sse</code>, and <code>cpu_has_sse2()</code> probes CPUID,
  123. sets <code>CR4.OSFXSR/OSXMMEXCPT</code> and is re-run on every AP.
  124. </p>
  125. <p>
  126. <code>gfxbench</code> times fill, copy, cell, canvas and flush paths
  127. and reports the bytes actually copied by the presenter.
  128. </p>
  129. </section>
  130. <section id="wm">
  131. <h2>Window manager</h2>
  132. <p>
  133. The WM is a process that owns the desktop and a fixed array of eight
  134. windows. It draws a system bar with the banner, launcher buttons,
  135. clock, the CPU temperature and, on ThinkPads, the AC adapter state
  136. and battery percentage, plus draggable
  137. terminal and canvas windows with a title bar and close button.
  138. Click-to-focus, drag-by-title-bar and close are routed through the
  139. same pointer path. The right-aligned date/time text is also a
  140. target: it highlights on hover and opens the Date &amp; Time editor
  141. (<code>bin/settime</code>) on a click. To its left, a network status
  142. icon shows a cable for wired Ethernet, signal bars for a connected
  143. WiFi link, a USB mark for RNDIS tethering, or a red X when no link
  144. is up. Clicking it opens the anchored <code>bin/netmenu</code>
  145. window: present NICs can be selected (then DHCP runs), and scanned
  146. WiFi networks can be joined, asking for the WPA2 passphrase first
  147. when the AP requires one.
  148. </p>
  149. <p>
  150. The WM is the ring-3 process loaded from
  151. <code>/ramfs/bin/wm</code>; there is no kernel WM fallback. The
  152. kernel window server in <span class="file">wm.c</span> owns the
  153. shadow mapping, the per-window canvases, input routing, the cursor,
  154. kexec detection and the boot-console handoff. It starts the first
  155. terminal after the storage mount
  156. (<code>wm_server_boot_ready</code>) and leaves the boot console
  157. visible until the WM attaches.
  158. </p>
  159. <figure class="figure">
  160. <svg viewBox="0 0 920 260" role="img" aria-labelledby="wm-title wm-desc">
  161. <title id="wm-title">Window anatomy</title>
  162. <desc id="wm-desc">A window consists of a title bar, a content area backed by an RGB565 canvas, and a close control. The WM composites the canvas into the screen shadow buffer.</desc>
  163. <rect class="svg-box" x="70" y="22" width="780" height="200" rx="10"/>
  164. <rect class="svg-box accent" x="70" y="22" width="780" height="34" rx="10"/>
  165. <text class="svg-text" x="88" y="45">Terminal · /ramfs/bin/ls</text>
  166. <text class="svg-text" x="800" y="45">×</text>
  167. <rect class="svg-box green" x="86" y="72" width="748" height="132" rx="6"/>
  168. <text class="svg-text" x="104" y="98">content canvas · RGB565 · mapped at 0x50000000</text>
  169. <text class="svg-sub" x="104" y="120">text cells, images, widgets, or arbitrary drawing</text>
  170. </svg>
  171. <figcaption>
  172. <strong>One window, one canvas.</strong> The title bar belongs to
  173. the WM; the content is whatever the owning process draws into its
  174. mapped back buffer. The WM never reads the process's memory — it
  175. composites the front canvas on <code>PRESENT</code>.
  176. </figcaption>
  177. </figure>
  178. <ul>
  179. <li><strong>Terminal windows</strong> are canvas clients: <span class="file">ucmd_term.c</span> renders its rows into the RGB565 window canvas; keys arrive through the generic <code>SYS_KEY_POLL</code> or <code>SYS_WIN_KEY_WAIT</code>. The WM no longer has a text-cell path.</li>
  180. <li><strong>Canvas windows</strong> are generic: the app gets a back buffer, pointer position/buttons/focus and queued press/release edges with capture.</li>
  181. <li><strong>Focus and z-order</strong> are explicit; closing a window kills its session and frees its canvas.</li>
  182. </ul>
  183. <h3>Compositing cost</h3>
  184. <p>
  185. The WM keeps a front-to-back order: the focused window first, then
  186. the remaining windows in descending slot order. For each damage
  187. rectangle it fills the desktop once and walks that order, subtracting
  188. every window in front from the region still uncovered. A window is
  189. drawn only in its exposed pieces, so a fully occluded window costs
  190. nothing and the stack is never repainted back-to-front. The kernel
  191. pointer hit-test uses the same order. A focused-window drag copies
  192. its shadow block to the new position and repaints only the old
  193. strips the new rectangle does not cover, instead of compositing the
  194. window at both positions. If either rectangle is partly off-screen
  195. the shadow has no valid pixels for the hidden part, so that step
  196. repaints the window rect instead of copying desktop background into
  197. it.
  198. </p>
  199. <p>
  200. <code>SYS_WIN_PRESENT</code> carries a content-relative damage
  201. rectangle. The terminal tracks its dirty rows, so a keystroke copies
  202. and composites one 8×16 row block instead of the whole 512×512
  203. canvas. The kernel unions the rectangles between WM polls and
  204. <code>SYS_WM_INFO</code> returns that union; a present with
  205. <code>w == 0</code> or <code>h == 0</code> still means the whole
  206. canvas. The WM overwrites <code>/share/wmstats</code> every two
  207. seconds with the interval's frame count, compositing time, worst
  208. frame and pixels drawn (<code>usurf_pixels</code>), which is the
  209. on-target compositor measurement.
  210. </p>
  211. </section>
  212. <section id="canvas">
  213. <h2>Canvas ABI</h2>
  214. <p>
  215. <code>SYS_WIN_CREATE</code> maps the calling command's window canvas
  216. read/write at <code>0x50000000</code>. The process draws RGB565
  217. pixels and publishes a finished frame with
  218. <code>SYS_WIN_PRESENT</code>, optionally passing a content-relative
  219. rectangle (<code>struct wm_win_present</code>) for the changed
  220. region. Present copies that region from the back buffer into the
  221. composited front canvas with interrupts off, so a half-drawn frame
  222. can never reach the screen. Frames carry monotonic sequence
  223. counters, so the WM cannot lose one.
  224. </p>
  225. <p>
  226. <code>SYS_WIN_CREATE</code> creates a generic canvas window. Input is
  227. exposed through <code>SYS_WIN_POINTER</code> (content-relative
  228. position, button mask, focus), <code>SYS_WIN_EVENT</code>, which
  229. queues press/release edges and a capture window, and
  230. <code>SYS_WIN_SCROLL</code>, which returns wheel/touchpad scroll
  231. accumulated for the window under the pointer in pixels
  232. (<code>+dy</code> is later content). A click that starts
  233. in a window still receives its release after the pointer leaves.
  234. The ngui helper <code>ngui_scroll_wheel()</code> applies the delta
  235. to a pane when the pointer is over it.
  236. The WM process itself is special: it maps the screen shadow at
  237. <code>0x58000000</code> and each front canvas at
  238. <code>0x59000000 + index × 4 MB</code>, which is how it composites
  239. without reading another process's memory. The 4 MB slot covers a
  240. full-screen 1280×1024 canvas minus the chrome; the old 2 MB slot
  241. capped a full-height window at roughly 1050 px wide.
  242. <code>SYS_WM_RESIZE</code> (WM side) and
  243. <code>SYS_WIN_ACK_RESIZE</code> (client side) swap in a resized
  244. canvas; the content size is bounded by the framebuffer, so a
  245. window can grow to the edge of the screen.
  246. </p>
  247. <p>
  248. Double-clicking a title bar maximizes the window to the work area
  249. below the system bar (<code>0, WM_BAR_H, fbw, fbh - WM_BAR_H</code>);
  250. the next double-click restores the saved geometry. Dragging a
  251. maximized window restores it first, and a manual resize clears the
  252. maximized state. Windows can be dragged past the left, right and
  253. bottom edges; <code>WM_DRAG_VISIBLE</code> (48 px) of the title bar
  254. always stays on-screen for recovery, and the top bar is never
  255. overlapped. The resize grip is clamped to the full work area and
  256. snaps to it when dragged into the screen corner.
  257. </p>
  258. <div class="note">
  259. <p><strong>Buttons fire on the release edge.</strong> The toolkit keeps
  260. a button <code>active</code> until <code>ui-&gt;released</code>;
  261. inferring release from the sampled button mask races the WM event
  262. queue and drops clicks.</p>
  263. </div>
  264. <p>
  265. When a process exits or crashes, the reaper releases its window
  266. mappings and the window server frees the canvas. Closing a window
  267. kills its owner. Keys go through the window key queue
  268. (<code>SYS_WIN_KEY_WAIT</code>) or the generic
  269. <code>SYS_KEY_POLL</code>; Ctrl+C still reaches the terminal.
  270. </p>
  271. </section>
  272. <section id="ngui">
  273. <h2>ngui toolkit</h2>
  274. <p>
  275. <span class="file">ngui.c</span>/<span class="file">ngui.h</span> is an
  276. immediate-mode GUI toolkit shared by the ring-3 apps. It provides
  277. clipped RGB565 primitives, buttons, menu items, text fields, nine
  278. procedural icons, a theme palette, layout cut helpers and scrolling
  279. panes. It is linked into <code>files</code>, <code>mines</code>,
  280. <code>solitaire</code>, <code>chat</code> and the real
  281. <code>gallery.elf</code>.
  282. </p>
  283. <ul>
  284. <li><strong>Text fields</strong> support drag-selection, double-click select-all, Ctrl+Left/Right word jumps, Ctrl+Delete/Backspace word deletion and Ctrl+A. They use the proportional face with measured caret, selection and horizontal scrolling; a field marked <code>disabled</code> ignores pointer and keyboard input, drops the focus and paints its value dimmed in the same proportional face. A field marked <code>secret</code> is measured and painted from a <code>*</code> mask (caret, selection and click positions follow the mask, Ctrl+Left/Right jump to the ends), and Tab returns <code>NGUI_INPUT_NEXT</code> together with <code>NGUI_INPUT_CANCEL</code> so forms can move the focus. Fields hold up to 511 characters (<code>NGUI_INPUT_MAX</code> 512). The terminal and editor keep the 8×16 cell grid.</li>
  285. <li><strong>Clip stack</strong> keeps nested panes and popups inside their parents.</li>
  286. <li><strong>Layout cuts</strong> split a rectangle into rows, columns and fixed margins without a constraint solver.</li>
  287. <li><strong>No retained widget tree.</strong> Apps redraw from state every frame, which keeps the toolkit small and predictable.</li>
  288. </ul>
  289. <p>
  290. Proportional text comes from <span class="file">ufont.c</span>: a
  291. fixed-point, integer-only TrueType parser and antialiased rasterizer.
  292. The pack carries Latin-1 Arimo regular/bold/italic/bold-italic and
  293. Liberation Mono subsets under <code>/ramfs/fonts</code>. The browser,
  294. ngui apps and the WM chrome use real pixel sizes from 8 to 48 px;
  295. the browser maps CSS <code>font-size</code>, <code>font-weight</code>,
  296. <code>font-style</code> and <code>font-family: monospace</code> to
  297. the faces. Missing fonts fall back to the 8×16 bitmap. The browser
  298. detaches the font arena before <code>heap_reset()</code> and
  299. reattaches after <code>heap_start()</code>; terminal content and
  300. the editor keep the bitmap cells.
  301. </p>
  302. </section>
  303. <section id="apps">
  304. <h2>Applications</h2>
  305. <div class="grid">
  306. <div class="card">
  307. <h3>files</h3>
  308. <p>File browser on the VFS: opens text in <code>edit</code>, images in <code>view</code> and HTML in <code>browser</code>. The first ngui user.</p>
  309. </div>
  310. <div class="card">
  311. <h3>gallery</h3>
  312. <p>Live showcase of every ngui control and primitive. Ships in the pack as <code>bin/gallery.app</code>; a launcher prefers <code>/share/GALLERY</code> for development.</p>
  313. </div>
  314. <div class="card">
  315. <h3>chat</h3>
  316. <p>Cloudron Chat client: native OIDC login, channel/DM sidebar with unread badges and presence dots, a wrapped timeline with avatars, day headers and reply snippets, a composer and browse/new-DM pickers. The top-bar <em>Chat</em> button launches it.</p>
  317. </div>
  318. <div class="card">
  319. <h3>solitaire</h3>
  320. <p>Klondike with drag-and-drop, draw-one/three, scoring, animation and pixel-art cards on the canvas service.</p>
  321. </div>
  322. <div class="card">
  323. <h3>view</h3>
  324. <p>Image viewer with fit, 1:1, zoom, pan, EXIF orientation and animated GIF playback.</p>
  325. </div>
  326. <div class="card">
  327. <h3>clock / colors / settime</h3>
  328. <p>Small canvas demos: an analog clock, a colour test window, and the Date &amp; Time editor. Clicking the top-bar date/time opens <code>settime</code>, which reads and writes the CMOS RTC and can fetch UTC from an SNTP server.</p>
  329. </div>
  330. <div class="card">
  331. <h3>browser</h3>
  332. <p>HTML/CSS renderer with flex rows, auto-fit card grids, inline-block runs, basic tables and inline SVG diagrams. It decodes inline images and fetches through a headless helper; inline and external scripts run on the vendored MuJS engine, while form submission is not supported. The largest ring-3 application in the tree.</p>
  333. </div>
  334. </div>
  335. </section>
  336. <section id="images">
  337. <h2>Image library</h2>
  338. <p>
  339. The codecs are independent of the OS and the toolkit: each takes an
  340. input buffer plus a caller-provided <code>struct img_mem</code> arena
  341. and returns an RGB565 image with an optional 8-bit alpha plane. The
  342. browser links the same objects and decodes HTTP bodies in memory.
  343. </p>
  344. <ul>
  345. <li><strong>Formats.</strong> PNG, baseline and progressive JPEG, animated GIF, BMP, TGA, PNM, ICO/CUR and PAM.</li>
  346. <li><strong>Drawing.</strong> <code>img_draw()</code> is a clipped, scaled RGB565 blit: box-filter downscale, bilinear upscale, alpha over a checkerboard or over the destination.</li>
  347. <li><strong>Orientation.</strong> EXIF orientation is applied; arithmetic JPEG and CMYK are rejected.</li>
  348. <li><strong>Cost.</strong> Downscaling runs on the CPU. The viewer caches one image; the browser keeps a display bitmap but still rescales on repaint.</li>
  349. </ul>
  350. </section>
  351. <section id="gpu">
  352. <h2>GPU quirks</h2>
  353. <ul>
  354. <li><strong>ATI Mach64.</strong> A CRTC fixup is applied after mode set. The legacy CRTC is skipped under UEFI.</li>
  355. <li><strong>NVIDIA.</strong> VBE-only. The legacy VGA CRTC is never touched. On the ION, the framebuffer is stolen DRAM behind a BAR3 aperture; <span class="file">nvidia.c</span> finds the stolen range and writes through a cached alias once a WC MTRR covers it.</li>
  356. <li><strong>Intel.</strong> The Ironlake-era path supports a hardware cursor where available; the software cursor is the common fallback.</li>
  357. <li><strong>UEFI.</strong> <code>vga.c</code> honours the RGBX flag and skips the VBE-only ATI CRTC reset and NVIDIA stolen-VRAM alias. <span class="file">mouse.c</span> forces the software cursor.</li>
  358. </ul>
  359. </section>
  360. <section id="sources">
  361. <h2>Source map</h2>
  362. <ul class="filelist">
  363. <li><code>vga.c</code>, <code>vga.h</code> — framebuffer and damage</li>
  364. <li><code>vga_copy.S</code> — copy primitives</li>
  365. <li><code>blit.c</code>, <code>blit.h</code> — pixel vtable</li>
  366. <li><code>cpu.c</code> — SSE2 probing</li>
  367. <li><code>wm.c</code>, <code>wm.h</code>, <code>wm_abi.h</code> — kernel window server and shared ABI</li>
  368. <li><code>ucmd_wm.c</code>, <code>uwm_lib.c</code> — ring-3 WM</li>
  369. <li><code>ucmd_netmenu.c</code> — top-bar network menu (NIC selection, DHCP, WiFi scan/passphrase)</li>
  370. <li><code>ucmd_settime.c</code>, <code>rtc.c</code>, <code>ntp.c</code> — Date &amp; Time editor, RTC access and SNTP sync</li>
  371. <li><code>gfx.c</code>, <code>gfx.h</code> — RGB565 window canvases</li>
  372. <li><code>sys_wm.c</code>, <code>sys_wm.h</code> — WM/window ABI</li>
  373. <li><code>browser.c</code>, <code>browser_html.c</code> — HTML/CSS renderer and inline SVG</li>
  374. <li><code>ngui.c</code>, <code>ngui.h</code> — toolkit</li>
  375. <li><code>ufont.c</code>, <code>ufont_ttf.c</code>, <code>ufont_draw.c</code> — fixed-point TrueType text</li>
  376. <li><code>fonts/*.ttf</code> — Arimo / Liberation Mono subsets in the pack</li>
  377. <li><code>ucmd_files.c</code>, <code>gallery.c</code>, <code>ucmd_solitaire.c</code>, <code>ucmd_chat.c</code></li>
  378. <li><code>img*.c</code> — image codecs</li>
  379. <li><code>ati.c</code>, <code>nvidia.c</code>, <code>intel.c</code>, <code>nv50.c</code></li>
  380. </ul>
  381. </section>
  382. </main>
  383. <footer class="site-footer">
  384. <div class="inner">
  385. <p><strong>NebOS 0.2.0</strong> · Architecture documentation · <a href="index.html">overview</a></p>
  386. <p>Maintain alongside <code>AGENTS.md</code> and the source files listed on each page.</p>
  387. </div>
  388. </footer>
  389. </body>
  390. </html>