Assorted Performance Enhancements (#9645)

A whole bunch of optimizations in hot paths in the IO processing areas
of our code (well, one of them covers everything). I validated that each
commit either improved one or more of our vtebench results, or improved
the time it takes to process 2 years worth (2.4GB) of data from
asciinema.

## vtebench
<img width="1278" height="903" alt="image"
src="https://github.com/user-attachments/assets/bad46777-4606-4870-b7d7-8df0c4bb3b39"
/>

(I decided to patch vtebench to report in nanoseconds instead of
milliseconds since clearly it was not designed for a machine as fast as
mine. Nanoseconds gives much more useful results when the numbers are
this low.)

Do note the *slight* regression in the "unicode" test, this is probably
because I added a branch hint in `Terminal.print` in order to optimize
for printing narrow characters, since they make up the vast majority of
characters typically printed in the terminal, but the vtebench "unicode"
test is pretty much all wide characters.

This shouldn't have a negative effect on users of CJK languages since
it's a *very* slight reduction in speed and they will still be printing
many narrow characters, especially in TUIs; spaces, box drawing
characters, symbols, punctuation, etc.

## asciinema processing
I wrote a program that uses libghostty to push 2 years worth (2.4GB) of
data from publicly uploaded asciinema recordings in to the terminal as
fast as possible- since it's just libghostty, there's no renderer
overhead happening, it's just the core terminal emulation, effectively
everything that io-reader thread does if it didn't have wait for the
renderer ever.

On main, this took roughly 26.1–26.7 seconds to process, on this branch
it takes just 18.4–18.6 seconds, that's a ~30% improvement in raw IO
processing speed when processing real world data!

## Summary of changes
In order of commits:
- Fixed a bug that I hit when trying to have Ghostty process all that
asciinema data, in certain bad cases it was possible to accidentally
insert the `0` hyperlink ID in to a page, which would then cause a
lockup in ReleaseFast mode when trying to clone that page since the
string alloc would try to iterate `1..0` to allocate 0 chunks.
- I noticed in profiling Ghostty that `std.debug.assert` was showing up
in the profile, which it should not have been since its doc comment
promises that it will be optimized out in ReleaseFast- but evidently
something is wrong with Zig, or that comment's promise is based on an
expectation from LLVM that it fails to meet - but either way, by
replacing all uses of `assert` with a version that is explicitly marked
`inline`, that function call overhead in tight loops and hotpaths is
avoided. This change alone accounts for like a third of the IO
processing time improvement, though it had minimal impact on vtebench
scores.
- I optimized the SGR parser somewhat by adding branch hints and
removing the `.reset_underline` action, replacing it with `.{ .underline
= .none }`.
- Gated a somewhat expensive assert in RefCountedSet behind a runtime
safety check.
- Improved the performance of `Style.eql` and `Style.hash` since these
are hot functions, called extremely frequently since adding styles to
the style set is a very common operation. Achieved this by making `eql`
less generic - explicitly comparing each part of the style rather than
looping over fields - and ordering checks from most likely to differ to
least likely to differ so that differences can be found as soon as
possible; and changed the hash from xxhash to simply folding the packed
struct down to 64 bits and then using `std.hash.int`. Also manually
inlined the code from `std.meta.activeTag` in `Packed.fromStyle`, since
profiling showed it in the callstack and it's a single cast so it really
should not have the function call overhead.
- Explicitly marked some trivial functions as inline, the optimizer
would already have been doing this (probably) but doing it explicitly
gives the optimizer more time to spend on other things. Added cold
branch hints to "should be impossible" and error-returning paths that
should be very rare, and unlikely branch hints to a lot of "invalid"
paths- to optimize for receiving valid data.
- Removed a branch in the parser csi param action, just unconditionally
multiply by 10 before adding digit value, even if it's the first digit.
This codepath is rarely hit since we have a fast path for this in the
stream code, but the stream code already has this optimization so I just
copied it over.
- `CharsetState.charsets` used to be an `EnumArray`, but the
layout/access logic for that was less-than-ideal, and the access
functions were not inlining-- and these are very hot since we access
this for every single print, so I wrote a bespoke struct to hold that
info instead, gained a couple percent of IO perf with that.
- Added branch hints based on the data I derived from the asciinema
dump, which gave big boost to vtebench results, especially for the
cursor movement and dense cells tests (which makes sense, since cursor
movement and setting attributes both got `likely` hints :p) -- data at
https://github.com/qwerasd205/asciinema-stats
- This is probably the most invasive change in this PR: I removed the
dirty bitset from `Page` and replaced it with a dirty flag on each row,
for the majority of operations this is faster to write, since the row
being dirtied is probably already loaded and probably will be written to
for other changes as well. This gave a couple percent IO processing
improvement. The only exception is scrolling-type operations, which are
extremely efficient by just moving rows around with a single memmov, so
looping through the rows to mark each dirty slows them down, and indeed
after this change the scrolling benchmarks in vtebench regressed,
*however*...
- Added a "full page dirty" flag on `Page`, which is set when an
operation is performed that dirties most or all the rows in the page,
which is used for scrolling-type operations. This *does* make the dirty
tracking slightly less precise for these operations, but with the
caching and stuff we do in the renderer, I don't think `rebuildCells` is
a bottleneck, so rebuilding a few extra rows shouldn't hurt. After this
change, all the scrolling benchmarks in vtebench improved drastically.
- Tiny micro-improvements to RefCountedSet; streamlined the control flow
in `lookup`, added an unlikely branch hint in `insert` for the branch
that resurrects dead items since dead items aren't that common.
- Improve SGR parser performance again by using `@call(.always_inline`
to explicitly inline calls to `StaticBitSet.isSet` (for the separator
list), since I noticed they weren't being inlined, causing function call
overhead in a hotpath.
- I noticed that `clearGrapheme` and `clearHyperlink` would check every
cell in the row after they were done in order to update the
`grapheme`/`hyperlink` flag on the row if there were none left, which
isn't great since `clearCells` called these functions for multiple cells
in the same row back-to-back, which leads to a ton of excess work. I
separated the flag updating parts of these functions out and called them
only if necessary (if the cells being cleared were the full row then the
flag could unconditionally be set to false) and only after all the cells
were cleared. This gave a nice improvement to IO processing since
clearCells is evidently a very hot function.
- Removed inline annotations on `Page.clearGrapheme` and
`Page.clearHyperlink` in favor of inlining directly at the one callsite
that benefited from inlining, this improved IO processing speed.
- Inlined trivial function `Charset.table`.
- Inlined `size.getOffset` and `size.intFromBase` as they are both
trivial pointer math that often benefits from surrounding context.

---

If you'd like me to separate out the trivial improvements (branch hints,
inline annotations, 1-line changes) from the functionality-changing ones
(pretty much just the changes to dirty tracking), just let me know!

src/App.zig

@@ -5,7 +5,7 @@ const App = @This();
 
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const build_config = @import("build_config.zig");
 const apprt = @import("apprt.zig");

src/Surface.zig

@@ -17,7 +17,7 @@ pub const Message = apprt.surface.Message;
 
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const ArenaAllocator = std.heap.ArenaAllocator;
 const global_state = &@import("global.zig").state;

src/apprt/action.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const build_config = @import("../build_config.zig");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const apprt = @import("../apprt.zig");
 const configpkg = @import("../config.zig");
 const input = @import("../input.zig");

src/apprt/embedded.zig

@@ -6,7 +6,7 @@
 
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const objc = @import("objc");
 const apprt = @import("../apprt.zig");

src/apprt/gtk/cgroup.zig

@@ -1,7 +1,7 @@
 /// Contains all the logic for putting the Ghostty process and
 /// each individual surface into its own cgroup.
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 const gio = @import("gio");

src/apprt/gtk/class/application.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const builtin = @import("builtin");
 const adw = @import("adw");

src/apprt/gtk/class/clipboard_confirmation_dialog.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const adw = @import("adw");
 const glib = @import("glib");
 const gobject = @import("gobject");

src/apprt/gtk/class/global_shortcuts.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const adw = @import("adw");
 const gio = @import("gio");

src/apprt/gtk/class/imgui_widget.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 
 const cimgui = @import("cimgui");
 const gl = @import("opengl");

src/apprt/gtk/class/resize_overlay.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const adw = @import("adw");
 const glib = @import("glib");
 const gobject = @import("gobject");

src/apprt/gtk/class/split_tree.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const build_config = @import("../../../build_config.zig");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const adw = @import("adw");
 const gio = @import("gio");

src/apprt/gtk/class/surface.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const adw = @import("adw");
 const gdk = @import("gdk");

src/apprt/gtk/class/surface_child_exited.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const adw = @import("adw");
 const glib = @import("glib");
 const gobject = @import("gobject");

src/apprt/gtk/class/surface_scrolled_window.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const adw = @import("adw");
 const gobject = @import("gobject");
 const gtk = @import("gtk");

src/apprt/gtk/class/tab.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const build_config = @import("../../../build_config.zig");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const adw = @import("adw");
 const gio = @import("gio");
 const glib = @import("glib");

src/apprt/gtk/class/window.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const build_config = @import("../../../build_config.zig");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const adw = @import("adw");
 const gdk = @import("gdk");
 const gio = @import("gio");

src/apprt/gtk/ext.zig

@@ -4,7 +4,7 @@
 //! helpers.
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const testing = std.testing;
 
 const gio = @import("gio");

src/apprt/gtk/ext/actions.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const testing = std.testing;
 
 const gio = @import("gio");

src/apprt/ipc.zig

@@ -2,7 +2,7 @@
 //! process.
 const std = @import("std");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 
 pub const Errors = error{
     /// The IPC failed. If a function returns this error, it's expected that

src/cli/args.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const mem = std.mem;
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = mem.Allocator;
 const ArenaAllocator = std.heap.ArenaAllocator;
 const diags = @import("diagnostics.zig");

src/cli/diagnostics.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const build_config = @import("../build_config.zig");
 

src/cli/edit_config.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const args = @import("args.zig");
 const Allocator = std.mem.Allocator;
 const Action = @import("ghostty.zig").Action;

src/cli/ssh-cache/DiskCache.zig

@@ -5,7 +5,7 @@ const DiskCache = @This();
 
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const internal_os = @import("../../os/main.zig");
 const xdg = internal_os.xdg;

src/config/CApi.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const cli = @import("../cli.zig");
 const inputpkg = @import("../input.zig");
 const state = &@import("../global.zig").state;

src/config/ClipboardCodepointMap.zig

@@ -4,7 +4,7 @@
 const ClipboardCodepointMap = @This();
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 // To ease our usage later, we map it directly to formatter entries.

src/config/Config.zig

@@ -13,7 +13,7 @@ const Config = @This();
 const std = @import("std");
 const builtin = @import("builtin");
 const build_config = @import("../build_config.zig");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const ArenaAllocator = std.heap.ArenaAllocator;
 const global_state = &@import("../global.zig").state;

src/config/conditional.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 /// Conditionals in Ghostty configuration are based on a static, typed

src/config/edit.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const ArenaAllocator = std.heap.ArenaAllocator;
 const internal_os = @import("../os/main.zig");

src/config/file_load.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const internal_os = @import("../os/main.zig");
 

src/config/io.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const ArenaAllocator = std.heap.ArenaAllocator;
 const string = @import("string.zig");

src/config/path.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const ArenaAllocator = std.heap.ArenaAllocator;
 

src/config/theme.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const global_state = &@import("../global.zig").state;
 const internal_os = @import("../os/main.zig");

src/datastruct/blocking_queue.zig

@@ -3,7 +3,7 @@
 
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 /// Returns a blocking queue implementation for type T.

src/datastruct/cache_table.zig

@@ -1,7 +1,7 @@
 const fastmem = @import("../fastmem.zig");
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 
 /// An associative data structure used for efficiently storing and
 /// retrieving values which are able to be recomputed if necessary.

src/datastruct/circ_buf.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const fastmem = @import("../fastmem.zig");
 

src/datastruct/lru.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 /// Create a HashMap for a key type that can be automatically hashed.

src/datastruct/segmented_pool.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const testing = std.testing;
 

src/datastruct/split_tree.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const build_config = @import("../build_config.zig");
 const ArenaAllocator = std.heap.ArenaAllocator;
 const Allocator = std.mem.Allocator;

src/fastmem.zig

@@ -1,6 +1,5 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
 
 /// Same as @memmove but prefers libc memmove if it is
 /// available because it is generally much faster?.

src/font/Atlas.zig

@@ -16,7 +16,7 @@
 const Atlas = @This();
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const testing = std.testing;
 const fastmem = @import("../fastmem.zig");

src/font/CodepointMap.zig

@@ -4,7 +4,7 @@
 const CodepointMap = @This();
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const discovery = @import("discovery.zig");
 

src/font/Collection.zig

@@ -16,7 +16,7 @@
 const Collection = @This();
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const config = @import("../config.zig");
 const comparison = @import("../datastruct/comparison.zig");

src/font/DeferredFace.zig

@@ -7,7 +7,7 @@
 const DeferredFace = @This();
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const fontconfig = @import("fontconfig");
 const macos = @import("macos");

src/font/SharedGrid.zig

@@ -19,7 +19,7 @@
 const SharedGrid = @This();
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const renderer = @import("../renderer.zig");
 const font = @import("main.zig");

src/font/SharedGridSet.zig

@@ -11,7 +11,7 @@ const SharedGridSet = @This();
 
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const ArenaAllocator = std.heap.ArenaAllocator;
 const font = @import("main.zig");

src/font/discovery.zig

@@ -1,7 +1,7 @@
 const std = @import("std");
 const builtin = @import("builtin");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const fontconfig = @import("fontconfig");
 const macos = @import("macos");
 const opentype = @import("opentype.zig");

src/font/face/coretext.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const macos = @import("macos");
 const harfbuzz = @import("harfbuzz");

src/font/face/freetype.zig

@@ -9,7 +9,7 @@ const builtin = @import("builtin");
 const freetype = @import("freetype");
 const harfbuzz = @import("harfbuzz");
 const stb = @import("../../stb/main.zig");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const testing = std.testing;
 const Allocator = std.mem.Allocator;
 const font = @import("../main.zig");

src/font/face/web_canvas.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const testing = std.testing;
 const Allocator = std.mem.Allocator;
 const js = @import("zig-js");

src/font/opentype/head.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const sfnt = @import("sfnt.zig");
 
 /// Font Header Table

src/font/opentype/hhea.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const sfnt = @import("sfnt.zig");
 
 /// Horizontal Header Table

src/font/opentype/os2.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const sfnt = @import("sfnt.zig");
 
 pub const FSSelection = packed struct(sfnt.uint16) {

src/font/opentype/post.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const sfnt = @import("sfnt.zig");
 
 /// PostScript Table

src/font/opentype/sfnt.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 /// 8-bit unsigned integer.

src/font/opentype/svg.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const font = @import("../main.zig");
 
 /// SVG glyphs description table.

src/font/shaper/Cache.zig

@@ -11,7 +11,7 @@
 pub const Cache = @This();
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const font = @import("../main.zig");
 const CacheTable = @import("../../datastruct/main.zig").CacheTable;

src/font/shaper/coretext.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const macos = @import("macos");
 const trace = @import("tracy").trace;

src/font/shaper/feature.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 const log = std.log.scoped(.font_shaper);

src/font/shaper/harfbuzz.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const harfbuzz = @import("harfbuzz");
 const font = @import("../main.zig");

src/font/shaper/noop.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const trace = @import("tracy").trace;
 const font = @import("../main.zig");

src/font/shaper/run.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const font = @import("../main.zig");
 const shape = @import("../shape.zig");

src/font/shaper/web_canvas.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const font = @import("../main.zig");
 const terminal = @import("../../terminal/main.zig");

src/font/sprite/Face.zig

@@ -14,7 +14,7 @@ const Face = @This();
 
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const wuffs = @import("wuffs");
 const z2d = @import("z2d");

src/font/sprite/canvas.zig

@@ -1,7 +1,7 @@
 //! This exposes primitives to draw 2D graphics and export the graphic to
 //! a font atlas.
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const z2d = @import("z2d");
 const font = @import("../main.zig");

src/font/sprite/draw/block.zig

@@ -6,7 +6,7 @@
 //!
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 const z2d = @import("z2d");

src/font/sprite/draw/box.zig

@@ -12,7 +12,7 @@
 //!
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 const z2d = @import("z2d");

src/font/sprite/draw/braille.zig

@@ -23,7 +23,7 @@
 //!
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 const font = @import("../../main.zig");

src/font/sprite/draw/branch.zig

@@ -16,7 +16,7 @@
 //!
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 const common = @import("common.zig");

src/font/sprite/draw/common.zig

@@ -4,7 +4,7 @@
 //! rather than being single-use.
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 
 const z2d = @import("z2d");

src/font/sprite/draw/special.zig

@@ -7,7 +7,7 @@
 
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const font = @import("../../main.zig");
 const Sprite = font.sprite.Sprite;

src/font/sprite/draw/symbols_for_legacy_computing.zig

@@ -21,7 +21,7 @@
 
 const std = @import("std");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 
 const z2d = @import("z2d");
 

src/font/sprite/draw/symbols_for_legacy_computing_supplement.zig

@@ -49,7 +49,7 @@
 
 const std = @import("std");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../../../quirks.zig").inlineAssert;
 
 const z2d = @import("z2d");
 

src/input/Binding.zig

@@ -4,7 +4,7 @@ const Binding = @This();
 
 const std = @import("std");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const build_config = @import("../build_config.zig");
 const uucode = @import("uucode");
 const EntryFormatter = @import("../config/formatter.zig").EntryFormatter;

src/input/command.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const Action = @import("Binding.zig").Action;
 

src/input/paste.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Terminal = @import("../terminal/Terminal.zig");
 
 pub const Options = struct {

src/inspector/Inspector.zig

@@ -4,7 +4,7 @@
 const Inspector = @This();
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const builtin = @import("builtin");
 const cimgui = @import("cimgui");

src/inspector/cell.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const cimgui = @import("cimgui");
 const terminal = @import("../terminal/main.zig");

src/inspector/page.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const cimgui = @import("cimgui");
 const terminal = @import("../terminal/main.zig");

src/lib/union.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const testing = std.testing;
 const Target = @import("target.zig").Target;
 

src/main_c.zig

@@ -8,7 +8,7 @@
 // it could be expanded to be general purpose in the future.
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("quirks.zig").inlineAssert;
 const posix = std.posix;
 const builtin = @import("builtin");
 const build_config = @import("build_config.zig");

src/os/args.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const objc = @import("objc");
 const macos = @import("macos");

src/os/cgroup.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const linux = std.os.linux;
 const posix = std.posix;
 const Allocator = std.mem.Allocator;

src/os/flatpak.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const builtin = @import("builtin");
 const posix = std.posix;

src/os/homedir.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const passwd = @import("passwd.zig");
 const posix = std.posix;
 const objc = @import("objc");

src/os/locale.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const macos = @import("macos");
 const objc = @import("objc");
 const internal_os = @import("main.zig");

src/os/macos.zig

@@ -1,7 +1,7 @@
 const std = @import("std");
 const builtin = @import("builtin");
 const build_config = @import("../build_config.zig");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const objc = @import("objc");
 const Allocator = std.mem.Allocator;
 

src/os/mouse.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const objc = @import("objc");
 
 const log = std.log.scoped(.os);

src/os/xdg.zig

@@ -3,7 +3,7 @@
 
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const posix = std.posix;
 const homedir = @import("homedir.zig");

src/quirks.zig

@@ -27,3 +27,20 @@ pub fn disableDefaultFontFeatures(face: *const font.Face) bool {
     //     error.OutOfMemory => return false,
     // };
 }
+
+/// We use our own assert function instead of `std.debug.assert`.
+///
+/// The only difference between this and the one in
+/// the stdlib is that this version is marked inline.
+///
+/// The reason for this is that, despite the promises of the doc comment
+/// on the stdlib function, the function call to `std.debug.assert` isn't
+/// always optimized away in `ReleaseFast` mode, at least in Zig 0.15.2.
+///
+/// In the majority of places, the overhead from calling an empty function
+/// is negligible, but we have some asserts inside tight loops and hotpaths
+/// that cause significant overhead (as much as 15-20%) when they don't get
+/// optimized out.
+pub inline fn inlineAssert(ok: bool) void {
+    if (!ok) unreachable;
+}

src/renderer/Metal.zig

@@ -2,7 +2,7 @@
 pub const Metal = @This();
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const builtin = @import("builtin");
 const objc = @import("objc");

src/renderer/OpenGL.zig

@@ -2,7 +2,7 @@
 pub const OpenGL = @This();
 
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const builtin = @import("builtin");
 const gl = @import("opengl");

src/renderer/Thread.zig

@@ -4,7 +4,7 @@ pub const Thread = @This();
 
 const std = @import("std");
 const builtin = @import("builtin");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const xev = @import("../global.zig").xev;
 const crash = @import("../crash/main.zig");
 const internal_os = @import("../os/main.zig");

src/renderer/cell.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const font = @import("../font/main.zig");
 const terminal = @import("../terminal/main.zig");
 const renderer = @import("../renderer.zig");

src/renderer/generic.zig

@@ -20,7 +20,7 @@ const Image = imagepkg.Image;
 const ImageMap = imagepkg.ImageMap;
 const ImagePlacementList = std.ArrayListUnmanaged(imagepkg.Placement);
 const shadertoy = @import("shadertoy.zig");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const ArenaAllocator = std.heap.ArenaAllocator;
 const Terminal = terminal.Terminal;
@@ -1190,12 +1190,14 @@ pub fn Renderer(comptime GraphicsAPI: type) type {
                 {
                     var it = state.terminal.screens.active.pages.pageIterator(
                         .right_down,
-                        .{ .screen = .{} },
+                        .{ .viewport = .{} },
                         null,
                     );
                     while (it.next()) |chunk| {
-                        var dirty_set = chunk.node.data.dirtyBitSet();
-                        dirty_set.unsetAll();
+                        chunk.node.data.dirty = false;
+                        for (chunk.rows()) |*row| {
+                            row.dirty = false;
+                        }
                     }
                 }
 

src/renderer/image.zig

@@ -1,6 +1,6 @@
 const std = @import("std");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const wuffs = @import("wuffs");
 
 const Renderer = @import("../renderer.zig").Renderer;

src/renderer/message.zig

@@ -1,5 +1,5 @@
 const std = @import("std");
-const assert = std.debug.assert;
+const assert = @import("../quirks.zig").inlineAssert;
 const Allocator = std.mem.Allocator;
 const configpkg = @import("../config.zig");
 const font = @import("../font/main.zig");

src/renderer/metal/Frame.zig

@@ -3,7 +3,7 @@ const Self = @This();
 
 const std = @import("std");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const builtin = @import("builtin");
 const objc = @import("objc");
 

src/renderer/metal/IOSurfaceLayer.zig

@@ -4,7 +4,7 @@ const IOSurfaceLayer = @This();
 
 const std = @import("std");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const builtin = @import("builtin");
 const objc = @import("objc");
 const macos = @import("macos");

src/renderer/metal/Pipeline.zig

@@ -3,7 +3,7 @@ const Self = @This();
 
 const std = @import("std");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const builtin = @import("builtin");
 const macos = @import("macos");
 const objc = @import("objc");

src/renderer/metal/RenderPass.zig

@@ -3,7 +3,7 @@ const Self = @This();
 
 const std = @import("std");
 const Allocator = std.mem.Allocator;
-const assert = std.debug.assert;
+const assert = @import("../../quirks.zig").inlineAssert;
 const builtin = @import("builtin");
 const objc = @import("objc");