Fix Cocoa startup crash when the VM has no console - #312
Open
charles-hood wants to merge 1 commit into
Open
charles-hood wants to merge 1 commit into
charles-hood wants to merge 1 commit into
Conversation
The texture-borrowing patch reads the default console's surface at the top of cocoa_display_init. When QEMU has no console at all (for example -nodefaults with no display device), qemu_console_lookup_default() returns NULL and qemu_console_surface() dereferences it, so QEMU segfaults before the window opens. Pristine QEMU shows its "This VM has no graphic display device." placeholder in that case. Leave surface NULL when there is no console. Listener registration then installs QEMU's placeholder, as it does upstream. With a console the statement is unchanged. Add a contract test that compiles the patched startup statements against a missing console.
charles-hood
marked this pull request as ready for review
October 4, 2026 00:54
themartiano
approved these changes
Oct 4, 2026
themartiano
left a comment
Collaborator
There was a problem hiding this comment.
Will be merged after v0.5.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
The runtime's QEMU segfaults at startup when the Cocoa display runs a VM that has no console at all:
It stops with
EXC_BAD_ACCESSat address0x38inqemu_console_surface, called fromcocoa_display_init. This happens withgl=offandgl=on.The cause is in
qemu-texture-borrowing-11.1.patch.cocoa_display_initcallsqemu_console_surface(dcl.con)straight afterqemu_console_lookup_default(), and that lookup returns NULL when there is no console. Pristine QEMU never reads the surface at that point. Its listener registration handles a NULL console by showing the "This VM has no graphic display device." placeholder.The fix is one statement in that patch:
Nothing reads
surfacebeforeqemu_console_register_listener()replaces it. I checked this with breakpoints on every reader inui/cocoa.mduring startup, with a GPU device in both GL modes. So with no console, registration installs QEMU's placeholder as upstream does, and with a console the statement does what it did before. Since nothing reads it there, the line could also be removed outright; I kept it, guarded, so the console case stays exactly as it was. Happy to switch to removing it if you prefer. I edited the patch that introduced the bug rather than adding a new one. Because the replacement keeps the same line count, every later patch applies at the same lines, with no fuzz or offsets.texture_patch_sha256is nowb8e107ecd82813854bac7f6bb110722f52937d64b6f98611c4dd02842977fc2f. Theui/cocoa.mhunks were regenerated with GNU diff 3.12 from a tree with the patch applied (the same tool reproduces the previous section byte for byte), and the section headers are unchanged.The app's own configuration always adds a virtio-gpu device and was never affected. The crash hits anyone who runs the runtime by hand without a display device. It needs a VM with no console of any kind. Without
-nodefaults, the default serial port and monitor arevctext consoles, so the crash needs-nodefaults, or both defaults replaced (for example-serial stdio -qmp ...).New test:
macos/Tests/test-cocoa-no-console.py, withcocoa-no-console-harness.c, added tomake test-contracts. It compiles the patched startup statements against stubs, with the console lookup returning NULL and then a real console. It also checks that the patch's only surface read is still among those statements and that the builder pins the patch's hash. It fails on the previous patch (abort at the NULL console) and passes now.Testing
MacBook Pro, Apple M4 Pro (Mac16,8), macOS 27.0.1 (26A434):
make runtimeon the base commit and on this branch. On both, the patch stack applies with no fuzz, offsets or rejects.python3 macos/Tests/test-cocoa-no-console.pypasses. With the previous patch it fails.make test-contractsandmake testpass.qemu_console_surfacewith-display cocoaand-display cocoa,gl=on. After the fix, both open a window with the placeholder (screenshot below), handle mouse movement, and exit with status 0 from the menu's Quit.-machine virt,accel=hvf,gic-version=3 -cpu host -m 1024 -nodefaults -S -device virtio-gpu-gl-pci,romfile= -display cocoa,gl=on) starts and shows "Display output is not active." before and after the change.make appand booting the guest (no Docker on this machine).vt100_keysym. Unpatched QEMU 11.1.1 (Homebrew) does the same, so that is upstream behaviour and this PR leaves it alone.