BACKENDS: OPENGLSDL: Incorrect resize of window when switching out of Fullscreen Mode

[macca8]

Tested with recent macOS Master builds from Buildbot:

• Intel x86 build on an Intel Mac running macOS 10.6.8 & 10.11.6,
• Apple Silicon build on a MacBook Air running macOS 15.7.

This is one of 3 inappropriate changes observed in Myst3 since the merger of the 3D & 2D Graphics Managers (see also #16254, #16255). I don't have any other 3D games to test, so I'm reporting this as a Myst3 bug.

When playing Myst3 (with or without the engine's Widescreen Mod enabled), using Alt+Enter to switch from fullscreen mode to windowed mode unexpectedly resizes the window (with title bar) to fill most of the screen area between the Menu Bar and the Dock (e.g. 1660x918 or 1660x926 on a 1680x1050 desktop, 1420x804 on a 1440x932 desktop).

This doesn't occur if the 3D Renderer is set to Software (but see #16254), or if the switch from fullscreen mode is completed after exiting Myst3.

It also doesn't occur when using Alt+Enter in the Launcher or 2D games, however the values stored in last_window_width/height are overwritten by this bug, so the Launcher and other games are affected indirectly.

This behaviour is similar to that described in #13054, except in this case, we're dealing with a normal unmaximized window, and it's limited to 3D OpenGL.

This specific issue affects all macOS versions listed above, but doesn't exist in the 2.9.1 macOS releases.

To reproduce, open ScummVM with a clean Config File (avoids any potential conflicts with other settings), add Myst3, click Start to launch the game, then use Alt+Enter to switch between window modes (toggling the game's main menu should trigger the bug).

[lephilousophe]

I can't reproduce the bug on Linux so this seems a MacOS specific bug.
I am not sure to understand why the behaviour is different between Launcher/2D games and Myst III as the backend is now unified.

[antoniou79]

This issue is reproducible in Windows too, with a recent build from master HEAD (2026.1.0git).

It does seem specific to Myst 3, ie. I don't get it with The Dig or with Grim Fandango.

[antoniou79]

As far as I can tell this issue is a result of two factors:
(A). The MYST3 engine is set to support the "kSupportsArbitraryResolutions" feature
(B). There's an added code in OpenGLSdlGraphicsManager::loadVideoMode() to prevent "unnecessary downscaling" starting here:
​https://github.com/scummvm/scummvm/blob/b48ac7ddcb3f161c94cbdda453af1b85deb0ba83/backends/graphics/openglsdl/openglsdl-graphics.cpp#L511

(which comes from this commit: ​https://github.com/scummvm/scummvm/commit/900d7632645498a4c5ffde135aa5db8d87bd5785
which was part of Pull Request: ​https://github.com/scummvm/scummvm/pull/3266 (the discussion there may be useful to provide context on these changes)

---
So when launching Myst 3 from a windowed ScummVM launcher, say of custom dimensions 1487x955 in a desktop screen resolution of 1920x1200:

1. OpenGLSdlGraphicsManager::loadVideoMode() will eventually be called with the engine-required-dimensions requestedWidth=853, and requestedHeight=480 (from OpenGLGraphicsManager::initSize() which also sets _currentState.gameWidth and _currentState.gameHeight to these values, ie 853x480), but the final line (setupMode()) will be called for dimensions 1487x955 (of the launcher window), which were saved in ScummVM configuration as last_window_width, last_window_height. Then, since Myst 3 supports kSupportsArbitraryResolutions, this piece of code (​https://github.com/scummvm/scummvm/blob/b48ac7ddcb3f161c94cbdda453af1b85deb0ba83/backends/graphics/opengl/opengl-graphics.cpp#L1859) "stores" the game's "new" width and height (1487x955) in _currentState.gameWidth and _currentState.gameHeight.

2. When switching Myst 3 to full screen, OpenGLSdlGraphicsManager::loadVideoMode() is now called with requestedWidth=1487, and requestedHeight=955 (because of the previous change to in _currentState.gameWidth and _currentState.gameHeight), and setupMode() will be called with the those same dimensions. And again, this piece of code (​https://github.com/scummvm/scummvm/blob/b48ac7ddcb3f161c94cbdda453af1b85deb0ba83/backends/graphics/opengl/opengl-graphics.cpp#L1859) "stores" the game's width and height which is now also the desktop full screen's width and height (1900x1200) in _currentState.gameWidth and _currentState.gameHeight.

3. Finally, when switching out of full screen, loadVideoMode() is now called with dimensions requestedWidth=1920, and requestedHeight=1200 (screen resolution, because of the previous change to _currentState.gameWidth and _currentState.gameHeight), gets the 1487x955 values stored in ScummVM config, but then qualifies for the condition of code (B) so it ignores those and keeps 1920x1200 calling setupMode(). Also, following that, OpenGLSdlGraphicsManager::notifyResize() is called, finally persisting (saving) the new dimensions (1900x1200) to the ScummVM config file.

---
In comparison, Grim Fandango, is not set to support the "kSupportsArbitraryResolutions" feature.
So when launching Grim Fandango from a windowed ScummVM launcher, of dimensions 1487x955 in a desktop screen resolution of 1920x1200:

1. OpenGLSdlGraphicsManager::loadVideoMode() will eventually be called with the engine-required-dimensions requestedWidth=640, and requestedHeight=480, but the final line (setupMode()) will be called for dimensions 1487x955. (Up to here this is similar behavior to Myst 3, but no change happens to _currentState.gameWidth and _currentState.gameHeight, which are set only in OpenGLGraphicsManager::initSize() to the game's required 640x480 resolution)

2. When switching Grim Fandango to full screen, OpenGLSdlGraphicsManager::loadVideoMode() is called with requestedWidth=640, and requestedHeight=480, and setupMode() will be called with those same dimensions.(This differs from Myst 3 because the _currentState.gameWidth and _currentState.gameHeight values were not changed in Grim Fandango's case)

3. Finally, when switching out of full screen, loadVideoMode() is called with dimensions requestedWidth=640, and requestedHeight=480, gets the 1487x955 values stored in ScummVM config, does not qualify for the condition of code (B) and calls setupMode() for the 1487x955 (original launcher resolution). Also, following that, OpenGLSdlGraphicsManager::notifyResize() is called, persisting (saving) these dimensions to the ScummVM config file (even though they've not changed in this case).

---

Hopefully the above will be useful in working towards resolving this issue.

[antoniou79]

From a quick search in our code, the engines that declare support for kSupportsArbitraryResolutions are:

• FREESCAPE
• MYST3
• STARK
• WINTERMUTE
• PLAYGROUND3D

These are probably all affected with this issue -- but I've only tested for Myst 3 so far.

[lephilousophe]

The pull request ​#7346 should fix it.

[Le Philousophe <lephilousophe@…>]

In f8902028:

SDL: Don't try to avoid downscaling on 3D with arbitrary resolution

Fix #16253