AGS: Invalid savegames created on big-endian systems

[kas1e]

For example i tried Space Quest IV.5, on both win32 and amigaos4 : on win32 it starts fine, on AmigaOS4 it bring such an output:

Running Space Quest IV.5 - Roger Wilco And The Voyage Home
SQ4,5.exe: 5cd8db602cedc8f04cd3ca290a4a2693, 6886082 bytes.
WARNING: TODO: SetCurrentDirectory: Work:games/Scumm_my/games/Space Quest IV.5 Roger Wilco And The Voyage Home/!
Initializing backend libs
Initializing game data
Located game data pak: SQ4,5.exe
Opened game data file: ac2game.dta
Game data version: 32
Compiled with: 2.72
Startup directory: Work:games/Scumm_my/games/Space Quest IV.5 Roger Wilco And The Voyage Home/
Data directory: ./
Setting up game configuration
WARNING: AmigaOSFilesystemNode::createDirectory() -> Not supported!
Voice pack found: speech.vox
music.vox found and initialized.
Initializing TTF renderer
Initializing mouse: number of buttons reported is 3
Install timer
Initialize legacy path finder library
Game title: 'Space Quest 4.5'
Game GUI version: 115
WARNING: font 'agsfnt2.wfn' has mistakes in data format, some characters may be displayed incorrectly
WARNING: font 'agsfnt3.wfn' has mistakes in data format, some characters may be displayed incorrectly
WARNING: font 'agsfnt4.wfn' has mistakes in data format, some characters may be displayed incorrectly
WARNING: font 'agsfnt5.wfn' has mistakes in data format, some characters may be displayed incorrectly
Checking for disk space
Game native resolution: 640 x 480 (16 bit) letterbox-by-design
Graphic settings: driver: OGL, windowed: no, screen size: 0 x 0, game scale: proportional
Requested graphics driver 'OGL' not found, will try existing drivers instead
WARNING: movement control not supported, mouse control can't be enabled!
Mouse speed control: disabled, unit: 1.000000, user value: 1.000000
Setting up window
Initialize sprites
Engine initialization complete
Starting game
WARNING: channel 2 - same clip assigned
Unable to load the room file 'room74.crm'.
There was an error reading room data..
Block data overlapping.
Block: 'Main', expected to end at offset: 30788, finished reading at 33085..!
Debugger started, type 'exit' to return to the game.
Type 'help' to see a little list of commands and variables.
ERROR: Unable to load the room file 'room74.crm'.
There was an error reading room data..
Block data overlapping.
Block: 'Main', expected to end at offset: 30788, finished reading at 33085..!

It does not matter if i use Opengl, OpengGLWithShaders or Software renders, all the same.

Which point out that it can be Endian issues (especially that phrase Block: 'Main', expected to end at offset: 30788, finished reading at 33085..! sounds like bad reading from/to memory)

[dwatteau]

I can't reproduce the room74.crm error on my old PowerBook G4 (big-endian too). It's incredibly slow (well, it's a debug build without optimizations) but the game looks playable.

However, it does crash when I quit the game to return to the launcher. It also crashes when loading a savegame I've made in the first scene. It looks like it always produces corrupted saves there.

Attaching some GDB logs below (gzip'd, since Trac issues slow down when I attach big .txt files to them).

[dwatteau]

crash on big-endian ppc32 when quitting the game during the intro to return to the launcher

[dwatteau]

crash on big-endian ppc32 when loading the sq45.000 save created on the same machine

[dwatteau]

gzip'd savegame which causes the crash. May have been corrupted when written from big-endian?

[dwatteau]

Exact same symptoms on PS3. The plugin problem is probably related to ​https://github.com/adventuregamestudio/ags/pull/1680 upstream.

[raziel-]

I dont get the room bug either anymore, but I can confirm the save oddities.

For me there isn't even a save state created (empty entry in the launcher after saved in-game -*not* CTRL+F5-, so I can't check if loading crashes)

[dwatteau]

I gave a look at that savegame problem on my big-endian G4 system.

It looks like the savegames created on big-endian systems are missing some (required) RICH_GAME_MEDIA_HEADER fields.

This part calls WriteArrayOfInt16():

<​https://github.com/scummvm/scummvm/blob/8def4f72455675bb712a125b447a04532cf34ad2/engines/ags/engine/ac/rich_game_media.cpp#L51-L54>

but the WriteArrayOfIntXX() functions behave differently on big-endian systems:

<​https://github.com/scummvm/scummvm/blob/8def4f72455675bb712a125b447a04532cf34ad2/engines/ags/shared/util/data_stream.h#L60-L73>

There, WriteAndConvertArrayOfIntXX() is called, instead:

​https://github.com/scummvm/scummvm/blob/8def4f72455675bb712a125b447a04532cf34ad2/engines/ags/shared/util/data_stream.cpp#L96-L120

but playing a bit with GDB, it looks like the loop immediately stops when writing these fields, since the condition is:

for (elem = 0; elem < count && !EOS(); ++elem, ++buffer) 
                            ^^^^^^^^^

but EOS() immediately returns true in this case, so that's why the fields are never written in the savegames.

If I completely remove the && !EOS() checks from the WriteAndConvertArrayOfIntXX() functions, big-endian savegames now work, but maybe there was a good reason why these checks were there...

[lephilousophe]

After some digging, the code here matches what is in the official project: ​https://github.com/adventuregamestudio/ags/blob/master/Common/util/datastream.cpp

IMO, there is a mixup in this code: EOS should be checked at reading and not at writing.
For example in musl libc, EOF flag is only set in read operations and never on write.
So EOS() should never return true when writing a file.

There is a bug in our code (in engines/ags/shared/util/file_stream.cpp, at line 67).
We return true for EOS() when stream is not a read stream.
I pushed PR ​#4700 for this.

[lephilousophe]

Fixed in master by commit 14d3afb297aa1b01b58894cbb69221913e229a31 and in 2.7 branch by b186a146ed88020a9b7d621aaa82564545c7a7f2.