Opened 2 years ago

Last modified 12 months ago

#9705 new defect

FULLPIPE: AmigaOS: Crash on start

Reported by: raziel- Owned by:
Priority: high Component: Engine: Fullpipe
Keywords: has-backtrace Cc:
Game: Full Pipe


ScummVM 1.10.0git (Feb 11 2017 10:16:18)
Features compiled in: Vorbis FLAC MP3 RGB zLib MPEG2 Theora AAC FreeType2 JPEG PNG cloud (servers, local)

I get a crash after starting the game.
No screen loads, just a black screen or window comes up, no sound no graphics.
No backtrace either, but a stacktrace (attached)

Not sure if it helps though, here's whataddr2line says:

addr2line --section=.text -C -fe scummvm 0x5220D8

Fullpipe::Picture::setPaletteData(unsigned char*)

Full Pipe (Windows/English)

AmigaOS4 - PPC - SDL - BE
gcc (adtools build 5.3.0) 5.3.0

Attachments (4)

Crashlog_scummvm_2017-02-12_23-44-32.txt (35.8 KB) - added by raziel- 2 years ago.
Debug 101 v1.1.2.png (230.8 KB) - added by raziel- 2 years ago.
Debug snapshot of crash
Crashlog_scummvm_2017-19-11_08-21-43.txt (15.7 KB) - added by raziel- 16 months ago.
log1.txt (16.2 KB) - added by raziel- 16 months ago.
memorypool.cpp crash

Download all attachments as: .zip

Change History (20)

Changed 2 years ago by raziel-


comment:1 Changed 2 years ago by sev-

I think it is just you're running out of memory. The line where you're getting a crash is '_paletteData = (byte *)malloc(1024);'

comment:2 Changed 2 years ago by raziel-

I still have 58% of memory free (2 GB Ram) when the crash happens, i don't believe i'm running out of it.
May htere be another reason?

Anything i can do to test?

comment:3 Changed 2 years ago by raziel-

Putting scummvm in a debugger and starting Fullpipe i get this stacktrace:

Your program has crashed! ip = 0x228ee7c
Program has stopped at an unknown point in memory (TRAP)

[engines/fullpipe/statics.cpp: line 1701]: _ZN8Fullpipe8Movement4loadERNS_10MfcArchiveEPNS_15StaticANIObjectE (scummvm)
[engines/fullpipe/statics.cpp: line 234]: _ZN8Fullpipe15StaticANIObject4loadERNS_10MfcArchiveE (scummvm)
[engines/fullpipe/statics.cpp: line 183]: _ZN8Fullpipe5Scene4loadERNS_10MfcArchiveE (scummvm)

Does this help?

Changed 2 years ago by raziel-

Attachment: Debug 101 v1.1.2.png added

Debug snapshot of crash

comment:4 Changed 2 years ago by raziel-

As you can see in the debug window (top right) there's "abadcafe" in one of the registers, which means there was access to memory that hasn't been initialised.

comment:5 Changed 2 years ago by raziel-

It's a game of luck with this engine/game.

I *once* was able to make it run (only when using the non-stripped debug version though, and only in window mode).

The timer counts down and displays the intro (slow)
The game screen comes up but all of the "sprites" are either reduced to gfx meshup lines or (as in the case of the cookoo's clock a big fat black rectangular. No sprites are actually drawn correct.
After the scene with the hole under the bed, a full screen display opens and seconds after that the upper half of that screen is covered in gfx meshup and my entire system goes down with a shriek

comment:6 Changed 16 months ago by csnover

Keywords: has-backtrace added

comment:7 Changed 16 months ago by csnover

Please give it a go with the latest master and let me know if things get any further than before. Thanks!

Changed 16 months ago by raziel-

comment:8 Changed 16 months ago by raziel-

Thanks for looking at this.

I'm afraid it's pretty much the same.
I can't reach any game screen.
Window stays at the launcher imager, screen stays black.

(new log attached)

comment:9 Changed 16 months ago by csnover

According to that backtrace, the crash is happening inside of newlib, again failing to allocate memory just like before, but at a different place. I had hoped the reduced number of heap allocations and significantly reduced memory leakage in the latest code would have improved the situation, but it still seems like the system is having memory trouble. If this is the case, it is probably not a ScummVM bug at this point since the game startup doesn’t leak at this point according to instrumentation.

There is some cruft in configure that changes the way that AmigaOS builds work which might be causing trouble, like forcing static linkage and making other changes to compiler flags which may not be needed with the more modern GCC 5 compiler you’re using, so maybe a place to start would be get rid of these lines for the AmigaOS backend in configure and try recompiling:

append_var CXXFLAGS "-mlongcall"
LDFLAGS=`echo $LDFLAGS | sed 's/-use-dynld//'`
append_var LDFLAGS "-static"

and maybe this one too:

append_var LDFLAGS "-Wl,--export-dynamic"

You might also try using configure --enable-plugins --default-dynamic to reduce the size of the ScummVM image in memory, to allow the game to have more memory, and/or -Os for the optimisation flag to have the compiler optimise the image size a bit more.

Let me know if any of that makes a difference, otherwise you will probably need to complain upstream (there’s not much we can reasonably do to work around bugs in the system’s memory allocator, if that’s what’s happening, which it appears so). Thanks!

comment:10 Changed 16 months ago by raziel-

Well, the first three settings you mentioned are for cross-compiling...i don't cross compile, i do native compiling

The export-dynamic one i will take out and see if that makes any difference.
But i only do static builds anyway (right now), as shared objects are even more error prone on my platform than static builds.

I just don't get why Full Pipe is the only game that causes memory probelms (not that i don't say it's possible), but why? Is there so much gfx or other mem needed as in all those other games?

Just curious...

comment:11 Changed 16 months ago by raziel-

So, i changed my configure according to your suggestion.

I still get a crash, but maybe the log gets clearer now?
I get a crash in memorypool and now i get this crash always in memorypool and not some random addresses scattered around the code or other running programs.

Not sure, but maybe that helps?

Changed 16 months ago by raziel-

Attachment: log1.txt added

memorypool.cpp crash

comment:12 Changed 16 months ago by csnover

Unfortunately it is still just consistently newlib crashing when allocating more memory.

This engine peaks at 180MiB on startup on my machine, which is quite a lot compared to Starship Titanic or SCI32 which both use less than 40MiB. I am not sure if there are any other engines in ScummVM that use so much memory.

If you are getting super unlucky, newlib might be doing a poor job of managing the many small allocations and running out of pages or something even if there is enough physical RAM to handle the load. I’m not blown away by the difference in number of allocations, it makes about 2.5x the number of allocations made by SCI32, though these are more frequently very small (16 byte) allocations. At face value, this doesn’t seem like something that should be breaking. You might try creating a small test app that makes a lot of small heap allocations and see what happens.

For reference here are some rough numbers on the top 10 allocations by total number of allocations at startup of the full version of Full Pipe on my system, which you might be able to use to create some synthetic test app to crash the allocator:

Category	Persistent Bytes	# Persistent	# Transient	Total Bytes	# Total	# Transient/Total
Malloc 16 Bytes	49.31 KiB	3156	22787	405.36 KiB	25943	Ratio: %0.01, %0.08
Malloc 32 Bytes	403.00 KiB	12896	9658	704.81 KiB	22554	Ratio: %0.05, %0.03
Malloc 4.00 KiB	300.00 KiB	75	9776	38.48 MiB	9851	Ratio: %0.00, %0.03
StdioStream	32 Bytes	1	9690	302.84 KiB	9691	Ratio: %0.00, %0.03
Common::MemoryReadStream	528 Bytes	11	9671	453.84 KiB	9682	Ratio: %0.00, %0.03
Fullpipe::MemoryObject2	992.14 KiB	9071	0	992.14 KiB	9071	Ratio: %0.03, %0.00
Malloc 1.00 KiB	8.57 MiB	8774	29	8.60 MiB	8803	Ratio: %0.03, %0.00
Fullpipe::DynamicPhase	1.84 MiB	8579	0	1.84 MiB	8579	Ratio: %0.03, %0.00
Fullpipe::GameVar	572.25 KiB	5232	0	572.25 KiB	5232	Ratio: %0.02, %0.00
Malloc 128 Bytes	152.00 KiB	1216	2685	487.62 KiB	3901	Ratio: %0.00, %0.01

There’s a fairly long tail of other allocations (total numbers are 69580 persistent, 84454 transient) but this seems like a place to start.

Hopefully we will be able to collect a bit more data during release testing to see if there are any other systems which have this problem, though I am not really sure outside of the big-five how many other platforms are even going to be powerful enough to run this game regardless.

comment:13 Changed 16 months ago by raziel-

I'm going to relay and quote answer from an OS dev:

There certainly might be a problem when there are so many small allocations. This is why on AmigaOS it is recommended to use memory pools and item pools for frequent allocations/deallocations of small-size buffers.

I quite doubt such a trivial test has never been done by the Amiga newlib maintainer(s)

One more thing: there's a whole world of difference if the small amounts just keep getting allocated (and are only freed when the program ends), or if they are constantly allocated and deallocated throughout program runtine. Because if the latter is the case, the "loss of memory" problem is much more likely. Could you possibly ask the authors about that?

Thank you

comment:14 Changed 16 months ago by csnover

Priority: blockerhigh

Yes, the game does these sorts of allocations all over the place throughout its lifetime, though whether or not this is so is not really relevant here since the game is crashing already on startup. :)

We do have code for creating memory pools, Common::MemoryPool. I’m happy to accept a patch to use MemoryPool where necessary in this engine, though unless this shows up in other platforms it’s not something that I plan on spending time working on and so won’t be addressed for the next release.

comment:15 Changed 15 months ago by dafioram

Game: Full Pipe
Summary: FULLPIPE: Crash on startFULLPIPE: AmigaOS: Crash on start

comment:16 Changed 12 months ago by raziel-

A core dev was nice enough to check the demo of FullPipe (which also crashes) with the latest newlib

I tried it with the latest beta version of newlib.

The crash is in the dlmalloc code but is not because of lack of memory.

Assuming that the dlmalloc is not bugged (seems a little unlikely since
many other programs are using it without problems) what could cause the
crash is if data before or after an allocation is trashed (as in a
buffer overflow for instance).

Aynthing i could do, to further corner the problem?

Note: See TracTickets for help on using tickets.