Opened 5 years ago

Last modified 13 months ago

#6686 new defect

SCI: General MIDI using SC-55 misses instruments

Reported by: SF/natedavis82 Owned by:
Priority: normal Component: Engine: SCI
Keywords: midi Cc:
Game: King's Quest 6

Description

I thought I would share an issue I discovered with version 1.7 of the Win32 software that was just released. Any game that utilizes General MIDI and using a real SC-55 the music is being mapped only to channels 10 through 17 on the device, thus causing instrument errors in the games. An example would be King's Quest VI; the title music is missing some trumpet sounds and in-game MIDI music and sound effects are either missing or are incorrect. I downgraded back to version 1.6 and all MIDI channels were once again mapped correctly on my SC-55. Thank you.

Ticket imported from: bugs/6686.

Attachments (7)

canvas.jpg (22.9 KB ) - added by SF/natedavis82 5 years ago.
scummvm.ini (13.0 KB ) - added by SF/natedavis82 5 years ago.
sci-log-bad.txt (32.0 KB ) - added by SF/natedavis82 5 years ago.
sci-log-good.txt (2.8 KB ) - added by SF/natedavis82 5 years ago.
1pt6.PNG (18.6 KB ) - added by SF/natedavis82 5 years ago.
1pt7.PNG (19.6 KB ) - added by SF/natedavis82 5 years ago.
sci-log-bad.2.txt (32.2 KB ) - added by SF/natedavis82 5 years ago.

Download all attachments as: .zip

Change History (66)

comment:1 by SF/natedavis82, 5 years ago

Sorry; channels 10 through 16. Typo.

comment:2 by digitall, 5 years ago

Summary: Discovered bug with version 1.7 using a real SC-55MIDI / SCI: General MIDI using SC-55 mapped to channels 10-17 only

comment:3 by digitall, 5 years ago

Component: Engine: SCI
Game: King's Quest 6

comment:4 by digitall, 5 years ago

natedavis82: Can you replicate this with an game using MIDI music other than SCI engine games?:
http://wiki.scummvm.org/index.php/Engines#Complete_Engines_in_the_Latest_Stable_Version

i.e. I am trying to work out if this is a regression in the common MIDI or midi output code, or a problem specific to the SCI engine sound/music code.

comment:5 by digitall, 5 years ago

Apart from this, are you capable of compiling ScummVM from source code?
http://wiki.scummvm.org/index.php/Developer_Central
http://wiki.scummvm.org/index.php/Compiling_ScummVM/MinGW

If so, then it would be best if you could do a git bisection to locate the specific commit between v1.6.0 and v1.7.0 release which introduced this.

If you are not capable of doing this, then I can look at preparing some test builds to allow you to do this... but this will be more painful and slow than if you can do this.

comment:6 by wjp, 5 years ago

10-17? That's strange. GM should use only 10-15 during the music at the KQ6 Sierra logo.

The new MIDI channel mapping code fills channels from the last one down, so if it fills up 10-17 (or 10-15), then channels aren't the limiting factor. Polyphony also isn't particularly tight there, so that seems an unlikely factor too.

Can you attach your ScummVM configuration file?

comment:7 by wjp, 5 years ago

I just noticed your forum post saying that a GS Synth does work. This probably means you have "Roland GS Mode" enabled in the MT-32 settings tab. Disable that if you have a GM device hooked up.

comment:8 by digitall, 5 years ago

Setting bug to pending (user response needed).

comment:9 by digitall, 5 years ago

Status: newpending

by SF/natedavis82, 5 years ago

Attachment: canvas.jpg added

comment:10 by SF/natedavis82, 5 years ago

Thanks for that suggestion. SCI appears to be the issue. The music plays on channels 10 through 16 while other engines play on 1 through 10. In version 1.6 SCI music plays on channels 1 through 10. I may have misspoke when I said "all" channels in the original post as this is not the case.

by SF/natedavis82, 5 years ago

Attachment: scummvm.ini added

comment:11 by SF/natedavis82, 5 years ago

In version 1.6 the music plays on channels 1-10 and all instruments play correctly. In version 1.7 the music plays on 10-16 and a trumpet sound is missing at the title screen. Also, in certain areas of the game sound effects and music do not play correctly. I have uploaded my configuration file.

comment:12 by SF/natedavis82, 5 years ago

I did as you suggested and it did not fix the issue. Music is still being played on channels 10-16.

comment:13 by SF/natedavis82, 5 years ago

I'm sorry, I do not know how to compile.

comment:14 by digitall, 5 years ago

Status: pendingnew

comment:15 by digitall, 5 years ago

Summary: MIDI / SCI: General MIDI using SC-55 mapped to channels 10-17 onlySCI: General MIDI using SC-55 mapped to channels 10-17 only

comment:16 by digitall, 5 years ago

natedavis82: OK. I think we need to bisect and locate the exact code change where this regression was introduced. This would be alot faster if you could compile and test yourself, but otherwise I can do a manual compilation and provide a build for you to test, you indicate if it shows the bug or not... and we repeat in a binary search until we locate the regression point (probably this will take about 4-5 test builds.).

comment:17 by SF/natedavis82, 5 years ago

I'd be happy to help in any way I can. I have never compiled before, but I think I understand what's involved. Do I simply download previous source code and build it for Win32? Did I say that right?

comment:18 by digitall, 5 years ago

Fairly close! Detailed instructions for getting the code and then compiling it into a binary to run are given at http://wiki.scummvm.org/index.php/Developer_Central, specifically as

http://wiki.scummvm.org/index.php/Git#Retrieving_the_code and http://wiki.scummvm.org/index.php/Compiling_ScummVM/MinGW

I really suggest you read these as they should give fairly clear guidelines.

I will give you a quick walkthrough assuming you are on Win32 (If you don't understand anything I suggest Wikipedia or Google for it as there is good quickstart documentation for most of these tools):

  1. First download and install Git from here: http://git-scm.com/download/win
    This is the tool we use to manage and work with source code revisions.

  2. From the Git Bash command line, change to a "temporary" directory on say
    your desktop using "cd ". This is very similar to the
    Windows cmd shell commands, but it is Unix Bash shell.

  3. You can then get our source code using the command:
    "git clone git://github.com/scummvm/scummvm.git"

  4. After a few minutes of downloading and unpacking you will have a new
    directory called "scummvm" in your temporary directory. This is our
    full source code and revision history as a Git repository, which is
    just a normal directory (with some special extra files hidden in a
    folder called .git at the top folder).

  5. You will need to install the MinGW (Minimal GCC (GNU C/C++ Compiler) for
    Windows) from here: http://www.mingw.org/
    and the relevant libraries.. Install these as per the guide I gave earlier:
    http://wiki.scummvm.org/index.php/Compiling_ScummVM/MinGW

  6. When you are ready, open a MinGW Bash shell, then use the "cd "
    command to change to the temporary folder and then "cd scummvm" to move into
    the scummvm folder.

  7. You should now be able to compile the source code using the command:
    "./configure && make clean && make"

This will try to "configure" i.e. run script to detect your operating
system, machine type, find the required libraries, any optional libraries
etc. and then "make" will compile the code (The "make clean" just cleans
up from previous or failed compile, so just a precaution).

If everything goes well, a scummvm.exe will end up in the final folder.
You may need to copy a SDL.dll from the released Win32 version folder
to this folder to run this.

You can run this by typing ./scummvm.exe at the MinGW command line or
browsing to the folder and double clicking on the executable.

comment:19 by digitall, 5 years ago

Once you can do this, you can do a Git bisection (binary search). See:
http://git-scm.com/docs/git-bisect

To see the commit history use gitk... You need to start the bisection using "git bisect"... then mark a bad commit (which is easy as you are currently on the latest revision of the master branch (HEAD) which from your previous compile still shows your bug and thus is BAD" ... so you mark that as bad using "git bisect bad" then change to a possible good commit using "git checkout " ... The id is the long hexadecimal value associated with each commit in the gitk window.. Choose 6e3510ac6154783a99b58bfae83c86145954435f to start. Then do a compile (./configure && make clean && make) as before... Then test this scummvm.exe. If the bug is absent, do "git bisect good".. otherwise you will have to mark it as "git bisect bad) and go back further with "git checkout " (but this should not be necessary).

Once the bisection has a good and bad commit... it will do the checkout automatically selecting the commit in the middle (thus binary search as each step halves the number of commits left in the good to bad range) and inform you. You do the compile... and test and then mark the commit as bad (has bug) or good (bug missing) with "git bisect bad/good"... the bisection will recalculate and give you another commit to compile and test... be careful to be accurate as it will switch a few times probably.

When you get down to a single good and bad commit, it will print "First Bad Commit is... ". Copy that information and post it here.

comment:20 by SF/natedavis82, 5 years ago

Thanks for the detailed instructions! I like a good challenge so I hope I can find the problem for you.

comment:21 by digitall, 5 years ago

If you get stuck, I have also now done a manual compile of a possible bisection point:
http://buildbot.scummvm.org/snapshots/other/scummvm-i586-mingw32msvc-8cc82d0a.zip
http://buildbot.scummvm.org/snapshots/other/scummvm-i586-mingw32msvc-857d2e7b.zip

If you could test these two builds, and we are lucky / right, then this will be the point of regression.

comment:22 by digitall, 5 years ago

You also be able to get more information on the difference purely from v1.6.0 and v1.7.0.

You will need to run both from a command line window and use "scummvm -d 4 --debugflags=Sound" or similiar before running the SCI game to test. See:
http://wiki.scummvm.org/index.php/Debugging_ScummVM

You can also change the debug level and debug flags can be listed and enabled and disabled from the interactive console in game using CTRL-D, then typing "help" to list commands and "debugflag_list" to list the available debug output channels. The actual detailed debug logs is dumped into the command window as standard output... This can be redirected to a file in either Windows or the MinGW shell... Google for how to do this.

comment:23 by digitall, 5 years ago

Sorry. CTRL-Shift-D in SCI. It does vary to avoid conflicting with the engine keyboard usage, but is normally CTRL-D.

comment:24 by SF/natedavis82, 5 years ago

You're good! 8cc82d0a maps correctly to channels 1 through 10! 857d2e7b maps to 10 through 16.

comment:25 by digitall, 5 years ago

Not really good... Just an educated guess as seen from 857d2e7b's commit log:
SCI: Rewrite MIDI channel remapping

This adds MIDI state tracking to allow channels to be temporarily
unmapped and later re-mapped when there are free device channels
available again.

and the 8cc82d0a is the previous commit to engines/sci/sound...

comment:26 by SF/natedavis82, 5 years ago

Well that was it! I guess I don't have to do any compiling now. Haha.

comment:27 by digitall, 5 years ago

Will do a test compile of the latest v1.8.0git code i.e. buggy with DEBUG_REMAP enabled in SCI. This should output debug as explained before, but including the MIDI channel remapping data.

comment:28 by digitall, 5 years ago

Or you can do this... :)

comment:29 by digitall, 5 years ago

Owner: set to wjp

comment:30 by digitall, 5 years ago

To be honest, there may be several patches to try to fix this and since we don't have a SC-55 to test with, getting you to check out a git branch to try them is probably a good idea..

comment:31 by SF/natedavis82, 5 years ago

I'll leave it in the capable hands of much more experienced programmers such as yourself. Will I be informed when it is included in a new build or should I just check back every so often?

comment:32 by SF/natedavis82, 5 years ago

Of course! I work during the day (I'm in the US) so I won't be able to test any changes until the evening hours. Thanks for all your help.

comment:33 by digitall, 5 years ago

This bug will updated and/or closed when a fix is prepared, so you should get a notification e-mail from that... but checking back is reasonable.

comment:34 by digitall, 5 years ago

Please test with http://buildbot.scummvm.org/snapshots/other/scummvm-i586-mingw32msvc-be9c3bf7-sci-debug-remap.zip running from a command window using "./scummvm.exe -d 4 --debugflags=Sound 1> sci-log-bad.txt 2>&1"

Then run say KQ-4 which you say shows the problem.. then close and zip the sci-log-bad and put it here with "Add attachments".

Do the same with v1.6.0 build using "./scummvm.exe -d 4 --debugflags=Sound 1> sci-log-good.txt 2>&1" from the correct directory... and then add that here as well for comparison.

by SF/natedavis82, 5 years ago

Attachment: sci-log-good.txt added

comment:35 by SF/natedavis82, 5 years ago

Here you go.

comment:36 by digitall, 5 years ago

Thanks. That should be sufficient for wjp to look at investigating this.

To explain, I am just doing bug "triage" i.e. ensuring that the basic information for the bug has been collected, ask the initial questions. It will take someone familar with the SCI MIDI code to work out why this regression has occurred and work out a fix. This may take some time i.e. 2-3 days minimum, maybe longer if the problem is complex or tricky to pin down. Please stand by... The bug will be updated when we have something further to test.

comment:37 by SF/natedavis82, 5 years ago

Awesome.

comment:38 by wjp, 5 years ago

You have "GS Device" enabled in the KQ6-specific settings. (Note the line "enable_gs=true" in your config file.) Disable that and it'll be fine.

comment:39 by SF/natedavis82, 5 years ago

It made no difference. Still mapping to 10-16.

comment:40 by wjp, 5 years ago

LordHoto points out privately (and correctly) that I should have said MT-32, not GS. Sorry for the confusion. Could you try disabling that option as well? (In the global configuration.)

by SF/natedavis82, 5 years ago

Attachment: sci-log-bad.2.txt added

comment:41 by SF/natedavis82, 5 years ago

No difference. Here is the debug file with global MT-32 and GS disabled.

Also, I'm not sure if this makes any difference, but the wording has changed on the GS option from version 1.6 to 1.7. I've attached those photos as well.

comment:42 by wjp, 5 years ago

Or actually you override that again in the KQ6 config, so that wouldn't be it either. Let us think about this a bit more.

comment:43 by SF/natedavis82, 5 years ago

Yes, in my global settings I have MT-32 enabled because I also have a real MT-32 that I use for many other games. So when I want to play a game that uses General MIDI I simply disable the option in the game settings so I can use my SC-55.

comment:44 by digitall, 5 years ago

The wording was changed to make it clearer in commit af87e49f9e963d3a5bcb24147b35411798f8db73 on 2013-03-23.

There was no other functional change associated with this and you tested a build of 8cc82d0a which was 2013-11-23 and indicated there was no issue, so that is not related to this.

comment:45 by digitall, 5 years ago

natedavis82: We have tracked this regression down to commit 857d2e7beff46fcd5d26e8590c9891a4959b8d1f.

To be completely sure that this is the cause of the difference you are seeing, can you please test a build from the commit immediately preceding this i.e. f777e54df291d1f39aefdc40d1d966e50d261efb and indicate the result.

The test build can be found here:
http://buildbot.scummvm.org/snapshots/other/scummvm-i586-mingw32msvc-f777e54d.zip

comment:46 by lordhoto, 5 years ago

Summary: SCI: General MIDI using SC-55 mapped to channels 10-17 onlySCI: General MIDI using SC-55 misses instruments

comment:47 by lordhoto, 5 years ago

This looks like a hardware bug in the SC-55 hardware. I can reproduce the missing trumpets in the title music in KQ6 CD Win with my SC-55mkII. When using lower channel numbers it works as expected. However, using Emu10k1 Wavetable Synth also works fine with the current channel mapping.

The original seems to be using channels 9 downwards with the "extended" MIDI setting on Win3.11. There it works fine. When I change ScummVM to use 9 downward it works fine with my SC-55mkII too.

However, the original also offers some "base" MIDI setting. This seems to be using channels 13 to 15 for the theme in KQ6 CD Win. There the trumpet works too. But since it uses less channels it must be some more primitive version of the theme...

comment:48 by SF/natedavis82, 5 years ago

I will be able to test it in about an hour and I will let you know.

comment:49 by SF/natedavis82, 5 years ago

The problem also occurs in the DOS version of KQ6 and in Quest for Glory 3, which is also a SCI game.

comment:50 by lordhoto, 5 years ago

I hacked DOSBox to remap channels 4 to 9 to 11 to 16 (and the other way round) and when doing that I also get broken sound with my SC-55mkII with KQ6 CD Windows. I really guess there's some bug in the SC-55 hardware here (or some limitation which I am not aware of).

The comment in midi.cpp line 353 ("NOTE: SSCI uses channels 1 through 8 for General MIDI as well, in the drivers I checked") seems to indicate it's known that the original SCI interpreters also use channels 2 to 9 for General MIDI. I guess that explains why KQ6 CD Win uses those for "extended level MIDI" too. Thus, I think we will need to limit ourselves to the same channels as the original to avoid such issues.

Allowing the current full map from 1 to 16 seems to have potentional other issues too. As far as I can tell the code in "SciMusic::remapChannels" would not take any precautions that it won't accidently map onto the precussion channel 10 (in case that one is not forced to be used already). But I don't really understand this enough to really know whether that might be an actual problem.

comment:51 by SF/natedavis82, 5 years ago

You are correct. This build works correctly with the SC-55.

comment:52 by SF/natedavis82, 5 years ago

Any updates to this problem?

comment:53 by SF/natedavis82, 4 years ago

Is this unfixable?

comment:54 by SF/natedavis82, 4 years ago

I would very much like to be able to use my SC-55. It used to work just fine with Scummvm so is there any way to fix it?

comment:55 by SF/spiffythedog, 4 years ago

Can confirm that the problem persists in the recent stable release (v1.8.0), and that the same issue also affects the soundtrack of QFG1VGA when played via General MIDI (with AND without MT-32 mappings). Missing/corrupted instruments in BGM for 'Erana's Peace' appear to match the ones from KQ6CD (used 'audience with Alhazred' scene near the beginning of the game as a point of reference)

comment:56 by SF/spiffythedog, 4 years ago

HOLD IT!

(Sorry about the double post)

It turns out that the issue HAS been fixed in the latest release. Simply leaving the 'Override global MT-32 settings' option in the MT-32 submenu (in KQ6's config) unchecked causes the game to load and play the proper instruments. KQ6 also takes slightly longer to boot up (about 5-10 seconds) under this setting, so that may help signify that it is working like it should.

The issue with QFG1VGA appears to be quite different, as that game's soundtrack was apparently composed natively for MT-32, unlike KQ6 which was composed for General MIDI (IIRC).

comment:57 by SF/spiffythedog, 4 years ago

Sigh.......I really jumped the gun on this one.

It turns out that the reason I could hear those correct instruments was because I had checked 'True Roland MT-32' under global settings, which explains the 5-10 second delay in starting the game (why didn't I see that?!). And furthermore, some of the other minor sound effects sound warped or corrupted, such as the seagulls chirping on the beach and opening the wooden box concealed by the log (produces an atonal honking sound). Issue still remains in current stable release,

Just disregard my second post. It's clear that I don't know what I'm doing, and this will be the last I say on this matter for now.

comment:58 by spiffythedog, 3 years ago

Issue of missing/borked instruments in General MIDI soundtrack for KQ6CD persists in recent stable release (v1.9.0). Used 'audience with Alhazred' scene near the very beginning of the game as point of reference, once again. Tested DOS version (both GOG release and old 'Sega Ozisoft' CD release) with high resolution Windows graphics. Used both FluidSynth and BASSMIDI with SGM V2.01 soundfont for music.

comment:59 by ndavis82, 2 years ago

I was hoping this would be fixed by now since I opened the ticket almost four years ago. MIDI instruments are still being mapped to channels 10-16 on a real SC-55 on all Sierra SCI games. Can someone please fix this?

comment:60 by lavadrop, 23 months ago

Maybe SCI is expecting to read a flag whether it's a Base or Extended MIDI device?

I tested the intro in DOSbox and there it plays from channels 2-10.
Did the same in today's daily build and both the Sierra fanfare and the intro are played using the systems MIDI Synth, but in-game music plays on channel 8-16 if and only if you disable high resolution graphics; during Alhazred's cutscene in the beginning, some instruments are wrong (samba whistles).
I also tested using Windows 3.1 inside DOSBox, the audio of the fanfare and intro is a VFW movie and in-game the music plays on channels 2-10 correctly.

comment:61 by csnover, 21 months ago

Keywords: midi added
Owner: changed from wjp to csnover

comment:62 by digitall, 13 months ago

Owner: csnover removed
Note: See TracTickets for help on using tickets.