Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#6720 closed defect (fixed)

SWORD1: Speech cluster file endian detection fails in some Mac variants

Reported by: SF/artphotodude Owned by: criezy
Priority: normal Component: Engine: Sword1
Keywords: Cc:
Game: Broken Sword 1

Description

Hello running into this same bug =>> http://forums.scummvm.org/viewtopic.php?t=5453&postdays=0&postorder=asc&start=0 in OSX ScummVM 1.7.0

Was a work-around ever found??

Ticket imported from: bugs/6720.

Attachments (3)

alternate (2.9 KB ) - added by SF/artphotodude 5 years ago.
alternate.2 (3.0 KB ) - added by SF/artphotodude 5 years ago.
alternate.3 (2.6 KB ) - added by SF/artphotodude 5 years ago.

Download all attachments as: .zip

Change History (34)

comment:1 by wjp, 5 years ago

Please describe the exact problem you're having.

Include as much detail as possible about which version of Broken Sword you're using, and the symptoms you are seeing/hearing.

comment:2 by SF/artphotodude, 5 years ago

Just as mentioned in the link above, am running into a digital static sound in Broken Sword 1 rather than hearing proper speech. Everything else is fine. Am running the original Mac Discs.

comment:3 by criezy, 5 years ago

The issue described in the forum post should be fixed. For the mac version is is now using an heuristic to determine if the speech data is in little or big endian. However while there was no report of it before, there is a possibility that this heuristic does not work in your case.
What language is your Broken Sword game? So far I only had reports about the English and French versions.

Can you also please run scummvm on the command line with the -d 8 flag. If you are on MacOS X use Terminal.app and the command should look like "/Applications/ScummVM.app/Contents/MacOS/scummvm -d 8" (without the quotes, but this may need to be adapted depending where your ScummVM application is). This should print something like the following four lines when starting to play Broken Sword 1. Please report what you get:
Opening hashed: SPEECH1.CLU
Using uncompressed Speech Cluster
Mac version: using little endian speech file
Speech endianness heuristic: average = 20986.150575 for BE and 5069.202101 for LE, computed on 2000 samples)

comment:4 by SF/artphotodude, 5 years ago

VERY sorry for the delay getting back to you - just starting a new job.

Anyway: here is the Terminal message you requested:


User picked target 'sword1mac' (gameid 'sword1mac')...
Looking for a plugin supporting this gameid... Sword1
Starting 'Broken Sword: The Shadow of the Templars (Mac)'
Starting resource manager
Opening hashed: swordres.rif
Starting object manager
File::open: opening 'SPEECH1.CLF' failed
File::open: opening 'SPEECH1.CLV' failed
File::open: opening 'SPEECH1.CL3' failed
Opening hashed: SPEECH1.CLU
Using uncompressed Speech Cluster
Mac version: using little endian speech file
Speech endianness heuristic: average = 2441.248624 for BE and 76.050025 for LE, computed on 2000 samples)


comment:5 by SF/artphotodude, 5 years ago

Not sure what it is asking for with
SPEECH1.CLF, SPEECH1.CLV & SPEECH1.CL3 these files where not on the discs. Could they be aliases (shortcuts) to files that are present, but named differently?

comment:6 by SF/artphotodude, 5 years ago

BTW: Someone else ran into this bug here: http://forums.scummvm.org/viewtopic.php?t=5453 and after analysis, your dev stated that the file might be fooling your system into thinking it is Little Endian, but is in fact Big Endian

Please see Creizy's reply - as he seemed to have figured out near the end of the first page what was happening.

comment:7 by criezy, 5 years ago

Yes I know of this thread. And I am criezy ;)

SPEECH1.CLF, SPEECH1.CLV & SPEECH1.CL3 are compressed versions of SPPECH1.CLU using FLAC, Ogg Vorbis and MP3 respectively. You can use our compression tool to generate one of those, which may be useful to get smaller files if you want to play on a phone for example. So the message about those files is not an issue (but now I know you are using the uncompressed SPEECH1.CLU which remove a few possibilities for the bug).

The forum thread you mention tells me the speech file in the mac version of the game can be little endian (as in the Windows version) or big endian. The mac version I have for example has little endian speech. But two persons in the thread reported that their English version uses big endian speech. So I added an heuristic in our code to automatically detect if the speech is big or little endian and support both. That worked for the English version reported in that forum thread. But it looks like the heuristic is failing in your case. It detects little endian speech, but you may actually have big endian speech.

Can you confirm you have the English version of the game (and not the French or German version for example)?

Also can you precise which version of OS X you are using (so that I can build a special version that forces big endian speech and that you can use to test if it works and if that is indeed the issue).

comment:8 by SF/artphotodude, 5 years ago

Thierry: Thanks so much for the reply! I would be happy to use the MP3 version, but I suppose if the program can't read Speech.clu to begin with, it also probably won't compress it properly either!

Anyway, if you have time, would really appreciate trying any patch or build you think would work. LOVE THIS GAME! To answer your question, I am on OSX 10.4.11 (G4 PowerPC version) and the game is in English (Can't tell from the discs if it is US or British release), but from playing this on the PSX, seems identical up to the point the problem begins.

P.S. am also running "Beneath a Steel Sky", "Day of the Tentacle", "Diskworld", "Full Throttle", "Indiana Jones and the Fate of Atlantis" and "The Dig" and none of the others have had ANY problems so far. MEGA KUDOS - ScummVM is probably the best Open Source Game-App out there!!!!

comment:9 by criezy, 5 years ago

Indeed the compression tool uses the same heuristic to guess the endianess of the speech data. So the compressed speech will not be correct either.

I made a small fix to the heuristic. But I don't think this will fix your issue (you can try tomorrow with the latest development build from http://buildbot.scummvm.org/builds.html in case I am wrong). There is a bigger issue with this heuristic (basically it tries both little and big endian on a small speech sample to see which one looks the most normal, but it may seriously underestimate the "wrongness" when using the incorrect endianess, which ends up looking better than the correct one). I may have to rewrite it completely. I will have a deeper look this weekend.

comment:10 by criezy, 5 years ago

I changed the code to make the speech endianess detection more robust. Please try the development build tomorrow (or later, there is no rush) from http://buildbot.scummvm.org/builds.html. Then let us know if this fixes the speech for you. If not please run it again with -d 8 and report the output here.

comment:11 by digitall, 5 years ago

Summary: 'Endian Bug' Broken Sword I speech.clu...SWORD1: Speech cluster file endian detection fails in some Mac variants

comment:12 by digitall, 5 years ago

artphotodude: Has Criezy's patches fixed this bug? i.e. 8eb82fd6c1c65ad0590462853ade281101068afa and 5450870326d56e42a80a20b5c91fbb07df21d86a

Am going to set this bug to pending-fixed. If no further update is received within 14 days, it will be closed.

comment:13 by digitall, 5 years ago

Owner: set to criezy
Resolution: fixed
Status: newpending

comment:14 by SF/artphotodude, 5 years ago

Thierry: Sorry for the delay, once again. The issue is still there, but the Terminal message was a bit different than last time.


User picked target 'sword1mac' (gameid 'sword1mac')...
Looking for a plugin supporting this gameid... Sword1
Starting 'Broken Sword: The Shadow of the Templars (Mac)'
Starting resource manager
Opening hashed: swordres.rif
Starting object manager
File::open: opening 'SPEECH1.CLF' failed
File::open: opening 'SPEECH1.CLV' failed
File::open: opening 'SPEECH1.CL3' failed
Opening hashed: SPEECH1.CLU
Using uncompressed Speech Cluster
Mac version: using little endian speech file
Speech endianness heuristic: average = 23004.605500 for BE (2000 samples) and 1380.266000 for LE (2000 samples)


comment:15 by SF/artphotodude, 5 years ago

digitali: the issue is still present. Left a reply to the dev working on it.

comment:16 by criezy, 5 years ago

Resolution: fixed
Status: pendingnew

comment:17 by criezy, 5 years ago

Let's first confirm that this is an endianness issue before I continue looking in this direction. I added a debugger command that you can use to override the endianness detected for the speech data in the mac version. You can use it with the development version from tomorrow.

Currently for you it detects little endian, so you can use it to tell ScummVM to assumes it is big endian instead and see if the speech is correct. To do so:
1) Start Broken Sword
2) Open the debugger using Crtl+d
3) In the console that pops up type (without the quotes) "speechEndianness be" and press enter.

Then report if after this the speech is correct.

You can also check which one is used by typing "speechEndianness" or set little endian speech by typing "speechEndianness le"

comment:18 by SF/artphotodude, 5 years ago

On 9/7/14 11:48 AM, Thierry Crozat wrote:

So just to verify, this won't be up till tomorrow morning??

Thanks Again!

comment:19 by digitall, 5 years ago

artphotodude: The daily builds are updated overnight automatically (as defined in UTC+0/GMT):
http://www.scummvm.org/downloads/#daily

As criezy added this 3 hours ago at 19.30, it will be included in the next nightly build. The other option would be to compile ScummVM yourself from the latest development source code to test:
http://wiki.scummvm.org/index.php/Developer_Central#Getting_started

However, this is not recommended unless you are comfortable with this i.e. Have done at least basic C/C++ and used a compiler before.

by SF/artphotodude, 5 years ago

Attachment: alternate added

comment:20 by SF/artphotodude, 5 years ago

On 9/7/14 2:32 PM, digitall wrote:

artphotodude: The daily builds are updated overnight automatically (as
defined in UTC+0/GMT):
http://www.scummvm.org/downloads/#daily

As criezy added this 3 hours ago at 19.30, it will be included in the
next nightly build. The other option would be to compile ScummVM
yourself from the latest development source code to test:
http://wiki.scummvm.org/index.php/Developer_Central#Getting_started

However, this is not recommended unless you are comfortable with this
i.e. Have done at least basic C/C++ and used a compiler before.


[bugs:#6720] http://sourceforge.net/p/scummvm/bugs/6720 SWORD1:
Speech cluster file endian detection fails in some Mac variants

Status: open
Group: Broken Sword 1
Created: Wed Aug 27, 2014 01:21 AM UTC by Adam Albrec
Last Updated: Sun Sep 07, 2014 06:48 PM UTC
Owner: Thierry Crozat

Hello running into this same bug =>>
http://forums.scummvm.org/viewtopic.php?t=5453&postdays=0&postorder=asc&start=0
in OSX ScummVM 1.7.0

Was a work-around ever found??


Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/scummvm/bugs/6720/
https://sourceforge.net/p/scummvm/bugs/6720

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/
https://sourceforge.net/auth/subscriptions

I am happy to wait, just wanted to make sure that I didn't download the
previous version by mistake. Not wanting to waste all of your time.

Thanks!!

comment:21 by SF/artphotodude, 5 years ago

••• THIS DID IT! Speech appears fine now. •••

comment:22 by criezy, 5 years ago

Resolution: fixed
Status: newpending

comment:23 by criezy, 5 years ago

Thank you for your feedback, this confirmed there was an issue with the heuristic to determine if the speech is stored using little endian or big endian.

I looked again at the code and I think I found and fixed the issue (I already had an inkling when you wrote you were using a PPC Mac, which is a big endian system, while I have been testing with a more recent intel mac which is a little endian systems; but I could not see what the problem was until this morning).

I have set this issue to pending. Please try again with a development built from tomorrow (tuesday) and report whether the speech is now OK without having to use the debugger. I am hopeful that it will.

by SF/artphotodude, 5 years ago

Attachment: alternate.2 added

comment:24 by SF/artphotodude, 5 years ago

On 9/8/14 1:21 AM, Thierry Crozat wrote:

  • status: open --> pending-fixed
  • Comment:

Thank you for your feedback, this confirmed there was an issue with
the heuristic to determine if the speech is stored using little endian
or big endian.

I looked again at the code and I think I found and fixed the issue (I
already had an inkling when you wrote you were using a PPC Mac, which
is a big endian system, while I have been testing with a more recent
intel mac which is a little endian systems; but I could not see what
the problem was until this morning).

I have set this issue to pending. Please try again with a development
built from tomorrow (tuesday) and report whether the speech is now OK
without having to use the debugger. I am hopeful that it will.


[bugs:#6720] http://sourceforge.net/p/scummvm/bugs/6720 SWORD1:
Speech cluster file endian detection fails in some Mac variants

Status: pending-fixed
Group: Broken Sword 1
Created: Wed Aug 27, 2014 01:21 AM UTC by Adam Albrec
Last Updated: Sun Sep 07, 2014 09:32 PM UTC
Owner: Thierry Crozat

Hello running into this same bug =>>
http://forums.scummvm.org/viewtopic.php?t=5453&postdays=0&postorder=asc&start=0
in OSX ScummVM 1.7.0

Was a work-around ever found??


Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/scummvm/bugs/6720/
https://sourceforge.net/p/scummvm/bugs/6720

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/
https://sourceforge.net/auth/subscriptions

The fix worked. Speech was fine without the debugger. Way Cool! Will
this change be in the main app, or do I need to use the dev build?

Thanks again!

comment:25 by criezy, 5 years ago

Status: pendingclosed

comment:26 by criezy, 5 years ago

The fix will be in the next release. Until then you can use the development build.

And thank you for reporting the bug and help us fix it. Now ScummVM is a little bit more awesome ;)

by SF/artphotodude, 5 years ago

Attachment: alternate.3 added

comment:27 by SF/artphotodude, 5 years ago

On 9/9/14 12:02 PM, Thierry Crozat wrote:

  • status: pending-fixed --> closed-fixed
  • Comment:

The fix will be in the next release. Until then you can use the
development build.

And thank you for reporting the bug and help us fix it. Now ScummVM is
a little bit more awesome ;)


[bugs:#6720] http://sourceforge.net/p/scummvm/bugs/6720 SWORD1:
Speech cluster file endian detection fails in some Mac variants

Status: closed-fixed
Group: Broken Sword 1
Created: Wed Aug 27, 2014 01:21 AM UTC by Adam Albrec
Last Updated: Mon Sep 08, 2014 08:21 AM UTC
Owner: Thierry Crozat

Hello running into this same bug =>>
http://forums.scummvm.org/viewtopic.php?t=5453&postdays=0&postorder=asc&start=0
in OSX ScummVM 1.7.0

Was a work-around ever found??


Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/scummvm/bugs/6720/
https://sourceforge.net/p/scummvm/bugs/6720

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/
https://sourceforge.net/auth/subscriptions

Do you know when the release will occur? I ask because the dev build
has a choppiness in the background music that is not present in the
standard build - seems like it isn't as optimized as the reg build is.

Thanks again!

Adam :0)

comment:28 by criezy, 5 years ago

There is no plans for the next release yet. If we release a version 1.7.1 it may happen in about 1 or 2 months. Otherwise the next release will be 1.8.0 and may be 6 months to a year from now.

I do have a solution for you though: use the compression tool to compress the speech to mp3, ogg vorbis or flac. Then you can play with the last release (1.7.0). You will have to wait a few days though as I have not yet fixed the endianness heuristic in the compression tool. I will let you know when I have and how to get and use it.

comment:29 by criezy, 5 years ago

Adam, I think I fixed the compression tool. Please can you try tomorrow (or later) to download the latest tools build from http://buildbot.scummvm.org/builds.html
Then you can use the following command (adapt the path to your game data files) in the terminal to compress the speech file to Ogg Vorbis (if you want MP3 or FLAC or define quality options see http://wiki.scummvm.org/index.php/User_Manual/Appendix:_Tools#Using_the_CLI):

./scummvm-tools-cli --tool compress_sword1 --vorbis --speech-only /Path/to/your/BS1/Game/swordres.rif

This will create two new files alongside the existing speech1.clu and speech2.clu and they should allow you play the game with the 1.7.0 release. Please let me know if it works.

comment:30 by SF/artphotodude, 5 years ago

On 9/11/14 2:10 PM, Thierry Crozat wrote:

Adam, I think I fixed the compression tool. Please can you try
tomorrow (or later) to download the latest tools build from
http://buildbot.scummvm.org/builds.html
Then you can use the following command (adapt the path to your game
data files) in the terminal to compress the speech file to Ogg Vorbis
(if you want MP3 or FLAC or define quality options see
http://wiki.scummvm.org/index.php/User_Manual/Appendix:_Tools#Using_the_CLI):

./scummvm-tools-cli --tool compress_sword1 --vorbis --speech-only
/Path/to/your/BS1/Game/swordres.rif

This will create two new files alongside the existing speech1.clu and
speech2.clu and they should allow you play the game with the 1.7.0
release. Please let me know if it works.


[bugs:#6720] http://sourceforge.net/p/scummvm/bugs/6720 SWORD1:
Speech cluster file endian detection fails in some Mac variants

Status: closed-fixed
Group: Broken Sword 1
Created: Wed Aug 27, 2014 01:21 AM UTC by Adam Albrec
Last Updated: Tue Sep 09, 2014 07:02 PM UTC
Owner: Thierry Crozat

Hello running into this same bug =>>
http://forums.scummvm.org/viewtopic.php?t=5453&postdays=0&postorder=asc&start=0
in OSX ScummVM 1.7.0

Was a work-around ever found??


Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/scummvm/bugs/6720/
https://sourceforge.net/p/scummvm/bugs/6720

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/
https://sourceforge.net/auth/subscriptions

Thierry: I've used terminal quite a bit, but something is not working
here. Is there an installation step? I get these two errors:

*dyld: Library not loaded: /opt/1028libs/lib/libvorbisfile.3.dylib
Referenced from:
/Users/adamalbrec/Desktop/osx_ppc-tools-master-4160509b/scummvm-tools-cli
Reason: image not found
Trace/BPT trap

*What am I missing?

Thanks Again!

Adam :0)

comment:31 by SF/artphotodude, 5 years ago

Thanks Thierry for the effort, but the tools won't work no matter what I try. Just not meant to be I guess.

Note: See TracTickets for help on using tickets.