#8162 closed patch (outdated)
COMI lip sync patch (experimental)
| Reported by: | SF/dschepler | Owned by: | aquadran |
|---|---|---|---|
| Priority: | normal | Component: | Engine: SCUMM |
| Version: | Keywords: | ||
| Cc: | Game: | Monkey Island 3 |
Description
This is a first version of a patch to implement lip syncing for COMI. I don't know whether the lip positions are exactly correct, but they do seem to move roughly in sync with the speech.
Ticket imported from: #671104. Ticket imported from: patches/267.
Attachments (1)
Change History (10)
by , 21 years ago
| Attachment: | lipsync.diff added |
|---|
comment:1 by , 21 years ago
First off, thanks for this :-)
Now, an idea on how to improve this possibly:
in voiceLipSyncData() you do a binary ssearch time it is called to find the correct lip sync entry. However, why not make use of the fact that the sound playback is always forward: initially, lipsync entry 0 is going to be used, then after the first threshold, entry 1, then entry 2 etc... so if you buffer the address of the last useed entry, and then do a linear search starting from that, it should actually perform better than doing a binary search each time (that is, it will usually find the correct entry in near constant time: it's either the current or the next entry).
I need to actually test this and look at the code more before I can make any other comments :-)
comment:2 by , 21 years ago
Actually, when I put in debugging statements, it turned out the lip sync functions were only getting called every 4 or 5 jiffies. Still, you're right on the linear search. I'll make that change when I get some time.
I also had reports on #scummvm that to other people the lip positions didn't look right -- Guybrush made the O position too often. I don't really know how to fix this -- switching width and height made it look to me like he was mumbling. There is a big-endian uint32 at the beginning of each SYNC data block which I'm just skipping for now; maybe it needs to be figured in somehow, but I'd have no idea how.
comment:3 by , 21 years ago
Also, it just occurred to me that I should probably somehow have the timing tied to where the playback actually is instead of wall clock time, in case the actor frequency is set.
comment:4 by , 21 years ago
Yeah, the results look pretty schewed here. Let me know when you have an improved version, and we'll probably stick that in CVS :)
comment:6 by , 20 years ago
Some notes:
* SYNC may also occur in music data. No surprise, since there are also opcodes for musicLipSyncWidth/Height * the first 4 bytes in the SYNC are equal to the first 4 bytes in the REGN block, and is simply the offset in bytes to the start of the DATA block, I think. So ignoring them should be safe
* In the patch, a jiffie is assumed to be 17ms. This isn't explained anywhere, but comparing with the rest of ScummVM; I'd guess 1000/60 = 16.666... is where this is from. In any case, I made some experiements (computed the ratio between data size, highest lip sync 'offset', and the sample size), which seem to confirm a jiffie length of 16-17ms
* Yes, the byte offset of the sound should be computed by looking at how many bytes are already used up, exactly. This is not easily possible when playRaw is used. However, using a custom AudioInputStream, this should be 'easily' possible to fix.
comment:7 by , 20 years ago
| Owner: | set to |
|---|---|
| Resolution: | → outdated |
| Status: | new → closed |
comment:8 by , 20 years ago
Pawel implemented proper lip sync support for both speech and music (I believe) in the new rewritten iMuse Digital. Enjoy :-)
comment:9 by , 5 years ago
| Component: | → Engine: SCUMM |
|---|---|
| Game: | → Monkey Island 3 |
Lip sync patch