#8594 closed patch
DXA: Improved encoder (filesize and speed)
| Reported by: | johndoe123 | Owned by: | Kirben |
|---|---|---|---|
| Priority: | normal | Component: | Tools |
| Version: | Keywords: | ||
| Cc: | Game: |
Description
Hi,
Long promised to Kirben I finally finished the new and improved version of the DXA encoder.
The changes include: - Frame type 12 was removed and replaced by type 13 - The new type produces files a lot smaller (depending on the input, of course) - Encoding should be faster (if only a bit) - The height scaling from Smacker animations (interlacing and line doubling) is now taken into account, too
The DXA files of the current SVN encoder can still be used in ScummVM as long as the code for type 12 isn't removed for some reason.
The DXA files from Feeble Files are now about 1.0 GB big, the Smacker animations 1.4 GB. I also compressed the videos from Broken Sword 2, the Smacker files are 130 MB big, the DXA files 96 MB. (Note that these do not include the audio files.)
Attached is the patch for the current SVN version, a win32 exe of the encoder and source/exe for the standalone DXA player (needs SDL and doesn't play files produced by the "old" encoder).
I haven't added the new code to the ScummVM DXA player yet as I wanted to hear what you think first.
Now for some technical details: The new type 13 uses the basic ideas from type 12 but adds two new block types:
1. Blocks with 4 colors or less now use a bitmap for each pixel and store the pixel data. E.g. if a block has only two colors, it stores a 16-bit bitmap and two bytes for each actual pixel. Now bit 0 in the map means 'use pixel 1' while bit 1 means 'use pixel 2'.
2. Blocks are now also divided into 4 subblocks, each 2x2 pixels. For each of the subblocks the same methods as for a 4x4 block are tried and the smallest one is chosen.
Also the data layout has been modified to improve compression with zlib. Before the data has been stored sequentially (e.g. "block type, block data, block type" etc.) which hurt compression. Now each type of data is stored in its own "section" which improves compression a lot.
Encoding should also be faster now since only so much data is compressed until compression wouldn't gain anything. This is simply done by limiting the available output size to zlib's compress2 function to the size of the previously compressed data (in the order type 13->xor->full).
The CPU requirements shouldn't be drastically different.
Ticket imported from: #1611503. Ticket imported from: patches/699.
Attachments (5)
Change History (15)
by , 17 years ago
| Attachment: | encode_dxa.patch added |
|---|
comment:3 by , 17 years ago
Sounds very nice, I must say. Unfortunately I can't work on this patch ("work" as in 'review, accept or reject it'), as I have no good way to test it. But definitely sounds good.
Hm, I just noticed we list you twice in our contributors list: Once with your real name, once under your nick... Guess I'll fix that now ;-)
comment:6 by , 17 years ago
| Owner: | set to |
|---|---|
| Status: | new → closed |
comment:7 by , 17 years ago
Excellent work, the DXA compression improvements offer much better fize sizes, compared to previous DXA compression used too.
Broken Sword 1: Old: 65.8MB New:45.4MB
Broken Sword 2: Old: 109MB New: 95.9MB
Feeble Files 4CD: Old: 1.25GB New: 1.05GB
I added the DXA encoder and decoder improvements to ScummVM SVN, keeping type 12 for backwards compatibility.
comment:8 by , 17 years ago
It looks like the encoder and player code aren't endian safe. I don't know if that's a deliberate choice, but if it isn't, the attached patches ais at "fixing" this. I choose to encode everything as big-endian to match the endianness of the original DXA tags.
I haven't "fixed" block type 12, since the code for encoding it isn't present anymore in encode_dxa.
File Added: dxa_endian_safe_diff.zip
comment:9 by , 17 years ago
Feeble Files 2CD: Old: 639MB New: 550MB
I added patches for endian issues to ScummVM SVN, encoding as big endian as suggested.
comment:10 by , 5 years ago
| Component: | → Tools |
|---|
The patch file