Opened 15 years ago

Closed 14 years ago

Last modified 2 years ago

#2172 closed defect

GUI: Accented characters

Reported by: eriktorbjorn Owned by: sev-
Priority: high Component: GUI
Keywords: Cc:
Game:

Description

ScummVM, CVS version.

Currently, the language selector in the ScummVM GUI has an entry for "Norwegian Bokmĺl". This works fine with the bigger version of the ScummVM GUI, since newfont_big.cpp contains, from the looks of it, the entire Latin-1 character set.

With the smaller version, the "ĺ" is rendered as a space because newfont.cpp only contains the ASCII character set.

The simplest solution would, of course, be to change the text to something else than "Norwegian Bokmĺl". Alternatively, I think we should settle on one character set for the GUI, and make sure we use only that. (In which case we should probably add an option to convbdf.c to strip out all unwanted characters.)

If we do settle on Latin-1, credits.pl will have to be updated, as well, to translate the HTML entities to Latin-1 instead of ASCII.

Ticket imported from: #1276480. Ticket imported from: bugs/2172.

Attachments (7)

gui-font.diff (48.3 KB ) - added by eriktorbjorn 14 years ago.
Partial patch
font-wip.diff (22.9 KB ) - added by eriktorbjorn 14 years ago.
Patch against current SVN
font-wip.v2.diff.bz2 (20.6 KB ) - added by sev- 14 years ago.
Patch v2 with fonts rendered.
bdf-fonts.tar.bz2 (41.3 KB ) - added by sev- 14 years ago.
BDF fonts used in ScummVM
font-wip.v3.diff.bz2 (28.2 KB ) - added by eriktorbjorn 14 years ago.
Patch v3 with fonts (re-)rendered
font-wip.v4.diff.bz2 (28.2 KB ) - added by sev- 14 years ago.
Patch v4 with updated fonts
bdf-fonts.v2.tar.bz2 (42.8 KB ) - added by sev- 14 years ago.
BDF fonts used in Patch v4 and in ScummVM

Download all attachments as: .zip

Change History (28)

comment:1 by eriktorbjorn, 15 years ago

Owner: set to fingolfin

comment:2 by fingolfin, 14 years ago

FYI, we are mostly ASCII-only in everything we do, but if we go for an 8-bit charset, then ISO Latin 1 indeed seems like the best choice right now.

comment:3 by sev-, 14 years ago

We should really resolve this before 0.9.0, i.e. come to some particular decision. Raising priority.

comment:4 by sev-, 14 years ago

Priority: normalhigh

by eriktorbjorn, 14 years ago

Attachment: gui-font.diff added

Partial patch

comment:5 by eriktorbjorn, 14 years ago

I started looking at it after sev provided me with font files. The attached patch does the following:

* Replaces newfont_big.cpp with a Latin-1 version. * Changes EditableWidget::tryInsertChar() to take a byte instead of a char, and to accept Latin-1 characters. * Changes the tryInsertChar() in launcher.cpp to take a byte instead of a char. * Adds a html_entities_to_cpp subroutine to credits.pl to generate a small subset of Latin-1 (as octal escape sequences). * Replaces credits.h with a regenerated version.

Problems:

* The font is buggy. For several accented characters, the top of the accent spills over into the previous chraracter. I don't know if this is a bug in the BDF file, or a bug in convbdf. The non-bold font in the modern theme has the same problem. * We still need a replacement for newfont.cpp. Maybe scummfont.cpp as well, though it seems to be used only by OSD.

comment:6 by sev-, 14 years ago

I investigated first problem. It appeared that convbdf misuses FONTBOUNDINGBOX key. Font width and height should be taken from it as well as x and y offsets. Fomrat is this:

FONTBOUNDINGBOX w h xoff yoff

xoff and yoff denote displacement of the lower left corner from origin.

See http://asitis.org/files/font_specs/bdf_spec.zip

Now it generates -1 offset for y coordinate of top row and thus renders it on previous glyph.

by eriktorbjorn, 14 years ago

Attachment: font-wip.diff added

Patch against current SVN

comment:7 by eriktorbjorn, 14 years ago

This patch is an attempt to fix the accented characters. Basically, it changes the font format so that the bitmaps contain everything inside the bounding box, and nothing else, and the bounding boxes are stored in a separate table and in the font header.

It's not very well tested - no doubt there are plenty of bugs left in it - but I'm getting rather sick and tired of it right now. Perhaps someone else can fix the remaining issues?

Issues I can think of are:

* The C++ fonts have to be re-generated. * The cached font data (in the theme ZIP) has to be re-generated. * How to calculate the width of a string accurately? The first and last characters may have to be treated as special cases. * Glyphs may extend above the font's height. Must make sure that the screen is properly updated.

comment:8 by sev-, 14 years ago

Owner: changed from fingolfin to eriktorbjorn

comment:9 by sev-, 14 years ago

I'm attaching updated patch with all fonts regenerated. Unfortunalely our 320x200 font is Cronyx, which is Cyrillic free font. I regenerated it as well, but of course, it has cyrillic glyphs instead of accented ones.

To force regeneration of fonts in modern theme, remove *.fcc from modern.zip and run ScummVM.

There are major display problems, as our font rendering functions aren't aware about bounding boxes.

by sev-, 14 years ago

Attachment: font-wip.v2.diff.bz2 added

Patch v2 with fonts rendered.

by sev-, 14 years ago

Attachment: bdf-fonts.tar.bz2 added

BDF fonts used in ScummVM

comment:10 by sev-, 14 years ago

Archive with all BDF fonts used in ScummVM and some more is attached.

by eriktorbjorn, 14 years ago

Attachment: font-wip.v3.diff.bz2 added

Patch v3 with fonts (re-)rendered

comment:11 by eriktorbjorn, 14 years ago

This new version of the patch fixes a couple of things. From what I remember:

* Editable widgets now accept valid Latin-1 characters. * A bug in reading cached font data has been fixed. (Fixes rendering bugs, and possibly crashes.) * A few bugs in convbdf.c have been fixed. This seems to fix rendering of the Cronyx font. (Used in 320x200 mode.) * Per-character BBX values in C++ fonts are stored as 8-bit values, which should save some space.

comment:12 by sev-, 14 years ago

Owner: changed from eriktorbjorn to fingolfin

comment:13 by sev-, 14 years ago

I updated fonts. I replaced cronyx font with Latin1 Schumacher-Clean font and mistakingly generated console font with Misc-Fixed. Both are free and got from Xorg project. These fonts are used in 320x200 mode. Since Schumacher font is 1pix taller, it may require slight adjustments of some widgets, particularly pop up and widths of checkboxes in theme config. For me the patch looks ok and ready to be committed to both trunk and branch-0-9-0.

Also with this patch applied we may finally render some developers' names properly but this is not included here.

Max, please, review it ASAP and give your opinion.

by sev-, 14 years ago

Attachment: font-wip.v4.diff.bz2 added

Patch v4 with updated fonts

comment:14 by sev-, 14 years ago

Attaching updated BDF fonts. These definitely have to be committed somewhere. Maybe vendor/ module? Max?

by sev-, 14 years ago

Attachment: bdf-fonts.v2.tar.bz2 added

BDF fonts used in Patch v4 and in ScummVM

comment:15 by eriktorbjorn, 14 years ago

The first patch, gui-font.diff, adds a function to credits.pl to generate octal escape sequences for the C++ version of the credits.

By the way, if we want a less intrusive change for 0.9.0, we could simply stick with the current font renderer/convbdf, and touch up the mis-rendered glyphs (most of which the user aren't likely to see anyway) manually.

comment:16 by fingolfin, 14 years ago

Sorry for not replying faster :-(.

As for the BDF font sources: I agree they should be stored in our repository . As to where... Well, vendor/ certainly would make sense, agreed. (Anybody not familiar with the notion of the 'vendor' branches can read up on those in the excellent online SVN book).

The patch looks fine to me, too -- I am sorry I can't try it out right now since I am at work, I'll try it once i am back home tonight. But I have no doubt you already checked it. So don't wait for m, just go ahead and add it.

comment:17 by eriktorbjorn, 14 years ago

Actually, last time I tried it there was still at least one small glitch present: For some characters, the bounding box extends above the top of the line, which can cause drawing glitches, at least in the list widget.

comment:18 by eriktorbjorn, 14 years ago

Sev asked me to apply this, so I've done most of it: The font-wip.v4.diff patch has been applied in its entirety, the credits.pl changes from gui-font.diff have been applied, and credits.h has been re-generated, and the .fcc files in modern.zip have been re-generated.

The reason I'm not closing this patch is that I have not added the BDF font files to SVN. I wasn't sure where sev meant for them to be.

I'll add a note about the glitch to the GUI TODO Wiki page before I go to sleep.

comment:19 by sev-, 14 years ago

Owner: changed from fingolfin to sev-
Status: newclosed

comment:20 by sev-, 14 years ago

I added contents of bdf-fonts.v2.tar.bz2 to SVN in vendor/fonts directory.

comment:21 by digitall, 2 years ago

Component: --Unset--GUI
Note: See TracTickets for help on using tickets.