Opened 14 years ago

Closed 13 years ago

Last modified 12 months ago

#2172 closed defect

GUI: Accented characters

Reported by: eriktorbjorn Owned by: sev-
Priority: high Component: GUI
Keywords: Cc:
Game:

Description

ScummVM, CVS version.

Currently, the language selector in the ScummVM GUI has
an entry for "Norwegian Bokmĺl". This works fine with
the bigger version of the ScummVM GUI, since
newfont_big.cpp contains, from the looks of it, the
entire Latin-1 character set.

With the smaller version, the "ĺ" is rendered as a
space because newfont.cpp only contains the ASCII
character set.

The simplest solution would, of course, be to change
the text to something else than "Norwegian Bokmĺl".
Alternatively, I think we should settle on one
character set for the GUI, and make sure we use only
that. (In which case we should probably add an option
to convbdf.c to strip out all unwanted characters.)

If we do settle on Latin-1, credits.pl will have to be
updated, as well, to translate the HTML entities to
Latin-1 instead of ASCII.

Ticket imported from: #1276480. Ticket imported from: bugs/2172.

Attachments (7)

gui-font.diff (48.3 KB ) - added by eriktorbjorn 13 years ago.
Partial patch
font-wip.diff (22.9 KB ) - added by eriktorbjorn 13 years ago.
Patch against current SVN
font-wip.v2.diff.bz2 (20.6 KB ) - added by sev- 13 years ago.
Patch v2 with fonts rendered.
bdf-fonts.tar.bz2 (41.3 KB ) - added by sev- 13 years ago.
BDF fonts used in ScummVM
font-wip.v3.diff.bz2 (28.2 KB ) - added by eriktorbjorn 13 years ago.
Patch v3 with fonts (re-)rendered
font-wip.v4.diff.bz2 (28.2 KB ) - added by sev- 13 years ago.
Patch v4 with updated fonts
bdf-fonts.v2.tar.bz2 (42.8 KB ) - added by sev- 13 years ago.
BDF fonts used in Patch v4 and in ScummVM

Download all attachments as: .zip

Change History (28)

comment:1 by eriktorbjorn, 14 years ago

Owner: set to fingolfin

comment:2 by fingolfin, 13 years ago

FYI, we are mostly ASCII-only in everything we do, but if we go for an 8-bit
charset, then ISO Latin 1 indeed seems like the best choice right now.

comment:3 by sev-, 13 years ago

We should really resolve this before 0.9.0, i.e. come to
some particular decision. Raising priority.

comment:4 by sev-, 13 years ago

Priority: normalhigh

by eriktorbjorn, 13 years ago

Attachment: gui-font.diff added

Partial patch

comment:5 by eriktorbjorn, 13 years ago

I started looking at it after sev provided me with font
files. The attached patch does the following:

* Replaces newfont_big.cpp with a Latin-1 version.
* Changes EditableWidget::tryInsertChar() to take a byte
instead of a char, and to accept Latin-1 characters.
* Changes the tryInsertChar() in launcher.cpp to take a byte
instead of a char.
* Adds a html_entities_to_cpp subroutine to credits.pl to
generate a small subset of Latin-1 (as octal escape sequences).
* Replaces credits.h with a regenerated version.

Problems:

* The font is buggy. For several accented characters, the
top of the accent spills over into the previous chraracter.
I don't know if this is a bug in the BDF file, or a bug in
convbdf. The non-bold font in the modern theme has the same
problem.
* We still need a replacement for newfont.cpp. Maybe
scummfont.cpp as well, though it seems to be used only by OSD.

comment:6 by sev-, 13 years ago

I investigated first problem. It appeared that convbdf
misuses FONTBOUNDINGBOX key. Font width and height should be
taken from it as well as x and y offsets. Fomrat is this:

FONTBOUNDINGBOX w h xoff yoff

xoff and yoff denote displacement of the lower left corner
from origin.

See http://asitis.org/files/font_specs/bdf_spec.zip

Now it generates -1 offset for y coordinate of top row and
thus renders it on previous glyph.

by eriktorbjorn, 13 years ago

Attachment: font-wip.diff added

Patch against current SVN

comment:7 by eriktorbjorn, 13 years ago

This patch is an attempt to fix the accented characters.
Basically, it changes the font format so that the bitmaps
contain everything inside the bounding box, and nothing
else, and the bounding boxes are stored in a separate table
and in the font header.

It's not very well tested - no doubt there are plenty of
bugs left in it - but I'm getting rather sick and tired of
it right now. Perhaps someone else can fix the remaining issues?

Issues I can think of are:

* The C++ fonts have to be re-generated.
* The cached font data (in the theme ZIP) has to be
re-generated.
* How to calculate the width of a string accurately? The
first and last characters may have to be treated as special
cases.
* Glyphs may extend above the font's height. Must make sure
that the screen is properly updated.

comment:8 by sev-, 13 years ago

Owner: changed from fingolfin to eriktorbjorn

comment:9 by sev-, 13 years ago

I'm attaching updated patch with all fonts regenerated.
Unfortunalely our 320x200 font is Cronyx, which is Cyrillic
free font. I regenerated it as well, but of course, it has
cyrillic glyphs instead of accented ones.

To force regeneration of fonts in modern theme, remove *.fcc
from modern.zip and run ScummVM.

There are major display problems, as our font rendering
functions aren't aware about bounding boxes.

by sev-, 13 years ago

Attachment: font-wip.v2.diff.bz2 added

Patch v2 with fonts rendered.

by sev-, 13 years ago

Attachment: bdf-fonts.tar.bz2 added

BDF fonts used in ScummVM

comment:10 by sev-, 13 years ago

Archive with all BDF fonts used in ScummVM and some more is
attached.

by eriktorbjorn, 13 years ago

Attachment: font-wip.v3.diff.bz2 added

Patch v3 with fonts (re-)rendered

comment:11 by eriktorbjorn, 13 years ago

This new version of the patch fixes a couple of things. From
what I remember:

* Editable widgets now accept valid Latin-1 characters.
* A bug in reading cached font data has been fixed. (Fixes
rendering bugs, and possibly crashes.)
* A few bugs in convbdf.c have been fixed. This seems to fix
rendering of the Cronyx font. (Used in 320x200 mode.)
* Per-character BBX values in C++ fonts are stored as 8-bit
values, which should save some space.

comment:12 by sev-, 13 years ago

Owner: changed from eriktorbjorn to fingolfin

comment:13 by sev-, 13 years ago

I updated fonts. I replaced cronyx font with Latin1
Schumacher-Clean font and mistakingly generated console font
with Misc-Fixed. Both are free and got from Xorg project.
These fonts are used in 320x200 mode. Since Schumacher font
is 1pix taller, it may require slight adjustments of some
widgets, particularly pop up and widths of checkboxes in
theme config. For me the patch looks ok and ready to be
committed to both trunk and branch-0-9-0.

Also with this patch applied we may finally render some
developers' names properly but this is not included here.

Max, please, review it ASAP and give your opinion.

by sev-, 13 years ago

Attachment: font-wip.v4.diff.bz2 added

Patch v4 with updated fonts

comment:14 by sev-, 13 years ago

Attaching updated BDF fonts. These definitely have to be
committed somewhere. Maybe vendor/ module? Max?

by sev-, 13 years ago

Attachment: bdf-fonts.v2.tar.bz2 added

BDF fonts used in Patch v4 and in ScummVM

comment:15 by eriktorbjorn, 13 years ago

The first patch, gui-font.diff, adds a function to
credits.pl to generate octal escape sequences for the C++
version of the credits.

By the way, if we want a less intrusive change for 0.9.0, we
could simply stick with the current font renderer/convbdf,
and touch up the mis-rendered glyphs (most of which the user
aren't likely to see anyway) manually.

comment:16 by fingolfin, 13 years ago

Sorry for not replying faster :-(.

As for the BDF font sources: I agree they should be stored
in our repository . As to where... Well, vendor/ certainly
would make sense, agreed. (Anybody not familiar with the
notion of the 'vendor' branches can read up on those in the
excellent online SVN book).

The patch looks fine to me, too -- I am sorry I can't try it
out right now since I am at work, I'll try it once i am back
home tonight. But I have no doubt you already checked it. So
don't wait for m, just go ahead and add it.

comment:17 by eriktorbjorn, 13 years ago

Actually, last time I tried it there was still at least one
small glitch present: For some characters, the bounding box
extends above the top of the line, which can cause drawing
glitches, at least in the list widget.

comment:18 by eriktorbjorn, 13 years ago

Sev asked me to apply this, so I've done most of it: The
font-wip.v4.diff patch has been applied in its entirety, the
credits.pl changes from gui-font.diff have been applied, and
credits.h has been re-generated, and the .fcc files in
modern.zip have been re-generated.

The reason I'm not closing this patch is that I have not
added the BDF font files to SVN. I wasn't sure where sev
meant for them to be.

I'll add a note about the glitch to the GUI TODO Wiki page
before I go to sleep.

comment:19 by sev-, 13 years ago

Owner: changed from fingolfin to sev-
Status: newclosed

comment:20 by sev-, 13 years ago

I added contents of bdf-fonts.v2.tar.bz2 to SVN in
vendor/fonts directory.

comment:21 by digitall, 12 months ago

Component: --Unset--GUI
Note: See TracTickets for help on using tickets.