Mercurial > sdl-ios-xcode
view CREDITS @ 3539:f2846bf19360
Fixed bug #896
John Popplewell 2009-12-08 23:05:50 PST
Originally reported by AKFoerster on the mailing list.
Error decoding UTF8 Russian text to UTF-16LE on Windows, but specifically on
platforms without iconv support (the default on Windows).
Valid UTF8 characters are flagged as being overlong and then substituted by the
UNKNOWN_UNICODE character.
After studying the testiconv.c example program, reading the RFCs and putting
some printf statements in SDL_iconv.c the problem is in a test for 'Maximum
overlong sequences', specifically 4.2.1, which is carried out by the following
code:
} else if ( p[0] >= 0xC0 ) {
if ( (p[0] & 0xE0) != 0xC0 ) {
/* Skip illegal sequences
return SDL_ICONV_EILSEQ;
*/
ch = UNKNOWN_UNICODE;
} else {
if ( (p[0] & 0xCE) == 0xC0 ) { <<<<<<<< here
overlong = SDL_TRUE;
}
ch = (Uint32)(p[0] & 0x1F);
left = 1;
}
} else {
Here is the 2-byte encoding of a character in range 00000080 - 000007FF
110xxxxx 10xxxxxx
The line in question is supposed to be checking for an overlong sequence which
would be less than
11000001 10111111
which should be represented as a single byte.
BUT, the mask value (0xCE) is wrong, it isn't checking the top-most bit:
11000001 value
11001110 mask (incorrect)
^
and should be (0xDE):
11000001 value
11011110 mask (correct)
making the above code:
} else if ( p[0] >= 0xC0 ) {
if ( (p[0] & 0xE0) != 0xC0 ) {
/* Skip illegal sequences
return SDL_ICONV_EILSEQ;
*/
ch = UNKNOWN_UNICODE;
} else {
if ( (p[0] & 0xDE) == 0xC0 ) { <<<<<<<< here
overlong = SDL_TRUE;
}
ch = (Uint32)(p[0] & 0x1F);
left = 1;
}
} else {
I can supply a test program and/or a patch if required,
best regards,
John Popplewell
| author | Sam Lantinga <slouken@libsdl.org> |
|---|---|
| date | Fri, 11 Dec 2009 08:03:43 +0000 |
| parents | 1ed5d432e468 |
| children |
line wrap: on
line source
Simple DirectMedia Layer CREDITS Thanks to everyone who made this possible, including: * Cliff Matthews, for giving me a reason to start this project. :) -- Executor rocks! *grin* * The Linux Fund, C Magazine, Educational Technology Resources Inc., Gareth Noyce, Jesse Pavel, Keith Kitchin, Jeremy Horvath, Thomas Nicholson, Hans-Peter Gygax, the Eternal Lands Development Team, Lars Brubaker, and Phoenix Kokido for financial contributions * Edgar "bobbens" Simo for his force feedback API development during the Google Summer of Code 2008 * Aaron Wishnick for his work on audio resampling and pitch shifting during the Google Summer of Code 2008 * Holmes Futrell for port of SDL to the iPhone and iPod Touch during the Google Summer of Code 2008 * Darren Alton for port of SDL to the Nintendo DS during the Google Summer of Code 2008 * Szymon "Wilku" Wilczek for adding support for multiple mice and tablets during the Google Summer of Code 2008 * Marty Leisner, Andrew, Will, Edgar Simo, Donny Viszneki, Andrea Mazzoleni, Dmytro Bogovych, and Couriersud for helping find SDL 1.3 bugs in the great SDL Bug Hunt of January 2009! * Donny Viszneki for helping fix SDL 1.3 bugs in the great SDL Bug Hunt of January 2009! * Luke Benstead for OpenGL 3.0 support * Gaëtan de Menten for writing the PHP and SQL behind the SDL website * Tim Jones for the new look of the SDL website * Ryan Gordon for helping everybody out and keeping the dream alive. :) * Mattias Engdegård, for help with the Solaris port and lots of other help * Eric Wing, Max Horn, and Darrell Walisser for unflagging work on the Mac OS X port * David Carré, for the Pandora port * Couriersud for the DirectFB driver * Jon Atkins for SDL_image, SDL_mixer and SDL_net documentation * Arne Claus, for the 2004 winning SDL logo, and Shandy Brown, Jac, Alex Lyman, Mikkel Gjoel, #Guy, Jonas Hartmann, Daniel Liljeberg, Ronald Sowa, DocD, Pekka Jaervinen, Patrick Avella, Erkki Kontilla, Levon Gavalian, Hal Emerich, David Wiktorsson, S. Schury and F. Hufsky, Ciska de Ruyver, Shredweat, Tyler Montbriand, Martin Andersson, Merlyn Wysard, Fernando Ibanez, David Miller, Andre Bommele, lovesby.com, Francisco Camenforte Torres, and David Igreja for other logo entries. * Bob Pendleton and David Olofson for being long time contributors to the SDL mailing list. * Everybody at Loki Software, Inc. for their great contributions! And a big hand to everyone else who gave me appreciation, advice, and suggestions, especially the good folks on the SDL mailing list. THANKS! :) -- Sam Lantinga <slouken@libsdl.org>