annotate upmana/mercurial/encoding.py @ 28:ff154cf3350c ornery-orc

Traipse 'OpenRPG' {100203-00} Traipse is a distribution of OpenRPG that is designed to be easy to setup and go. Traipse also makes it easy for developers to work on code without fear of sacrifice. 'Ornery-Orc' continues the trend of 'Grumpy' and adds fixes to the code. 'Ornery-Orc's main goal is to offer more advanced features and enhance the productivity of the user. Update Summary (Stable) New Features: New Bookmarks Feature New 'boot' command to remote admin New confirmation window for sent nodes Miniatures Layer pop up box allows users to turn off Mini labels, from FlexiRPG New Zoom Mouse plugin added New Images added to Plugin UI Switching to Element Tree New Map efficiency, from FlexiRPG New Status Bar to Update Manager New TrueDebug Class in orpg_log (See documentation for usage) New Portable Mercurial New Tip of the Day, from Core and community New Reference Syntax added for custom PC sheets New Child Reference for gametree New Parent Reference for gametree New Gametree Recursion method, mapping, context sensitivity, and effeciency.. New Features node with bonus nodes and Node Referencing help added New Dieroller structure from Core New DieRoller portability for odd Dice New 7th Sea die roller; ie [7k3] = [7d10.takeHighest(3).open(10)] New 'Mythos' System die roller added New vs. die roller method for WoD; ie [3v3] = [3d10.vs(3)]. Included for Mythos roller also New Warhammer FRPG Die Roller (Special thanks to Puu-san for the support) New EZ_Tree Reference system. Push a button, Traipse the tree, get a reference (Beta!) New Grids act more like Spreadsheets in Use mode, with Auto Calc Fixes: Fix to allow for portability to an OpenSUSE linux OS Fix to mplay_client for Fedora and OpenSUSE Fix to Text based Server Fix to Remote Admin Commands Fix to Pretty Print, from Core Fix to Splitter Nodes not being created Fix to massive amounts of images loading, from Core Fix to Map from gametree not showing to all clients Fix to gametree about menus Fix to Password Manager check on startup Fix to PC Sheets from tool nodes. They now use the tabber_panel Fix to Whiteboard ID to prevent random line or text deleting. Fixes to Server, Remote Server, and Server GUI Fix to Update Manager; cleaner clode for saved repositories Fixes made to Settings Panel and now reactive settings when Ok is pressed Fixes to Alternity roller's attack roll. Uses a simple Tuple instead of a Splice Fix to Use panel of Forms and Tabbers. Now longer enters design mode Fix made Image Fetching. New fetching image and new failed image Fix to whiteboard ID's to prevent non updated clients from ruining the fix. default_manifest.xml renamed to default_upmana.xml
author sirebral
date Wed, 03 Feb 2010 22:16:49 -0600
parents
children
rev   line source
28
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
1 # encoding.py - character transcoding support for Mercurial
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
2 #
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
3 # Copyright 2005-2009 Matt Mackall <mpm@selenic.com> and others
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
4 #
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
5 # This software may be used and distributed according to the terms of the
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
6 # GNU General Public License version 2, incorporated herein by reference.
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
7
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
8 import error
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
9 import sys, unicodedata, locale, os
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
10
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
11 _encodingfixup = {'646': 'ascii', 'ANSI_X3.4-1968': 'ascii'}
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
12
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
13 try:
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
14 encoding = os.environ.get("HGENCODING")
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
15 if sys.platform == 'darwin' and not encoding:
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
16 # On darwin, getpreferredencoding ignores the locale environment and
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
17 # always returns mac-roman. We override this if the environment is
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
18 # not C (has been customized by the user).
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
19 locale.setlocale(locale.LC_CTYPE, '')
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
20 encoding = locale.getlocale()[1]
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
21 if not encoding:
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
22 encoding = locale.getpreferredencoding() or 'ascii'
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
23 encoding = _encodingfixup.get(encoding, encoding)
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
24 except locale.Error:
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
25 encoding = 'ascii'
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
26 encodingmode = os.environ.get("HGENCODINGMODE", "strict")
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
27 fallbackencoding = 'ISO-8859-1'
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
28
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
29 def tolocal(s):
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
30 """
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
31 Convert a string from internal UTF-8 to local encoding
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
32
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
33 All internal strings should be UTF-8 but some repos before the
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
34 implementation of locale support may contain latin1 or possibly
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
35 other character sets. We attempt to decode everything strictly
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
36 using UTF-8, then Latin-1, and failing that, we use UTF-8 and
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
37 replace unknown characters.
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
38 """
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
39 for e in ('UTF-8', fallbackencoding):
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
40 try:
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
41 u = s.decode(e) # attempt strict decoding
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
42 return u.encode(encoding, "replace")
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
43 except LookupError, k:
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
44 raise error.Abort("%s, please check your locale settings" % k)
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
45 except UnicodeDecodeError:
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
46 pass
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
47 u = s.decode("utf-8", "replace") # last ditch
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
48 return u.encode(encoding, "replace")
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
49
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
50 def fromlocal(s):
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
51 """
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
52 Convert a string from the local character encoding to UTF-8
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
53
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
54 We attempt to decode strings using the encoding mode set by
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
55 HGENCODINGMODE, which defaults to 'strict'. In this mode, unknown
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
56 characters will cause an error message. Other modes include
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
57 'replace', which replaces unknown characters with a special
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
58 Unicode character, and 'ignore', which drops the character.
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
59 """
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
60 try:
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
61 return s.decode(encoding, encodingmode).encode("utf-8")
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
62 except UnicodeDecodeError, inst:
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
63 sub = s[max(0, inst.start-10):inst.start+10]
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
64 raise error.Abort("decoding near '%s': %s!" % (sub, inst))
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
65 except LookupError, k:
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
66 raise error.Abort("%s, please check your locale settings" % k)
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
67
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
68 def colwidth(s):
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
69 "Find the column width of a UTF-8 string for display"
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
70 d = s.decode(encoding, 'replace')
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
71 if hasattr(unicodedata, 'east_asian_width'):
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
72 w = unicodedata.east_asian_width
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
73 return sum([w(c) in 'WF' and 2 or 1 for c in d])
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
74 return len(d)
ff154cf3350c Traipse 'OpenRPG' {100203-00}
sirebral
parents:
diff changeset
75