Mercurial > lcfOS

comparison python/c3/lexer.py @ 221:848c4b15fd0b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
pointers
author Windel Bouwman
date Mon, 08 Jul 2013 22:21:44 +0200
parents 003c8a976fff
children 1c7364bd74c7
comparison
equal deleted inserted replaced
220:3f6c30a5d234 221:848c4b15fd0b
7 """ 7 """
8 8
9 keywords = ['and', 'or', 'not', 'true', 'false', \ 9 keywords = ['and', 'or', 'not', 'true', 'false', \
10 'else', 'if', 'while', 'return', \ 10 'else', 'if', 'while', 'return', \
11 'function', 'var', 'type', 'const', \ 11 'function', 'var', 'type', 'const', \
12 'struct', \ 12 'struct', 'cast', \
13 'import', 'package' ] 13 'import', 'package' ]
14 14
15 def tokenize(s): 15 def tokenize(s):
16 """ 16 """
17 Tokenizer, generates an iterator that 17 Tokenizer, generates an iterator that