Mercurial > lcfOS

comparison python/c3/lexer.py @ 149:74241ca312cc

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fixes on parser and semantics
author Windel Bouwman
date Fri, 01 Mar 2013 11:43:52 +0100
parents e5263f74b287
children b73bc14a3aa3
comparison
equal deleted inserted replaced
148:e5263f74b287 149:74241ca312cc
8 # Token is used in the lexical analyzer: 8 # Token is used in the lexical analyzer:
9 Token = collections.namedtuple('Token', 'typ val loc') 9 Token = collections.namedtuple('Token', 'typ val loc')
10 10
11 keywords = ['and', 'or', 'not','true', 'false', \ 11 keywords = ['and', 'or', 'not','true', 'false', \
12 'else', 'if', 'while', 'return', \ 12 'else', 'if', 'while', 'return', \
13 'public', 'function', 'var', 'type', \ 13 'function', 'var', 'type', \
14 'import', 'package' ] 14 'import', 'package' ]
15 15
16 def tokenize(s): 16 def tokenize(s):
17 """ 17 """
18 Tokenizer, generates an iterator that 18 Tokenizer, generates an iterator that