annotate doc/compiler.rst @ 368:d2ddfe134c48

Remove yield from for python < 3.3
author Windel Bouwman
date Fri, 21 Mar 2014 11:21:50 +0100
parents 39bf68bf1891
children
rev   line source
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
1
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
2
314
38f5f298ce0e Add log for interference graph
Windel Bouwman
parents: 310
diff changeset
3 .. toctree::
38f5f298ce0e Add log for interference graph
Windel Bouwman
parents: 310
diff changeset
4
38f5f298ce0e Add log for interference graph
Windel Bouwman
parents: 310
diff changeset
5 ir
38f5f298ce0e Add log for interference graph
Windel Bouwman
parents: 310
diff changeset
6
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
7 Compiler
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
8 ========
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
9
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
10 This chapter describes the design of the compiler.
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
11 The compiler consists a frontend, mid-end and back-end. The frontend deals with
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
12 source file parsing and semantics checking. The mid-end performs optimizations.
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
13 This is optional. The back-end generates machine code. The front-end produces
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
14 intermediate code. This is a simple representation of the source. The back-end
300
Windel Bouwman
parents: 299
diff changeset
15 can accept this kind of representation.
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
16
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
17 .. graphviz::
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
18
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
19 digraph x {
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
20 rankdir="LR"
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
21 1 [label="c3 source file"]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
22 10 [label="c3 front end" ]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
23 11 [label="language X front end" ]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
24 20 [label="mid end" ]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
25 30 [label="back end for X86" ]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
26 31 [label="back end for ARM" ]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
27 40 [label="object file"]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
28 1 -> 10
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
29 10 -> 20 [label="IR-code"]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
30 11 -> 20 [label="IR-code"]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
31 20 -> 30 [label="IR-code"]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
32 20 -> 31 [label="IR-code"]
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
33 30 -> 40
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
34 }
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
35
299
674789d9ff37 Added a doc
Windel Bouwman
parents: 297
diff changeset
36
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
37 IR-code
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
38 -------
300
Windel Bouwman
parents: 299
diff changeset
39
299
674789d9ff37 Added a doc
Windel Bouwman
parents: 297
diff changeset
40 The intermediate representation (IR) of a program de-couples the front end
674789d9ff37 Added a doc
Windel Bouwman
parents: 297
diff changeset
41 from the backend of the compiler.
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
42
310
e95e5572cd6d Added utils doc page
Windel Bouwman
parents: 308
diff changeset
43 See :doc:`ir` for details about all the available instructions.
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
44
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
45
274
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
46 C3 Front-end
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
47 ------------
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
48
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
49 For the front-end a recursive descent parser is created for the c3 language.
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
50 This is a subset of the C language with some additional features.
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
51
274
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
52 .. graphviz::
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
53
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
54 digraph c3 {
300
Windel Bouwman
parents: 299
diff changeset
55 rankdir="LR"
274
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
56 1 [label="source text"]
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
57 10 [label="lexer" ]
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
58 20 [label="parser" ]
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
59 40 [label="code generation"]
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
60 99 [label="IR-code object"]
299
674789d9ff37 Added a doc
Windel Bouwman
parents: 297
diff changeset
61 1 -> 10
674789d9ff37 Added a doc
Windel Bouwman
parents: 297
diff changeset
62 10 -> 20
310
e95e5572cd6d Added utils doc page
Windel Bouwman
parents: 308
diff changeset
63 20 -> 40
274
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
64 40 -> 99
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
65 }
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
66
305
0615b5308710 Updated docs
Windel Bouwman
parents: 304
diff changeset
67 .. autoclass:: ppci.c3.Lexer
0615b5308710 Updated docs
Windel Bouwman
parents: 304
diff changeset
68
301
6753763d3bec merge codegen into ppci package
Windel Bouwman
parents: 300
diff changeset
69 .. autoclass:: ppci.c3.Parser
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
70
301
6753763d3bec merge codegen into ppci package
Windel Bouwman
parents: 300
diff changeset
71 .. autoclass:: ppci.c3.CodeGenerator
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
72
306
b145f8e6050b Start on c3 rewrite
Windel Bouwman
parents: 305
diff changeset
73 .. autoclass:: ppci.c3.Builder
b145f8e6050b Start on c3 rewrite
Windel Bouwman
parents: 305
diff changeset
74
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
75 Back-end
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
76 --------
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
77
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
78 The back-end is more complicated. There are several steps to be taken here.
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
79
357
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
80 1. Canonicalization
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
81 2. Tree creation
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
82 3. Instruction selection
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
83 4. register allocation
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
84 5. Instruction emission
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
85 6. TODO: Peep hole optimization?
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
86
306
b145f8e6050b Start on c3 rewrite
Windel Bouwman
parents: 305
diff changeset
87 .. automodule:: ppci.codegen
273
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
88 :members:
6b3a874edd6e Added some docs
Windel Bouwman
parents:
diff changeset
89
357
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
90 Canonicalize
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
91 ~~~~~~~~~~~~
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
92
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
93 During this phase, the IR-code is made simpler. Function calls are pulled pulled
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
94 to top level and the frame pointer is introduced.
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
95
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
96 Tree building
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
97 ~~~~~~~~~~~~~
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
98
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
99 From IR-code a tree is generated which can be used to select instructions.
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
100
274
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
101 Instruction selection
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
102 ~~~~~~~~~~~~~~~~~~~~~
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
103
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
104 The instruction selection phase takes care of scheduling and instruction
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
105 selection. The output of this phase is a one frame per function with a flat
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
106 list of abstract machine instructions.
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
107
304
fa99f36fabb5 Fix docs
Windel Bouwman
parents: 301
diff changeset
108 // .. autoclass:: ppci.irmach.Frame
274
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
109
304
fa99f36fabb5 Fix docs
Windel Bouwman
parents: 301
diff changeset
110 // .. autoclass:: ppci.irmach.AbstractInstruction
274
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
111
366
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
112 To select instruction, a tree rewrite system is used. This is also called
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
113 bottom up rewrite generator (BURG). See pyburg.
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
114
274
ea93e0a7a31e Move docs
Windel Bouwman
parents: 273
diff changeset
115
357
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
116 Register allocation
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
117 ~~~~~~~~~~~~~~~~~~~
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
118
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
119 The selected instructions are used to select correct registers.
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
120
818be710e13d Added acceptance function to burg
Windel Bouwman
parents: 314
diff changeset
121
366
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
122 code emission
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
123 ~~~~~~~~~~~~~
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
124
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
125 Code is emitted using the outputstream class. The assembler and compiler use
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
126 this class to emit instructions to. The stream can output to object file
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
127 or to a logger.
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
128
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
129 .. autoclass:: ppci.outstream.OutputStream
39bf68bf1891 Fix sample tests and deterministic build
Windel Bouwman
parents: 357
diff changeset
130