:r4 web site

View on GitHub

:r4 Compiler

The Post Tokenizer

When have the dictionary and the token string fill then you can start to make more analysis to the programa to compile/evaluate/convert.

There are two word for make this, ::tokenpostall analize all the word and ::tokenpostusa only analyze the words used by the program, if you convert or compile the program, not need all the words analysed but when you use this information for and editor, for example, you need analize all, for show whats happen in all words.

The way in which the word analysis works is as follows: The list of tokens is calculated by calculating the movement of the stacks, of return and of data and marking different uses of the words , like access to memory, use of records, etc.

The calculation of movement of the data stack is made using two indicators, the use and the difference, the use maintains that cells are used and the difference is how much is removed or put on the stack.

Compile to plain code

The next stage is convert the code making some optimization


::makeplain is the main word, this generate a file in “r4/compiler/plain.txt” with some optimizations from the tokenized source code.

Make some transformation to source code without modify behavior.

[ 1 'v +! ; ] --> :_a1 1 'v +! ;  '_a1
#var 1
.. var + ..   -->    .. 1 + ..
3 4 + -->  7
13 / --> $4ec4ec4f 34 *>> dup 31 >> -
4 / --> 2 >> dup 31 >> -
8 * --> 3 >> dup 31 >> -

When have a plain.txt generate, I reset the tokenizer and tokenize again with this version for generate code.

This convertions are transformation from code in :r4 to code in :r4, this is in part, because the low level words for manipulate integer, *>> /<< and */ are very clever, the last one is clasical in forth, but the first two born when need this behavior and not have access to asm, this decision is the key.

Bytecode compile

You can compile to bytecode and use the interprete avoid the compile phase, if yoou generate first the plain code, the tokens result are more eficient, more compact and more fast!, the plain button in the editor is for this.