This work is licensed under a Creative Commons
Attribution-ShareAlike 4.0 International License

Write your own programming language

Andy Balaam, OpenMarket
artificialworlds.net/blog

Workshop structure

Sit near a laptop with Python 2 or 3
Endure 3 slides
Then we do test-driven development
Work together!

What is a programming language?

Our language

> today
2017-06-28 (Tuesday)

Our language

> today
2017-06-28 (Tuesday)

> 2 weeks
14 days

Our language

> today
2017-06-28 (Tuesday)

> 2 weeks
14 days

> tomorrow + 2 weeks
2017-07-13 (Wednesday)

Code structure

Create a Python file datecalc.py:

# -- imports --

# -- functions --

# -- tests --

# -- main --

Lex a single word

# -- tests --
assert lex("today") == [("WordToken", "today")]

Make it pass!

Lex a single word

My version:

# -- functions --
def lex(chars):
    return [("WordToken", chars)]

Parse a single word

# -- tests --
def p(x):
    return parse(lex(x))

assert p("today") == ("WordTree", "today")

Make it pass!

Parse a single word

My version:

# -- functions --
def parse(tokens):
    tok = tokens[0]
    return ("WordTree", tok[1])

Eval a single word

# -- imports --
from datetime import date, timedelta

# -- tests --
def e(x):
    return evaluate(parse(lex(x)))

today = date.today()

assert e("today") == ("DateValue", today)

Make it pass!

Parse a single word

My version (for now):

# -- functions --
def evaluate(tree):
    return ("DateValue", date.today())

Another single word

# -- tests --
assert (
    lex("tomorrow") ==
    [("WordToken", "tomorrow")]
)

assert (
    p("tomorrow") ==
    ("WordTree", "tomorrow")
)

These may already pass!

Another single word

# -- tests --
def days(n):
    return timedelta(days=n)

assert (
    e("tomorrow") ==
    ("DateValue", today + days(1))
)

Make it pass!

Another single word

My version:

# -- functions --
def evaluate(tree):
    if tree[1] == "today":
        return ("DateValue", date.today())
    elif tree[1] == "tomorrow":
        return (
            "DateValue",
            date.today() + timedelta(days=1)
        )

Two different tokens

# -- tests --
assert (
    lex("2 days") ==
    [
        ("NumberToken", "2"),
        ("WordToken", "days")
    ]
)

Make it pass!

Two different tokens

My version:

# -- functions --
def make_token(s):
    if s[0] in "0123456789":
        return ("NumberToken", s)
    else:
        return ("WordToken", s)

def lex(chars):
    return [
        make_token(s)
        for s in chars.split(" ")
    ]

Parse time length

# -- tests --
assert (
    p("2 days") ==
    ("LengthTree", "2", "days")
)

Make it pass!

Parse time length

My version:

def parse(tokens):
    tok = tokens[0]
    if tok[0] == "NumberToken":
        next_tok = tokens[1]
        return (
            "LengthTree",
            tok[1],
            next_tok[1]
        )
    else: # Must be WordToken
        return ("WordTree", tok[1])

Eval time length

# -- tests --
assert e("2 days") == ("LengthValue", 2)

Make it pass!

Eval time length

My version:

def evaluate(tree):
    if tree[0] == "LengthTree":
        return ("LengthValue", int(tree[1]))
    elif tree[0] == "WordTree":
        if tree[1] == "today":
            return ("DateValue", date.today())
        elif tree[1] == "tomorrow":
            return (
                "DateValue",
                date.today() + timedelta(days=1)
            )

Weeks are 7 days

# -- tests --
assert (
    lex("3 weeks") ==
    [
        ("NumberToken", "3"),
        ("WordToken", "weeks")
    ]
)
assert (
    p("3 weeks") ==
    ("LengthTree", "3", "weeks")
)

These may already pass!

Weeks are 7 days

# -- tests --
assert e("3 weeks") == ("LengthValue", 21)

Make it pass!

Weeks are 7 days

def length_tree_in_days(length_tree):
    number = int(length_tree[1])
    unit = length_tree[2]
    if unit == "weeks":
        return number * 7
    else:
        return number

def evaluate(tree):
    if tree[0] == "LengthTree":
        return (
            "LengthValue",
            length_tree_in_days(tree)
        )
    ...

Plus expression

assert (
    lex("today + 3 days") ==
    [
        ("WordToken", "today"),
        ("OperatorToken", "+"),
        ("NumberToken", "3"),
        ("WordToken", "days")
    ]
)

Make it pass!

Plus expression

My version:

def make_token(s):
    if s[0] in "0123456789":
        return ("NumberToken", s)
    elif s[0] in "+":
        return ("OperatorToken", s[0])
    else:
        return ("WordToken", s)

Plus expression

assert (
    p("today + 3 days") ==
    ("OperatorTree",
        "+",
        ("WordTree", "today"),
        ("LengthTree", "3", "days")
    )
)

Challenge: make it pass!

Plus expression

My version:

def parse(tokens, so_far=None):
    if len(tokens) == 0:
        return so_far
    ...

Plus expression

My version:

def parse(tokens, so_far=None):
    ...
    tok = tokens[0]
    other_toks = tokens[1:]
    elif tok[0] == "OperatorToken":
        return (
            "OperatorTree",
            tok[1],
            so_far,
            parse(other_toks)
        )
    ...

Plus expression

My version:

def parse(tokens, so_far=None):
    ...
    else:
        return parse(
            other_toks,
            ("WordTree", tok[1])
        )

Plus expression

def parse(tokens, so_far=None):
    if len(tokens) == 0:
        return so_far
    tok = tokens[0]
    other_toks = tokens[1:]
    if tok[0] == "NumberToken":
        next_tok = tokens[1]
        return ("LengthTree", tok[1], next_tok[1])
    elif tok[0] == "OperatorToken":
        return ("OperatorTree",
            tok[1], so_far, parse(other_toks))
    else: # Must be WordToken
        return parse(
            other_toks, ("WordTree", tok[1]))

Plus expression

assert (
    e("today + 3 days") ==
    ("DateValue", today + days(3))
)
assert (
    e("tomorrow + 1 day") ==
    ("DateValue", today + days(2))
)

Make them pass!

Plus expression

def evaluate(tree):
    if tree[0] == "LengthTree":
        ...
    elif tree[0] == "OperatorTree":
        left = evaluate(tree[2])
        right = evaluate(tree[3])
        return (
            "DateValue",
            left[1] + timedelta(days=right[1])
        )
    elif tree[0] == "WordTree":
        ...

Extension: main

$ python datecalc.py
2 weeks
14 days
today + 3 days
2017-08-01 (Saturday)

Make it so.

Extension: main

# -- imports --
import sys

# -- functions --
def pretty(value):
    if value[0] == "DateValue":
        return value[1].strftime("%Y-%m-%d (%A)")
    else:
        return "%s days" % value[1]

# -- main --
while True:
    ln = sys.stdin.readline()
    if ln is None or ln.strip() == "":
        break
    sys.stdout.write(
        "%s\n" %
        pretty(evaluate(parse(lex(ln.strip()))))
    )

Homework: error-handling

> 2 elephants
Unknown time unit 'elephants'.

> penguin
Unknown word 'penguin'.

Homework: minus

> today - 3 days
2017-08-25 (Sunday)

Homework: date input

> 5th July 2015 - 3 days
2017-07-02 (Thursday)

Nice work.

OpenMarket is hiring!	openmarket.com/company/careers
Code:	github.com/andybalaam/datecalc
Slides:	github.com/andybalaam/videos-write-your-own-language

More:
@andybalaam
@andybalaam@mastodon.social
youtube.com/user/ajbalaam
artificialworlds.net/blog