tartar

Kotlin trie-based lexical analysis and pratt-parsing as a DSL.

Installation

Using in Gradle:

repositories {
  maven { url = 'https://maven.cafeteria.dev' }
}

dependencies {
  implementation 'net.notjustanna:tartar:VERSION'
}

Using in Maven:

<repositories>
    <repository>
        <id>cafeteria</id>
        <name>cafeteria</name>
        <url>https://maven.cafeteria.dev</url>
    </repository>
</repositories>

<dependencies>
<dependency>
    <groupId>net.notjustanna</groupId>
    <artifactId>tartar</artifactId>
    <version>VERSION</version>
</dependency>
</dependencies>

Usage

To create a lexer (also called a tokenizer) use createLexer { ... }, which exposes an interface to configure and add matchers to the lexer.

To create a grammar use createGrammar { ... }, which exposes an interface to create and add prefix parsers and infix parsers to the grammar.

To create a parser use createParser(grammar) { ... }, which will run the block of code on parser.parse().

Example code

// Create a lexer as simple as a method call.
val lexer = Lexer.create<Token<TokenType>> {
    // Implement a token type per line.
    '+' { processToken(PLUS) }
    // Built-in extension functions.
    '"' { processToken(STRING, readString(it), offset = 2) }
    // NOOP tokens.
    ' '()
    '\n'()
}

// Create a pratt-parser grammar. Dead simple.
val grammar = Grammar.create<TokenType, String> {
    // Create a prefix parselet as a lambda function.
    prefix(STRING) { token -> token.value }
    // Create an infix parselet, with support to precedence as a lambda function.
    infix(PLUS, 1) { left, _ -> left + parseExpression() }
}

// Use your grammar to create a pratt-parser.
val parser = SourceParser.create(lexer, grammar) { // Actual code run by the parser.
    // Extension function: Throws if there's still tokens.
    ensureEOF {
        // Parses a expression using this parsers' grammar.
        parseExpression()
    }
}

// One line of code to rule them all.
val result = parser.parse(Source.classpath { "input.str" })
println(result)

See the full code here, as well as other examples here.

Special thanks

Special thanks to Avarel, this library wouldn't be possible without his help, feedback and source code of Lobos and Kaiper.