Lexer

It then asks the Lexer for its first two tokens, and if those Tokens contain the Lexemes Identifier("case"), and Identifier("class") the it knows its found a case class.

Archive 2009-05-01 Jack Cough 2009

It creates a new Lexer with its input, an IdentifierOnlyLexer technically, it could fire up a Lexer with all the finders, but it's overkill.

A Scala Lexer Jack Cough 2009

In my next post I'm going to cover a bunch of major refactorings I just did to my Lexer, and how easy it was.

Scala over Ruby - My Debate Ends Jack Cough 2009

I'll show the remainder of the Lexer code in the next section.

Refactoring in Scala Jack Cough 2009

The finders need the current line, Tokens and errors need currentLineNumber and currentOffset, and the Lexer itself needs to know if it processed the whole file eofReached_?

Refactoring in Scala Jack Cough 2009

I wanted to do a general cleanup of my Lexer trait, and extract the mutability from it.

Refactoring in Scala Jack Cough 2009

Separate Scala lexing from Lexer core, providing a reusable Lexer framework for Lexing any language.

Refactoring in Scala Jack Cough 2009

The code for calling into the syntax error handler in the Lexer actually shaped up nicely as well:

Refactoring in Scala Jack Cough 2009

Previously, the Lexer just worked with a single array.

Refactoring in Scala Jack Cough 2009

My Lexer is now approaching 500 lines with roughly 350 lines of test code.

Refactoring in Scala Jack Cough 2009