Skip to content

A small and easy to use .NET library for building lexical analyzers at run-time

License

Notifications You must be signed in to change notification settings

edazdarevic/Lesk

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

What is Lesk?

Lesk is a lexical analyzer library. It helps you easily build your lexical analyzers at runtime. Lexical analyzers are also called lexers or tokenizers or scanners. Lexer transforms a sequence of characters to a sequence of tokens.

How to use Lesk?

Lesk uses regular expressions for defining tokens. For example a JSON lexer would be defined with something like this:

var jsonLexer = LeskInstance.Configure
              .DefineToken(@"\s+", () => new WhitespaceToken())
              .DefineToken(":", () => new ColonToken())
              .DefineToken(",", () => new CommaToken())
              .DefineToken("{", () => new LBraceToken())
              .DefineToken("}", () => new RBraceToken())
              .DefineToken("true", () => new TrueFalseToken())
              .DefineToken("false", () => new TrueFalseToken())
              .DefineToken("null", () => new NullToken())
              .DefineToken("-?[0-9]+", () => new IntToken())
              .DefineToken("\".*?\"", () => new StringToken())
              .DefineToken(@"(-?[0-9]+)(\.[0-9]+)",()=> new DoubleToken())
              .AsCompiled()
              .Done();

To perform actual lexing call the Tokenize method

List<Token> tokens = jsonLexer.Tokenize(yourStringHere); 

How is Lesk implemented?

Lesk internally relies on default .NET regular expression implementation. However, this might change in the future.

Roadmap

0.4

  • Support for tokenizing directly from a Stream and yielding tokens as they become available

0.5 etc.

  • Recursive definition of tokens
  • Other cool stuff

License

Lesk is released under MIT License.

About

A small and easy to use .NET library for building lexical analyzers at run-time

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages