AntlrVSIX is an extension for Visual Studio 2019 and Visual Studio Code to support editing and refactoring of Antlr v4 grammars. It is implemented using Microsoft's Language Server Protocol (LSP) client and server APIs, Antlr, Antlr4BuildTasks, and a number of other tools. Most of the extension is implemented in C#, while the client for VS Code is in Typescript. Supported are colorized tagging, hover, go to def, find all refs, replace, command completion, reformat, and go to visitor/listener.

Instead of providing railroad diagrams and ATN graphs, this extension focuses on features useful for maintaining grammars, i.e., refactoring grammars in order to make them cleaner, more readable, and more efficient.

The source code for the extension is open source, free of charge, and free of ads. For the latest developments on the extension, check out my blog.

• Install Java tool chain, either OpenJDK or Oracle JDK SE.

• Downloaded the Java-based Antlr tool chain. Complete ANTLR 4.8 Java binaries jar.

• Set the environment variable "JAVA_HOME" to the directory of the java installation. See this for some instructions on how to do this on Windows.

• Set the environment variable "Antlr4ToolPath" to the path of the downloaded Antlr jar file.

• Do not include the generated .cs Antlr parser files in the CSPROJ file for your program. The generated parser code is placed in the build temp output directory and automatically included.

• Make sure you do not have a version skew between the Java Antlr tool and the runtime versions.

Please verify that you have these variables set up as expected. Try "$JAVA_HOME/bin/java.exe" -jar "$Antlr4ToolPath" from a Git Bash or "%JAVA_HOME%\bin\java.exe" -jar "%Antlr4ToolPath%" from a Cmd.exe. That should execute the Antlr tool and print out the options expected for the command. If it doesn't work, adjust JAVA_HOME and Antlr4ToolPath. JAVA_HOME should be the full path of the JDK; Antlr4ToolPath should be the full path of the Antlr tool jar file. If you look at the generated .csproj file for the Antlr Console program generated, you should see what it defaults if they aren't set.

For information on how to use AntlrVSIX, see the User Guide, which is geared specifically for Antlr programs. Note--all this needs updating due to the new slimmed-down UI, which is now implemented using the Visual Studio LSP Client code.

• I am in the process of reviewing the collection of Java grammars in Antlr's grammar github repository. Currently, Antlrvsix uses the slower Java9 grammar. The faster, "official" Java grammar is here.

• Support for VS2017 and older editions has been removed. If you are interested in those, you can try using an older version of the extension.

• The grammar used is the standard Antlr4 grammar in the examples: https://github.com/antlr/grammars-v4/tree/master/antlr4.

• If you want to make modifications for yourself, you should reset your Experimental Hive for Visual Studio. To do that, Microsoft recommends using CreateExpInstance.exe. Unfortunately, I've found CreateExpInstance doesn't always work because it copies from previous hives stored under the AppData directory. It is often easier to just recursively delete all directories ...\AppData\Local\Microsoft\VisualStudio\16.0_*.

• Use Visual Studio 2019 to build the extension. Note, the extension builds in two steps: (1) open VS on the solution and perform a build, then quit. (2) reopen the solution in VS and perform a build. VS computes the objects to pack into the .vsix file at the time it opens the project, not at build time (a really bad thing MS). If your debug version of the .vssix is 5M in size, it did not compute that the server executable needs to be added into the .vsix. Quit VS, then reopen. Do not do "rebuild", only "build".

• Add in Piggy for grammar rewriting and checking. The addition of Piggy will allow for rules to transform the grammar e.g., warning/removal of LHS predicates in lexer, replacing factored grammar rules with Kleene star rules (which result in faster parsers).

• Add in more refactorings: a) Sort rules alphabetically, DFS or BFS traversals. b) Separate/combine grammars.
• Bug fixes.

• Add in a few refactoring transformations (remove useless parser productions, convert parser string literals to lexer token symbols, move start rule to top of grammar). Note: Lexer rules are prioritized, so transformations on these types may not be totally correct at the moment.
• Fix goto visitor/listener.
• Add stability fixes.

• Fixed "Find references" of grammar symbols when opening only lexer grammar file.
• Fixed Format Document.
• Fixed Go To Listener/Visitor. Caveat: only works for C# and you must save solution before using.
• Templates updated, Antlr4BuildTasks 2.2.

• Re-added About Box, Options Box, and next/previous grammar symbol.
• Options are now contained in ~/.antlrvsixrc, a JSON file.

• Re-added colorized tagging of grammar. When I switch to the LSP implementation, this functionality was lost because it's not directly supported by the LSP Client/Server API that Microsoft provides. Microsoft says to implement this functionality using TextMate files, but that is not how it should be done--it duplicates the purpose of the LSP server, and it's hard to get right. Instead, it's implemented with a Visual Studio ITagger<> and a custom message to the LSP server.

• Restructuring the code as a Language Server Protocol client/server implementation with extensions for VS 2019 (IDE) and VS Code.

• Templates for C# and C++. Note, Antlr 4.8 currently does not have a C++ pre-built binary release for Windows. You will need to build the runtime and update the generated .vcxproj file with path information. The template will generate a project that is expecting Debug Static and x64 target.

• These are the LSP features currently implemented. Note, Midrosoft.VisualStudio.LanguageServer.Protocol version 16.4.30 does not implement LSP version 3.14, rather something around version 3.6. What is missing is color tagging. I'm not sure how to deal with this other than pitch the inferior API, then use OmniSharp's API, or as usual write everything myself.

• Computing the completion symbols is not complete. It only gives token types for completion, not the actual symbols that could be inserted.

• Fixing #26.

• Fixing solution loading crash.
• Removing parse tree print code from Python--it was used for debugging and should not have been there.
• Stubbing out Python and Rust targets because they could interfere with (currently) much better extensions. Will add them back once they are up to snuff.

A shout out to all those folks using this extension and have opted-in to the automatic reporting!!

• Restructuring the code for options.
• Adding in reporting of caught exceptions to server.
• Some changes for the stability for Java soruce.

• Updated symbol table with new classes to represent files, directories, and search paths. This will help support for better modeling of Java's ClassPath, Antlr's imports, etc., so that scopes can be cleared out quickly and easily when the sources have changed.
• Fixing issues in stability with Antlr and Java files.
• Fixing kaby76#23
• Fixing tool tips and highlighting for Java.
• Updated symbol table to allow ambigous code editing.
• Changing About and Options boxes to non-modal.

• Fixing stability issues with Antlr and Java files.

• Major changes to architecture, focusing on separation of GUI from a backend that works like Language Server Protocol (LSP).
• In the GUI, tagging was improved in quality and speed.
• In the backend, a symbol table was added to represent all symbols in a project and solution.
• For Antlr grammars, imports and tokenVocab are now used to determine the scope of the symbol table.
• For Antlr grammars, Intellisense pop-ups now give the rule definition and location of the file defining the symbol.
• No specific improvements were made for Python, Rust, and Java support, in favor of focusing on the improved overall design and implementation of the extension.

• Supports Java, Python, Rust, Antlr in various stages. Description of languages abstracted into a "Grammar Description".

• The extension will support VS 2019 and VS 2017.

• A menu for the extension will be added to a submenu under Extensions. The functionality provided will duplicate that in context menus.

• The source code build files will be updated and migrated to the most recent version of .csproj format that is compatible with VS extensions. Unfortunately, updating to the latest (version 16) is not possible.

• With my Antlr4BuildTasks NuGet package, .g4 files can be automatically compiled to .cs input within VS 2019 without having to manually run the Antlr4 Java tool on the command line. Building of the extension itself will be upgraded to use the Antlr4BuildTasks package.

• Listener and Visitor classes are generated for a grammar with a right-click menu operation. For Listeners, there are two methods associated with a nonterminal. Depress the control key to select the Exit method, otherwise it will select the Enter method.

• An options menu is provided to turn on incremental parsing. By default, incremental

• parsing is off because it is very slow.

• The extension is now both VS 2017 and 2015 compatible.

• The results windows of Antlr Find All References is now "Antlr Find Results".

• Color selection through VS Options/Environment/Fonts and Colors. Look for "Antlr ..." named items.

• Bug fixes with Context Menu entries for AntlrVSIX. AntlrVSIX commands are now only visible when cursor positioned at an Antlr symbol in the grammar. This fixes the segv's when selecting AntlrVSIX commands in non-Antlr files.

Any questions, email me at ken.domino gmail.com