MoveToCode is an open source mixed reality visual programming language (VPL) created by the Interaction Lab led by Maja J Matric. The VPL is used to study student kinesthetic curiostiy. The VPL can be thought of similar to Scratch in 3D.

MoveToCode is built for the Microsoft Hololens 2 using the Mixed Reality Toolkit. Currently it is set up to work with the Mayfield Kuri robot but can be modified for any ROS supported robot via ROS# and rosbridge.

Here is a video of MoveToCode + a robot tutor.

Currently the study commit for all papers: 0765f1d1ab46373f1bc1be90d9d5b07fafc2a533

• Arrays
• CodeBlock to Python Text
• Robot speech via Amazon Polly
• Adding virtual robot arms

• Singleton class that reads/interprets connected codeblocks line-by-line from top to bottom, beginning at the "Start" codeblock.
• Utilizes an instruction stack to ensure that the reaching the end of a codeblock control flow statement (if, while) does not end the program unless the stack is empty.

• Singleton class that holds a list of buttons which spawn variables created by the user.
• Visualizes changes in the values stored in variables as interpreter reads/runs code.

• Abstract class from which all visual/block coding components are derived.
• Of all the classes that inherit from IArgument, only the bottom classes/leaf nodes are non-abstract
• Child classes - main partition for code blocks based on function
  • IDataType: All data types (primitive and derived) inherit from this class
    • int, float, string, bool, char, array
  • Instruction: All instructions inherit from this class
    • print, set variable value, conditionals, arithmetic

Versions of packages are pushed directly to the repository to avoid versioning issues, licensing permitting.

• ROS#; dwhit UWP branch (Apache 2.0 License)
  • Newtonsoft.Json (MIT License)
  • Newtonsoft.Json.Bson (MIT License)
  • websocket-sharp (MIT License)
• Mixed Reality Toolkit v2.1(MIT License)
• ProBuilder (Unity Companion License)
• TextMeshPro (Unity Companion License)
• NugetForUnity (MIT License)

The following are currently tested versions of required software. Higher versions of each software may work but have not been tested.

• Unity 3D v2018.4.14f
• .NET Framework 4.6 and Visual Studio 2017
• Built for Hololens 2

MoveToCode is open source under the MIT License

For any questions, you can contact Thomas Groechel at groechel@usc.edu. Feel free to fork for your own projects and/or submit pull requests.