Simulation can significantly speed up the development of new robots. Simulation can for example be used to test hardware that is physically not available yet. It can also be used to generate sensor data that can be used during development. In this project we primarily focus on generating sensor data.
Eventually, we want to test agricultural robots in simulation. The robots need to recognize plants and pull them out of the ground or harvest them. In this project we focus on the first aspect: having a realistic simulation of plants that move and look realistically. Ideally, these simulations can run in real-time with 100s of plants. In parallel, we need to explore how to combine the simulation in Unity with the robotic simulation in ROS and Gazebo.
The Realistic Plant Simulation (RPS) application is built as a suite of editor tools within Unity. These tools offer the possibility to the ROS comunity to test their code within a virtualized environment inside of Unity. This is less time consuming and more graphically realistic than the current ways of testing their code using programs like Gazebo or going out to a field with an actual robot.
For a full manual to our project with instruction on how to set it up and how to use it. Including links to external parts of the project that aren't on this repository. Please refer to the user manual: User Manual
To address all of these issues an application needs to be created that can implement all of these requirements and possibly more in the future. This application needs to contain at least the following features:
To correctly simulate agricultural robots there needs to be a virtual environment for it to be simulated in. This environment will exist in a virtual farmland field with plants. The field and plants will both be procedurally generated at editor time. A virtual robot that can move through the environment and be controlled by outside scripts/connections To do something with the virtual environment there needs to be a robot that can move around the scene and get visual data from it. This robot must also be able to be controlled by external scripts or connections such as ROS
The unity project needs to be able to connect to ROS to be able to interface with actual robot software. It needs to have two way communication to achieve the desired result. Data must be able to be sent and received over the different types of methods ROS supplies. It must also be possible for the ROS connection to receive video data from Unity. This means color and depth data from 1 or more Unity cameras
It is important that the application is modular so many different types of robots and connections can be simulated. Interaction The robot needs to be able to interact with the scene. The interaction can mean leaving tracks behind in the sand, or interacting with the plants to cut them etc. This interaction is important for the realism and usability of the project.
Realistic simulation, plants, crops, agriculture, robotic simulation, Unity, ROS, Gazebo
During the process you will be guided by experts from Saxion and Aeres hogeschool. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).