Skip to content

This tutorial series provides step-by-step instructions for how to perform human pose estimation in Unity with the Barracuda inference library.

License

Notifications You must be signed in to change notification settings

cj-mills/Barracuda-PoseNet-Tutorial

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Barracuda PoseNet Tutorial 2nd Edition

Tutorial Links

  • Part 1: Install the Barracuda package in a Unity project and import the required video files and PoseNet models.
  • Part 2: Set up a video player and webcam in Unity.
  • Part 3: Implement the preprocessing steps for the MobileNet and ResNet PoseNet models.
  • Part 4: Load, modify, and execute the PoseNet models.
  • Part 5: Implement the post-processing steps for single pose estimation with PoseNet.
  • Part 6: Implement the post-processing steps for multi-pose estimation with PoseNet.
  • Part 7: Create pose skeletons and manipulate them using output from a PoseNet model.
  • WebGL: Modify the Barracuda PoseNet project to run in a browser using WebGL.

Single Pose Estimation

single-pose-test.mp4

Multi-Pose Estimation

multi-pose-test.mp4

Update - 3/10/22

Added a new branch and tutorial for adapting the project to run in a browser using WebGL.

You can try out a live demo at the link below.


Update - 12/7/21

Added a new branch that uses Barracuda version 2.3.1. Version 2.3.1 contains some improvements over version 2.1.0 used in the main branch that may be especially relevant when building for non-Windows platforms.

It has support for creating a tensor from a RenderTexture even when compute shaders or GPUs are not available.

It also now has a Pixel Shader worker. This allows models to run on GPU without requiring support for compute shaders. It has limited support at the moment, but seems to fully support the PoseNet models used in this project. My testing shows that it is much more performant than using the CSharpBurst worker type for both the MobileNet and ResNet50 versions of the model. Although, it is still not as performant as using compute shaders when those are available.


Version 1

GitHub Repository

About

This tutorial series provides step-by-step instructions for how to perform human pose estimation in Unity with the Barracuda inference library.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages