Automatic multimodal acquisition and understanding of social signals is an essential building block for natural and effective human-machine collaboration and communication. OpenSense is a platform for real-time multimodal acquisition and recognition of social signals. OpenSense enables precisely synchronized and coordinated acquisition and processing of human behavioral signals. Powered by the Microsoft's Platform for Situated Intelligence, OpenSense supports a range of sensor devices and machine learning tools and encourages developers to add new components to the system through straightforward mechanisms for component integration. This platform also offers an intuitive graphical user interface to build application pipelines from existing components. OpenSense is freely available for academic research.

Please refer to files under the Documents sub-directory of this repository.

You can file an issue to get support.

OpenSense is available under an USC Research License.

3rd-party components may have their respective licenses. Please contact their respective authors to obtain licenses.

We are moving to one of BSD licenses, we will switch the license file after we get approved.

OpenSense is freely available for academic research. Please cite OpenSense in your publications by citing the following paper

@inproceedings{10.1145/3382507.3418832,
  author = {Stefanov, Kalin and Huang, Baiyu and Li, Zongjian and Soleymani, Mohammad},
  title = {OpenSense: A Platform for Multimodal Data Acquisition and Behavior Perception},
  booktitle = {Proceedings of the 2020 International Conference on Multimodal Interaction},
  publisher = {Association for Computing Machinery},
  pages = {660–664},
  numpages = {5},
  year = {2020}
}