Sarah B, A Mira C and Eric F

)

Reinforcement learning system as a submissive-dominant controller of VR experience-space.

• “Branching” of the state based on user head movement
• Machine trying to maximize the time the user stay in the VR environment

Deep Q Learning https://github.com/keon/deep-q-learning

Reinforcement learning in python: Theano + Keras

Source code originally written for playing CartPole game on OpenAI Gym

https://gym.openai.com/envs/CartPole-v1

“A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track”

We replaced CartPole game with our Unity VR system

• Form: head direction (coordinate in 3d-space), head movement (acceleration data)
• Dataset: collected from a human subject - 50+ trials

• When a human subject is bored, they push the “END” button.
• This resets the scene, the machine will retry to maximize the rewards

by collecting this data array:

• state 1 (VR gear readings)
• action (determined by the machine)
• state 2 (VR gear readings)
• rewards: for a unit time stay within the VR system, the system gets +1 reward; when terminated by the user, the system gets -10 penalty

The model selected 1 action from our discrete action space (306442=5760 possible actions) which determined each object's:

• visibility,
• position,
• size (scale), and
• angular orientation

)

Best record?

• Sarah: sound + training + documentation
• Alice: python(keras) + Unity + documentation
• Eric: Unity + VR + visuals