Augmented Reality Robot Interaction
June 2017 - August 2017
I completed this project within a summer, despite not knowing much about Augmented Reality (AR) at the start of this internship. There were more than a few bumps and challenges along the way but that's what made this project even more meaningful and fun!
​
Check out the video below to see the AR experience that I developed!
Building the Physical Robot
I spent the few days of my internship building an adaptation of the Bioloid Humanoid Robot from the company Robotis. In addition to following the assembly instructions, I installed an additional set of servo motors and grippers for the robot's hands.
 |
|---|
 |
 |
 Mechanics Analysis |
|---|
 Editing joint definition |
 Editing regeneration value in Simulation |
 Calibration of Center of Gravity |
Digital Twin CADÂ Assembly and Analysis
I calibrated the digital twin in Creo Parametric and ran mechanics analysis for 900 different standing positions for four different weights.
Data Analysis and Animation
After collecting data from the Creo mechanics simulations, I filtered the data for optimal standing joint angles for each weight. In Creo Illustrate, I generated animations that mimic the motion of the weightlifting robot for each optimal standing posture.
 |
|---|
 |
 |
 |
 Screen Shot 2017-08-16 at 12.34.01 AM.png |
 |
 Animation step example |
 |
|---|
 |
 |
 |
AR Testing and IoT communication
Although I was familiar with Solidworks, I had no previous experience with Creo Parametric (CAD), ThingWorx (IoT platform), and ThingWorx Studio (AR platform). It did not take me long to get familiarized with both softwares. Once we were able to debug the code allowing for bluetooth connection between the physical robot and a computer, we implemented services and properties of the weightlifter in ThingWorx to use later in ThingWorx Studio. I also populated a home mashup with the joint angles for the optimal standing positions for each weight.
User Experience Design
The final part of this project was designing the AR interface in which the user interacts with both the digital twin and the physical robot. Some features include:
Weight Gauge: displays the number that the physical robot estimated after evaluating the weight
Red ball: indicates the robot's center of gravity - this ball also moves during the animated motion of standing up
Grey buttons: actions performed by the physical robot
Green buttons: actions performed by the digital twin
Toggle button (top right): allows the user to switch between an overlapping view of the robots and a side by side comparison view of the robots
 |
|---|
 |
 |
 |
 |
|---|
 |
 |
Final Presentation at PTC Haifa Office
I presented my work to PTC engineers. My project was presented to the VP EVC of ThingWorx Studio, and will be demonstrated at PTC headquarters in Needham, MA and potentially at the AR in Action Event at the MIT Media Lab.
To discuss the projects I have worked on, contact me today.