ESE 6500 | LEARNING IN ROBOTICS
This course covered the mathematical fundamentals and applications of machine learning algorithms to mobile robotics - specifically for signal filtering and ground robot localization.
In the course I implemented several daunting algorithms from scratch: Extended Kalman Filtering and Unscented Kalman Filtering for sensor fusion & quadrotor state estimation, and Simultaneous Localization and Mapping (SLAM) using a particle filter on LiDAR data collected from a bipedal robot. In addition, I explored Policy Iteration algorithms, LQR/iLQR control, and Q-Learning and other RL approaches.
Example SLAM result:
Final Project: Indoor-Outdoor Localization for Ground Robots Using Factor Graphs
Read the report and look through our solution at our GitHub repo
DRONE NAVIGATION LIBRARY
MEAM 6200 SEMESTER PROJECT
MEAM 620: Advanced Robotics was ultimately focused on techniques for quadrotor control, path planning, trajectory generation, sensor filtering, and pose estimation/odometry. Each project built upon the previous, and by the end of the semester, I had created a full drone navigation library in Python.
During the first half of the semester, I implemented the control, planning, and trajectory generation aspects of the drone library. There were many opportunities to implement custom, creative solutions to the problems posed.
For the controller, I chose a geometric nonlinear controller with decoupled position and attitude control loops. This intuitively encourages the drone to orient it's vertical axis toward the direction of desired acceleration (and in fact enables more acrobatic maneuvers), and the decoupled outer (position) and inner (attitude) control loops enable frequency separation in control response, as a much quicker reaction is required from the drone's roll/pitch/yaw.
I chose the efficient A* graph search algorithm to populate the environment with dense waypoints during the early planning steps. Then, I implemented a dual pruning approach to cleverly downselect the final waypoints. Then, I generated a minimum snap (4th derivative) polynomial trajectory and imposed maximal smoothness criterion to achieve a fluid and efficient trajectory. I also developed a progressive scaling function attempting to optimize the time allocation between waypoints to minimize large excursions from the linear path and prevent tight cornering requirements.
These features are explained further in the following report:
The remainder of the course focused on quadrotor odometry. Ultimately, I implemented visual-inertial odometry (VIO) by using an Error State Kalman Filter to optimally blend between stereo vision and IMU data. This was evaluated in simulation (the bottom 2 videos in this section employ VIO in a simulated environment in which the drone is casting rays from it's virtual camera to identify environment obstacles), where the quadrotor used only this form of odometry to update its state.
The other video just below is from the physical lab where I tested my controller + planner + trajectory generator. Environment obstacles were given as priors and a VICON system was used for localization of the Crazyflie 2.0 microdrone.
REINFORCEMENT LEARNING FOR QUADROTORS
CIS 5190 FINAL PROJECT
CIS 519: Applied Machine Learning gave a broad overview of machine learning methods including decision trees, Bayesian methods, ensemble learning, supervised learning, unsupervised learning, clustering/dimensionality reduction, neural networks & CNNs, NLP, and some RL.
Despite the only brief coverage of RL in lecture, myself and two friends also in the ROBO MSE program became interested in Reinforcement Learning for robotics applications. Thus, we chose to explore RL in our final project - settling upon a comparative study of modern RL applications and reward functions for quadrotor navigation. We utilized a python-based repository developed by the University of Toronto which integrates a physics engine with an OpenAI Gym environement which allowed us to train flexibly using several algorithms (PPO, SAC, DDPG, etc.), various custom reward schemes, action space types, and obstacles.
Abstract:
Reinforcement learning (RL) allows for control of complex systems without knowledge of the system’s
inherent dynamics and predefined control structure and eliminates the need for vast amounts of data collection. Current approaches to reinforcement learning on drone control focus on the technical feasibility of a single algorithm for a task, such as inclined drone landing with PPO or whether a drone can recover from perturbations. As our primary contribution in this project, we analyze the performance of modern RL algorithms such as Proximal Policy Optimization (PPO) and Soft Actor Critic (SAC) on a mini-drone, the CrazyFlie 2.X quadcoptor. Additionally, the algorithms are modified with various custom rewards functions that are implemented to analyze the robustness of the algorithms. The work in this project highlights which algorithms and rewards schemes work best for the CrazyFlie 2.X platform for simple trajectories and obstacle avoidance. It was observed that SAC works best out of all algorithms with a Euclidean based rewards scheme and a small living cost.
inherent dynamics and predefined control structure and eliminates the need for vast amounts of data collection. Current approaches to reinforcement learning on drone control focus on the technical feasibility of a single algorithm for a task, such as inclined drone landing with PPO or whether a drone can recover from perturbations. As our primary contribution in this project, we analyze the performance of modern RL algorithms such as Proximal Policy Optimization (PPO) and Soft Actor Critic (SAC) on a mini-drone, the CrazyFlie 2.X quadcoptor. Additionally, the algorithms are modified with various custom rewards functions that are implemented to analyze the robustness of the algorithms. The work in this project highlights which algorithms and rewards schemes work best for the CrazyFlie 2.X platform for simple trajectories and obstacle avoidance. It was observed that SAC works best out of all algorithms with a Euclidean based rewards scheme and a small living cost.
BEYOND THE BINARY
WEARABLE ELECTROMECHANICAL ART
From left to right in the photo above are architecture student Beikel Rivas, choir soloist Lucas DeJesus, and myself doing a final fitting of the wearable several hours before the performance
This project was the culmination of a semester in a project-based collaborative program involving the Penn School of Engineering and Applied Sciences, GRASP Lab, the Weitzman School of Design, and the Mendelssohn Chorus of Philadelphia. Cross-listed as MEAM 516 - Adv Mechatronics Reactive Spaces and Architecture elective ARCH746, the course partnered engineers and designers to develop pieces of electromechanical art. In the final months of the program, teams worked to develop pieces for the Mendelssohn Chorus' latest performance - a novel composition from Andrea Clearfield and librettist Ellen Frankel about artificial intelligence: “Beyond the Binary: A Meditation on Humans and Machines.” My work was worn by a soloist and incorporated into his performance before a several-hundred member audience.
Official Course Description:
"This course combines performance art and advanced mechatronics concepts that include the design and implementation of large-scale actuation, advanced sensing, actuation and control. This course pairs design school and engineering students to form interdisciplinary teams that together design and build electro-mechanical reactive spaces and scenic/architectural elements in the context of the performing arts... Engineering students gain design sensibilities and advanced mechatronics in the form of networked embedded processing and protocols for large scale actuation and sensing"
My team's final wearable features two back-mounted, joystick-controlled robotic continuum arms each driven by a pair of antagonistic tendons which move in an organic way behind the wearer. Along the right shoulder and down the arm, a silicone skin rests above two balloons which inflate and deflate as if breathing - driven by a series of small air pumps and solenoid vales tucked away in backpiece. The piece is meant to be a statement on robotic augmentation in both a human's life and more broadly in the workforce. The soloist's "character" was a harbinger of the capitalist potential of robotics to supplant human beings altogether.
Through several Covid-related setbacks in the small team of three comprising myself, EE student Martin Alijaj, and design student Beikel Rivas, I took the lead on developing prototypes and spent nearly two weeks beyond the end of the semester preparing the piece for the performance in May. My responsibilities included all mechanical design and fabrication, design of the motor driver and pnuematic circuits, programming and tuning of behaviors, and more.
This project was so creatively fulfilling and rewarding. I was able to interface with many people across a wide array of disciplines to contribute to a production with a myriad of moving parts. I really appreciated the creative freedom and technical challenges that helped me to grow as an engineer and designer.
MOIRé LAMP
ELECTROMECHANCIAL ART WARMUP
This was an earlier project in the same MEAM 516 Reactive Spaces course for which I created the wearable art piece. This was in partnership with architecture student Miguel Matos and fellow engineering student Tim Kulesza. The idea was to create a lamp with an interesting rotating Moiré pattern and a set of interactive functionalities.