This week I spent a lot of time fixing stability bugs with the ORCAS system and gathering data with the reinforcement learning system. With a learning system, stability is extremely important as learning can takes days or even weeks which means that any small issue can completely bring down the whol...

This week the ORCAS team finished the communication layer between Gazebo and the reinforcement learning algorithm. With this, we were able to start training robots. We created a parameterized four legged robot and starting training a dense neural network (DNN) controller for it. Video In this sim...

The RoCo team is planning to submit two papers with deadlines in early April. The plan is to submit a paper on an interface for the co-design of mechanical and electro-functional structures to UIST 2018 and a paper on a unified design environment for multiple fabrication output forms for SCF 2018...

A recent blog post by Christian outlines an algorithm that we formulated for the co-optimization of structure and control of robots. This algorithm seeks to more tightly couple the co-optimization as compared to the algorithm described in [1]. Let's call the algorithm we propose Algorithm 1. Algo...

Pictured: A simulation of the learning algorithm in a CartPole-v0 environment provided by OpenAI gym. This last week I started playing around with Tensorflow for reinforcement learning. I was able to implement a policy gradient with parameter-based exploration(PGPE) algorithm for learning a policy...

This last weekend, I was finally able to fix the bug that had been eluding me for the past two weeks. Click here to read my previous blog post about this bug. My approach to fixing the bug was to painstakingly keeping track of the reference states of the different variables after each deserializ...

I spent the majority of this past week debugging a single issue in RoCo. We noticed that when we constrained subcomponent parameters in RoCo and rebuilt the component, the drawing would not reflect the constraint. The representation of the faces in RoCo are parameterized with Variables that define t...

My main progress this week was getting hierarchical composition to finally work again. There were issues in the old robot compiler backend that seemed to have sorted themselves out when we rewrote it. The main issue I had to deal with to get hierarchical composition working again was fixing yaml ser...

This week I worked on fixing and documenting bugs in RoCo to prepare it for an upcoming handover to a new team. I will be moving on to a new project later this quarter so in addition to working on RoCo, I performed a literature search for the current state of the art of reinforcement learning and le...

At the end of last year I had a constraint solving solution that did not scale well with complexity. It quickly became obvious that it would not suffice. Therefore, I began this year by tackling that problem once more. After trying a few more methods, we decided that the best solution would be to si...