Course homepage for CS287 at Berkeley
https://people.eecs.berkeley.edu/~pabbeel/cs287-fa19/
CS 287: Advanced Robotics, Fall 2019
1995-01-01
None
<div><div>
<tbody><tr>
<td> Lecture </td>
<td> Slides | YouTube</td>
<td> Topic </td>
<td> Optional Readings </td>
</tr>
<tr>
<td> Th Aug 29 </td>
<td> <a href="slides/Lecture1-Intro.pdf">slides</a>
| <a href="https://youtu.be/xWPViQ6LI-Q">youtube</a></td>
<td> Course Introduction</td>
<td></td>
</tr>
<tr>
<td> Tu Sep 3</td>
<td> <a href="slides/Lec2-mdps-exact-methods.pdf">slides</a>
| <a href="https://youtu.be/H_9YQBN45fo">youtube</a></td>
<td> MDPs: Exact Methods</td>
<td><a href="http://incompleteideas.net/book/the-book.html">Sutton and Barto, Reinforcement Learning</a>, Chapters 3 and 4
</td>
</tr>
<tr>
<td> Th Sep 5 </td>
<td> <a href="slides/Lec3-discretization-of-continuous-state-space-MDPs.pdf">slides</a>
| <a href="https://youtu.be/mJlAfKc4990">youtube</a></td>
<td> Discretization of Continuous State Space MDPs </td>
<td> 
<a href="optreadings/MooreAtkeson-1993-prioritized-sweeping.pdf">Moore and Atkeson, 1993</a>,
<a href="optreadings/MunosMoore-MLJ2001.pdf">Munos and Moore, MLJ 2001</a>,
</td>
</tr>
<tr>
<td> Tu Sep 10 </td>
<td><a href="slides/Lec4-function-approximation.pdf">slides</a>
| <a href="https://youtu.be/I0LykEdXDPk">youtube</a></td>
<td>Function Approximation / Feature-based Representations</td>
<td> 
<a href="optreadings/ChowTsitsiklis-1991.pdf">Chow and Tsitsiklis, 1991</a>,
<a href="optreadings/Gordon-1995.pdf">Gordon, 1995</a>,
<a href="optreadings/TsitsiklisVanRoy-1996.pdf">Tsitsiklis and Van Roy, 1996</a>,
Kushner and Dupuis, 1992/2001
</td>
</tr>
<tr>
<td> Th Sep 12 </td>
<td> <a href="slides/Lec5-LQR.pdf">slides</a> 
| <a href="https://youtu.be/S5LavPCJ5vw">youtube</a></td>
<td> LQR, iterative LQR / Differential Dynamic Programming </td>
<td> </td>
</tr>
<tr>
<td> Tu Sep 17 </td>
<td> <a href="slides/Lec6-unconstrained-optimization.pdf">slides</a>
| <a href="https://youtu.be/77dOsZbAGgU">youtube</a></td>
<td> Unconstrained Optimization </td>
<td> <a href="http://www.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf">Boyd and Vandenberghe, Chapters 9-11</a> </td>
</tr>
<tr> 
<td> Th Sep 19 </td>
<td> <a href="slides/Lec7-constrained-optimization.pdf">slides</a>
| <a href="https://youtu.be/gFewyKcDFXI">youtube</a></td>
<td> Constrained Optimization</td>
<td> <a href="https://www.google.com/search?q=nocedal+and+wright+numerical+optimization">Nocedal and Wright, Chapter 18</a></td>
</tr>
<tr>
<td> Tu Sep 24 </td>
<td></td>
<td> No Lecture </td>
<td></td>
</tr>
<tr>
<td> Th Sep 26 </td>
<td> <a href="slides/Lec8-optimization-for-optimal-control.pdf">slides</a> 
| <a href="https://youtu.be/ha60pQdepMY">youtube</a></td>
<td> Optimization-based Control: Collocation, Shooting, MPC (basics/foundations)</td>
<td> </td>
</tr>
<tr>
<td> Tu Oct 1</td>
<td> <a href="slides/Lec9-direct-collocation-Mordatch.pdf">slides</a>
| <a href="https://youtu.be/pBCVQbZtv78">youtube</a></td>
<td> Optimization-based Control: Collocation, Shooting, MPC, Contact-Invariant Optimization (advanced) -- Igor Mordatch </td>
<td></td>
</tr>
<tr>
<td> Th Oct 3</td>
<td> <a href="slides/Lec10-motion-planning.pdf">slides</a>
| <a href="https://youtu.be/ZDuoQRutcfk">youtube</a></td>
<td>Motion Planning: RRT, PRM, Trajopt, 3-d poses</td>
<td><a href="http://planning.cs.uiuc.edu/">Steven M. Lavalle, Motion Planning</a>, Chapters 5, 14, <a href="optreadings/rrtstar.pdf">RRT*, Karaman and Frazzoli</a>, <a href="optreadings/lqrtrees.pdf">LQR trees, Tedrake</a></td> 
</tr>
<tr> 
<td> Tu Oct 8 </td>
<td> <a href="slides/Lec11-probability-bayes-filters-gaussians.pdf">slides</a>
| <a href="https://youtu.be/xamzdNUN1o0">youtube</a></td>
<td> Probability Review, Bayes Filters, Multivariate Gaussians</td>
<td> </td>
</tr>
<tr>
<td> Thu Oct 10 </td>
<td> <a href="slides/Lec12-KalmanFiltering.pdf">slides</a>
| <a href="https://youtu.be/eCjffhEeQyw">youtube</a></td>
<td> Kalman Filtering, EKF, UKF -- Ignasi Clavera</td>
<td> <a href="optreadings/Sorenson-FromGaussToKalman-1970.pdf">From Gauss to Kalman</a>, <a href="optreadings/JulierUhlmann-UKF.pdf">Julier and Uhlmann, the UKF</a></td>
</tr> 
<tr>
<td> Tu Oct 15 </td>
<td> <a href="slides/Lec13-KalmanSmoother-MAP-ML-EM.pdf">slides</a>
| <a href="https://youtu.be/qHLLMg0Teg4">youtube</a></td>
<td> Smoother, MAP, Maximum Likelihood, EM, KF parameter estimation</td>
<td> </td>
</tr>
<tr>
<td> Th Oct 17 </td>
<td> <a href="slides/Lec14-particle-filters.pdf">slides</a>
| <a href="https://youtu.be/8k--yWn8_ds">youtube</a></td>
<td> Particle Filters -- Wolfram Burgard</td>
<td></td>
</tr>
<tr>
<td> Tu Oct 22 </td>
<td> <a href="slides/Lec15-POMDPs.pdf">slides</a>
| <a href="https://youtu.be/2dNp7QyoF_k">youtube</a></td>
<td> POMDPs </td>
<td></td>
</tr>
<tr>
<td> Th Oct 24 </td>
<td> n/a | n/a</td>
<td> INDUSTRY: Ike (self-driving trucks) Jur van den Berg and/or Sachin Patil</td>
<td> </td> 
</tr>
<tr>
<td> Tu Oct 29 </td>
<td> <a href="slides/Lec17-imitation-learning.pdf">slides</a>
| <a href="https://youtu.be/8uQkk-JFHtA">youtube</a></td>
<td> Imitation Learning -- Laura Smith </td>
<td> </td>
</tr>
<tr>
<td> Th Oct 31 </td>
<td> <a href="slides/Lec18-RL-I-policy-gradients.pdf">slides</a>
| <a href="https://youtu.be/LsKXJl-_kzk">youtube</a></td>
<td> RL1: Policy Gradients</td>
<td> </td>
</tr>
<tr>
<td> Tu Nov 5</td>
<td> <a href="slides/Lec19-RL-II-off-policy.pdf">slides</a>
| <a href="https://youtu.be/QASqaj_HUZw">youtube</a></td>
<td> RL2: Off-policy RL</td>
<td></td>
</tr>
<tr>
<td> Th Nov 7 </td>
<td> <a href="slides/Lec20-RL-III-model-based.pdf">slides</a> 
| <a href="https://youtu.be/Y2XBiUtZo1k">youtube</a></td>
<td> RL3: Model-based RL</td>
<td> </td>
</tr>
<tr>
<td> Tu Nov 12</td>
<td> <a href="slides/Lec21-physics-simulation.pdf">slides</a>
| <a href="https://youtu.be/HRp6DH5M7Co">youtube</a></td>
<td> How do simulators work? </td>
<td> </td>
</tr>
<tr>
<td> Th Nov 14</td>
<td> </td>
<td> no lecture (midterm at night)</td>
<td> </td>
</tr>
<tr>
<td> Tu Nov 19</td>
<td><a href="slides/Lec22-sim2real-JoshTobin.pdf">slides</a>
| <a href="https://youtu.be/ac_W9IgKX2c">youtube</a></td>
<td> Sim2Real -- Josh Tobin</td>
<td> </td>
</tr>
<tr> 
<td>Th Nov 21</td>
<td> n/a | n/a </td>
<td> INDUSTRY: Waymo: Drago Anguelov
</td><td></td>
</tr>
<tr>
<td> Tu Nov 26 </td>
<td> n/a
| <a href="https://youtu.be/Yizyv8MpYfg">youtube</a></td>
<td> INDUSTRY: Skydio (drones): Adam Bry/Hayk Martirosyan </td>
<td> </td>
</tr>
<tr>
<td> Th Nov 28 </td>
<td> </td>
<td> Happy Thanksgiving! </td>
<td> </td>
</tr>
<tr>
<td> Tu Dec 3</td>
<td> n/a
| <a href="https://youtu.be/kr6k_SyKsh0">youtube</a></td>
<td> Backstories behind how some papers were originated and came together</td> 
<td> </td>
</tr>
<tr>
<td> Th Dec 5 </td>
<td> <a href="slides/Lec25-helicopter.pdf">slides</a>
| <a href="https://youtu.be/qKZhtYviqSU">youtube</a></td>
<td> Autonomous Helicopters and Course Wrap-Up</td>
<td> <a href="http://www.cs.berkeley.edu/~pabbeel/papers/AbbeelCoatesNg_IJRR2010.pdf">Abbeel, Coates, Ng, IJRR 2010</a>, <a href="http://heli.stanford.edu/">videos and data</a></td>
</tr>
<tr>
<td> Week of Dec 9-13</td>
<td></td>
<td> Project Presentations</td>
<td> </td>
</tr>
</tbody></div>||||I||||


University of California at Berkeley
Dept of Electrical Engineering & Computer Sciences

CS 287: Advanced Robotics, Fall 2019

Fall 2015 offering (reasonably similar to current year's offering)
Fall 2013 offering (reasonably similar to current year's offering)
Fall 2012 offering (reasonably similar to current year's offering)
Fall 2011 offering (fairly similar to current year's offering)
Fall 2009 offering (not particularly closely matched to current year's offering)


Instructors:
Professor: Pieter Abbeel
TAs: Ignasi Clavera, Laura Smith, Huazhe (Harry) Xu
Lectures: Tuesdays and Thursdays, 11am-12:30pm in 306 Soda Hall
Office Hours:
Posted on Piazza: here
Communication: Piazza is our primary digital channel for communication about the course. To sign up, go to the Piazza website and sign up with "UC Berkeley" and "CS287" for your school and class.
 * Announcements
 * Assignments, Midterm, Final Project Schedule
 * Prerequisites
 * Class goals
 * Grading
 * Assignment policy
 * Syllabus and materials
 * Related materials

Announcements

 * For all future announcements, please sign up on Piazza for CS287 Advanced Robotics.
 * Welcome to the Fall 2019 edition of CS287!

Assignments, Midterm, and Final Project Schedule (subject to change)

 * Due: Wed 9/18     (Out: Thu 9/5)     Problem Set 1: Value Iteration, Discretization-based Optimal Control, Function Approximation Optimal Control (hw1.zip)
 * Due: Wed 10/2     (Out: Thu 9/19)   Problem Set 2: LQR, iLQR, DDP, Feedback Linearization (hw2.zip)
 * Due: Mon 10/21   (Out: Sun 10/6)   Problem Set 3: Convex Optimization, Sequential Convex Programming, Optimization-based Motion Planning and Control (hw3.zip)
 * Due: Mon 11/4     (Out: Sat 10/20) Problem Set 4: Multivariate Gaussians, Kalman Filtering, Maximum Likelihood, EM, Particle Filtering, Belief Space Planning (hw4.zip)
 * Due: Wed 11/6     Final Project Proposal
 * Due: Wed 12/4     (Out: 11/15)   Problem Set 5: Reinforcement Learning (hw5.zip)
 * Thursday 11/14   Midterm (study handout)
 * Week of 12/9-13   Final Project Presentations
 * Due: Sun 12/15     Final Project Reports

Assignment policy

 * Collaboration: Students may discuss assignments. However, each student must code up and write up their solutions independently.
 * Late assignments: Recognizing that students may face unusual circumstances and require some flexibility in the course of the semester, each student will have a total of seven free late (calendar) days to use as s/he sees fit. Late days are counted at the granularity of days: e.g., 3 hours late is one late day. If an assignment is submitted beyond the late-day budget, you will lose 20 (out of 100) points per day over budget (but you cannot go below zero).
 * Drop-the-lowest: Students are strongly encouraged to complete all 5 assignments. Our grading policy will drop your lowest scoring assignment.

Final Project

The final project could be either of the following, where in each case the topic should be closely related to the course:
 * An algorithmic or theoretical contribution that extends the current state of the art.
 * An implementation of a state-of-the-art algorithm using real-world data and/or robots.
Ideally, the project covers interesting new ground and might be the basis for a future conference paper submission or product. You are encouraged to come up with your own project ideas. Course staff can also help brainstorm and shape ideas in office hours.

Logistics and Timeline

 * 1 or 2 students per project.
 * November 6th: Project Proposals due: 1 page.
 * Week of December 9-13: Project Presentations.
 * December 15th: Final Paper due. This should be a 6 page paper, structured like a conference paper. I.e., focus on the problem setting, why it matters and what's interesting/novel about it, your approach, your results, analysis of results, limitations, future directions. Cite and briefly survey prior work as appropriate but don't re-write prior work when not directly relevant to understand your approach.
 * Late days cannot be used for the final project.


Prerequisites

 * Familiarity with mathematical proofs, probability, algorithms, linear algebra; ability to implement algorithmic ideas in code.
 * Key Berkeley courses that can build your background: CS188, EE126, EE127. If your foundations are rusty, we recommend you refreshing them through those courses
 * Priority goes to PhD students. Consent of instructor required for undergraduate masters students.

Class Goals

 * Learn the math and algorithms underneath state-of-the-art robotic systems. The majority of these techniques are heavily based on optimization and probabilistic reasoning --- both areas with wide applicability in modern Artificial Intelligence. An intended side-effect of the course is to generally strengthen your expertise in these areas.
 * Implement, and experiment with, these algorithms.
 * Be able to understand research papers in the field of robotics:
 + Main conferences: ICRA, IROS, RSS, CoRL, ISER, ISRR.
 + Main journals: IJRR, T-RO, Autonomous Robots.
 * Try out some ideas/extensions of your own.
 * Note: the focus of this course is on math and algorithms. We will not study mechanical or electrical design of robots.

Grading

 * Assignments (60%)
 + There are 5 assignments.
 + We'll drop your lowest scoring assignment.
 + Your remaining 4 (highest scoring) assignments will each equally contribute 60/4 of the points.
 * Open-ended final project (30%)
 + 1/5 quality of presentation, 1/5 quality of writing of the final paper, 3/5 quality of the results themselves.
 * Midterm (10%)

Syllabus and materials

Slides are made available as the semester progresses.

Tentative schedule (subject to change):

Lecture Slides | YouTube Topic  Optional Readings  
Th Aug 29 slides | youtube Course Introduction    
Tu Sep 3 slides | youtube MDPs: Exact Methods  Sutton and Barto, Reinforcement Learning, Chapters 3 and 4 
Th Sep 5 slides | youtube Discretization of Continuous State Space MDPs  Moore and Atkeson, 1993, Munos and Moore, MLJ 2001, 
Tu Sep 10 slides | youtube Function Approximation / Feature-based Representations Chow and Tsitsiklis, 1991, Gordon, 1995, Tsitsiklis and Van Roy, 1996, Kushner and Dupuis, 1992/2001
Th Sep 12 slides | youtube LQR, iterative LQR / Differential Dynamic Programming   
Tu Sep 17 slides | youtube Unconstrained Optimization  Boyd and Vandenberghe, Chapters 9-11 
Th Sep 19 slides | youtube Constrained Optimization  Nocedal and Wright, Chapter 18  
Tu Sep 24 No Lecture    
Th Sep 26 slides | youtube Optimization-based Control: Collocation, Shooting, MPC (basics/foundations)   
Tu Oct 1 slides | youtube Optimization-based Control: Collocation, Shooting, MPC, Contact-Invariant Optimization (advanced) -- Igor Mordatch  
Th Oct 3 slides | youtube Motion Planning: RRT, PRM, Trajopt, 3-d poses  Steven M. Lavalle, Motion Planning, Chapters 5, 14, RRT*, Karaman and Frazzoli, LQR trees, Tedrake 
Tu Oct 8 slides | youtube Probability Review, Bayes Filters, Multivariate Gaussians   
Thu Oct 10 slides | youtube Kalman Filtering, EKF, UKF -- Ignasi Clavera  From Gauss to Kalman, Julier and Uhlmann, the UKF 
Tu Oct 15 slides | youtube Smoother, MAP, Maximum Likelihood, EM, KF parameter estimation   
Th Oct 17 slides | youtube Particle Filters -- Wolfram Burgard   
Tu Oct 22 slides | youtube POMDPs    
Th Oct 24 n/a | n/a INDUSTRY: Ike (self-driving trucks) Jur van den Berg and/or Sachin Patil   
Tu Oct 29 slides | youtube Imitation Learning -- Laura Smith   
Th Oct 31 slides | youtube RL1: Policy Gradients    
Tu Nov 5 slides | youtube RL2: Off-policy RL    
Th Nov 7 slides | youtube RL3: Model-based RL    
Tu Nov 12 slides | youtube How do simulators work?    
Th Nov 14 no lecture (midterm at night)   
Tu Nov 19 slides | youtube Sim2Real -- Josh Tobin    
Th Nov 21 n/a | n/a INDUSTRY: Waymo: Drago Anguelov   
Tu Nov 26 n/a | youtube INDUSTRY: Skydio (drones): Adam Bry/Hayk Martirosyan   
Th Nov 28 Happy Thanksgiving!    
Tu Dec 3 n/a | youtube Backstories behind how some papers were originated and came together   
Th Dec 5 slides | youtube Autonomous Helicopters and Course Wrap-Up  Abbeel, Coates, Ng, IJRR 2010, videos and data 
Week of Dec 9-13 Project Presentations    




Related materials

Most relevant book for estimation:
 * Thrun, Burgard, Fox, Probabilistic Robotics
There is no direct match for other lectures, but here are some related resources:
 * If you want to brush up your linear algebra background, I suggest working through this course (video lectures and homeworks available online) at your own pace: Stephen Boyd's EE263: Introduction to Linear Dynamical Systems.
 * If you want to learn more about the linear systems aspects (Kalman filtering, LQR), I recommend Stephen Boyd's EE363: Linear Dynamical Systems.
 * If you want go deeper into the theory of linear systems, I recommend: Claire Tomlin's EE221a: Linear System Theory
 * If you want to learn more about convex optimization, I recommend: Stephen Boyd's EE364a: Convex Optimization I and Stephen Boyd's EE364b: Convex Optimization II. Both of them have all course materials, including lecture videos, available online.
 * For (although draft-status) more about optimal control and motion planning, Russ Tedrake's class: Underactuated Robotics: Learning Planning, and Control for Efficient Agile Machines could give you a somewhat different angle, some complementary ideas, and more examples.
 * A more traditional book on control theory: Astrom and Murray, Feedback Systems
 * A more traditional book on nonlinear control: Slotine and Li, Applied Nonlinear Control.
 * A great introductory text on reinforcement learning: Sutton and Barto, Reinforcement Learning
 * A more mathematically oriented text on reinforcement learning: Bertsekas and Tsitsiklis, Neuro-dynamic programming
 * Earlier offerings of my graduate class had more emphasis on reinforcement learning / approximate dynamic programming than the current offering: CS294-40, Fall 2008 and CS287, Fall 2009
Lecture 
 Slides | YouTube
 Topic 
 Optional Readings 
 Th Aug 29 
 slides
| youtube 
 Course Introduction
 
 Tu Sep 3
 slides
| youtube 
 MDPs: Exact Methods
 Sutton and Barto, Reinforcement Learning, Chapters 3 and 4
 Th Sep 5 
 slides
| youtube 
 Discretization of Continuous State Space MDPs 
 
Moore and Atkeson, 1993,
Munos and Moore, MLJ 2001,
 Tu Sep 10 
 slides
| youtube 
Function Approximation / Feature-based Representations
 
Chow and Tsitsiklis, 1991,
Gordon, 1995,
Tsitsiklis and Van Roy, 1996,
Kushner and Dupuis, 1992/2001
 Th Sep 12 
 slides 
| youtube 
 LQR, iterative LQR / Differential Dynamic Programming 
 
 Tu Sep 17 
 slides
| youtube 
 Unconstrained Optimization 
 Boyd and Vandenberghe, Chapters 9-11 
 
 Th Sep 19 
 slides
| youtube 
 Constrained Optimization
 Nocedal and Wright, Chapter 18 
 Tu Sep 24 
 
 No Lecture 
 
 Th Sep 26 
 slides 
| youtube 
 Optimization-based Control: Collocation, Shooting, MPC (basics/foundations)
 
 Tu Oct 1
 slides
| youtube 
 Optimization-based Control: Collocation, Shooting, MPC, Contact-Invariant Optimization (advanced) -- Igor Mordatch 
 
 Th Oct 3
 slides
| youtube 
Motion Planning: RRT, PRM, Trajopt, 3-d poses
 Steven M. Lavalle, Motion Planning, Chapters 5, 14, RRT*, Karaman and Frazzoli, LQR trees, Tedrake  
 
 Tu Oct 8 
 slides
| youtube 
 Probability Review, Bayes Filters, Multivariate Gaussians
 
 Thu Oct 10 
 slides
| youtube 
 Kalman Filtering, EKF, UKF -- Ignasi Clavera
 From Gauss to Kalman, Julier and Uhlmann, the UKF 
 
 Tu Oct 15 
 slides
| youtube 
 Smoother, MAP, Maximum Likelihood, EM, KF parameter estimation
 
 Th Oct 17 
 slides
| youtube 
 Particle Filters -- Wolfram Burgard
 
 Tu Oct 22 
 slides
| youtube 
 POMDPs 
 
 Th Oct 24 
 n/a | n/a
 INDUSTRY: Ike (self-driving trucks) Jur van den Berg and/or Sachin Patil
  
 Tu Oct 29 
 slides
| youtube 
 Imitation Learning -- Laura Smith 
 
 Th Oct 31 
 slides
| youtube 
 RL1: Policy Gradients
 
 Tu Nov 5
 slides
| youtube 
 RL2: Off-policy RL
 
 Th Nov 7 
 slides 
| youtube 
 RL3: Model-based RL
 
 Tu Nov 12
 slides
| youtube 
 How do simulators work? 
 
 Th Nov 14
 
 no lecture (midterm at night)
 
 Tu Nov 19
 slides
| youtube 
 Sim2Real -- Josh Tobin
 
 
Th Nov 21
 n/a | n/a 
 INDUSTRY: Waymo: Drago Anguelov
  
 Tu Nov 26 
 n/a
| youtube 
 INDUSTRY: Skydio (drones): Adam Bry/Hayk Martirosyan 
 
 Th Nov 28 
 
 Happy Thanksgiving! 
 
 Tu Dec 3
 n/a
| youtube 
 Backstories behind how some papers were originated and came together 
 
 Th Dec 5 
 slides
| youtube 
 Autonomous Helicopters and Course Wrap-Up
 Abbeel, Coates, Ng, IJRR 2010, videos and data 
 Week of Dec 9-13
 
 Project Presentations
CS 287: Advanced Robotics, Fall 2019
https://people.eecs.berkeley.edu/~pabbeel/cs287-fa19/
http://www.fields.utoronto.ca/talks/Deep-Reinforcement-Learning-Robotics
Deep Reinforcement Learning for Robotics
2017-02-07
None
<div><div>
 
<div>
 <p>
 Speaker: 
 </p>
 
 <p>
 Pieter Abbeel, University of California Berkeley </p>
</div>
<div><p>Date and Time: </p><div><p>Thursday, February 16, 2017 - 12:00pm to 1:00pm</p></div></div><div><p>Location: </p><div><p>Fields Institute, Room 230</p></div></div><div><p>Abstract: </p><div><p>Programming robots remains notoriously difficult. Equipping robots with the ability to learn would by-pass the need for what otherwise often ends up being time-consuming task specific programming. This talk will describe recent progress in deep reinforcement learning, in which robots learn through their own trial and error, and resulting capabilities in robotics. I will discuss technical advances in policy gradient methods, in learning to reinforcement learn, and in safe reinforcement learning. And I will discuss resulting robotic capabilities in manipulation, locomotion, and flight.</p>
<p>Pieter Abbeel (Associate Professor, UC Berkeley EECS) works in machine learning and robotics, in particular his research is on making robots learn from people (apprenticeship learning) and how to make robots learn through their own trial and error (reinforcement learning). His robots have learned: advanced helicopter aerobatics, knot-tying, basic assembly, and organizing laundry. He has won various awards, including best paper awards at ICML and ICRA, the Sloan Fellowship, the Air Force Office of Scientific Research Young Investigator Program</p>
<p>(AFOSR-YIP) award, the Office of Naval Research Young Investigator Program (ONR-YIP) award, the DARPA Young Faculty Award (DARPA-YFA), the National Science Foundation Faculty Early Career Development Program Award (NSF-CAREER), the Presidential Early Career Award for Scientists and Engineers (PECASE), the CRA-E Undergraduate Research Faculty Mentoring Award, the MIT TR35, the IEEE Robotics and Automation Society (RAS) Early Career Award, and the Dick Volz Best U.S. Ph.D. Thesis in Robotics and Automation Award.</p>
</div></div> </div>||||I|||| * About Us
 * Sponsors and Partners
 * People and Contacts
 * Calendar
 * FieldsLive
 * Mailing List
 * Donate

 The Fields Institute for
 Research in Mathematical Sciences

 * Activities
 + Thematic and Focus Programs
 o Current and Upcoming
 o Past
 + Fields Medal Symposium
 o Public Opening
 o Scientific Program
 + Mathematical Modelling of COVID-19
 + Mathematics for Public Health
 + Workshops and Conferences
 o Current and Upcoming
 o Past
 + Special Lecture Series
 o Current and Upcoming
 o Past
 + Fields Academy
 + Seminar Series
 o Current
 o Past
 + Courses and Schools
 + Commercial and Industrial Mathematics Program
 + Outreach
 * Centres
 + Centre for Quantitative Analysis and Modelling
 + Centre for Financial Industries
 + Centre for Mathematical Medicine
 + Centre for Mathematics Education
 + NRC-Fields Collaboration Centre
 * Honours and Fellowships
 + Fields Institute Fellows
 + Fellowships and Visitorships
 o Fields Research Fellowship
 o Postdoctoral Fellowships
 o Fields Elliott Distinguished Visitorship
 + Simons Distinguished Visitor
 + Dean's Distinguished Visiting Professorship
 + CAIMS-Fields Industrial Mathematics Prize
 + CRM-Fields-PIMS Prize
 + Margaret Sinclair Memorial Award
 * Publications
 + Annual Reports and Newsletters
 + Communications and Monographs
 + The Fields Institute Turns Twenty-Five
 + Fields Mathematics Education Journal
 + Mathematics-in-Industry Case Studies Journal
 * Resources
 + Proposals and Applications
 + Research and Training Opportunities
 o Fields Research Exchange Program
 o Other Opportunities
 + For Program Organizers
 + For Event Organizers
 + For Speakers
 + For Visitors
 + Ontario Mathematics Calendar
 Home

 Deep Reinforcement Learning for Robotics

 Speaker:
 Pieter Abbeel, University of California Berkeley
 Date and Time:
 Thursday, February 16, 2017 - 12:00pm to 1:00pm
 Location:
 Fields Institute, Room 230
 Abstract:

 Programming robots remains notoriously difficult. Equipping robots with the ability to learn would by-pass the need for what otherwise often ends up being time-consuming task specific programming. This talk will describe recent progress in deep reinforcement learning, in which robots learn through their own trial and error, and resulting capabilities in robotics. I will discuss technical advances in policy gradient methods, in learning to reinforcement learn, and in safe reinforcement learning. And I will discuss resulting robotic capabilities in manipulation, locomotion, and flight.

 Pieter Abbeel (Associate Professor, UC Berkeley EECS) works in machine learning and robotics, in particular his research is on making robots learn from people (apprenticeship learning) and how to make robots learn through their own trial and error (reinforcement learning). His robots have learned: advanced helicopter aerobatics, knot-tying, basic assembly, and organizing laundry. He has won various awards, including best paper awards at ICML and ICRA, the Sloan Fellowship, the Air Force Office of Scientific Research Young Investigator Program

 (AFOSR-YIP) award, the Office of Naval Research Young Investigator Program (ONR-YIP) award, the DARPA Young Faculty Award (DARPA-YFA), the National Science Foundation Faculty Early Career Development Program Award (NSF-CAREER), the Presidential Early Career Award for Scientists and Engineers (PECASE), the CRA-E Undergraduate Research Faculty Mentoring Award, the MIT TR35, the IEEE Robotics and Automation Society (RAS) Early Career Award, and the Dick Volz Best U.S. Ph.D. Thesis in Robotics and Automation Award.

 Fields Live Video

 Scheduled as part of

 * 2016-2017 Machine Learning Advances and Applications Seminar

 Who we are

 The Fields Institute is a centre for mathematical research activity - a place where mathematicians from Canada and abroad, from academia, business, industry and financial institutions, can come together to carry out research and formulate problems of mutual interest. Our mission is to provide a supportive and stimulating environment for mathematics innovation and education. Learn more about us.

 The Fields Institute promotes mathematical activity in Canada and helps to expand the application of mathematics in modern society.

 Everyone is welcome to register and participate in events at the Fields Institute.

 Find us on the following social media sites.

 E-mail: inquiries@fields.utoronto.ca

 People and Contacts

 Contacting the Institute, Staff, and Visiting Members of the Fields Institute.

 View Contacts

 Calendar and Events

 Stay up to date with our upcoming events and news by viewing our calendar.

 View Events

 * Activities
 + Thematic and Focus Programs
 o Current and Upcoming
 o Past
 + Fields Medal Symposium
 o Public Opening
 o Scientific Program
 + Mathematical Modelling of COVID-19
 + Mathematics for Public Health
 + Workshops and Conferences
 o Current and Upcoming
 o Past
 + Special Lecture Series
 o Current and Upcoming
 o Past
 + Fields Academy
 + Seminar Series
 o Current
 o Past
 + Courses and Schools
 + Commercial and Industrial Mathematics Program
 + Outreach
 * Centres
 + Centre for Quantitative Analysis and Modelling
 + Centre for Financial Industries
 + Centre for Mathematical Medicine
 + Centre for Mathematics Education
 + NRC-Fields Collaboration Centre
 * Honours and Fellowships
 + Fields Institute Fellows
 + Fellowships and Visitorships
 o Fields Research Fellowship
 o Postdoctoral Fellowships
 o Fields Elliott Distinguished Visitorship
 + Simons Distinguished Visitor
 + Dean's Distinguished Visiting Professorship
 + CAIMS-Fields Industrial Mathematics Prize
 + CRM-Fields-PIMS Prize
 + Margaret Sinclair Memorial Award
 * Publications
 + Annual Reports and Newsletters
 + Communications and Monographs
 + The Fields Institute Turns Twenty-Five
 + Fields Mathematics Education Journal
 + Mathematics-in-Industry Case Studies Journal
 * Resources
 + Proposals and Applications
 + Research and Training Opportunities
 o Fields Research Exchange Program
 o Other Opportunities
 + For Program Organizers
 + For Event Organizers
 + For Speakers
 + For Visitors
 + Ontario Mathematics Calendar
 * About Us
 * Sponsors and Partners
 * People and Contacts
 * Calendar
 * FieldsLive
 * Mailing List
 * Donate

 222 College Street · Toronto, Ontario · M5T 3J1 · Canada

Speaker: 
 
 
 
 Pieter Abbeel, University of California Berkeley 
 Date and Time:   Thursday, February 16, 2017 - 12:00pm to 1:00pm    Location:   Fields Institute, Room 230    Abstract:   Programming robots remains notoriously difficult. Equipping robots with the ability to learn would by-pass the need for what otherwise often ends up being time-consuming task specific programming. This talk will describe recent progress in deep reinforcement learning, in which robots learn through their own trial and error, and resulting capabilities in robotics. I will discuss technical advances in policy gradient methods, in learning to reinforcement learn, and in safe reinforcement learning. And I will discuss resulting robotic capabilities in manipulation, locomotion, and flight.
Pieter Abbeel (Associate Professor, UC Berkeley EECS) works in machine learning and robotics, in particular his research is on making robots learn from people (apprenticeship learning) and how to make robots learn through their own trial and error (reinforcement learning). His robots have learned: advanced helicopter aerobatics, knot-tying, basic assembly, and organizing laundry. He has won various awards, including best paper awards at ICML and ICRA, the Sloan Fellowship, the Air Force Office of Scientific Research Young Investigator Program
(AFOSR-YIP) award, the Office of Naval Research Young Investigator Program (ONR-YIP) award, the DARPA Young Faculty Award (DARPA-YFA), the National Science Foundation Faculty Early Career Development Program Award (NSF-CAREER), the Presidential Early Career Award for Scientists and Engineers (PECASE), the CRA-E Undergraduate Research Faculty Mentoring Award, the MIT TR35, the IEEE Robotics and Automation Society (RAS) Early Career Award, and the Dick Volz Best U.S. Ph.D. Thesis in Robotics and Automation Award.
Deep Reinforcement Learning for Robotics
http://www.fields.utoronto.ca/talks/Deep-Reinforcement-Learning-Robotics
https://harvard-ml-courses.github.io/cs287-web/
CS287: Machine Learning for Natural Language
2015-05-21
None
<div><div>
 <div>
 <div>
 <p>Time: Mon/Wed 10:30-11:45pm</p>
 <p>Location: Pierce 301</p>
 </div>
 <div>
 <h3>Course Info</h3>
 <div>
 <dl>
 <dt>Application</dt>
 <dd><ul>
 <li><a href="https://docs.google.com/forms/d/e/1FAIpQLSfZcGeC2WenocQLkLC5acfBq5PeqKeXORwiRxk9GZxGQZX5SA/viewform?usp=sf_link">Course Application</a> (CS 287 will be capped at 35
 students this semester. We are
 looking for interested and
 dedicated students. If you think
 this is you, please take the time
 to fill out the application
 carefully.)
 </li></ul></dd>
 <dt>Links</dt>
 <dd><ul>
 <li><a href="https://cs-287.slack.com/">Slack </a></li>
 <li> <a href="syllabus.pdf">Syllabus</a></li>
 <li><a href="https://canvas.harvard.edu/courses/46821">Canvas Site</a></li>
 <li><a href="https://docs.google.com/document/d/1dntIvJ7RlOsPbFcnJyDuQBBGHE7Q9kL1a8c9VUmBgTA/edit">Project Ideas</a></li>
 </ul></dd>
 <dt>Lab Times </dt>
 <dd>
 <ul>
 <li>Friday 2:30-3:30pm, MD 2nd Floor Lounge</li>
 </ul>
 </dd>
 
 <dt>Instructor</dt>
 <dd> <ul>
 <li>Alexander "Sasha" Rush <br />
 </li>
 <li>Email: Slack preferred or srush at seas.harvard.edu
 </li></ul>
 </dd>
 <dt>Teaching Assistants</dt>
 <dd>
 <ul>
 <li>
 
 <a href="mailto:justinchiu@g.harvard.edu">Justin</a> (OH: Monday 3pm-4pm, MD 2nd floor lounge),
 
 <a href="mailto:dengyuntian@g.harvard.edu">Yuntian</a> (OH: Wednesday 4pm-5pm, MD 2nd floor lounge)
 
 </li>
 </ul>
 </dd>
 <dt>Office Hours</dt>
 <dd>
 <ul>
 
 <li>Monday 12-1:30pm: MD 217 (Sasha)</li>
 
 </ul>
 </dd>
 <dt>Grading</dt>
 <dd>
 <ul>
 <li> Assignments (20%) </li>
 <li> Presentation and Participation (15%) </li>
 <li> Midterm Exam (15%) </li>
 <li> Final Project (50%) </li>
 </ul>
 </dd>
 </dl>
 </div>
 </div>
 </div>
 
 <div>
 <h3>Time and Location</h3>
 <div>
 <ul>
 <li>Thursday 5-6pm: Pierce Hall 320</li>
 <li>Friday 11-11:59am: MD 223</li>
 </ul>
 </div>
 <hr />
 <div>
 <table>
 <thead>
 <tr>
 <th>Date</th>
 <th>Location</th>
 <th>Topic</th>
 <th>Materials</th>
 </tr>
 </thead>
 <tbody>
 <tr>
 <td>Sep. 1, 10-11am (Mark)</td>
 <td>Pierce 301</td>
 <td>
 <a href="https://github.com/harvard-ml-courses/cs287-sections/tree/master/00">Math Review (Linear Algebra, Calculus, Probabilistic Theory)</a>
 </td>
 <td>
 </td>
 </tr>
 <tr>
 <td>Sep. 4, 5-6pm (Zhirui)</td>
 <td>Pierce 301</td>
 <td>
 <a href="https://github.com/harvard-ml-courses/cs287-sections/tree/master/00">Math Review (Linear Algebra, Calculus, Probabilistic Theory)</a>
 </td>
 <td>
 </td>
 </tr>
 <tr>
 <td>Sep. 7, 5-6pm (Rachit)</td>
 <td>Pierce 320</td>
 <td>
 Code Review (Python, Numpy, Matplotlib, PyTorch)
 </td>
 <td>
 </td>
 </tr>
 <tr>
 <td>Sep. 8, 11-11:59am (Rachit)</td>
 <td>MD 223</td>
 <td>
 Code Review (Python, Numpy, Matplotlib, PyTorch)
 </td>
 <td>
 </td>
 </tr>
 </tbody>
 </table>
 </div>
 </div>
 <div>
 <p>
 You will form groups of 3 (preferably, for exceptions please ask Sasha) to work on a project. The ideal outcome of this project
 would be a paper that could be submitted to a top-tier natural language or machine learning conference such as ACL, EMNLP, NIPS,
 ICML, or UAI. There are different ways to approach this project, which are discussed in a more comprehensive document that is available on the course website. There are four
 separate components of the project.
 </p>
 <p>
 You will upload these materials via Canvas. Please see the syllabus (linked
 in the course website) for a more thorough description of the final project and policies related to
 collaboration, etc.
 </p>
 <hr />
 <h3>Important Dates</h3>
 <div>
 <table>
 <thead>
 <tr>
 <th>Date</th>
 <th>Due</th>
 <th>Descriptions</th>
 </tr>
 </thead>
 <tbody>
 <tr>
 <td>March 27</td>
 <td>Abstract and Status Report</td>
 <td>
 This is a three to four page document that contains a
 draft of your final abstract, as well as a brief status report on the progress of your project.
 </td>
 </tr>
 <tr>
 <td></td>
 <td></td>
 <td>
 </td>
 </tr>
 <tr>
 <td>May 13</td>
 <td>Final Report</td>
 <td>
 You will write a report of up to ten pages, in the style of a mainstream CS
 conference paper. Please use the provided template (see <a href="https://github.com/harvard-ml-courses/cs287-s18/tree/master/projects/template">here</a>)
 </td>
 </tr>
 </tbody>
 </table>
 </div>
 </div>
 <div>
 <p>
 Our syllabus this semester consists of two parts.
 The first part of the semester will be an accelerated
 background on applied deep learning for natural language
 processing with a series of Kaggle competitions. The second
 part of the semester will consist of student led paper
 presentations on the topic of text generation and transfer.
 </p>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 </div>
 
 
 </div>||||I|||| CS287: Machine Learning for Natural Language

 Alexander Rush, Harvard University

 * Home
 * Schedule
 * Project

 Time: Mon/Wed 10:30-11:45pm

 Location: Pierce 301

 Course Info

 Application
 * Course Application (CS 287 will be capped at 35 students this semester. We are looking for interested and dedicated students. If you think this is you, please take the time to fill out the application carefully.)
 Links
 * Slack
 * Syllabus
 * Canvas Site
 * Project Ideas
 Lab Times
 * Friday 2:30-3:30pm, MD 2nd Floor Lounge
 Instructor
 * Alexander "Sasha" Rush
 * Email: Slack preferred or srush at seas.harvard.edu
 Teaching Assistants
 * Justin (OH: Monday 3pm-4pm, MD 2nd floor lounge), Yuntian (OH: Wednesday 4pm-5pm, MD 2nd floor lounge)
 Office Hours
 * Monday 12-1:30pm: MD 217 (Sasha)
 Grading
 * Assignments (20%)
 * Presentation and Participation (15%)
 * Midterm Exam (15%)
 * Final Project (50%)

 Time and Location

 * Thursday 5-6pm: Pierce Hall 320
 * Friday 11-11:59am: MD 223
 Date Location Topic Materials
 Sep. 1, 10-11am (Mark) Pierce 301 Math Review (Linear Algebra, Calculus, Probabilistic Theory) 
 Sep. 4, 5-6pm (Zhirui) Pierce 301 Math Review (Linear Algebra, Calculus, Probabilistic Theory) 
 Sep. 7, 5-6pm (Rachit) Pierce 320 Code Review (Python, Numpy, Matplotlib, PyTorch) 
 Sep. 8, 11-11:59am (Rachit) MD 223 Code Review (Python, Numpy, Matplotlib, PyTorch) 
 

 You will form groups of 3 (preferably, for exceptions please ask Sasha) to work on a project. The ideal outcome of this project would be a paper that could be submitted to a top-tier natural language or machine learning conference such as ACL, EMNLP, NIPS, ICML, or UAI. There are different ways to approach this project, which are discussed in a more comprehensive document that is available on the course website. There are four separate components of the project.

 You will upload these materials via Canvas. Please see the syllabus (linked in the course website) for a more thorough description of the final project and policies related to collaboration, etc.

 Important Dates

 Date Due Descriptions   
 March 27 Abstract and Status Report This is a three to four page document that contains a draft of your final abstract, as well as a brief status report on the progress of your project.
    
 May 13 Final Report You will write a report of up to ten pages, in the style of a mainstream CS conference paper. Please use the provided template (see here) 
 

 Our syllabus this semester consists of two parts. The first part of the semester will be an accelerated background on applied deep learning for natural language processing with a series of Kaggle competitions. The second part of the semester will consist of student led paper presentations on the topic of text generation and transfer.

 Date Area Topic Demos Required Readings  Assignment 
 Jan. 28 Intro   
  * Fundamental: Baselines and Bigrams  
  * Word Representations in Vector Space  
 Jan. 30 Classification Basics notebook * Blog: Deep Learning, NLP, and Representations 
  * Coding: Deep Learning Tutorial  
  * Named Tensor  
  * Fundamental: NLP (almost) from scratch  
 Feb. 4 CNNs notebook * CNNs for Sentence Classification  
  * Blog: Understanding CNNs for NLP  
  * Coding: NLP Tutorial (Basic)  
  * Fundamental: A Neural Probabilistic Language Model 
 Feb. 6 Sequences NNLMs notebook * Hierarchical Probabilistic Neural Network Language Model 
  * Coding: Learning Embeddings  
  * Fundamental: RNN Language Model  
  * Generating Text with RNNs  
 Feb. 11 RNNs notebook * Blog: Understanding LSTM Networks Classification (Kaggle) 
  * The Unreasonable Effectiveness of RNNs  
  * Coding: NLP Tutorial RNNs  
  * Extra: A Bit of Progress in Language Modeling 
  * Coding: Translation with a Sequence to Sequence Network and Attention 
 Feb. 13 Translation (Yuntian) * Fundamental: NMT by Jointly Learning to Align and Translate 
  * Show, Attend, and Tell  
  * Blog: Attention and Augmented Recurrent Neural Networks 
  * Attention is All You Need  
  * BERT  
 Feb. 20 Attention notebook * BLEU Score  Modeling (Kaggle) 
  * Sequence-level Training with Recurrent Neural Networks 
  * Blog: Annotated Transformer  
 Feb. 21 Talk Sasha (Thurs 3pm G115)   
  * Blog: OpenAI Model Blog  
 Feb. 25 Guest Lecture: Alec Radford * Coding: Distributions and Sampling  
  * Tutorial: Gradient Estimation Using Stochastic Computation Graphs 
  * Tutorial: Variational Inference: A Review for Statisticians 
 Feb. 27 Latent Variables Variational Autoencoders (Yoon Kim) * Coding: Variational Autoencoders  
  * Fundamental: Auto-Encoding Variable Bayes  
 Mar. 4 Latent Variables 2 notebook * Tutorial: Tutorial Slides  
  * Coding: Pytorch Distributions  
  * Tasks: GLUE Tasks  
  * HotPot QA  
 Mar. 6 Frontier of Tasks Problem and Datasets * DROP: Paragraph Understanding Translation (Kaggle) 
  * Low Resource Translation  
  * Visual Reasoning from Photographs  
 Mar. 11 Midterm * Guide  
 Mar. 13 Projects Discussion Sign-up 9:30am-2pm * Presentation Sign-ups  
 Mar. 13 Yann LeCunn Talk   
 Mar. 25 Project / Office Hours * Rationalizing Neural Predictions  
 Mar. 27 EthiCS (Fairness + Bias) * Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings Final Project Abstracts 
  * Women also Snowboard  
 Mar. 28 Timnit Gebru (3pm MD 115)   
 Apr. 1 Probing Models * Identifying and Controlling Important Neurons in Neural Machine Translation 
 Apr. 3 Project * Pretraining-Based Natural Language Generation for Text Summarization Attention Ethics (Kaggle)
  * Non-parametric adaptation for neural machine translation 
 Apr. 8 Project * Learning Latent Permutations with Gumbel-Sinkhorn Networks 
  * Bottom-Up Abstractive Summarization  
 Apr. 10 Project * GQA: A New Dataset for Compositional Question Answering over Real-World Images 
  * Neural Discrete Representation Learning  
 Apr. 15 Project * Learning protein sequence embeddings using information from structure 
  * Sequence Level Training with Recurrent Neural Networks 
 Apr. 17 Project * A Comprehensive Survey on Graph Neural Networks 
  * Improving Grammatical Error Correction via Pre-Training 
 Apr. 22 Project * Neural Discrete Representation Learning  
  * Gtr-lstm: A triple encoder for sentence generation from rdf data 
 Apr. 24 Project * Improving neural machine translation models with monolingual data. 
  * Fast Decoding in Sequence Models Using Discrete Latent Variables. 
 Apr. 29 Project * Learning to reason: End-to-end module networks for visual question answering. 
  * Learning libraries of subroutines for neurally–guided bayesian program induction. 
 May. 1 Project   
 May 11   
 May 13 Final Paper Due   
 
Time: Mon/Wed 10:30-11:45pm
 Location: Pierce 301
 
 
 Course Info
 
 
 Application
  
  Course Application (CS 287 will be capped at 35
 students this semester. We are
 looking for interested and
 dedicated students. If you think
 this is you, please take the time
 to fill out the application
 carefully.)
   
 Links
  
  Slack  
  Syllabus 
  Canvas Site 
  Project Ideas 
  
 Lab Times 
 
 
 Friday 2:30-3:30pm, MD 2nd Floor Lounge
 
 
 
 Instructor
  
 Alexander "Sasha" Rush 
 
 Email: Slack preferred or srush at seas.harvard.edu
  
 
 Teaching Assistants
 
 
 
 
 Justin (OH: Monday 3pm-4pm, MD 2nd floor lounge),
 
 Yuntian (OH: Wednesday 4pm-5pm, MD 2nd floor lounge)
 
 
 
 
 Office Hours
 
 
 
 Monday 12-1:30pm: MD 217 (Sasha)
 
 
 
 Grading
 
 
  Assignments (20%) 
  Presentation and Participation (15%) 
  Midterm Exam (15%) 
  Final Project (50%) 
 
 
 
 
 
 
 
 
 Time and Location
 
 
 Thursday 5-6pm: Pierce Hall 320
 Friday 11-11:59am: MD 223
 
 
 
 
 
 
 
 Date
 Location
 Topic
 Materials
 
 
 
 
 Sep. 1, 10-11am (Mark)
 Pierce 301
 
 Math Review (Linear Algebra, Calculus, Probabilistic Theory)
 
 
 
 
 
 Sep. 4, 5-6pm (Zhirui)
 Pierce 301
 
 Math Review (Linear Algebra, Calculus, Probabilistic Theory)
 
 
 
 
 
 Sep. 7, 5-6pm (Rachit)
 Pierce 320
 
 Code Review (Python, Numpy, Matplotlib, PyTorch)
 
 
 
 
 
 Sep. 8, 11-11:59am (Rachit)
 MD 223
 
 Code Review (Python, Numpy, Matplotlib, PyTorch)
 
 
 
 
 
 
 
 
 
 
 You will form groups of 3 (preferably, for exceptions please ask Sasha) to work on a project. The ideal outcome of this project
 would be a paper that could be submitted to a top-tier natural language or machine learning conference such as ACL, EMNLP, NIPS,
 ICML, or UAI. There are different ways to approach this project, which are discussed in a more comprehensive document that is available on the course website. There are four
 separate components of the project.
 
 
 You will upload these materials via Canvas. Please see the syllabus (linked
 in the course website) for a more thorough description of the final project and policies related to
 collaboration, etc.
 
 
 Important Dates
 
 
 
 
 Date
 Due
 Descriptions
 
 
 
 
 March 27
 Abstract and Status Report
 
 This is a three to four page document that contains a
 draft of your final abstract, as well as a brief status report on the progress of your project.
 
 
 
  
  
 
 
 
 
 May 13
 Final Report
 
 You will write a report of up to ten pages, in the style of a mainstream CS
 conference paper. Please use the provided template (see here)
 
 
 
 
 
 
 
 
 Our syllabus this semester consists of two parts.
 The first part of the semester will be an accelerated
 background on applied deep learning for natural language
 processing with a series of Kaggle competitions. The second
 part of the semester will consist of student led paper
 presentations on the topic of text generation and transfer.
CS287: Machine Learning for Natural Language
https://harvard-ml-courses.github.io/cs287-web/
https://github.com/mj-hwang/cs287-advanced-robotics
GitHub - mj-hwang/cs287-advanced-robotics: Optimal Control, Reinforcement Learning & Robotics Projects in CS 287: Advanced Robotics (fall 2019)
2023-01-01
Mj-Hwang
<div><div>
 
 <p> A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
</p>
 </div>||||I|||| Skip to content
 Toggle navigation
 Sign up
 * Product
 + Actions
 Automate any workflow
 + Packages
 Host and manage packages
 + Security
 Find and fix vulnerabilities
 + Codespaces
 Instant dev environments
 + Copilot
 Write better code with AI
 + Code review
 Manage code changes
 + Issues
 Plan and track work
 + Discussions
 Collaborate outside of code
 + Explore
 + All features
 + Documentation
 + GitHub Skills
 + Blog
 * Solutions
 + For
 + Enterprise
 + Teams
 + Startups
 + Education
 + By Solution
 + CI/CD & Automation
 + DevOps
 + DevSecOps
 + Case Studies
 + Customer Stories
 + Resources
 * Open Source
 + GitHub Sponsors
 Fund open source developers
 + The ReadME Project
 GitHub community articles
 + Repositories
 + Topics
 + Trending
 + Collections
 * Pricing
 * In this repository All GitHub ↵
 Jump to ↵
 * No suggested jump to results
 * In this repository All GitHub ↵
 Jump to ↵
 * In this user All GitHub ↵
 Jump to ↵
 * In this repository All GitHub ↵
 Jump to ↵
 Sign in
 Sign up
 {{ message }}
 mj-hwang / cs287-advanced-robotics Public
 * Notifications
 * Fork 4
 * Star 19

 Optimal Control, Reinforcement Learning & Robotics Projects in CS 287: Advanced Robotics (fall 2019)

 19 stars 4 forks
 Star
 Notifications
 * Code
 * Issues 0
 * Pull requests 1
 * Actions
 * Projects 0
 * Security
 * Insights
 More
 * Code
 * Issues
 * Pull requests
 * Actions
 * Projects
 * Security
 * Insights

 mj-hwang/cs287-advanced-robotics

 This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
 master
 Switch branches/tags
 Branches Tags
 Could not load branches
 Nothing to show
 {{ refName }} default
 View all branches
 Could not load tags
 Nothing to show
 {{ refName }} default
 View all tags

 Name already in use

 A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
 Cancel Create
 2 branches 0 tags
 Code
 * Local
 * Codespaces
 * Clone
 HTTPS GitHub CLI

 Use Git or checkout with SVN using the web URL.

 Work fast with our official CLI. Learn more.

 * Open with GitHub Desktop
 * Download ZIP

 Sign In Required

 Please sign in to use Codespaces.

 Launching GitHub Desktop

 If nothing happens, download GitHub Desktop and try again.

 Launching GitHub Desktop

 If nothing happens, download GitHub Desktop and try again.

 Launching Xcode

 If nothing happens, download Xcode and try again.

 Launching Visual Studio Code

 Your codespace will open once ready.

 There was a problem preparing your codespace, please try again.

 Latest commit

 Git stats

 * 21 commits

 Files

 Permalink
 Failed to load latest commit information.
 Type
 Name
 Latest commit message
 Commit time
 cs287hw1
 cs287hw2
 cs287hw3
 cs287hw4
 cs287hw5
 README.md
 View code

 README.md

 Optimal Control, Reinforcement Learning & Robotics Projects in CS 287: Advanced Robotics (fall 2019)

 This repository contains past projects I've completed in CS 287, which I took in Berkeley EECS department under Prof. Pieter Abbeel, in fall 2019. (https://people.eecs.berkeley.edu/~pabbeel/cs287-fa19/)

 Followings are those projects and related materials covered/implemented in each project.

 * Project 1

 + Value Iteration
 + Dicretization-based Optimal Control
 + Function Approximation Optimal Control

 * Project 2

 + LQR, iLQR
 + DDP
 + Feedback Linearization

 * Project 3

 + Convex Optimiazation
 + Sequential Convex Programming
 + Motion Planning and Control (w/ convex opt.)

 * Project 4

 + Multivariate Gaussians
 + Kalman Filtering
 + EM & MLE
 + Particle Filtering
 + Belief Space Planning

 * Project 5

 + Policy Gradient
 o Trust Region Policy Optimization (TRPO)
 o Proximal Policy Optimization (PPO)
 + Deep Q-Learning
 o DQN
 o Double DQN
 o Dueling DQN

 About

 Optimal Control, Reinforcement Learning & Robotics Projects in CS 287: Advanced Robotics (fall 2019)

 Resources

 Readme

 Stars

 19 stars

 Watchers

 1 watching

 Forks

 4 forks

 Releases

 No releases published

 Packages 0

 No packages published

 Languages

 * Jupyter Notebook 97.8%
 * Python 1.6%
 * TeX 0.2%
 * JavaScript 0.2%
 * HTML 0.1%
 * CSS 0.1%

 Footer

 © 2023 GitHub, Inc.

 Footer navigation

 * Terms
 * Privacy
 * Security
 * Status
 * Docs
 * Contact GitHub
 * Pricing
 * API
 * Training
 * Blog
 * About
 You can’t perform that action at this time.
 You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Skip to content
 Toggle navigation
 Sign up
 * Product
 + Actions
 Automate any workflow
 + Packages
 Host and manage packages
 + Security
 Find and fix vulnerabilities
 + Codespaces
 Instant dev environments
 + Copilot
 Write better code with AI
 + Code review
 Manage code changes
 + Issues
 Plan and track work
 + Discussions
 Collaborate outside of code
 + Explore
 + All features
 + Documentation
 + GitHub Skills
 + Blog
 * Solutions
 + For
 + Enterprise
 + Teams
 + Startups
 + Education
 + By Solution
 + CI/CD & Automation
 + DevOps
 + DevSecOps
 + Case Studies
 + Customer Stories
 + Resources
 * Open Source
 + GitHub Sponsors
 Fund open source developers
 + The ReadME Project
 GitHub community articles
 + Repositories
 + Topics
 + Trending
 + Collections
 * Pricing
 * In this repository All GitHub ↵
 Jump to ↵
 * No suggested jump to results
 * In this repository All GitHub ↵
 Jump to ↵
 * In this user All GitHub ↵
 Jump to ↵
 * In this repository All GitHub ↵
 Jump to ↵
 Sign in
 Sign up
 {{ message }}
 mj-hwang / cs287-advanced-robotics Public
 * Notifications
 * Fork 4
 * Star 19
 Optimal Control, Reinforcement Learning & Robotics Projects in CS 287: Advanced Robotics (fall 2019)
 19 stars 4 forks
 Star
 Notifications
 * Code
 * Issues 0
 * Pull requests 1
 * Actions
 * Projects 0
 * Security
 * Insights
 More
 * Code
 * Issues
 * Pull requests
 * Actions
 * Projects
 * Security
 * Insights
 mj-hwang/cs287-advanced-robotics
 This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
 master
 Switch branches/tags
 Branches Tags
 Could not load branches
 Nothing to show
 {{ refName }} default
 View all branches
 Could not load tags
 Nothing to show
 {{ refName }} default
 View all tags
 Name already in use
 A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
 Cancel Create
 2 branches 0 tags
 Code
 * Local
 * Codespaces
 * Clone
 HTTPS GitHub CLI
 Use Git or checkout with SVN using the web URL.
 Work fast with our official CLI. Learn more.
 * Open with GitHub Desktop
 * Download ZIP
 Sign In Required
 Please sign in to use Codespaces.
 Launching GitHub Desktop
 If nothing happens, download GitHub Desktop and try again.
 Launching GitHub Desktop
 If nothing happens, download GitHub Desktop and try again.
 Launching Xcode
 If nothing happens, download Xcode and try again.
 Launching Visual Studio Code
 Your codespace will open once ready.
 There was a problem preparing your codespace, please try again.
 Latest commit
 Git stats
 * 21 commits
 Files
 Permalink
 Failed to load latest commit information.
 Type
 Name
 Latest commit message
 Commit time
 cs287hw1
 cs287hw2
 cs287hw3
 cs287hw4
 cs287hw5
 README.md
 View code
 README.md
 Optimal Control, Reinforcement Learning & Robotics Projects in CS 287: Advanced Robotics (fall 2019)
 This repository contains past projects I've completed in CS 287, which I took in Berkeley EECS department under Prof. Pieter Abbeel, in fall 2019. (https://people.eecs.berkeley.edu/~pabbeel/cs287-fa19/)
 Followings are those projects and related materials covered/implemented in each project.
 * Project 1
 + Value Iteration
 + Dicretization-based Optimal Control
 + Function Approximation Optimal Control
 * Project 2
 + LQR, iLQR
 + DDP
 + Feedback Linearization
 * Project 3
 + Convex Optimiazation
 + Sequential Convex Programming
 + Motion Planning and Control (w/ convex opt.)
 * Project 4
 + Multivariate Gaussians
 + Kalman Filtering
 + EM & MLE
 + Particle Filtering
 + Belief Space Planning
 * Project 5
 + Policy Gradient
 o Trust Region Policy Optimization (TRPO)
 o Proximal Policy Optimization (PPO)
 + Deep Q-Learning
 o DQN
 o Double DQN
 o Dueling DQN
 About
 Optimal Control, Reinforcement Learning & Robotics Projects in CS 287: Advanced Robotics (fall 2019)
 Resources
 Readme
 Stars
 19 stars
 Watchers
 1 watching
 Forks
 4 forks
 Releases
 No releases published
 Packages 0
 No packages published
 Languages
 * Jupyter Notebook 97.8%
 * Python 1.6%
 * TeX 0.2%
 * JavaScript 0.2%
 * HTML 0.1%
 * CSS 0.1%
 Footer
 © 2023 GitHub, Inc.
 Footer navigation
 * Terms
 * Privacy
 * Security
 * Status
 * Docs
 * Contact GitHub
 * Pricing
 * API
 * Training
 * Blog
 * About
 You can’t perform that action at this time.
 You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
GitHub - mj-hwang/cs287-advanced-robotics: Optimal Control, Reinforcement Learning & Robotics Projects in CS 287: Advanced Robotics (fall 2019)
https://github.com/mj-hwang/cs287-advanced-robotics