Teaching Team

Lead Instructor: Dr. Song Chaoyang, Dr. Wan Fang (Invited)
Teaching Assistant: Mr. Liu Xiaobo
Office: Room 606, 7 Innovation Park
Lecture: Wednesday 10:20-12:10, Location @ #208, 2 Lychee Park
Lab: Every Friday 10:20-12:10, Location @ #607, 7 Innovation Park

Course Objectives

This course is intended for students advancing in the study of robotic engineering. The focus is on the problems of how a robot can learn to perceive the physical world well enough to act in it and make reliable plans. Subjects covered by this course include robotic collaboration, kinematics, Robot Operating Systems (ROS), robotic vision, calibration, RGB-D sensing, object recognition, artificial intelligence (AI) and deep learning (DL). Specific projects will be carried out throughout this course regarding the simulation of robot picking using fundamental kinematics and robot vision, an AI robot to play tic-tac-toe game, and a DL robot to play an arcade claw game.

To teach students how to conduct the basic kinematic formulation of a robotic system in simulation.
To teach students how to use robotic vision, including algorithm, hardware, and software, in simulation.
To teach students how to program artificial intelligence into a robot hardware performing interactive tasks.
To teach students how to use deep learning methods to program a robot hardware to perform advanced tasks.
To reinforce students’ team skills through various team project, including problem formulation, problem solutions and written reporting of results.
To reinforce students’ visualization and hands-on skills through project virtual prototyping and/or physical construction exercises.

Learning Outcomes

As an elective course for robotic engineering major, this course lays the foundation for students to use widely adopted Robot Operating System (ROS) to perform advanced robot control including basic mathematical formulation, hardware usage, and intelligence integration. The following learning outcomes are expected for students taking this course:

Given functional and environmental requirement, utilize concepts generation methods within a team setting to achieve a consensus for a robot design concept.
Design and develop functional robot programs from the perspectives of function, hardware, algorithm, and physical environment.
Apply basics of disciplines including mechanical engineering, electrical engineering, applied mathematics, computer science to understand the use of robots in action.
Communicate engineering decisions, justification for those decisions, designs, programming, and test results in multi-media presentation and report writing.

Grading

60%: project marking
- 15% per project for the first four projects, including 10% code submission and 5% video presentation.
10%: individual marking
30%: final project marking, including
- 10% final paper | 10% final video demo | 5% final poster | 5% code submission

Pre-requisites

MEE5101 Introduction to Robotics and Automation
MEE5104 Robotic Modeling and Control

Robotic Engineering Project

Project 1: Simulate picking using kinematics [Github]
- Simulate a robot arm picking objects from a shelf and placing them in a bin.
  - Similar to Amazon Picking Challenge, but much simplified in ROS;
  - Without using any image data, focus mainly on inverse kinematics for planning;
  - Focus on the understanding of analytical method for common robotic grasping;
  - To be completed using the students’ own laptop.
Project 2: Simulate picking using robotic vision [Github]
- Simulate a robot arm processing RGB-D images to grasp objects from a bin.
  - Similar to industrial bin-picking scenario, but much simplified in ROS;
  - Process simulated images in ROS to pick up simple object autonomously;
  - Focus on the use of robotic vision techniques for common robotic grasping;
  - To be completed using the students’ own laptop.
Project 3: Program an AI robot to play the Tic-Tac-Toe game [Github]
- Use AI to control a real robot to play tic-tac-toe with a human player.
  - Similar to the Alpha-Go scenario, but much simplified using a real desktop robot;
  - Design and implement an AI algorithm on a robot for tic-tac-toe;
  - Focus on the design of AI algorithms to control a real robot arm autonomously;
  - To be completed using a real desktop robot hardware (in progress).
Project 4: Program a DL robot to play the arcade claw game [Github]
- Use Deep Learning to control a real robot to play the arcade claw machine game.
  - Similar to the DeepClaw scenario, but much simplified using a real desktop robot;
  - Design and implement a deep neural network to play the game for data acquisition;
  - Focus on the use of DL techniques on a real robot for data-driven robotics;
  - To be completed using a real desktop robot hardware (in progress).
Project 5: Autonomous Robot Manipulation Competition [Github]
- Design a competitive and autonomous picking robot
  - Design a robot system to autonomously play a board game competitively;
  - In the final class, each team will present your robot system design in details by
    - Writing a full paper describing your robot system in technical details;
    - Submit a video demo of your robot system design;
    - Submit a posted to present your robot system design;
    - Open-source your codes by uploading on GitHub;
    - Compete live and win! (Good luck)
  - To be completed using a real desktop robot hardware.
Due to the limitation of the robotic hardware, a total of 9 students will be limited to this course.

Lecture & Lab Content

How to Video Record Your Presentation using PowerPoint

There is a simple solution in Microsoft PowerPoint, as a recording function is built-in the software.
- The content of the video should revolve around your presentation, but you are welcome to be creative about how to present it.
- This is a Team assignment, and the question about who to present makes no sense at all.
- You need to prepare and submit a PDF version of the slides at this [link].

Late Homework Policy

Each student is granted four unpenalized late days for the semester. Homework can be handed in no more than four days late and will receive a 25% penalty for each day late (excluding unpenalized late days used). Homework are due at 3 PM on the due date, and each late day extends the deadline by exactly 24 hours. All homework must be done to pass the course.

Course Materials

Required:
- Introduction to Robotics – Mechanics and Control (4th Edition) by John Craig. [PDF]
Optional:
- Robotics, Vision & Control – Fundamental Algorithms in MATLAB® by Peter Corke.
- Artificial Intelligence – A Modern Approach (3rd Edition) by Stuart Russel, and Peter Norvig.
- Mastering ROS for Robotics Programming by Lentin Joseph. [Website]
- A Mathematical Introduction to Robotic Manipulation by Richard M. Murray, Zexiang Li, and S. Shankar Sastry. [PDF]

Academic Integrity

This course follows the SUSTech Code of Academic Integrity. Each student in this course is expected to abide by the SUSTech Code of Academic Integrity. Any work submitted by a student in this course for academic credit will be the student’s own work. Violations of the rules (e.g., cheating, copying, non-approved collaborations) will not be tolerated.

Course Schedule & Lecture Notes

Register your Office 365 account using SUSTech Email first, otherwise, you won’t be able to access the files.

Week	Date	Day	Class	Topic	Subject
01	February 20	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Course Introduction	Course Introduction
01	February 22	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Programming Basics	Linux and ROS
02	February 27	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part I Robot Foundations	Mathematical Foundations
02	March 01	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project I Simulate Picking Kinematics	ROS Basics
03	March 06	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part I Robot Foundations	Kinematics & Jacobian
03	March 08	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project I Simulate Picking Kinematics	ROS Simulation
04	March 13	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part I Robot Foundations	Dynamics & Control
04	March 15	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project I Simulate Picking Kinematics	Seminar [Hu Chunxu] ROS Picking Project I Submission
05	March 20	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part II Robotic Vision	Seminar [Dr. Qiu Qiang]
05	March 22	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project II Simulate Picking with Vision	Camera & Images RealSense with ROS
06	March 27	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part II Robotic Vision	Calibration & Servoing
06	March 29	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project II Simulate Picking with Vision	Seminar [Dr. Gao Lei]
07	April 03	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part II Robotic Vision	Features & Filters OpenCV & PCL
07	~~April 05~~	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	~~Project II Simulate Picking with Vision~~	Canceled due to holiday Project II Submission
08	April 10	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part III Artificial Intelligence	Rational Agents
08	April 12	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project III Program a Tic-Tac-Toe Robot	Seminar [Liu Sicong]
09	April 17	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part III Artificial Intelligence	Search Problems
09	April 19	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project III Program a Tic-Tac-Toe Robot	Seminar [Zhang Min]
10	April 24	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part III Artificial Intelligence	Adversarial Search
10	April 26	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project III Program a Tic-Tac-Toe Robot	Seminar [Dr. Liang Conghui] Project III Submission
11	~~May 01~~	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	~~Part IV Deep Learning~~	Canceled due to holiday
11	May 05	Sunday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project IV Program an Arcade Claw Robot	Neural Networks Tensorflow Basics
12	May 08	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part IV Deep Learning	Network Tuning
12	May 10	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project IV Program an Arcade Claw Robot	Learning Picking
13	May 15	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part IV Deep Learning	Seminar [Liu Guoqing]
13	May 17	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project IV Program an Arcade Claw Robot	Robot Learning Project IV Submission
14	May 22	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part V Special Topics	Shareability & Reproducibility
14	May 24	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project V Collaborative Robot Learning	[Seminar] Li Miao
15	May 29	Wednesday 10:20-12:10	Lecture @ #208, 2 Lychee Park	Part V Special Topics	Industrial vs. Collaborative
15	May 31	Friday 10:20-12:10	Lab @ #607, 7 Innovation Park	Project V Collaborative Robot Learning	Seminar [Albert Causo] Final Presentation Project V Submission