ARAS Autonomous Robotics Research Group is going to hold a workshop on Deep Learning, with the title Deep Learning for Self-Driving Cars. Deep learning, and in particular Convolutional Neural Networks, has become the main component of many intelligent vehicle algorithms. Self-Driving Cars (Autonomous Cars) need to choose actions, e.g. steering commands which will affect the driving scenes encountered. This setting is well-suited to apply deep learning to determine the optimal commands. Recent advances in deep learning techniques in computer vision applied to a broad range of applications including classification, detection, and segmentation.
These techniques are used in Self-Driving cars in order to detect objects from camera images such as other cars, bicycles, pedestrians and etc. In camera-based sensing systems for intelligent vehicles, object detection offers the fundamental ability to real-time environment perception. In this workshop we explain what is deep learning and introduce open source software like Python, Keras and Tensorflow to build deep learning models on real world data. The workshop is conceived to maximize the learning experience for everyone and includes 50% theory and 50% hands-on practice. Previous experience programming in Python or in other languages is advised to make best use of the workshop. Additionally, some familiarity with machine learning is necessary. The goal of this workshop is to bring together researchers and practitioners in the field of autonomous driving to address core challenges with machine learning. The contents of this workshop include, but are not limited to titles below :
Workshop Context:
- Introduction to Deep Neural Networks
- Finding lane lines by low-level image processing techniques
- Introduction to TensorFlow
- Traffic sign classification by TensorFlow
- Introduction to Keras
- Behavioral cloning
- Accurate and efficient vehicle detection
Speakers:
Hamid D. Taghirad has received his B.Sc. degree in mechanical engineering from Sharif University of Technology, Tehran, Iran, in 1989, his M.Sc. in mechanical engineering in 1993, and his Ph.D. in electrical engineering in 1997, both from McGill University, Montreal, Canada. He is currently a Professor and the dean of the Faculty of Electrical Engineering, and the Director of the Advanced Robotics and Automated System (ARAS) at K.N. Toosi University of Technology, Tehran, Iran. He is a senior member of IEEE, the chairman of IEEE control system chapter in Iran section, member of the board of Industrial Control Center of Excellence (ICCE), at K.N. Toosi University of Technology, editor in chief of Mechatronics Magazine, and Editorial board of International Journal of Robotics: Theory and Application, and International Journal of Advanced Robotic Systems. His research interest is robust and nonlinear control applied to robotic systems. His publications include five books, and more than 200 papers in international Journals and conference proceedings.
Hamid D. Taghirad
Alireza Norouzzadeh Ravari received the B.Sc. and M.Sc. degrees in electrical engineering from Shahid Bahonar University of Kerman, Kerman, Iran, and K. N. Toosi University of Technology, Tehran, Iran, respectively. He is pursuing the Ph.D. degree in electrical engineering at K. N. Toosi University. He is currently a member of the Advanced Robotics and Automated System at K. N. Toosi University of Technology. His current research interests include machine vision, mobile robotics and information theory.
Alireza Norouzzadeh Ravari
ICRoM 2017 / 24 Oct.
Amirkabir University of Technology
IRAN
| Official Poster |
| Conference Website |
Note: In-person registration is going to be done on Tuesday morning, 24 Oct. 2017.
Student Registration : 500,000 Rials
Full Registration: 2000,0000 Rials
Workshop Materials
In this section all the materials presented at the workshop is available for download.
Download presentation file from here.
Donwload .ZIP file
CarND Term1 Starter Kit
The purpose of this project is to provide unified software dependency support for students enrolled in Term 1 of the Udacity Self-Driving Car Engineer Nanodegree.
Python 3 is used for entirety of term 1.
There are two ways to get up and running:
Anaconda Environment
Get started here. More info here.
Supported Sytems: Linux (CPU), Mac (CPU), Windows (CPU)
Pros | Cons |
---|---|
More straight-forward to use | AWS or GPU support is not built in (have to do this yourself) |
More community support | Implementation is local and OS specific |
More heavily adopted |
Docker
Get started here. More info here.
Supported Systems : AWS (CPU, GPU), Linux (CPU), Mac (CPU), Windows (CPU)
Pros | Cons |
---|---|
Configure once for all environments | More challenging to use |
AWS, GPU support | Less community support |
Practice with Docker | Have to manage images and containers |
Download .ZIP file
#Finding Lane Lines on the Road
Overview
When we drive, we use our eyes to decide where to go. The lines on the road that show us where the lanes are act as our constant reference for where to steer the vehicle. Naturally, one of the first things we would like to do in developing a self-driving car is to automatically detect lane lines using an algorithm.
In this project you will detect lane lines in images using Python and OpenCV. OpenCV means “Open-Source Computer Vision”, which is a package that has many useful tools for analyzing images.
To complete the project, two files will be submitted: a file containing project code and a file containing a brief write up explaining your solution. We have included template files to be used both for the code and the writeup.The code file is called P1.ipynb and the writeup template is writeup_template.md
To meet specifications in the project, take a look at the requirements in the project rubric
Creating a Great Writeup
For this project, a great writeup should provide a detailed response to the “Reflection” section of the project rubric. There are three parts to the reflection:
- Describe the pipeline
- Identify any shortcomings
- Suggest possible improvements
We encourage using images in your writeup to demonstrate how your pipeline works.
All that said, please be concise! We’re not looking for you to write a book here: just a brief description.
You’re not required to use markdown for your writeup. If you use another method please just submit a pdf of your writeup. Here is a link to a writeup template file.
The Project
CarND Term1 Starter Kit you should be good to go! If not, you should install the starter kit to get started on this project.
If you have already installed theStep 1: Set up the CarND Term1 Starter Kit if you haven’t already.
Step 2: Open the code in a Jupyter Notebook
You will complete the project code in a Jupyter notebook. If you are unfamiliar with Jupyter Notebooks, check out Cyrille Rossant’s Basics of Jupyter Notebook and Python to get started.
Jupyter is an Ipython notebook where you can run blocks of code and see results interactively. All the code for this project is contained in a Jupyter notebook. To start Jupyter in your browser, use terminal to navigate to your project directory and then run the following command at the terminal prompt (be sure you’ve activated your Python 3 carnd-term1 environment as described in the CarND Term1 Starter Kit installation instructions!):
> jupyter notebook
A browser window will appear showing the contents of the current directory. Click on the file called “P1.ipynb”. Another browser window will appear displaying the notebook. Follow the instructions in the notebook to complete the project.
Step 3: Complete the project and submit both the Ipython notebook and the project writeup
Download .ZIP file
Behavioral Cloning Project
Overview
This repository contains starting files for the Behavioral Cloning Project.
In this project, you will use what you’ve learned about deep neural networks and convolutional neural networks to clone driving behavior. You will train, validate and test a model using Keras. The model will output a steering angle to an autonomous vehicle.
We have provided a simulator where you can steer a car around a track for data collection. You’ll use image data and steering angles to train a neural network and then use this model to drive the car autonomously around the track.
We also want you to create a detailed writeup of the project. Check out the writeup template for this project and use it as a starting point for creating your own writeup. The writeup can be either a markdown file or a pdf document.
To meet specifications, the project will require submitting five files:
- model.py (script used to create and train the model)
- drive.py (script to drive the car – feel free to modify this file)
- model.h5 (a trained Keras model)
- a report writeup file (either markdown or pdf)
- video.mp4 (a video recording of your vehicle driving autonomously around the track for at least one full lap)
This README file describes how to output the video in the “Details About Files In This Directory” section.
Creating a Great Writeup
A great writeup should include the rubric points as well as your description of how you addressed each point. You should include a detailed description of the code used (with line-number references and code snippets where necessary), and links to other supporting documents or external references. You should include images in your writeup to demonstrate how your code works with examples.
All that said, please be concise! We’re not looking for you to write a book here, just a brief description of how you passed each rubric point, and references to the relevant code :).
You’re not required to use markdown for your writeup. If you use another method please just submit a pdf of your writeup.
The Project
The goals / steps of this project are the following:
- Use the simulator to collect data of good driving behavior
- Design, train and validate a model that predicts a steering angle from image data
- Use the model to drive the vehicle autonomously around the first track in the simulator. The vehicle should remain on the road for an entire loop around the track.
- Summarize the results with a written report
Dependencies
This lab requires:
- CarND Term1 Starter Kit
The lab enviroment can be created with CarND Term1 Starter Kit. Click here for the details.
The following resources can be found in this github repository:
- drive.py
- video.py
- writeup_template.md
The simulator can be downloaded from the classroom. In the classroom, we have also provided sample data that you can optionally use to help train your model.
Details About Files In This Directory
drive.py
Usage of drive.py
requires you have saved the trained model as an h5 file, i.e. model.h5
. See the Keras documentation for how to create this file using the following command:
model.save(filepath)
Once the model has been saved, it can be used with drive.py using this command:
python drive.py model.h5
The above command will load the trained model and use the model to make predictions on individual images in real-time and send the predicted angle back to the server via a websocket connection.
Note: There is known local system’s setting issue with replacing “,” with “.” when using drive.py. When this happens it can make predicted steering values clipped to max/min values. If this occurs, a known fix for this is to add “export LANG=en_US.utf8” to the bashrc file.
Saving a video of the autonomous agent
python drive.py model.h5 run1
The fourth argument, run1
, is the directory in which to save the images seen by the agent. If the directory already exists, it’ll be overwritten.
ls run1
[2017-01-09 16:10:23 EST] 12KiB 2017_01_09_21_10_23_424.jpg
[2017-01-09 16:10:23 EST] 12KiB 2017_01_09_21_10_23_451.jpg
[2017-01-09 16:10:23 EST] 12KiB 2017_01_09_21_10_23_477.jpg
[2017-01-09 16:10:23 EST] 12KiB 2017_01_09_21_10_23_528.jpg
[2017-01-09 16:10:23 EST] 12KiB 2017_01_09_21_10_23_573.jpg
[2017-01-09 16:10:23 EST] 12KiB 2017_01_09_21_10_23_618.jpg
[2017-01-09 16:10:23 EST] 12KiB 2017_01_09_21_10_23_697.jpg
[2017-01-09 16:10:23 EST] 12KiB 2017_01_09_21_10_23_723.jpg
[2017-01-09 16:10:23 EST] 12KiB 2017_01_09_21_10_23_749.jpg
[2017-01-09 16:10:23 EST] 12KiB 2017_01_09_21_10_23_817.jpg
...
The image file name is a timestamp of when the image was seen. This information is used by video.py
to create a chronological video of the agent driving.
video.py
python video.py run1
Creates a video based on images found in the run1
directory. The name of the video will be the name of the directory followed by '.mp4'
, so, in this case the video will be run1.mp4
.
Optionally, one can specify the FPS (frames per second) of the video:
python video.py run1 --fps 48
Will run the video at 48 FPS. The default FPS is 60.
Why create a video
- It’s been noted the simulator might perform differently based on the hardware. So if your model drives succesfully on your machine it might not on another machine (your reviewer). Saving a video is a solid backup in case this happens.
- You could slightly alter the code in
drive.py
and/orvideo.py
to create a video of what your model sees after the image is processed (may be helpful for debugging).
Download .ZIP file
Vehicle Detection
In this project, your goal is to write a software pipeline to detect vehicles in a video (start with the test_video.mp4 and later implement on full project_video.mp4), but the main output or product we want you to create is a detailed writeup of the project. Check out the writeup template for this project and use it as a starting point for creating your own writeup.
Creating a great writeup:
A great writeup should include the rubric points as well as your description of how you addressed each point. You should include a detailed description of the code used in each step (with line-number references and code snippets where necessary), and links to other supporting documents or external references. You should include images in your writeup to demonstrate how your code works with examples.
All that said, please be concise! We’re not looking for you to write a book here, just a brief description of how you passed each rubric point, and references to the relevant code :).
You can submit your writeup in markdown or use another method and submit a pdf instead.
The Project
The goals / steps of this project are the following:
- Perform a Histogram of Oriented Gradients (HOG) feature extraction on a labeled training set of images and train a classifier Linear SVM classifier
- Optionally, you can also apply a color transform and append binned color features, as well as histograms of color, to your HOG feature vector.
- Note: for those first two steps don’t forget to normalize your features and randomize a selection for training and testing.
- Implement a sliding-window technique and use your trained classifier to search for vehicles in images.
- Run your pipeline on a video stream (start with the test_video.mp4 and later implement on full project_video.mp4) and create a heat map of recurring detections frame by frame to reject outliers and follow detected vehicles.
- Estimate a bounding box for vehicles detected.
Here are links to the labeled data for vehicle and non-vehicle examples to train your classifier. These example images come from a combination of the GTI vehicle image database, the KITTI vision benchmark suite, and examples extracted from the project video itself. You are welcome and encouraged to take advantage of the recently released Udacity labeled dataset to augment your training data.
Some example images for testing your pipeline on single frames are located in the test_images
folder. To help the reviewer examine your work, please save examples of the output from each stage of your pipeline in the folder called ouput_images
, and include them in your writeup for the project by describing what each image shows. The video called project_video.mp4
is the video your pipeline should work well on.
As an optional challenge Once you have a working pipeline for vehicle detection, add in your lane-finding algorithm from the last project to do simultaneous lane-finding and vehicle detection!
If you’re feeling ambitious (also totally optional though), don’t stop there! We encourage you to go out and take video of your own, and show us how you would implement this project on a new video!
Download .ZIP file
Advanced Lane Finding
In this project, your goal is to write a software pipeline to identify the lane boundaries in a video, but the main output or product we want you to create is a detailed writeup of the project. Check out the writeup template for this project and use it as a starting point for creating your own writeup.
Creating a great writeup:
A great writeup should include the rubric points as well as your description of how you addressed each point. You should include a detailed description of the code used in each step (with line-number references and code snippets where necessary), and links to other supporting documents or external references. You should include images in your writeup to demonstrate how your code works with examples.
All that said, please be concise! We’re not looking for you to write a book here, just a brief description of how you passed each rubric point, and references to the relevant code :).
You’re not required to use markdown for your writeup. If you use another method please just submit a pdf of your writeup.
The Project
The goals / steps of this project are the following:
- Compute the camera calibration matrix and distortion coefficients given a set of chessboard images.
- Apply a distortion correction to raw images.
- Use color transforms, gradients, etc., to create a thresholded binary image.
- Apply a perspective transform to rectify binary image (“birds-eye view”).
- Detect lane pixels and fit to find the lane boundary.
- Determine the curvature of the lane and vehicle position with respect to center.
- Warp the detected lane boundaries back onto the original image.
- Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.
The images for camera calibration are stored in the folder called camera_cal
. The images in test_images
are for testing your pipeline on single frames. If you want to extract more test images from the videos, you can simply use an image writing method like cv2.imwrite()
, i.e., you can read the video in frame by frame as usual, and for frames you want to save for later you can write to an image file.
To help the reviewer examine your work, please save examples of the output from each stage of your pipeline in the folder called ouput_images
, and include a description in your writeup for the project of what each image shows. The video called project_video.mp4
is the video your pipeline should work well on.
The challenge_video.mp4
video is an extra (and optional) challenge for you if you want to test your pipeline under somewhat trickier conditions. The harder_challenge.mp4
video is another optional challenge and is brutal!
If you’re feeling ambitious (again, totally optional though), don’t stop there! We encourage you to go out and take video of your own, calibrate your camera and show us how you would implement this project from scratch!
Download .ZIP file
Project: Build a Traffic Sign Recognition Program
Overview
In this project, you will use what you’ve learned about deep neural networks and convolutional neural networks to classify traffic signs. You will train and validate a model so it can classify traffic sign images using the German Traffic Sign Dataset. After the model is trained, you will then try out your model on images of German traffic signs that you find on the web.
We have included an Ipython notebook that contains further instructions and starter code. Be sure to download the Ipython notebook.
We also want you to create a detailed writeup of the project. Check out the writeup template for this project and use it as a starting point for creating your own writeup. The writeup can be either a markdown file or a pdf document.
To meet specifications, the project will require submitting three files:
- the Ipython notebook with the code
- the code exported as an html file
- a writeup report either as a markdown or pdf file
Creating a Great Writeup
A great writeup should include the rubric points as well as your description of how you addressed each point. You should include a detailed description of the code used in each step (with line-number references and code snippets where necessary), and links to other supporting documents or external references. You should include images in your writeup to demonstrate how your code works with examples.
All that said, please be concise! We’re not looking for you to write a book here, just a brief description of how you passed each rubric point, and references to the relevant code :).
You’re not required to use markdown for your writeup. If you use another method please just submit a pdf of your writeup.
The Project
The goals / steps of this project are the following:
- Load the data set
- Explore, summarize and visualize the data set
- Design, train and test a model architecture
- Use the model to make predictions on new images
- Analyze the softmax probabilities of the new images
- Summarize the results with a written report
Dependencies
This lab requires:
The lab environment can be created with CarND Term1 Starter Kit. Click here for the details.
Dataset and Repository
- Download the data set. The classroom has a link to the data set in the “Project Instructions” content. This is a pickled dataset in which we’ve already resized the images to 32×32. It contains a training, validation and test set.
- Clone the project, which contains the Ipython notebook and the writeup template.
git clone https://github.com/udacity/CarND-Traffic-Sign-Classifier-Project
cd CarND-Traffic-Sign-Classifier-Project
jupyter notebook Traffic_Sign_Classifier.ipynb