Tracking lanes on real hardware

The car hardware can be controlled easily from the drive module. I am using arduino to control the drive wheel and steering. I use COM over USB  to communicate with arduino from drive module.

The localization is not working yet and I need to create a PID controller. I am excited to finish last bit of kalman filter to track lanes correctly, localization using particle filter and PID controller to drive the car.

Tracking lanes

I have taken the caltech cordova1 dataset (search of the term will find the location to download) to test the algorithm I develop to detect lanes. Here is the algorithm in short (without kalman filter and noise elimination):

First convert to bird eye view of the frame

Then convert the image to grayscale
Apply Gaussian filter to smooth out outliers
Use canny method to create binary image (detect edges)
Run probabilistic hough transform to get the lines
Remove lines with wrong angle and size
Use perspective transform to get image co-ordinates
Draw the lines on original frame

Draw a green line to show the vehicle path and yellow circle to show the vanishing point.

Next step would be to kalman filter to deal with noises present in the detection using vehicle speed/ acceleration (and may be other sensors) to establish prior belief. Then will use the current detected lanes and update the belief. I'll be using estimated vehicle speed and use dataset 1 for training and then test using other datasets to measure its performance.  Once it works I'll be using RANSAC to avoid going through all the points - its kind of slow otherwise (if Jetson TX1 performs around 20 fps I probably wont bother).