tag:blogger.com,1999:blog-35417135690394911622024-03-23T03:14:39.137-07:00COPOTRONCreating simple robotsMAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-3541713569039491162.post-61686780367963988732020-08-03T23:54:00.005-07:002020-08-03T23:56:49.062-07:00Student registration reached 365 in less than a weekAs I have posted previously, I asked students to register for Autonomous Vehicle course and within less than a week we have 365 registrations / subscriptions for the course. My intention was to create the course if we have a class of 50 people, but we have 7 times more students interested about the course. So, the plan is now to complete the lectures.<div><br /></div><div>I have started to upload videos to the course youtube channel and you may subscribe to the channel using following link:</div><div><br /></div><a href="https://www.blogger.com/#">https://www.youtube.com/c/copotron?sub_confirmation=1</a><div><br /><div>See you in class soon.</div><div><span style="background-color: white; color: #1c1e21; font-family: helvetica, arial, sans-serif; font-size: 14px; white-space: pre-wrap;"><br /></span></div><div><span style="background-color: white; color: #1c1e21; font-family: helvetica, arial, sans-serif; font-size: 14px; white-space: pre-wrap;"><br /></span></div></div>MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-3541713569039491162.post-91872420769820060402020-07-11T07:50:00.002-07:002020-07-12T16:41:21.674-07:00Robotics class: building an autonomous vehicle (self driving car)I am planning to create a series of tutorials / videos to discuss about the theories and implementation details about how to build an autonomous vehicle.<br />
<br />
Topics will include:<br />
<br />
<ul>
<li>Localization,</li>
<li>Perception,</li>
<li>Sensor fusion,</li>
<li>Control</li>
</ul>
<br />
And more...<br />
<br />
If you are interested subscribe your email on the right of this page and like the facebook copotron page to stay connected.<br />
<br />MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-3541713569039491162.post-41488650372084862692018-04-29T00:04:00.003-07:002018-04-29T00:04:59.903-07:00Photos: Building a frame for the LIDAR, Camera and GPS antenna - 4 hours and 65$ aproximately<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjY35aGimFZUyW_Z0DD7PSDYXiuEkxq4BcHIRAosyqTHqdcyjIzZukcLbJmdKA2i5z6lQCF5HhfbAZ82BA9ZRJRAwH01L5cSVSHk3WxYoABIdy6zqz-xsLY6efsxt8e6fmJ7bY8c_NfprM/s1600/frame3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="960" data-original-width="720" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjY35aGimFZUyW_Z0DD7PSDYXiuEkxq4BcHIRAosyqTHqdcyjIzZukcLbJmdKA2i5z6lQCF5HhfbAZ82BA9ZRJRAwH01L5cSVSHk3WxYoABIdy6zqz-xsLY6efsxt8e6fmJ7bY8c_NfprM/s400/frame3.jpg" width="300" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmmP4MmDLlsVhkQyCmk5eFRhN3_66NIwUkZDH7TTl18kjCm-zpZkDyuvOore96Ppm6I-SrcMcYVIA5XaN0mxgjPJub6XqJRvUhedNJT7lH4LOoDTbG6NLf2uFce-uq5Yo36SOXc9Akjfo/s1600/frame1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="720" data-original-width="960" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmmP4MmDLlsVhkQyCmk5eFRhN3_66NIwUkZDH7TTl18kjCm-zpZkDyuvOore96Ppm6I-SrcMcYVIA5XaN0mxgjPJub6XqJRvUhedNJT7lH4LOoDTbG6NLf2uFce-uq5Yo36SOXc9Akjfo/s400/frame1.jpg" width="400" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgf41zKeZ92VBupGU_i2vOz98tkCSkI1LKzp2Rr-tyF0F1Q3vx8PPbRblvJoBuWI60U-aykRflPV3_pwaw7lr8KsBrZ2YVXjoF1I3LHz7BqxihYFW6OILru4EEj2DPOvK471cTAqoT3Bn0/s1600/frame2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="720" data-original-width="960" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgf41zKeZ92VBupGU_i2vOz98tkCSkI1LKzp2Rr-tyF0F1Q3vx8PPbRblvJoBuWI60U-aykRflPV3_pwaw7lr8KsBrZ2YVXjoF1I3LHz7BqxihYFW6OILru4EEj2DPOvK471cTAqoT3Bn0/s400/frame2.jpg" width="400" /></a></div>
<br />MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-3541713569039491162.post-41839593397292649682018-01-31T19:16:00.002-08:002018-01-31T19:16:46.637-08:00Driving toyota prius using joystick<div class="separator" style="clear: both; text-align: center;">
This is initial test of car control using a joystick.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/B-HI2lZPgIY/0.jpg" frameborder="0" height="400" src="https://www.youtube.com/embed/B-HI2lZPgIY?feature=player_embedded" width="600"></iframe></div>
<br />MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com1tag:blogger.com,1999:blog-3541713569039491162.post-21669254571544084012017-08-06T09:38:00.006-07:002023-07-15T09:42:06.259-07:00$8 car control for self driving car<p>See it at: <a href="https://kuasha.github.io/ccwriteup/carcontrol.html">https://kuasha.github.io/ccwriteup/carcontrol.html</a></p><p><br /></p><p><br /></p>MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-3541713569039491162.post-9451740164870013192017-04-05T08:05:00.003-07:002017-04-05T08:05:53.759-07:00Building a realistic car simulatorAs part of the Udacity Self Racing Cars team I worked with the simulator to test our models without the real car. Looking at ebay listing of toyota prius steering column I have found that it is cheap to buy the real parts to build a more realistic simulator.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxzHWRMWYwNkyI8AwwvYA6-_Wo8j7es_tiWvTngbvXaGLbZrxyKAa0Ot86P-YCSIBF1uoZOunXn8dbP4wvdRGqRDmcgSHXdHiiOtN1oVezggwNGp-IO9F7yhmUVpP6HoNyN709huqNaWU/s1600/prius_power_str.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="239" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxzHWRMWYwNkyI8AwwvYA6-_Wo8j7es_tiWvTngbvXaGLbZrxyKAa0Ot86P-YCSIBF1uoZOunXn8dbP4wvdRGqRDmcgSHXdHiiOtN1oVezggwNGp-IO9F7yhmUVpP6HoNyN709huqNaWU/s320/prius_power_str.jpg" width="320" /></a></div>
<br />
I am thinking of buying an ECU and other parts to build a more realistic simulator where I can steer/ brake / accel the car from these hardware setup in front of the computer.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-3541713569039491162.post-84128762609321736412017-02-07T10:12:00.000-08:002017-02-11T03:16:21.804-08:00Self Driving Car Tutorial (Part N) : Developing convolutional neural network<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">I have been talking to a friend about building a self driving car and told him that I can show him how to build it it few short sessions. He seemed interested. So, I decided to write up some tutorials to show how its done.I am starting from the end- because its the one on which I am currently working. This and a lot of interesting stuff is taught in Udacity Self Driving Car Engineer Nanodegree program. Check that out at <a href="http://udacity.com/">Udacity.com</a>.</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">
</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">This is how it looks when I used the network (with some max-pooling and dropout layers) for a fully autonomous drive on Udacity simulator:</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">
</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/s3a40NSstm0/0.jpg" frameborder="0" height="366" src="https://www.youtube.com/embed/s3a40NSstm0?feature=player_embedded" width="520"></iframe></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">I am taking the convolutional neural network developed at NVIDIA research (this is a tutorial - so we should take existing research work rather than creating our own) in this tutorial. The paper can be found at <a href="http://images.nvidia.com/content/tegra/automotive/images/2016/solutions/pdf/end-to-end-dl-using-px.pdf">NVIDIA Self Driving Car</a>.</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">Below is how their neural network looks.</pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6OBHTdcWsksxFe7CR9cNCBKRkLUOstYuzYjIssCs6ezUAg5A2x_piojhLXgjF05-Dbvuo_GX1d1y3f7gfcVlcvoHIDrVUPWz-SlDHVDXx6U0LNas3mynJWdd2aPsT7bHopg2gZucnzto/s1600/nvidia_cnn.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="628" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6OBHTdcWsksxFe7CR9cNCBKRkLUOstYuzYjIssCs6ezUAg5A2x_piojhLXgjF05-Dbvuo_GX1d1y3f7gfcVlcvoHIDrVUPWz-SlDHVDXx6U0LNas3mynJWdd2aPsT7bHopg2gZucnzto/s640/nvidia_cnn.jpg" width="640" /></a></div>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">I'll go step by step how to build the network. The network is shown in the bottom up structure in the image. At the bottom we provide a 66x200 size image that has 3 color layers (RGB). Then it is normalized. We'll start from the normalized layer. So our input size is 66x200 and depth is 3.</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">First lets take a camera image.</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJEvDAKcOYx64JkWjdE8RHFcd4f3Mq9cBhJtqoLBb9jyJxPpgatf09oxM6mVPEd2AIj1P7T8ZPZMXiYaJQgjDK-XroKLjvTfmfWilTYyW3ygPgUrrp3onsQtHQjSSaTiOtjTRnaviapZY/s1600/sample.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJEvDAKcOYx64JkWjdE8RHFcd4f3Mq9cBhJtqoLBb9jyJxPpgatf09oxM6mVPEd2AIj1P7T8ZPZMXiYaJQgjDK-XroKLjvTfmfWilTYyW3ygPgUrrp3onsQtHQjSSaTiOtjTRnaviapZY/s400/sample.jpg" width="400" /></a></div>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">This image is of size: 160x320. We resize it to 100x200 and crop out top 34 pixels. This can be done using OpenCV like below:</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre>image = cv2.imread("./sample.jpg")
img = cv2.resize(image, (200,100))
crp=img[34:,:]
plt.imshow(crp)
</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">This can be done in the model so that the cropping is done on the GPU:</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">model.add(Cropping2D(cropping=((68,0),(0,0))))</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;">And we get an image like this with shape 3@66x200:</pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilNkH7hbZqU0QwxuGp9rkLOxAZ_dIQO1HxjLMId_lvatWaAg-TlYCYQm6WeHj82Wlq3D_QzE_jTPXy-LJvPFON91jJlvV4va9VFcBO2nsdztBGync_9sXu0m_JAHIaX8Q2fx02tt7q9eI/s1600/input.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="105" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilNkH7hbZqU0QwxuGp9rkLOxAZ_dIQO1HxjLMId_lvatWaAg-TlYCYQm6WeHj82Wlq3D_QzE_jTPXy-LJvPFON91jJlvV4va9VFcBO2nsdztBGync_9sXu0m_JAHIaX8Q2fx02tt7q9eI/s320/input.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Now we will use keras to build the neural network. In my setup I am using tensorflow as the keras back end. Lets create a sequential network:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<pre>input_shape = (66, 200, 3)
net = Sequential()
</pre>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Now lets add the normalization layer:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both;">
model.add(Lambda(lambda x: x / 255.0 - 0.5, input_shape=(160,320,3)))</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
From the network image above we need a 5x5 convolutional layer - we'll use ReLU activation which is a function that basically sets all negative values to zero:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<pre>layer1 = Convolution2D(24, 5, 5,
input_shape=input_shape, border_mode='valid', activation='relu')
net.add(layer1)
#output size = 24@31x94
</pre>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
From network we see that we have 4 more convolutional layers:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<pre>net.add(Convolution2D(36, 5, 5, border_mode='valid', activation='relu'))
#output size = 36@14x47
net.add(Convolution2D(48, 5, 5, border_mode='valid', activation='relu'))
#output size = 48@5x22</pre>
<br />
net.add(Convolution2D(64, 3, 3, border_mode='valid', activation='relu'))
<br />
#output size = 64@3x20
<br />
<br />
net.add(Convolution2D(64, 3, 3, border_mode='valid', activation='relu'))
<br />
#output size = 64@1x18
<br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Now we add the flatten layer:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<pre>net.add(Flatten())
</pre>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Simple huh? Now we add a fully connected layer (Dense layer) of size 1156:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<pre>net.add(Dense(115))
</pre>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
We then add remaining 4 dense layer as shown in the network image:</div>
<pre>net.add(Dense(100))
net.add(Dense(50))
net.add(Dense(10))
net.add(Dense(1))
</pre>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
That's it. We have build our network. Lets compile the network and see the summary of the network to make sure we have done it right:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<pre>net.compile(loss='mean_squared_error', optimizer='adam')
net.summary()
</pre>
<div class="separator" style="clear: both; text-align: left;">
Here is the summary output:</div>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; font-size: 14px; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><span style="font-size: x-small;">____________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
====================================================================================================
convolution2d_1 (Convolution2D) (None, 62, 196, 24) 1824 convolution2d_input_1[0][0]
____________________________________________________________________________________________________
convolution2d_2 (Convolution2D) (None, 58, 192, 36) 21636 convolution2d_1[0][0]
____________________________________________________________________________________________________
convolution2d_3 (Convolution2D) (None, 54, 188, 48) 43248 convolution2d_2[0][0]
____________________________________________________________________________________________________
convolution2d_4 (Convolution2D) (None, 52, 186, 64) 27712 convolution2d_3[0][0]
____________________________________________________________________________________________________
convolution2d_5 (Convolution2D) (None, 50, 184, 64) 36928 convolution2d_4[0][0]
____________________________________________________________________________________________________
flatten_1 (Flatten) (None, 588800) 0 convolution2d_5[0][0]
____________________________________________________________________________________________________
dense_1 (Dense) (None, 1156) 680653956 flatten_1[0][0]
____________________________________________________________________________________________________
dense_2 (Dense) (None, 100) 115700 dense_1[0][0]
____________________________________________________________________________________________________
dense_3 (Dense) (None, 50) 5050 dense_2[0][0]
____________________________________________________________________________________________________
dense_4 (Dense) (None, 10) 510 dense_3[0][0]
____________________________________________________________________________________________________
dense_5 (Dense) (None, 1) 11 dense_4[0][0]
====================================================================================================
Total params: 680,906,575
Trainable params: 680,906,575
Non-trainable params: 0</span><span style="font-size: 14px;">
________________________</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><span style="font-size: 14px;">Looks good. Now we can generate inputs by driving our car with camera attached and a way to measure steering angle and train the network.</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><span style="font-size: 14px;">
</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><span style="font-size: 14px;">The output of the network is steering angle. So given a new image the network will tell what should be the cars steering angle. With right training the car should be bale to steer a car given there is mechanical / electrical components to steer the wheel.</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><span style="font-size: 14px;">
</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><span style="font-size: 14px;">Now sit back and relax while the car is being driven by the network.</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; white-space: pre-wrap; word-break: break-all; word-wrap: break-word;"><span style="font-size: 14px;">
</span></pre>
<pre style="background-color: white; border-radius: 0px; border: 0px; box-sizing: border-box; line-height: inherit; overflow: auto; padding: 0px; vertical-align: baseline; word-break: break-all; word-wrap: break-word;"></pre>
MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com21tag:blogger.com,1999:blog-3541713569039491162.post-63174008175116717762017-01-07T11:17:00.000-08:002017-01-07T14:26:23.311-08:00Driving Udacity car simulator with home made steering wheel (!)OK, its really a bar than a wheel-<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFdKpklodSBOIHqGl4ZsGhvcHe2rQPWeaAuTfgkBoNfnrazBJ7v9eaOA58AgS0d8UdYjZXnNimpVfunw63-2ojtW1qJyhpehSkEGvyc5Qx-2HO-0c73MSqCcRw-ocLghEAIjAXQ3v6_LY/s1600/DSC_0626.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="262" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFdKpklodSBOIHqGl4ZsGhvcHe2rQPWeaAuTfgkBoNfnrazBJ7v9eaOA58AgS0d8UdYjZXnNimpVfunw63-2ojtW1qJyhpehSkEGvyc5Qx-2HO-0c73MSqCcRw-ocLghEAIjAXQ3v6_LY/s400/DSC_0626.JPG" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
I took an old servo motor and attached a wooden bar and removed all circuitry keeping the variable resistor. Then measured the position of the wheel using an arduino and sending that data to PC using serial port.
<br />
The middle pin of variable resistor is attached to A0 pin of the arduino and other two pins are attached to ground and +5v. This is creating a voltage divider and the measurement ranges from 0 to 1024. The measurement of 512 is received when the steering position is straight ahead.
<h3>The code</h3>
Here is arduino code that reads analog input from A0 and send the value to serial port. The code is adapted from Arduino IDE Analog sketch demo with minor changes.
<pre>
int sensorPin = A0; // select the input pin for the potentiometer
int sensorValue = 0; // variable to store the value coming from the sensor
int newSensorValue = 0;
void setup() {
Serial.begin(9600);
}
void loop() {
// read the value from the sensor:
newSensorValue = analogRead(sensorPin);
if(abs(newSensorValue - sensorValue) > 5){
sensorValue = newSensorValue;
Serial.println(sensorValue);
}
delay(10);
}
</pre>
Here is python code that reads the data from arduino serial port and update angle variable. This angle is that is being sent to the simulator:
<pre>
import serial
from time import sleep
import threading
port = "/dev/ttyACM1"
ser = serial.Serial(port, 9600)
angle = 0.0
def read_steering_angle():
print("Monitoring steering angle")
global angle
while True:
data = ser.readline()
if len(data) > 0:
#print('Got serial data:', data)
angle = float(int(data) - 512) / 50.0
return angle
t = threading.Thread(target=read_steering_angle)
t.daemon = True
t.start()
while True:
print(angle)
sleep(1)
</pre>
<h3>Demo</h3>
Here is how I am driving using the steering.
<iframe allowfullscreen="" frameborder="0" height="315" src="https://www.youtube.com/embed/LcCBC-bmA_A" width="560"></iframe>
<br />
<br />
Here is screen capture of driving with the steering.
<iframe allowfullscreen="" frameborder="0" height="315" src="https://www.youtube.com/embed/A6wJpPMSRLw" width="560"></iframe>MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-3541713569039491162.post-61387533881577203442016-11-15T07:03:00.000-08:002016-11-15T07:03:19.436-08:00Tracking lanes on real hardwareThe car hardware can be controlled easily from the drive module. I am using arduino to control the drive wheel and steering. I use COM over USB to communicate with arduino from drive module.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/gJe5q0rGlk8/0.jpg" src="https://www.youtube.com/embed/gJe5q0rGlk8?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
The localization is not working yet and I need to create a PID controller. I am excited to finish last bit of kalman filter to track lanes correctly, localization using particle filter and PID controller to drive the car.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-3541713569039491162.post-29212460729064694572016-11-15T06:53:00.001-08:002016-11-15T06:53:22.425-08:00Tracking lanes<div style="background-color: white; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
I have taken the caltech cordova1 dataset (search of the term will find the location to download) to test the algorithm I develop to detect lanes. Here is the algorithm in short (without kalman filter and noise elimination):</div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
<br /></div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
First convert to bird eye view of the frame</div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
Then convert the image to grayscale<br />Apply Gaussian filter to smooth out outliers<br />Use canny method to create binary image (detect edges)<br />Run probabilistic hough transform to get the lines<br />Remove lines with wrong angle and size<br />Use perspective transform to get image co-ordinates<br />Draw the lines on original frame</div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
Draw a green line to show the vehicle path and yellow circle to show the vanishing point.</div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/aGuImk-75bE/0.jpg" src="https://www.youtube.com/embed/aGuImk-75bE?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<div style="background-color: white; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
<br /></div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
<br /></div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
Next step would be to kalman filter to deal with noises present in the detection using vehicle speed/ acceleration (and may be other sensors) to establish prior belief. Then will use the current detected lanes and update the belief. I'll be using estimated vehicle speed and use dataset 1 for training and then test using other datasets to measure its performance. Once it works I'll be using RANSAC to avoid going through all the points - its kind of slow otherwise (if Jetson TX1 performs around 20 fps I probably wont bother).</div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
<br /></div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
<br /></div>
MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com1tag:blogger.com,1999:blog-3541713569039491162.post-61305280118335609562016-10-09T15:01:00.000-07:002016-10-20T10:51:09.504-07:00Calculating Cross Track Error (CTE) and steering the robot<span style="background-color: #f6f7f6; color: #333333; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 13px;">I am one step ahead to build a full self driving toy car which is a big step toward creating a full size car.</span><br />
<span style="background-color: #f6f7f6; color: #333333; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 13px;"><br /></span>
<span style="background-color: #f6f7f6; color: #333333; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 13px;">I am now able to calculate cross track error and steer the robots front wheel to correct it. The steering driver is very crude at this moment- it has three positions Left, Middle and Right. I'll have to send more positions using a second parameter that tells exact angle. The servo motor can position itself between 0 to 180 degree angle with 1 degree resolution where 90 degree is straight forward. Wanted to make it work first then make it precise. Here is the video- please pay attention to the robots front wheel and where the lane is:</span><br />
<span style="background-color: #f6f7f6; color: #333333; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 13px;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/_TvjE8v6KqI/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/_TvjE8v6KqI?feature=player_embedded" width="320"></iframe><br />
<span style="background-color: #f6f7f6; color: #333333; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 13px;"><br /></span>
<br />
<div style="background-color: #f6f7f6; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
I first detect two lane marking lines and group them together (if one line is in 100 pixel, configurable, from another line they form one group). Then calculate the mid point for each group that gives me the list of lane marking. The lanes can be curved (I have to calculate a 3rd degree bezier spline instead of straight lines that I use right now) - but still all lines should be within 100 pixel of each other for one lane marking- its like the snake algorithm to track the lanes while not exact. </div>
<div style="background-color: #f6f7f6; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
<br /></div>
<div style="background-color: #f6f7f6; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
After I get all the lane markings I take the middle point of the lane from each neighboring pair of lane marking. Then from the image position I get the current car/ camera position and calculate the cross track error (CTE) by getting the difference from the lane mid point. </div>
<div style="background-color: #f6f7f6; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
<br /></div>
<div style="background-color: #f6f7f6; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
I use kalman filter to eliminate single frame errors and over multiple frames the values stables (in less than 20 frames it becomes almost correct). Jetson TX1 GPU array can process 20 frames per second without any optimization effort. Here is how it looks for a single frame (this is bird eye view- transformed using wrap transform of original frame):</div>
<div style="background-color: #f6f7f6; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKpaWPT3yuBes3ajoCuruhg0zXcbJHLTQKnveZ3_ISwhJ_ItqPrhpUIHK6hZeODhtQsJQlFFNA1uDHEsDQdFAzke5MMDmI5cS8iiXujcePpbk9O5fqJv4NBfhr33h-jQSj4JuPLTS7wtk/s1600/cte.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKpaWPT3yuBes3ajoCuruhg0zXcbJHLTQKnveZ3_ISwhJ_ItqPrhpUIHK6hZeODhtQsJQlFFNA1uDHEsDQdFAzke5MMDmI5cS8iiXujcePpbk9O5fqJv4NBfhr33h-jQSj4JuPLTS7wtk/s400/cte.png" width="350" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="background-color: #f6f7f6; color: #333333; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 13px;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="background-color: #f6f7f6; color: #333333; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 13px;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="background-color: #f6f7f6; color: #333333; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 13px;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="background-color: #f6f7f6; color: #333333; font-family: "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 13px;">The image shows detected lane marking using thicker straight line, lane mid point (green) and current car position (pink) and on top it shows frame #, car position, CTE value and which way it should steer the the car.</span></div>
<div style="background-color: #f6f7f6; color: #333333; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; margin-bottom: 2px; min-height: 15px; word-break: break-word;">
<br /></div>
MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-3541713569039491162.post-46701914135523497152016-08-24T09:14:00.001-07:002016-08-24T09:14:31.324-07:00Detecting lines using proabilistic hough transformWe use <code>Canny</code> algorithm to create a binary image and then use the <code>HoughLinesP</code> function detects line in binary image.
<p>
First we take the bird eye view image of straight labe for processing.
<p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmHQ6pq2AiKshLkUmrGRj16O4AHW9XZhRYPldUxlkqkqOy5ptBwzY0ESfL4c8op84IFzNaPQ2gMfDyF4VyOn8YoQOp4k-SPCi8qWXBpxw8qXy-OAYS6GWOMDoLL_K_bwW1nDFSjgmbRwk/s1600/filtered.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmHQ6pq2AiKshLkUmrGRj16O4AHW9XZhRYPldUxlkqkqOy5ptBwzY0ESfL4c8op84IFzNaPQ2gMfDyF4VyOn8YoQOp4k-SPCi8qWXBpxw8qXy-OAYS6GWOMDoLL_K_bwW1nDFSjgmbRwk/s400/filtered.jpg" width="400" height="177" /></a></div>
<p>
When the <code>HoughLinesP</code> is applied we get following resulting lines:
</p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6FX78h02c_54ecSWJg330vbWy-rC-W0bvy0tyhxUmWXTSFw1IbST6Z0bu7t0dBp1O76_pDDUuXH5CaOBkCqH6due3_68OxfNmK-alco9gbBWkWfFCVDMXQenMTe7UFe6nqRxatixCmxE/s1600/hough1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6FX78h02c_54ecSWJg330vbWy-rC-W0bvy0tyhxUmWXTSFw1IbST6Z0bu7t0dBp1O76_pDDUuXH5CaOBkCqH6due3_68OxfNmK-alco9gbBWkWfFCVDMXQenMTe7UFe6nqRxatixCmxE/s400/hough1.jpg" width="400" height="177" /></a></div>
<p>
Then we take a curved lane image
</p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8zN7QQ2b706ilxpd0SEovDabthxu8uvWcYuoUlcwjOq-UjhYHcK7gyfSsmAhKWGoTYbSYB8ASwJsgkyDa1gWRz6psHI0Cfs-6DCZRc-IPf3BINdY8HuJanSMUqmVFiLthUA1MceD5Qzw/s1600/curved.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8zN7QQ2b706ilxpd0SEovDabthxu8uvWcYuoUlcwjOq-UjhYHcK7gyfSsmAhKWGoTYbSYB8ASwJsgkyDa1gWRz6psHI0Cfs-6DCZRc-IPf3BINdY8HuJanSMUqmVFiLthUA1MceD5Qzw/s400/curved.jpg" width="400" height="252" /></a></div>
<p>
When the <code>HoughLinesP</code> is applied we get following resulting lines:
</p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvkYl0JPOFKJKAjJ6g8U46ooQVo3Urz3mRmHtoawI1BJ6RzamZjeDHh2SEj3X0ezrl1K6xbXWwQ5IZGiJx17bUH9PJrEaf2hhUrR9uURyqQ0Ar8xvF87Mw0tg-S0aTqz-6UMkQGNNWhOA/s1600/hough2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvkYl0JPOFKJKAjJ6g8U46ooQVo3Urz3mRmHtoawI1BJ6RzamZjeDHh2SEj3X0ezrl1K6xbXWwQ5IZGiJx17bUH9PJrEaf2hhUrR9uURyqQ0Ar8xvF87Mw0tg-S0aTqz-6UMkQGNNWhOA/s400/hough2.jpg" width="400" height="252" /></a></div>
<p>
Here is the code.
</p>
<pre>
int hough(const char *input_path, const char *output_path)
{
Mat src_o, src, dst;
src_o = imread(input_path);
cvtColor(src_o, src, CV_BGR2GRAY );
Canny(src, dst, 200, 255, 3 );
vector<Vec4i> lines;
HoughLinesP(dst, lines, 1, CV_PI/180, 20, 10, 20 );
std::map <int, bool> candidate;
for( size_t i = 0; i < lines.size(); i++ )
{
line( src_o, Point(lines[i][0], lines[i][1]),
Point(lines[i][2], lines[i][3]), Scalar(0,0,255), 3, 8 );
}
imwrite(output_path, src_o);
return 0;
}
</pre>
<p>
Next step is to take the lines and calculate the lane midpoint.
</p>MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-3541713569039491162.post-82659478566510140072016-08-21T00:07:00.000-07:002016-08-21T22:02:14.069-07:00The self driving car platform (GTAR)We are using the Traxxas Slash 4x4 and Jetson TX1 (which provides 1 TeraFLOPs peak performance) to build the self driving car platform.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgebXz8-7sQ5r3XG9JzbkqVo4Chn7eWbclrpnQO6qzUnBrFgDuKMSwAdgzws6glgr6JN52OUGFwW5NzWCs_hs5aEAmFz_gvR1lyH_hYK3SRgROQ55rlnezXV90qVDVmhRBl1jFAiXrZVf0/s1600/1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgebXz8-7sQ5r3XG9JzbkqVo4Chn7eWbclrpnQO6qzUnBrFgDuKMSwAdgzws6glgr6JN52OUGFwW5NzWCs_hs5aEAmFz_gvR1lyH_hYK3SRgROQ55rlnezXV90qVDVmhRBl1jFAiXrZVf0/s400/1.JPG" width="400" height="265" /></a></div>
First we remove the body and the radio controller from the car and added 4 narrow bolts to hold to plastic platforms.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzeckIbpnhy58vL0Nz1Yu5aPpdsGY2hymk-yYD0McN2cyOw73ibNO8WzpazN1Jkb5zR2duAMJJF7rp5PC9tQKuLGjRBU0UuTlDAQB4nsqObkb2nuRA6_OrntvRJRPDCyRqB5bfqS9jqTc/s1600/2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzeckIbpnhy58vL0Nz1Yu5aPpdsGY2hymk-yYD0McN2cyOw73ibNO8WzpazN1Jkb5zR2duAMJJF7rp5PC9tQKuLGjRBU0UuTlDAQB4nsqObkb2nuRA6_OrntvRJRPDCyRqB5bfqS9jqTc/s400/2.JPG" width="400" height="265" /></a></div>
We are going to use two piece of plastic sheets like below. Both has a few holes.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinA6pZSF81f_Okbda0Yk9v8nIAVPJnHLxJ-S1EDGoWFm2u8AniA0ShUlK36HeCFO2VbxD1g1HVALPfljsH8cwhHxP22qObSositrroc-63i0MuAhouxI6xNKiZFaEVJXhgcgIp8caYJqA/s1600/3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEinA6pZSF81f_Okbda0Yk9v8nIAVPJnHLxJ-S1EDGoWFm2u8AniA0ShUlK36HeCFO2VbxD1g1HVALPfljsH8cwhHxP22qObSositrroc-63i0MuAhouxI6xNKiZFaEVJXhgcgIp8caYJqA/s400/3.JPG" width="400" height="265" /></a></div>
<p>
Here is how it looks after adding the lower platform.
</p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMTM48XWNK3K9QTzVnkEiF0Lv_gve51HJZj8tPHT_44WwNDWD0BXzPJh5_rTnbqiM0_q1i2SWD46L7Y4gMAIjGqAlvixSe7u2ssxr2iF-rUBOo3gGAaRPBfeUp6l2rMKXJUKV9gYcQsBE/s1600/4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMTM48XWNK3K9QTzVnkEiF0Lv_gve51HJZj8tPHT_44WwNDWD0BXzPJh5_rTnbqiM0_q1i2SWD46L7Y4gMAIjGqAlvixSe7u2ssxr2iF-rUBOo3gGAaRPBfeUp6l2rMKXJUKV9gYcQsBE/s400/4.JPG" width="400" height="265" /></a></div>
<p>Now we detach the Jetson TX1 bottom plastic base and remove the plastic legs.</p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAvEnDVj5I7szkGR-mXlEvMzZKenSSO3eGQqu6PZq_2bSVIm0JhlscdWu7dVQOTs83v2FaaeaVD9aS6_QUCGBXPKlo3q85kJpUggxO9MctvtqU6Ji93vGjJQyxmMFJpXyrn1dYAyVfjeU/s1600/5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjAvEnDVj5I7szkGR-mXlEvMzZKenSSO3eGQqu6PZq_2bSVIm0JhlscdWu7dVQOTs83v2FaaeaVD9aS6_QUCGBXPKlo3q85kJpUggxO9MctvtqU6Ji93vGjJQyxmMFJpXyrn1dYAyVfjeU/s400/5.JPG" width="400" height="265" /></a></div>
<p>Attach the Jetson TX1 plastic base to the lower platform. The onboard camera is facing front.</p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj3BV9BmHzJVcD7r2Xl7j1TiAAGpRv6ZEsOErKydN9SDWn8o4xfoCkDrVht-BY1YetA8PtMgdAUo3H4Kv8uZxOsmFPE2CsN9Fw2i8_Hsh1qvVJ3H9_WGjNRDrpCVZMe2-6xRfFnma62pY/s1600/6.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj3BV9BmHzJVcD7r2Xl7j1TiAAGpRv6ZEsOErKydN9SDWn8o4xfoCkDrVht-BY1YetA8PtMgdAUo3H4Kv8uZxOsmFPE2CsN9Fw2i8_Hsh1qvVJ3H9_WGjNRDrpCVZMe2-6xRfFnma62pY/s400/6.JPG" width="400" height="265" /></a></div>
<p>Add the Jetson TX1 board and the power supply to the platform<p>.
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1-pQGV9FlzXS9WHQsAl5SBQSziHrqkHHnu5CK1QHx6oMu3EiD9ryN9MKA35qylz6_lm4RcV2aABugcETpJySq2jC-3w3NnSumeAvGPW9857l91o4Tsc3cHy7BbLwbnRd6_mAgWSTcsmY/s1600/7.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1-pQGV9FlzXS9WHQsAl5SBQSziHrqkHHnu5CK1QHx6oMu3EiD9ryN9MKA35qylz6_lm4RcV2aABugcETpJySq2jC-3w3NnSumeAvGPW9857l91o4Tsc3cHy7BbLwbnRd6_mAgWSTcsmY/s400/7.JPG" width="400" height="265" /></a></div>
<p>And add the upper platform and the sonar sensor in the front.</p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpmLYtHVohHSHH3JsHB0uZj6hXdzKdPpF9KmnNdmG6YoRNh5oXs6mLer-5MZsxY6Q66bgBT_9SbtsuWUSErJKiNW3uCFeADAj2F41Wp0nMKNl4dB0ininj0GNu6XysTNCZIZMnCZCvSmE/s1600/8.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpmLYtHVohHSHH3JsHB0uZj6hXdzKdPpF9KmnNdmG6YoRNh5oXs6mLer-5MZsxY6Q66bgBT_9SbtsuWUSErJKiNW3uCFeADAj2F41Wp0nMKNl4dB0ininj0GNu6XysTNCZIZMnCZCvSmE/s400/8.JPG" width="400" height="265" /></a></div>
<p>I have not received the camera yet. The camera will be placed on the upper platform and the GPS, gyroscope and compass will also be placed on the top platform.</p>
<p>
The lower platform needs some cut to free the front and rear wheels. Other than that its complete and we are ready to add the remaining electronics and start coding.
</p>
<p>Finally the sensor detecting obstacle in front of it:</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/vUZyq_xn7DE" frameborder="0" allowfullscreen></iframe>MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com3tag:blogger.com,1999:blog-3541713569039491162.post-15846917418356092792016-08-18T15:43:00.001-07:002016-08-25T16:05:25.869-07:00Lane Detection: Step 1: Inverse/Wrap perspective mappingTo detect lane we first do some image transformation. First lets consider the following image. It has lane marking (green). The lanes are almost parallel in real world, but as we see in the image the lanes seem wider at the bottom and narrower at the top.
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8CkLQyNrYoJVutLxoTN2gZ4MbjUHAZatypQT609bUvQmfYIgp0Gvs4MqCnGC8zvGthsK0_2ENQ_KhdivZ9wQHOzJlfEybrhnAA5WcSdRVmchsbua6mUlADrRRdqHFe2jQaGbq6wHauqs/s1600/lane1s_labeled.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8CkLQyNrYoJVutLxoTN2gZ4MbjUHAZatypQT609bUvQmfYIgp0Gvs4MqCnGC8zvGthsK0_2ENQ_KhdivZ9wQHOzJlfEybrhnAA5WcSdRVmchsbua6mUlADrRRdqHFe2jQaGbq6wHauqs/s400/lane1s_labeled.jpg" width="400" /></a></div>
<br />
To make it easier we want to have a bird eye view so that the lane lines become parallel to each other. OpenCV has built-in function to do the transformation. So we just use that. The process is called wrap perspective mapping (WPM) and we need to provide 4 points on the original image (I have marked them red in the image above) and a corresponding real world co-ordinate. I have just made up the numbers to make them parallel.
<br />
Here is the result:
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOLcASsUf7h-IKx_x4ksJcX6ksYSqiK-ZzSJBNUnqbOwyy7Pum63korJsInu0Yohpe9ljShNm2JzsAbmgOJ4S9Z76iuRjNahmDm4tNMJiquy_SE36K08ImA4FSBkpfYrFH6T0dtPBHKxg/s1600/ipm.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOLcASsUf7h-IKx_x4ksJcX6ksYSqiK-ZzSJBNUnqbOwyy7Pum63korJsInu0Yohpe9ljShNm2JzsAbmgOJ4S9Z76iuRjNahmDm4tNMJiquy_SE36K08ImA4FSBkpfYrFH6T0dtPBHKxg/s400/ipm.jpg" width="400" /></a></div>
And here is the code (just a demo- without error handling):
<br />
<pre>#include "iostream"
#include "cv.h"
#include "highgui.h"
using namespace std;
using namespace cv;
void ipm(const char *input_path, const char *output_path)
{
Mat src, dst;
src=imread(input_path, 1);
Point2f img_vertices[4];
img_vertices[0] = Point(213, 104);
img_vertices[1] = Point(410, 106);
img_vertices[2] = Point(6, 372);
img_vertices[3] = Point(540, 400);
Point2f world_vertices[4];
world_vertices[0] = Point(26, 104);
world_vertices[1] = Point(540, 106);
world_vertices[2] = Point(26, 400);
world_vertices[3] = Point(540, 400);
Mat warpMatrix = getPerspectiveTransform(img_vertices, world_vertices);
warpPerspective(src, dst, warpMatrix, dst.size(), INTER_LINEAR);
imwrite(output_path, dst);
}
int main(int argc, char** argv)
{
ipm(argv[1], "ipm.jpg");
return 0;
}
At the end of all steps the goal is to detect all paths and objects </pre>
<pre>in the http://www.cvlibs.net/datasets/kitti/index.php dataset.</pre>
MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com0tag:blogger.com,1999:blog-3541713569039491162.post-23326418560649601292015-04-03T08:18:00.001-07:002015-05-15T20:39:27.227-07:00Self driving toy car<p>
While companies are building self driving cars for adults we are building self driving cars for kids. We took a car like this image and converted it into a self driving one.
</p>
<p>
</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWXDVRGKnD4bXYHKI0GUXoRZFcopw-veDKkvCC-MudiEAhUz2bwA9b7RhvOTCLqZv0GTlTIv64SsK6r9Cdcu1kwXgqqYYR8t8WrBiADhVx9KNrc00pjVZyW7AAK5yJQQQPMoSOkE-qWCs/s1600/IMG_0111.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWXDVRGKnD4bXYHKI0GUXoRZFcopw-veDKkvCC-MudiEAhUz2bwA9b7RhvOTCLqZv0GTlTIv64SsK6r9Cdcu1kwXgqqYYR8t8WrBiADhVx9KNrc00pjVZyW7AAK5yJQQQPMoSOkE-qWCs/s320/IMG_0111.JPG" /></a></div>
</p>
<p>
Kids (3-5 years) can ride on it. The car runs at 5mph maximum speed and uses rechargeable batteries as power source.
</p>
<!--p>
We are not sure about pricing yet, but we are trying to keep our basic model under 500$.
</p-->
<p>
If you are interested about the first edition of the cool car (or interested to get the source code of the probabilistic algorithm) subscribe to the mailing list on the right (Follow by Email). We will not use it for any other purpose than infrequent communication about the self driving toy car and we will not give it to others without your permission. You will be able to unsubscribe anytime.MAhttp://www.blogger.com/profile/15647592837919229742noreply@blogger.com1