That is the ultimate installment in a sequence concerning the Midwest Defenders’ construct for the OpenCV Spatial AI Competitors.
In regards to the Writer
Jesse Brockmann is a senior software program engineer with over 20 years of expertise. Jesse works for a big company designing real-time simulation software program, began programming on an Apple IIe on the age of six and has received a number of AVC occasion through the years. That is the final installment within the Spatial AI Competitors Collection, thanks for tuning in everybody!
Learn the first and second blogs if you’re all in favour of or have to atone for the logistics of this undertaking!
Curses primarily based knowledge show and rover management program
Persevering with from the place we left off…
A run datalog file is created throughout every run that logs the x and y place of the rover, together with the relative positions of the markers through the run. This might be used to generate a map with the places of every marker. As a result of noisy nature of the situation knowledge of the markers, some larger stage algorithm would have to be used to find the markers with a excessive diploma of certainty primarily based on run knowledge from a number of runs. Location knowledge is collected utilizing encoder output and heading knowledge from the BNO-055.
Instance Run Information
rover: 499.995667 499.780090 356.750000
impediment: marker -31.577566 -145.256805 912.055542 0.993661 0.049720 0.038363
finish
rover: 499.995667 499.780090 356.750000
impediment: marker -28.925745 -146.694870 895.138916 0.995610 0.050098 0.037815
impediment: marker 189.767776 -114.743301 1912.000000 0.697999 0.030058 0.016747
finish
rover: 499.995667 499.780090 356.750000
impediment: marker -28.925745 -146.694870 895.138916 0.995610 0.050098 0.037815
impediment: marker 189.767776 -114.743301 1912.000000 0.697999 0.030058 0.016747
finish
rover: 499.995667 499.780090 356.750000
impediment: marker -28.925745 -146.694870 895.138916 0.995610 0.050098 0.037815
impediment: marker 189.767776 -114.743301 1912.000000 0.697999 0.030058 0.016747
finish
rover: 499.995667 499.780090 356.312500
impediment: marker -28.668959 -147.440353 887.192322 0.995123 0.052451 0.037603
finish
Path rover navigated utilizing markers and indicators
An excellent challenge was to detect unknown objects and keep away from them. Code was examined to transform the depth map from the Oak-D-Lite into a degree cloud that might be used to find obstacles.
Watch Jesse’s Webinar with OpenCV about this undertaking!
A hand may be seen within the level cloud created from Oak-D-Lite depth map
This end result proved that such an answer could also be attainable, however a lot work must be performed to interpret the purpose cloud to supply helpful data to be used with navigation and the timeframe of this undertaking didn’t enable us to finish an answer.
Based mostly on our consequence, it’s clear it is a nicely outlined drawback that may be solved utilizing an Oak-D product. A ultimate answer would seemingly use the depth output or a lidar to keep away from obstacles not detected by the neural community. Use of this undertaking might be an excellent motivational instrument for teenagers all in favour of STEM.
The ultimate video with an indication of the rover
Right now the supply code for this undertaking just isn’t open supply. This is because of issues about any reused code such because the DepthAI instance code, pdcurses supply and so forth. Nevertheless the supply code will likely be supplied to anybody who requests it as a part of the overview course of for our entry into this contest. As time permits a correct audit of the code will likely be performed, and launched as soon as any licensing points are resolved.
Points That Got here Up Throughout Improvement
The Lego Land Rover was not accessible on this time-frame from buy from Lego, so was acquired by way of third get together sellers. The conversion course of from a local darknet output to a Oak-D-Lite appropriate blob is a bit troublesome, and I truly needed to create two conda environments to get the conversion working. A part of the method is in tensorflow 2, and a part of it’s in tensorflow 1. If tensorflow 2 was used for all steps the blob couldn’t be created attributable to unsupported choices. The depth data reported by Oak-D-Lite appears to have many limitations and the primary one I encountered is {that a} small object by itself in house can have very poor Z distances reported. These are sometimes a lot additional away than the article truly is. Our rovers would truly run into indicators and the space reported can be as a lot as two meters away. At no time whereas touring in direction of objects did the reported distance ever develop into lower than one meter. Placing a field, or one other object behind the indicators appears to unravel this challenge. One other challenge is the autofocus Oak-D-Lite, which is consistently searching and sometimes will get confused and the entire scene will likely be blurry. Additionally, a rover is a poor place for an autofocus digicam because the vibrations will most likely forestall it from working anyway. In consequence we used the mounted focus digicam for almost all of testing.
One other challenge was the facility necessities of the Oak-D-Lite. It appears unstable with out a splitter for the facility/knowledge. The Raspberry Pi V4 simply would not have sufficient energy to maintain it working. The problem is intermittent operation. It’d work advantageous for a lot of minutes earlier than failing and having to restart this system and even restart the Oak-D-Lite by unplugging and replugging in. Ideally, it ought to report an influence challenge such because the Pi does when underpowered. There additionally appears to be points attempting to debug when utilizing the Depth AI core. Sitting at a breakpoint for some time will trigger the method to crash attributable to points with the varied Depth AI threads. Lastly, utilizing a Lego primarily based car just isn’t really helpful for basic use in this sort of testing. Even an RC automotive of comparable price is a a lot better alternative because of the frail nature of a lego drivetrain in a car of this dimension.
We wish to thank Sparkfun for offering elements for the construct and Roboflow for his or her assist and use of the roboflow software program throughout this contest. Lastly, we wish to thank OpenCV, Intel, Microsoft and Luxonis for this competitors.
Jesse’s Rover Scout
Jesse has plans to make use of the Oak-D-Lite and the code from this competitors to compete in F1TENTH and Robo-Columbus occasions sooner or later.
