Halide for ADAS

Luggage space will decide the winner of autonomous vehicle race.

Automakers have two choices towards meeting sensor fusion compute requirements for autonomous vehicles - take up all the trunk space for server-grade computers OR optimize processing so that compute can fit in a pizza box and leave room in the trunk for luggage.

As I described in my earlier post on ADAS framework challenges, optimizing sensor processing tasks is a time intensive process and may well be the rate-determining factor in deciding on how to use the trunk.

I have looked at image processing frameworks available in the industry and I have come to the conclusion that Halide is the most suitable framework to quickly connect sensor processing algorithms and experiment with their optimal load distribution on an SoC. I encourage system designers to learn about Halide directly from the link above. The key features that make Halide attractive for an ADAS sensor compute framework are:

• Split algorithm from implementation
• This allows the algorithm designer to focus on the algorithm and not on how to optimally construct it for a particular SoC.
• Enable implementation iteration over cores, threads, rows, tiles, SIMD sizes
• An SoC expert can take a working algorithm and then experiment with various load distributions without worrying about changing the algorithm. Thus if a particular SoC has a mix of CPU, GPU, DSP, vision-specific accelerators, the implementation can quickly take advantage of these elements. 
• Function-call type pipeline constructs
• Frameworks such as OpenVX mistakenly focus on pipeline connections and conformance as the problems to be solved, but these are rather trivial aspects as compared to the bigger problem of quick experimentation with load-distribution.
• Defined interfaces between Halide and C++ based functions such as that in OpenCV
• Halide isn't suitable for control-intensive algorithms such as finding contours. One can easily integrate OpenCV with Halide. 
• The ability to visualize performance
• This is an often overlooked aspect of optimization. Halide has a built-in, elegant way of visualizing performance.

Finally, although the Halide run-time and tools haven't yet been qualified for ISO26262, the way the language is constructed makes it well suited for ADAS usage nonetheless. Image boundaries (a common source of bugs) are syntactically handled. Bounds inference on input and output buffers is automated. 

All said and done, Halide offers a faster path to move from server-sized compute to optimized compute in a smaller form factor. I'd rather buy a car that I can load my luggage into. 

And it might be a while until my next post.

Kuntal. 

Machine Learning - more 5 year old milestones

Sundar spoke about the expected impact of machine learning at I/O. I'd like to put forth my observations of my 5yo daughter, who has just learnt to read, continuing from my earlier post on the topic. My sensitivity of this topic has been heightened as I work on autonomous driving in my day job.

So here we are reading Dick and Jane, which is still an excellent book for beginner readers.


As one progresses through the book you get introduced to the family. Here are their pets, on the page to the right

The cat's name is Puff, the teddy bear is Tim and the dog's name is Spot.

At one point in the book my daughter sees these pictures:

Without reading the text, my daughter said that it looks like they're making Spot and Puff out of PlayDoh. She thought Sally (the girl in the middle) was going to make a Tic Tac Toe game. She was thrilled when she read the text and found that they were indeed making their pets!

The memory from her own past experience of making things with PlayDoh was applied effortlessly by her. She chose to label the pets with names, not leaving them as generic "dogs" and "cats".

After seeing the final Tim, she shook her head and said that it didn't quite look like him.


I think it's still pretty hard for CV algorithms to get to this point without significant human help.
To add to it, she saw the text in the following image and said that it looks like an upside down staircase. This sort of lateral thinking is probably further away than teaching AlphaGo to play chess too.



That's all for today.
Kuntal.