ADAS framework challenges

After studying the ADAS solution providers based on computer vision, one thing stands out to me. A lot of vendors have excellent demos of individual algorithms, such as lane detection, pedestrian detection, etc. But when asked if they're putting it all together, a common answer is either we're working on it or we have a fixed set of algorithms running together. I believe that putting algorithms together in an efficient manner will be a fundamental challenge to be resolved for mass-market low-cost low-power ADAS.

Here's what I see as the primary challenges that need to be solved:

1. Syntactical expression of pipelines, that enable re-use of processing blocks
2. There are too many technologies to learn for a developer:
1. Algorithms
2. Operating systems (for scheduling the algorithms to take advantage of multi-core architectures)
3. SIMD / Vectorization (for taking advantage of special architectures for vision processing)
4. GPU usage - either for simple parallelism or with a deep learning framework.
3. The need to iterate quickly - as algorithms get updated and workloads shift with addition of custom hardware.

I built the audio concurrency framework that's been running on all of Qualcomm's 7K (from the first Android phone onwards), 8K and 9K platforms. The challenges there were: audio and voice concurrency, the need to handle different sampling rates and the long list of post-processing blocks. I believe that we solved it elegantly with the right balance of re-configurability and simplicity, in resource constrained (MHz and Memory) platforms.  I see similar challenges in CV based solutions for ADAS. 

It will be a fun ride.

Kuntal. 

Is your Machine Learning algorithm smarter than a 5 year old?

I'd like to offer some thoughts on how "learning" differs between the new poster child for machine learning - "Convolutional Neural Networks" and my 5 year old daughter, who is learning how to read. If you want to skip the rest of the post, here's the conclusion - fears of AI taking over the world are vastly overblown.

While reading, she started reading words one letter at at time, not restarting for every syllable. Now, when she reads, she groups more letters together and then reads them aloud at at time. Once she has read aloud the written pronunciation, she maps it to colloquial pronunciation with some hesitation. When she understands the context, the hesitation is less. Also, she has now become faster with sight words - familiar words which we can read without reading the spelling. When she began, she wouldn't remember which words she has already read so it would be a new exercise every time. Another interesting observation is that when reading "3-D," she ignored the "-" first, read 3D and then asked, what is the "-."

I showed her a puzzle book where one page had upper case letters printed on tea cups and lower case letters printed on saucers. Simply by looking at the page, she figured out that the goal was to match the upper and lower case forms and she started drawing lines from the tea cups to the saucers. She also noticed, and said aloud, that the since the colors of some of the saucers were the same, you don't have to match by color.

Then we flipped a few pages and found different pictures of two objects on two sides of a scale. Her question was - do I circle the heavier side or the lighter side?

Flip a few more pages and there is a picture of a maze. She exclaimed "Oh I love to do mazes" and traced a path through from a point that she thought was the starting point to the end. She didn't notice the start arrow at first so she simply built a path that made sense to her. Then when I said that start from the arrow, she said "oh" and effortlessly redrew the path.

In the last few days I've learnt about CNNs. I have a DSP background and have known about "traditional" ML methods such as Extended Kalman Filters for a while but CNN was new to me. The hierarchical representation does present a nice conceptual simplification of learning and classification. However, as I think about how rigorous the training procedure is for image classification, compared to how my daughter learns reading, I can safely draw the conclusion that we will be able to harness AI to our benefit. Conscious harm from an AI entity is in the far off future.

And I'll teach her to play Go. Watch out Google.

Later,
Kuntal.

Cruising "altitude" reached - you are now free to move around the cabin

California designates special lanes for High Occupancy Vehicles (HOV) to promote car-pooling and thereby achieve less congestion on the highway. Certain hybrid and electric vehicles have been allowed to use these lanes even if they have a single occupant to reduce their climate impact. One can expect that these lanes will be re-purposed or expanded to "Autonomous Vehicles Only" (AVO) lanes. For a vehicle traveling in such lanes, what kind of rules may apply? This blog post extends my previous post on "Rules for autonomous cars."

I think we can draw an analogy from aircraft travel - where passengers are allowed to move around once the airplane reaches cruising altitude. While current highway rules mandate that the front passengers must wear seatbelts at all times, one can expect that rules around vehicles in autonomous mode in dedicated lanes may be different. For instance, on a long distance inter-city journey, one  may expect to stay in an AVO lane for hours at a stretch and during that time vehicle occupants can be allowed to stretch their legs safely by moving around the vehicle.

The notification to enter and exit "cruise" mode could be automated through DSRC with appropriate margins to ensure that occupants can return to their seats and wear seatbelts well before the vehicle exits the AVO lane.

The USDoT's Intelligent Transport System's office is already considering speed limited "Eco" lanes to reduce emissions and AVO lanes may already be under consideration.

That's all for today.
Kuntal.

ADAS in pop culture

Last night's episode of The Good Wife (S7, E7) appears to be the first depiction of autonomous cars in popular culture. The episode touched upon many of the subjects currently being discussed in the automobile industry - liability, security and machine learning. It turned out that the key to assign liability in the case was how the hard-disk that maintained the running log of the vehicle got erased.


This brings us to the next rule for autonomous vehicles, that essentially mirrors the proverbial "black box" inside airplanes. The "black box" is supposed to keep a running log of all that's happening in the cockpit, including aircraft performance metrics and cockpit conversations and radio communications.

Autonomous vehicles must be required to log a standard set of material events up to the point of an untoward event. The types of the data/events that must be stored are:
1. Sensor inputs
2. Drivetrain commands
3. Radio communications
4. CAN bus / Ethernet traffic
5. Central Processing Unit - memory, scheduling and execution logs

Once an event occurs, that memory needs to become non-overwritable until it is uploaded to a server which stores the information for the lifetime of the vehicle.

That's all for the day.
Kuntal.

Rules for autonomous cars

While policy develops and researchers put ethical decision making in software, continuing my last week's post on how life might change with autonomous cars, here is one rule to start with:

• An adult must accompany a minor in an autonomous car.

The reason for this is to handle eventualities in case of an accident or a malfunction, such as the car running out of fuel/charge. As a consequence, some government policy is needed to provide the necessary safeguards preventing an adult from putting an accompanied minor in an autonomous car:

1. Biometric / driver's license based unlock of the car's navigation system.
2. Visual monitoring for the presence of the adult in the vehicle

I will need to collect my thoughts on the safety framework needed in place for adults with diminished capacity for driving. What are your thoughts on the topic?

School drop offs with autonomous cars

After I read the announcement by Tesla earlier this week about new auto pilot features being made available via a software upgrade to the Model S, I thought how our daily routines might change when fully autonomous cars become the norm.

On the way to drop my kids off to school, I asked my middle-school son: "If this car were fully autonomous, would you still need me to drop you to school?" He thought for a moment and then he said that no - then he'd be fine with taking the car by himself.

Clearly, when things go fine there's no need to have an adult chaperone along. But if the car were to get into an accident then I wouldn't want my child to be all by himself. Since he does take the bus on the way back home, it made me realize that the comfort factor of letting him take the school bus home actually arises from there being a trusted human driver of the bus. Now as he gets older, I may feel OK with him taking an autonomous car by himself, probably before he comes of driving age. But for my younger one, who still has a few years to go before she gets to middle school, I'm sure I'd be her chaperone.


In conclusion, even with fully autonomous cars, I'd still be dropping my kids off to school. No change expected to that daily routine.

Regards,
Kuntal.

Defeat Device and Open Source

As the story about Volkswagen's emission test failure evolves and "defeat device" enters into everyday parlance, I believe I have some insight to provide on software testing.
The Electronic Frontier Foundation says that "Researchers Could Have Uncovered Volkswagen’s Emissions Cheat" if they had access to the source code. As a professional software developer, I feel that this may be a hard argument to make.

The process of building of high quality software is no different from that involved in building something more physical such as say Gorilla Glass or even a new pharmaceutical drug. Simply making it available for others to copy would eventually de-incentivize the development of Intellectual Property that I believe is the foundation of the higher living standards in the United States. Instead I propose what I consider a reasonable middle ground

                                                          Open Interfaces

Instead of making the source open, companies making products that affect public safety must be asked to:
1. Modularize their code (this is a necessary step to identify #2)
2. Publish the interface to each module
3. Provide simulation based test platforms and test cases, where the vehicle is simulated but the ECU software is identical to the software on the vehicle.

VW's emission controller software would monitor the steering wheel position, how long the engine was at a steady speed, etc. and deduce that the vehicle was in test mode and accordingly reduce emissions. When the vehicle was driven in regular conditions, the emissions would fall back to a much higher level.
If VW had published the interface to their emissions ECU, then that itself would have raised questions - clearly, why would an emissions ECU need to know about the steering wheel position?  By providing additional information required in #3 above, regulators and/or researchers could have further deduced that something was amiss.
Apart from keeping proprietary code secret, the Open Interfaces method has the following advantages:
1. A review of the test cases is easier than the review of every line of code
2. The system becomes testable and the tests can be used across manufacturers. If test case input and output data formats were standardized then the EPA/CARB could test the simulated behavior of the vehicle. This incidentally can be used to "fuzz" input data and enhance security too.
I'm already 15 minutes past lunch-time, so that's all for this time.
Kuntal.