|Velodyne has become the gold standard for automotive LIDAR, used by almost all the major players trying to produce driverless cars.|
The billion-dollar widget steering the driverless car industry
by Ben Popper
Oct 18, 2017, 8:02am EDT
No matter what it took, David Hall was going to kill that clown. He maneuvered Drillzilla for another ramming run. The robot was squat and heavy, with serrated blades coming off one end and a sharp drill whirling on the other. Across the arena, Conquering Clown awaited. It had the face of a goofy jokester, but its hands were a pair of smashing hammers and its body was equipped with a pair of circular saws.
Drillzilla managed to flank the clown, then ram it, sneaking blades under its body and lifting it up off its wheels. Its opponent was helpless, and Drillzilla pushed it onto a waiting geyser of flames. As the audience cheered, the clown’s grinning face melted away. “We were in there to make great TV,” recalls Hall with a chuckle, “and damn it, we were successful.”
The year was 2001, and this was the third round of the Robot Wars Annihilator Challenge. The show pitted homemade gladiators against one another. Hall, an eccentric inventor, was best known as the creator of a high-end subwoofer. His company, Velodyne, had around 60 employees and a few million dollars in annual sales. But Hall had grown bored with the audio industry, and was trying his hand at building robots. Perhaps, he thought, Velodyne could find a new product to manufacture. At the very least he could have some fun.
Hall admits he “bent the rules a little” to win the competition — he camouflaged the robot’s wheels as “legs” — and, after other teams complained, the judges banned Hall’s approach for future seasons. He took that as a sign to move on. But he was eager to continue his work on vehicles in the public eye.
He got his chance in 2004, when the US Army’s research division, DARPA, held its first Grand Challenge. Teams were asked to design an automobile that could autonomously navigate its way through a 150-mile course. Hall took a bunch of the motor controls and code from his robots and got to work on a self-driving truck.
There were 15 vehicles in that first race, all competing for a $1 million prize. Not a single one finished the course. Hall used stereo cameras to see the road and avoid obstacles. “I could see some of the road all the time, and all the road some of the time,” he says now. But the vehicles were constantly confused. “There was all kinds of weird artifacts. It would see a fence five feet in front of it and throw on the brakes.”
A few of the teams were using a technology called LIDAR, which uses laser beams to sense objects and measure their range. “I didn’t know what a LIDAR was,” admits Hall, “but Jim McBride from Ford kept bending my ear about how LIDAR was the solution for all his problems. I kinda made a mental note — maybe I’ll look into it when I’m bored. A few months later, I started looking into it, and the more I did, the more intrigued I was.”
Hall returned the next year with a custom LIDAR unit he had designed and built. Instead of putting a laser scanner on the front of his car, as most teams had done, he put it on the roof. And instead of looking forward, it scanned in all directions at once. It was a radically different approach to the technology.
He didn’t win the race, but his design impressed competitors, and the DARPA challenge convinced a number of the participants that driverless cars were no longer an unattainable fantasy. The teams from Stanford and CMU would go on to help found the self-driving projects at Google and Uber. And as those massive corporations built out their self-driving ambitions, they turned to Velodyne for LIDAR.
At age 66, Hall finds himself in an enviable position. LIDAR has eclipsed Velodyne’s audio business; the company now has over 400 full-time employees people and generates hundreds of millions in annual revenue. Velodyne’s most recent round of funding made Hall a billionaire. And earlier this year, the company announced it was on the verge of rolling out a new kind of LIDAR, one that would make the technology radically cheaper, allowing it to move from expensive test vehicles to a standard piece of an affordable consumer sedan.
After four decades of inventing, five product categories, and a lot of wrong turns, Hall has stumbled onto a high-tech widget that is poised to become an essential piece of a trillion-dollar self-driving auto industry. Setting aside Steve Jobs’ return to Apple, it might be Silicon Valley’s greatest second act.
There are many ways of seeing the world. Humans rely on our eyes, which interpret incoming rays of light. On a sunny day, they allow us to see a richly nuanced view of the world. In total darkness, they aren’t much use. Some animals, like bats, use echolocation. While hunting small insects at night, they emit high-pitched noises, then interpret the sound waves that bounce back to get a picture of the world.
Humans have created technologies that mimic echolocation. Sonar, used by submarines, emits a pulse of sound, then reads the waves that bounce back. Radar does the same thing, using radio waves instead. It’s found countless applications, from spotting incoming missiles to catching drivers who break the speed limit. LIDAR, short for “light detection and ranging,” adopts Radar’s approach, but uses lasers in place of radio waves. Scientists at Hughes Research Lab demonstrated the first functioning lasers in 1960, and LIDAR attempts quickly followed. Early on, it was used by government research agencies to map and measure the natural world, from cloud formations to sea floors to the surface of the Moon.
By the late 1980s, LIDAR had found its way into autonomous cars. Researchers at Carnegie Mellon University’s Navlab used a laser scanner to help detect obstacles and determine their range back in 1989, but it was not their primary sensor. “We would see, with our scanning lasers, these very grainy updates every half second,” says Dean Pomerleau, a Navlab researcher. “A kid on a bicycle would just look like a blob.” The lab used LIDAR primarily because it was good at detecting reflective materials, like lane markings and road signs. In the late ‘90s, Mitsubishi actually tried LIDAR in its driver assistance system, but its high cost made it prohibitive. As the 21st century arrived, the auto industry moved to radar and cameras, which were much cheaper.
There was a lot of LIDAR at the first DARPA Grand Challenge in 2004, but it was still a blunt instrument. It could be used to see the edges of a short tunnel, allowing cars to navigate through a space where there was little light and no GPS. But it didn’t give anything like a complete picture of the world. “Mechanically there was a limitation. How do you deflect the beam fast enough?” says Dirk Langer, who worked at CMU’s Navlab and now designs LIDAR systems for Continental Automotive. LIDAR could capture a highly detailed map of any terrain, but that required numerous scans over hours or days. It was impossible, the thinking went, to get a high-resolution picture without a delay of at least several seconds. “The scanning mechanism was just too slow for autonomous navigation,” says Langer.
Still, it was clear that LIDAR had potential. Ingesting and interpreting video data was difficult work, especially for the processors at the time. Cameras could also be easily fooled — to the computer navigation system, a speck of dust on the lens might appear like a rock on the road ahead — and as lighting conditions changed, the cameras struggled. The advantage of LIDAR was that it returned data that was easy for a computer to read: there is an object this wide, this tall, this far ahead, and this far to the left. And it didn’t matter if you were driving at dawn or at dusk, because a laser is its own light source. “Everyone knew LIDAR was the key, so any real improvement to that technology was a big deal,” says Joseph Bebel, a professor of computer science at USC who competed in the 2004 DARPA race.
Hall didn’t know the first thing about LIDAR on the eve of the race, but within a few years, he would produce the best LIDAR system in the market.
Hall grew up in Connecticut, and he was a tinkerer from an early age. Engineering ran in the family: his father helped design nuclear power plants; his grandfather, a physicist created a process for producing color photographs in the 1930s. Family legend has it that Hall’s grandfather was also a key figure in the Manhattan project, as a secret agent who had worked on the atomic bomb and kept a fake ID and PO box.
At age four, Hall built his first amplifier. “He taught himself to read schematics for science projects before he learned to read words,” his wife Marta says. By the time he was a teenager, he was wearing thick, coke-bottle glasses and spending thousands of hours in his workshop, a refuge for a kid who preferred soldering to school dances or team sports.
Hall went on to get a degree in mechanical engineering, and while he was still a student, he designed an improved version of the tachometer, a device used for measuring the spin of something like a wheel or drive shaft. “Most [tachometers] just showed you the basic RPMs. Mine could read out revolutions per hour or inches per second,” Hall says. He originally built the device for his grandfather, who used it on his boat. But Hall was also granted a patent on it in 1979, and he licensed the tech to several companies that sold it. With the income stream from those royalties, he moved straight from college to his own machine shop in Boston. “I’ve never had a real job, or a boss, and I never will,” he says.
For the first few years, Hall focused on building devices that appealed to medical and industrial customers. If a company like Raytheon had a government contract for a laser test bed, Hall would bid on the contract to manufacture the prototype. There was plenty of work, but little recognition from the wider world. Hall was drawn to the idea of building a brand name, and garnering a little bit of fame.
Hall’s brother-in-law suggested they get into the speaker business as a challenge. The two men, both audiophiles, became obsessed with the idea of crafting a perfect loudspeaker. Hall’s grandfather gave him a $250,000 loan to jump-start his business. In 1979, Hall left Boston to found his new company in California. He named it Velodyne, after his favorite style of racing bike.
Hall spent the next three years living off the royalties from his tachometer and working to solve the problem. It was his comfort zone: heads down on a problem that no one else had yet solved, something with a technical solution that could be measured objectively. By 1983, he had cracked it. That year, Hall filed a patent for a “loudspeaker with high frequency motional feedback.” It was an improvement on an approach pioneered by Philips. Hall’s design used a tiny piezoelectric accelerometer to measure movement in the speaker cone and adjust the frequency of the signal to eliminate distortion.
Velodyne’s first product, the ULD-18, was a hit with critics and consumers. “As I heard the solid, gut-punching bass note, I felt the wood floor flex under my feet! The impact of the floor shock reminded me of the shift caused by a Richter 4.5 earthquake that had hit the New York area 22 months before,” wrote a critic for Stereophile. “Yet there were no unnecessary overtones, no overhang, no disturbance of the midrange or treble sounds, just that big bass note with its clean, well-defined leading edge. All the overtones of the bass drum were there, as I had not heard them before.”
“The work we did in audio, it was the perfect playground for getting ready to do LIDAR,” Culkin says now. LIDAR presented a similar challenge to speakers: how do you measure the world around you and, with the smallest possible delay, and use that data to adjust your input signal?
Over the decades, Velodyne produced seven different speaker lines, including some of the most expensive subwoofers ever brought to market: the signature edition of its 1812 cost $25,000. But Velodyne struggled to stay healthy, and to hold the interest of its founder and CEO.
While Hall loved engineering, he wasn’t much of a businessman; his history as a polymath would rival Elon Musk, but he lacked Musk’s skills as a pitchman. He’s more Steve Wozniak than Steve Jobs.“I’ve never successfully sold one of my products,” he admits. Velodyne never managed to leverage its brand into new products or sell cheaper units that might have appealed to a broader base of consumers.
By the mid-‘90s, Hall had fallen out of love with the audio industry. “It became more and more about brute force, not clean sound. The direction I was taking things wasn’t really appreciated, so I thought I better get into a different business.” Globalization also dulled the appeal. To keep prices reasonably competitive, Velodyne had moved the production of its speakers overseas. “As soon as we shipped all the manufacturing to China, that was the end of our ability to see anything we could improve on,” Hall recalls. “You go from an inventor to an importer. I realized it’s not for me as a person.”
He began spending time exploring new fields. After a brief stint in the semiconductor business, and a couple years playing with robots, Hall landed on self-driving cars, and then LIDAR.
The second year of the DARPA Grand Challenge took place in 2005, and it was not at all like the first. A lot of teams managed to actually finish the course, and several did it more than once. Though Hall’s team didn’t win, it was clear that Velodyne was onto something special. “The race starts and all the other cars are just crawling,” says Culkin. “Meanwhile, Dave’s truck is going 50 miles an hour, straight across the desert.” Deep into the race, the Velodyne vehicle suffered a critical mechanical failure. But his LIDAR caught everyone’s attention.
What set Hall’s LIDAR apart was that it rotated, firing off short laser pulses as it spun. A sensor measured how long it took each beam of light to strike an object and then return. In that way, each burst produced a picture of a small slice of the world. A computer program then assembled the data together into a full, 360-degree picture of the environment around the car. It was a groundbreaking design, one that Hall patented in 2007, that meant you could not only see and avoid obstacles, you could localize yourself on a real-time map, allowing for navigation, even if you lost GPS.
The design was audacious. “Most people wouldn’t attempt something like that,” says Langer, the LIDAR system developer for Continental Automotive. Hall had packed 64 separate lasers inside a sphere rotating 900 times a minute. If the data was bad, there was no way to see what was going wrong inside the scanner during operation, “but the results were quite impressive.”
By 2007, when DARPA’s Urban Grand Challenge rolled around, Hall had requests from more than 10 other teams looking to utilize his technology. He took over part of the manufacturing space Velodyne typically used to build subwoofers, and dragged a few engineers onto a special team. He was able to charge about $80,000 for each LIDAR system. The new business emerged just in time to save Velodyne’s audio division in the midst of the 2008 financial crisis.
Early customers included mapping companies like NavTech and construction companies like Caterpillar
“When the recession comes, the first thing people don’t buy are fancy subwoofers,” Hall says. In 2009, Hall and his brother were in talks to sell the entire audio business for $6 million, and the company went through several rounds of layoffs. “Those were a little traumatic, to go through that a few times in a year.” But they managed to sell enough LIDAR units to keep the stereo side afloat.
By 2010, orders for LIDAR were starting to roll in. Early customers included mapping companies like NavTech and construction companies like Caterpillar, which wanted to use LIDAR for autonomous vehicles it could deploy in areas with few humans, like deep underground in mines. By 2012, the idea of putting self-driving cars onto ordinary roads was gaining steam, with Google tapping Velodyne for its LIDAR systems. Uber followed a few years later. Apple’s self-driving cars use six Velodyne LIDAR per vehicle.
In years since, Velodyne has become the gold standard for automotive LIDAR, used by almost all the major players trying to produce driverless cars. While cameras and radar can play a role in future systems, most agree LIDAR will remain a critical piece of the puzzle. “I wouldn’t put an autonomous car on the road that didn’t have LIDAR,” says Aston Martin CEO Andy Palmer. That leaves the multibillion-dollar question: can a company that lucked into its lead stay ahead of an ever-growing crowd of competitors?
In July, I visited Velodyne’s new advanced manufacturing facility in San Jose, California. On the ground floor, teams of engineers in lab coats and protective glasses calibrated new LIDAR units, measuring the time of flight for laser pulses as they bounced back and forth along a firing range. Nearby, workers wearing rubber gloves stared through magnifying glasses as they assembled circuit boards.
Offices for executives and workstations for administrative staff sit one flight above the factory floor. At the time of my visit, several banks of desks sat empty, and an entire section of the floor was bare, set aside for new hires. Mike Jellen, the company’s president, says Velodyne’s goal is to increase the company’s revenues by three to five times this year, and to staff up accordingly. He claims the 13 largest automakers and at least 22 very large tech companies are using Velodyne LIDAR. “There are 30 projects ongoing, many registered with California DMV. Whenever we run a forecast for the demand, things are still heating up.” In the battle between Detroit and Silicon Valley, Velodyne plans to be an arms dealer to both sides.
Several banks of desks sat empty, and an entire section of the floor was bare, set aside for new hires
But nearly every major automaker is now experimenting with multiple suppliers and in-house production, and that means Velodyne faces new competition almost every week. Toyota, the world’s largest automaker, recently announced it would launch test vehicles powered by the LIDAR startup Luminar. Volkswagen, the second biggest player in the market, is partnering with French automotive supplier Valeo for its LIDAR. And GM just bought its own LIDAR maker.
Meanwhile, companies like Alphabet, Uber, and Chinese search giant Baidu are all trying to prove that the next era of automotive breakthroughs will come from the tech sector. Chip makers like Intel and Nvidia are working on self-driving solutions powered by artificial intelligence. And not a week goes by without a new startup announcing its secured funding to build LIDAR it claims will outperform the rest.
Velodyne’s long history in the business is a major advantage. “The big OEMS tend to trust suppliers they have long relationships with,” says Akhilesh Kona, a LIDAR industry analyst. “So that gives Velodyne an edge over many of the startups getting into the market.” But its approach to the technology will have to fundamentally evolve. “Automakers have used Velodyne for the development of their automated driving platforms. Now they want to get production volumes ready,” says Kona. “That means they have to massively reduce the size and the cost, and manufacture it in large volumes.”
Velodyne’s spinning LIDAR has come a long way since the 2005 DARPA Grand Challenge. That first unit was two feet wide and weighed over 50 pounds. It had a range of just over 80 meters, and now sits as a museum piece in the Smithsonian. The models Velodyne sold to other competitors in the 2007 were roughly the same size and cost upwards of $80,000 each. Since then, the company has managed to drop the size and cost of its product. Today, it sells the VLP-16, a 16-channel LIDAR with 100 meters of range, for just $7,999. That unit weighs less than two pounds and fits easily into a large purse.
LIDAR is the sort of product Hall loves, because you can measure improvements in performance. Just as he did with the subwoofer, Hall and his team are working to optimize their device; one prototype in the works would vastly increase the number of laser channels crammed onto a single sphere, allowing it to capture a incredibly detailed view of the world. “We are taking the puck form factor and extending it to the highest-resolution sensor will ever be needed for autonomous driving,” says Anand Gopalan, Velodyne’s chief technical officer.
But just like subwoofers, Hall may be pushing past the point of diminishing returns. Even at $8,000, Velodyne’s rotating LIDAR is too expensive to include on most consumer vehicles. Worse, its design has raised concerns about longevity. Most consumers keep a car for 10 years or more. “Many OEMS have issues with mechanical scanning LIDAR,” says Kona. “They say upwards of 60 percent need to be sent back to the manufacturer for recalibration.”
Automakers care about achieving safe cars, but they don’t care how they get there. As safety requirements are established, they become the mark suppliers must hit. “OEMS don’t take a look at what’s happening inside. To them it’s a box,” says Kona. “It’s all about who reaches those requirements first and at the lowest cost … That’s why the industry is moving toward a LIDAR with no moving parts in it.”
As Velodyne increases its size, it can reduce cost. The company announced earlier this month that it has managed to quadruple its manufacturing capacity over the last year, allowing it to offer immediate availability across Europe, Asia, and North America. But cutting costs also means moving away from the rotating design that Hall pioneered in 2005.
In December of last year, Velodyne announced it was getting into solid state LIDAR with a new product called the Velarray. Instead of spinning around, the new device will face one direction. Velodyne has promised the devices will be extremely cheap: under $50 for the key components once volumes ramp up. That would let carmakers put these sensors on every side of the car. A custom chip inside would help to make sense of the world and fuse together the data from each sensor. Self-driving taxis or trucks might still invest in a Velodyne puck for maximum reliability, but for a few hundred dollars, vehicles aimed at consumers could achieve a view of the world good enough for high-level autonomous driving without Hall’s spinning LIDAR.
The custom chip in the Velarray can also be used to craft LIDAR with advanced functionality tailored to specific vehicles. Hall hopes to build LIDAR that works not just for driverless cars, but for agricultural drones, security robots, and whatever comes next. “When we get the cost down these will be in everything,” he says. “Forklifts, golf carts, anything that can run into something else.”
But almost a year after its big announcement, Velodyne has yet to deliver a finished product to market. Employees told The Verge that resources had been redirected to the production of high-end rotating LIDAR, and that the press release touting the solid state unit had been premature. But the longer Velodyne delays, the greater the chance a competitor will be first to market with a solid state unit that meets automakers’ demands.
When I asked Hall about solid state LIDAR, and how it would move the industry forward, I expected a triumphant discourse on his company’s next step. But he sounded bored by the prospect. “For Velodyne, we’re going to be a full-service LIDAR company, so whatever anybody wants: big ones, small ones, expensive ones, cheap ones. One of them is the solid state. How that competes with the rest of them, I don’t know. I still am fond of my rotating design. I like that.” For Hall, the best part of the job is still perfecting his own creations, not economizing to find a unit that can sell at scale.
Hall is now a wealthy man, but he doesn’t live like one. He and his wife Marta, an artist, make their home on a converted barge overlooking a boatyard in Alameda. That puts him close to his latest project: the Martini 1.5.
Over the last few years, Hall’s interest has migrated once again, this time to a venture he calls Velodyne Marine. Its star product is a self-balancing boat with pontoons that rapidly expand and contract to keep the deck of the boat perfect level, even in rough seas. He’s also been sketching some ideas for outer space, too. “David has ideas for designing a space launch system using electro magnetics.” Marta tells me. “He thinks going to Mars is a waste of time because of the harsh environment. One could create more comfortable and inhabitable environments in space stations and position them anywhere in the galaxy.”
Though Hall would never relinquish control of Velodyne, he doesn’t appear to relish his role as CEO either. “When I come down here, I put my CEO hat on,” he says, when I meet him at the factory in San Jose. “Now I got to think about corporate finance, growth, employment, employees, sigh, structure, management, hierarchy, these sorts of challenges. When I go up to Alameda, I put my engineering hat on, and I get to talk about FPGAs, design, and new products. I was doing that this morning, so it’s a bit of a culture shock coming down here.”
Velodyne LIDAR has graduated into a massive company, thanks to a $150 million investment by Baidu and Ford, and it has hired veterans from the world of automobiles and semiconductors. Hall even has a board of directors. “I made it through my entire career without getting a job or reporting to anyone. I’m too old to change now, so if anyone thinks they’re going to be my boss, it’s out of the question. But I have my board of directors. It’s a novel experience. I kind of enjoy it, in a strange way.”
Marta is Velodyne’s chief creative officer, and she heads up its business development arm. “I don’t drag him anywhere,” she tells me. “We coax him into executive meetings.” Gopalan, Velodyne’s chief technical officer, described it to me this way: “Essentially, I’m the David Hall translator. David has an idea, I try to understand, flesh it out, and make it happen.”
I asked Hall whether he felt secure in Velodyne’s role as the predominant LIDAR producer in the automotive industry. Hall compared his company’s position to its namesake sport: cycling. “We’re in a breakaway, out in front all by ourselves, there’s nothing but clean air, the finish line is over there, it’s a beautiful day,” says Hall. “You turn around, there’s a hundred angry guys huffing and puffing, wanting to kill you,” he laughs. “I think I’ll just stare forward. It’s a much better view.”