AlphaGo Zero Goes From Rank Beginner to Grandmaster in Three Days—Without Any Help

In the 1970 sci-fi thriller Colossus: The Forbin Project, a computer designed to control the United States’ nuclear weapons is switched on, and immediately discovers the existence of a Soviet counterpart.

The two machines, Colossus and Guardian, trade equations, beginning with  “1 + 1 = 2.”  The math moves faster and faster, advancing through calculus and beyond until suddenly the blurry cascade of symbols stops. The two machines have become one, and it has mankind by the throat.

Hah, you say. Development work takes a lot longer than that.

Maybe not. Today DeepMind, a London-based subsidiary of Google, announced that it has developed a machine that plays the ancient Chinese game of Go much better than its predecessor, AlphaGo, which last year beat Lee Sedol, a world-class player, in Seoul.

The earlier program was trained for months on a massive database of master games and got plenty of pointers—training wheels, as it were—from its human creators. Then it improved further by playing countless games against itself. But the new one, called AlphaGo Zero, received no training wheels; it trained itself all the way from tyro to grandmaster.

In three days.

After a few more days of training, the new machine was pitted against the old one in games held at the standard tournament speed of two hours per player, per game. AlphaGo Zero won by 100 games to zero.

To understand the new system, we must first review last year’s version. It has three parts: a search algorithm, a Monte Carlo simulator, and two deep neural networks. 

Search algorithms dictate the moves within computer chess programs. The algorithm begins by listing every possible move, then every possible rejoinder, and so on, generating a tree of analysis. Next, it uses a second algorithm to evaluate the final position on each branch of the tree. Finally, it works its way back to select the move that leads to the best outcome should the opponent also play the best moves. Search is of only limitred value in Go, because it’s so hard to evaluate final positions, a problem explained in IEEE Spectrum 10 years ago by Feng Hsiung-Hsu, who programmed IBM’s Deep Blue, which defeated then-World Chess Champion Garry Kasparov.

Monte Carlo simulation instead generates great numbers of possible games to get an idea of how often a given move leads to good results. This is what financial planners do when they take the known statistical variance for stocks, bonds, and inflation and use it to generate far more alternative histories than the actual periods for which good records exist. Other Go programmers had already tried this method, with decent results, as described in Spectrum in 2014 by Jonathan Shaeffer, Martin Miller, and Akihiro Kishimoto.

Deep neural networks were applied to Go for the first time by DeepMind’s engineers, led by ​CEO Demis​ ​Hassabis​ ​and​ ​David​ ​Silver. On top of search and Monte Carlo, their original AlphaGo system used two networks, one trained to imitate the play of masters, as exemplified in a huge database of games, and another to evaluate positions. Then the program played millions of times against itself to improve beyond the level of mere human players.

DeepMind calls such self-training reinforced learning, and AlphaGo Zero relied on this technique alone. No search or Monte Carlo here. The machine played itself repeatedly, looking only at the board and at the black and white stones that take their places, move by move, at the intersections of 19 vertical and 19 horizontal lines. And it used one neural network rather than two. 

“After 72 hours, we evaluated AlphaGo Zero against the exact version... that defeated Lee Sedol, under the same 2-hour time controls and match conditions that were used in the man-machine match in Seoul,” write Silver, Hassabis and their co-authors today in Nature. “AlphaGo Zero used a single machine with 4 tensor processing units (TPUs), whereas AlphaGo Lee was distributed over many machines and used 48 TPUs. AlphaGo Zero defeated AlphaGo Lee by 100 games to 0.”

Don’t get the idea that this stuff is easy. The authors explain their work with a jungle of symbols reminiscent of the conversation between Guardian and Colossus. Here’s a sample:

“MCTS may be viewed as a self-play algorithm that, given neural
network parameters θ and a root position s, computes a vector of search
probabilities recommending moves to play, π = αθ(s), proportional to
the exponentiated visit count for each move, πa ∝ N(s, a)1/τ, where τ is
a temperature parameter.”

Image: DeepMind    AlphaGo's power consumption has lowered with each generation.

I looked it up: “Temperature” is a concept derived from statistical mechanics. 

To a Go player, the result is a mixture of the familiar and the strange. In a commentary in Nature, Andy Okun and Andrew Jackson of the American Go Association write: “At each stage of the game, it seems to gain a bit here and lose a bit there, but somehow it ends up slightly ahead, as if by magic.”

And, the commentators add, the machine’s self-taught methods in the early and later parts of the game confirm the lore that grandmasters have accumulated over centuries of play. “But some of its middle-game judgements are truly mysterious.”

The DeepMind researchers discovered another bit of weirdness. When they had a neural network train itself to predict the moves of expert players it did very well, though it took a bit longer to reach the standard of a system trained with human supervision. However, the self-trained network played better overall, suggesting “that AlphaGo Zero may be learning a strategy that is qualitatively different to human play.”

Different and better. Toward the end of Colossus: The Forbin Project, the computer says, “What I am began in Man's mind, but I have progressed further than Man.”

But before we welcome our new overlords, a splash of cold water may be in order. And there is no better supplier of such coolant for AI hype than Rodney Brooks, who recently wrote for Spectrum on the limitations of self-driving cars.

In his keynote at the IEEE TechEthics Conference, held on Friday in Washington, D.C., Brooks said he’d asked the creators of the original AlphaGo how it would have fared against Lee Sedol if, at the last minute, the board had been enlarged by 10 lines, to 29 x 29. They told him the machine couldn’t have managed even if the board had been shrunk by a single line, to 18 x 18. That’s how specialized these deep neural networks are.

Brooks showed the audience a photo that Google’s AI system had labeled as a scene of people playing frisbee. “If a person had done this,” Brooks said, “we’d assume that he’d know a lot of other things about frisbees—that he could tell us whether a three-month-old can play frisbee, or whether you can eat a frisbee. Google’s AI can’t!”

AlphaGo Zero can’t tell us that Go is harder than checkers, that it involves occupying territory rather than checkmating the opponent’s king, or that it is a game. It can’t tell us anything at all. 

But no human can stand against it.

Read the full article here by Computing: IEEE Spectrum

Self-Driving Wheelchairs Debut in Hospitals and Airports

Autonomous vehicles can add a new member to their ranks—the self-driving wheelchair. This summer, two robotic wheelchairs made headlines: one at a Singaporean hospital and another at a Japanese airport.

The Singapore-MIT Alliance for Research and Technology, or SMART, developed the former, first deployed in Singapore’s Changi General Hospital in September 2016, where it successfully navigated the hospital’s hallways. It is the latest in a string of autonomous vehicles made by SMART, including a golf cart, electric taxi and, most recently, a scooter that zipped more than 100 MIT visitors around on tours in 2016.

The SMART self-driving wheelchair has been in development for about a year and a half, since January 2016, says Daniela Rus, director of MIT’s Computer Science and Artificial Intelligence Laboratory and a principal investigator in the SMART Future Urban Mobility research group. Today, SMART has two wheelchairs in Singapore and two wheelchairs at MIT being tested in a variety of settings, says Rus.

The robot’s computer uses data from three lidars to make a map.  A localization algorithm then determines where it is in the map. The chair’s six wheels lend stability, and the chair is designed to make tight turns and fit through normal-sized doorframes. “When we visited several retirement communities, we realized that the quality of life is dependent on mobility. We want to make it really easy for people to move around,” said Rus in a recent MIT statement.

A second autonomous wheelchair recently premiered at Haneda Airport in Tokyo, designed by Panasonic and Whill, Inc., creator of the Model A Whill wheelchair, a sleek, hi-tech wheelchair now on the market in Japan and the United States.

According to a recent press release, Panasonic is planning to conduct technical trials of the WHILL NEXT this year. Like the SMART wheelchair, the WHILL NEXT uses sensors to detect nearby obstacles. It also employs automation technology developed for Panasonic’s autonomous (and adorable) hospital delivery robot, HOSPI. The wheelchair identifies its position, selects routes, and moves to a chosen destination based on a user’s input into a smartphone app. It can even be hailed with the app – the Uber of wheelchairs.

The WHILL NEXT is also able to sync up with nearby wheelchairs to travel in a column, which is useful for a family or a group, the company notes. Best of all, each wheelchair automatically returns to its home base, reducing the need for airport staff to collect the chairs.

Beyond use in hospitals and airports, the SMART team says they envision a connected autonomous mobility system, where a user could use a scooter or wheelchair indoors at an office, zip outside and pick up a golf cart to cross the parking lot, and slip into an autonomous car to drive home. Recent studies with the scooter suggest the control algorithms work indoors as well as out, according to a press release last year. “The autonomous wheelchair could be very useful in any pedestrian environmen—including hospitals and airports —and we are exploring all these possibilities,” Rus tells IEEE Spectrum.

Yet the field faces the challenge of commercialization. Not all hi-tech wheelchairs have sold well, such as Dean Kamen’s stair-climbing iBot, whose $25,000 price tag was one reason the device was discontinued in 2009. But hopefully the next generation of wheelchairs won’t be as expensive, says Rus. “The system consists of an off-the-shelf wheelchair augmented with an autonomy package. We hope the price point of the autonomy package can come down to make the system affordable.”

Read the full article here by Computing: IEEE Spectrum

Renesas Unveils Its Open Autonomous Vehicle Platform

Tokyo-based Renesas Electronics announced Monday that it is launching a new open platform for advanced driving assistance (ADAS) and autonomous driving systems. 

Dubbed Renesas autonomy, the platform will employ ADAS and autonomous driving technologies being developed by Renesas and a number of partners, including AutonomousStuff, Cogent Embedded, Polysync, eTrans, and the University of Waterloo in Ontario, Canada. In addition, the platform is using technologies produced by the R-Car Consortium, that Renesas established in 2010. The Consortium now has 195 entities, including hardware manufacturers, software companies, and research institutes as members, and include NEC, Hitachi and QNX Software Systems. 

These alliances have produced, among other things, Renesas’ Lincoln (model MKZ) demonstration car for autonomous driving, which was showcased at the Consumer Electronics Show in Las Vegas in January.

“The Renesas autonomy Platform offers end-to-end solutions scaling from secure cloud connectivity and sensing to vehicle control,” says Uwe Westmeyer, Principal Engineer at Renesas Global ADAS Centre in Dusseldorf, Germany. “It connects everything we are offering under ADAS and autonomous driving. This will help customers to reduce integration efforts.”

Accompanying the announcement, Renesas released its first product under the Renesas autonomy brand: the R-Car V3M image recognition system-on-chip (SoC) targeting smart camera applications. The chipmaker, which claims to be the world’s leading supplier of SoCs and microcontroller units, shipped almost one billion units to the automotive industry alone in 2015.

The company noted that the new sensor incorporates an image signal processor (ISP), which frees up circuit-board space and reduces system-manufacturing costs. In addition, the device complies with the ISO26262 safety standard for electronic systems.

“The R-Car V3M is a good example of our strategy,” said Westmeyer. “It is based on discussions we had with Tier 1 [companies] and OEMs. It will help our customers develop leading-edge, cost-efficient smart camera applications, surround view systems, even lidars.”

Renesas exhibited the new SoC, its autonomy platform, and the Lincoln demonstration car at its DevCon Japan trade fair held in Tokyo on April 11.  

During his keynote speech at the fair, Renesas President Bunsei Kure revealed impetus for the R-Car Consortium and autonomy platform when he admitted that it was “difficult to survive on our own,” in such a competitive industry. He added that Renesas was working with so many partners to ensure “there will still be a semiconductor manufacturer left in Japan that can supply the automotive industry.”

No doubt on Kure's mind is smartphone chipmaker Qualcomm’s bid to buy NXP Semiconductors NV in the Netherlands, a leading supplier of chips to the automotive industry. The acquisition will cost a reported $47 billion, the biggest ever in the semiconductor industry. Qualcomm, which is seeking to diversify away from smartphones, is waiting on the approval of US antitrust regulators to complete its purchase.

Meanwhile, Intel is also seeking to gain ground in the automotive industry. It has agreed to purchase Mobileye, an Israeli supplier of software safety products for the ADAS and autonomous vehicle markets.

Faced with this kind of international competition, Renesas is emphasizing its autonomy platform as open, given that customers can choose where to start ADAS and autonomous driving development based either in part on their own differentiating technologies, or on the technologies provided by the Renesas autonomy platform and R-Car Consortium.

Read the full article here by Computing: IEEE Spectrum

Open Source Clues to Google's Mysterious Fuchsia OS

Google Details Tensor Chip Powers

In January’s special Top Tech 2017 issue, I wrote about various efforts to produce custom hardware tailored for performing deep-learning calculations. Prime among those is Google’s Tensor Processing Unit, or TPU, which Google has deployed in its data centers since early in 2015.

In that article, I speculated that the TPU was likely designed for performing what are called  “inference” calculations. That is, it’s designed to quickly and efficiently calculate whatever it is that the neural-network it’s running was created to do. But that neural network would also have to be “trained,” meaning that its many parameters would be tuned to carry out the desired task. Training a neural network normally takes a different set of computational skills: In particular, training often requires the use of higher-precision arithmetic than does inference.

Yesterday, Google released a fairly detailed description of the TPU and its performance relative to CPUs and GPUs. I was happy to see that the surmise I had made in January was correct: The TPU is built for doing inference, having hardware that operates on 8-bit integers rather than higher-precision floating-point numbers.

Yesterday afternoon, David Patterson, an emeritus professor of computer science at the University of California, Berkeley and one of the co-authors of the report, presented these findings at a regional seminar of the National Academy of Engineering, held at the Computer History Museum in Menlo Park, Calif. The abstract for his talk summed up the main point nicely. It reads in part: “The TPU is an order of magnitude faster than contemporary CPUs and GPUs and its relative performance per watt is even larger.”

Google’s blog post about the release of the report shows how much of a difference in relative performance there can be, particularly in regard to energy efficiency. For example, compared with a contemporary GPU, the TPU is said to offer 83 times the performance per watt.  That might be something of an exaggeration, because the report itself claims only that there’s a range of between 41 times and 83 times. And that’s for a quantity the authors call incremental performance. The range of improvement for total performance is less: from 14 to 16 times better for the TPU compared with that of a GPU.

The benchmark tests used to reach these conclusions are based on a half dozen of the actual kinds of neural-network programs that people are running at Google data centers. So it’s unlikely that anyone would critique these results on the basis of the tests not reflecting real-world circumstances. But it struck me that a different critique might well be in order.

The problem is this: These researchers are comparing their 8-bit TPU with higher-precision GPUs and CPUs, which are just not well suited to inference calculations. The GPU exemplar Google used in its report is Nvidia’s K80 board, which performs both single-precision (32-bit) and double-precision (64-bit) calculations. While they’re often important for training neural networks, such levels of precision aren’t typically needed for inference.

In my January story, I noted that Nvidia’s newer Pascal family of GPUs can perform “half-precision” (16-bit) operations and speculated that the company may soon produce units fully capable of 8-bit operations, in which case they might be much more efficient when carrying out inference calculations for neural-network programs.

The report’s authors anticipated such a criticism in the final section of their paper; there they considered the assertion (which they label a fallacy) that “CPU and GPU results would be comparable to the TPU if we used them more efficiently or compared to newer versions.” In discussing this point, they say they had tested only one CPU that could support 8-bit calculations, and the TPU was 3.5 times better. But they don’t really address the question of how GPU’s tailored for 8-bit calculations would fare—an important question if such GPUs soon became widely available.

Should that come to pass, I hope that these Googlers will re-run their benchmarks and let us know how TPUs and 8-bit-capable GPUs compare.

Read the full article here by Computing: IEEE Spectrum

How to Detect a GPS Spoof On a Superyacht