5 things that happened in Data Science in 2018

2018 was a hot year for Data Science and AI. Here we picked out 5 highlights, which in our opinion shaped the field in the past year.

OpenAI Five achieves human-level DOTA team gameplay

OpenAI has started to achieve professional-level 1v1 playback in 2017, but 2018 was the year when team gameplay became human level. In DOTA, two teams of 5 players (or bots) battle for domination of resources on a map and try to kill each other off. OpenAI uses Reinforcement Learning with an LSTM network and self-play for learning, playing 180 years against itself every day on 256 GPUs and 128,000 CPU cores.

At a high level, one interesting aspect of OpenAI’s success is that no deep theoretical breakthrough was necessary (so far) to achieve this; the basic ideas of the AI are similar to Google DeepMind’s AlphaGo, which defeated Lee Sedol in 2016. But Go, although a very deep, strategic game, still seemed very “discrete” compared to a Real Time Strategy game like DOTA. It will be interesting to see how far the OpenAI architecture will scale: can it defeat the top professional human team in 2019? I wouldn’t bet against the AI.

OpenAI resources:

• OpenAI Five
• OpenAI Gym

Human-level performance in Quake 3 Arena

Staying with games, 2018 was also the year when DeepMind achieved human-level performance in Quake 3, using only pixels and game points as input. Although this result received less coverage than the earlier AlphaGo and the OpenAI successes, for people who grew up playing Doom and Quake, this is perhaps even cooler.

Like the other game playing bots, this one is also based on Reinforcement Learning, but here thousands of bots are “alive” at any given time, playing in teams against themselves in randomly generated environments. Each agent in the population learns its own internal reward signal to complement the sparse delayed reward from winning and selects actions using a hierarchical representation that enables the agent to reason at multiple timescales. The videos show that the bots display human-like behaviors such as navigating, following, and defending:

• DeepMind paper: Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
• Video supplement to the paper
• Video explanation of the DeepMind Quake3 architecture

Self-driving cars did not arrive in 2018

Self-driving cars are the “killer-app” of the current AI hype cycle, and many people look at it as an indicator tent pole. Everybody understand this application, most people's life would be affected, and it’s hard to argue with the cool-factor. The stakes are high, many big players are working on this disruptive advancement: who will supply the brain of these self-driving cars? who will sell the most self-driving cars? which self-driving car service will dominate? and equally importantly, what will drivers who no longer have to drive do? whose content will they consume?

Progress is steady, with Tesla pushing out new versions over the air, and other major car manufacturing also experimenting with self-driving features, but it seems Google’s Waymo is the most far ahead in terms of commercial deployment: in 2018 Waymo launched its first commercial self-driving car service called "Waymo One", where users in the Phoenix metropolitan area can use an app to request the service.

But in 2018, we still were not able to get in a self-driving car, put our kids in the back, and get from San Francisco to Los Angeles without touching the wheel.

Instead, we had to cope with sad news. There were fatal crashes related to self-driving technology of Uber and Tesla, which triggered widespread debate about how the technology should be marketed, tested and deployed.

References:

• Uber crash
• Tesla crash

PyTorch reached 1.0

Data Scientists and ML Researchers have a wide variety of tools and frameworks to chose from, with Tensorflow, Keras and PyTorch being the most widely used Deep Learning ones. PyTorch stands out to us because it is a “hacker’s framework”. Tensorflow came out of Google, where most ML jobs operate on vast amounts of data, which lead to design decisions to trade ease of use and debuggability for raw performance.

For smaller scale, more lean problems and teams, we believe PyTorch is a better fit. PyTorch is easier to work with than Tensorflow, the execution model is nicely integrated with Python, for example, step-by-step print()s work---at the expense of some performance. But unless you’re working at Google scale, the speed of iterating on your model and code should come before the speed of training. To get an introduction to PyTorch, check out the excellent tutorial lecture by Stefan Otte:

• Deep Neural Networks with PyTorch - Stefan Otte

Reinforce AI Conference announced!

Reinforce Conf gives product managers, data scientists and engineers insights into success stories, lessons learned, best practices and new approaches, in the beautiful European city of Budapest, on March 20-22, 2019.

Reinforce brings together a diverse group of leading experts and practitioners to share their knowledge and experience on the fast-moving field of Machine Learning and Artificial Intelligence. Some of the speakers:

• Andrea Pasqua, Data Science Manager, Uber
• Christian Szegedy, Staff Research Scientist, Google
• Cibele Montez Halasz, Machine Learning Engineer, Twitter
• Kush R. Varshney, Principal Research Staff Member, and Manager, IBM Watson
• Patrick van der Smagt, Director Artificial Intelligence, Volkswagen

Use promo code KDNuggets to get a 20% discount.