Elon Musk, the founder of Tesla Motors and Space X, has openly shared his belief that artificial intelligence could be the “biggest existential threat” to humanity if not developed carefully. He has advocated for artificial intelligence that benefits everyone and not only a small group of companies using it to crunch piles of data.
His concerns were perhaps not entirely clear until he helped found OpenAI, an artificial intelligence research company that has promised to release its research to the public. Last December, Musk and other investors pledged $1 billion to the non-profit, which has hired several high-profile researchers over the last few months. Now, the company has revealed its first toolkit, OpenAI Gym, for developing and comparing so-called reinforcement learning algorithms.
Reinforcement-learning programs are unique in that they teach themselves abstract concepts by cycling through huge amounts of data. Usually, the program performs a specific task over and over again, learning through trial and error, without human supervision. Reinforcement learning is one of the strongest candidates for new programs that enable anything from autonomous cars to personal assistants in our smartphones.
The new tool consists of several basic environments where users can experiment with different reinforcement learning programs. In one environment, users can control virtual robots, which can train themselves to do things like walk. Users can also build programs that teach themselves to solve basic math problems or play anything from simple Atari video games to more complex board games like Go.
Reinforcement learning is turning into a major trend in artificial intelligence. The research team from DeepMind, a machine learning company under Google parent Alphabet, applied the training regimen to its AlphaGo program, which taught itself to play the ancient board game Go. The program developed new strategies and an almost instinctual feel for the game, defeating one of the world’s top players earlier this year.
The technique could eventually factor into programs built by Facebook and IBM, which are using machine learning to help chatbots have conversations with users on social media and custom programs make sense of electronic medical records. Other companies, like Geometric Intelligence, have rejected the reinforcement learning model, saying that it is too simple to teach computers the abstraction necessary to interact with humans.
OpenAI Gym was first developed for the internal research team, but the company decided to release the toolkit to the public. The company hopes that research projects inside OpenAI Gym will help forge new standards for testing machine learning programs, so that researchers can easily duplicate results published in journals.
The ability to recreate test results could help the companies investing in OpenAI to close the gap with more experienced companies like Facebook and Google. The non-profit is also supported by Paypal founder Peter Thiel and startup incubator Y-Combinator founder Sam Altman. Amazon Web Services, the online retailer’s cloud computing division, has also invested in OpenAI and partnered on the toolkit launch.
The virtual training gym is compatible with algorithms written in any framework, including Google’s Tensorflow software library, an open-source resource for building machine learning programs. It can also be used with programs built in Theano, another open-source machine learning library.
The toolkit also provides a way for users to upload and share their projects with the wider community. Rather than creating a leaderboard for users to compare results of different programs, OpenAI researchers will curate a list of algorithms that work in interesting ways.
The company eventually plans to hand control of that list to the community, with the intent of building a more collaborative space for research. “What matters for research isn’t your score,” the company writes in a blog post, “but instead the generality of your technique.”
Looking for parts? Go to SourceESB.