NIPS Conversational Intelligence Challenge Call for Human Evaluators
Join ConvAI Competition as a Human Evaluator to make next generation chatbots truly intelligent.
Dialogue systems and conversational agents – including chatbots, personal assistants and voice control interfaces – are becoming increasingly widespread in our daily lives. Nevertheless, conversational intelligence remains one of the major unsolved tasks in artificial intelligence. This year, NIPS is sponsoring an open competition the Conversational Intelligence Challenge ( ) to create a chatbot that can hold an intelligent conversation with a human partner. Goals of the competition are evaluation of state-of-the-art dialogue systems, and collection of an open-source dataset for future training of end-to-end systems.

"Learning to communicate with humans is a crucial ability for intelligent agents. Among the primary forms of communication between humans is natural language dialogue. As such, building systems that can naturally and meaningfully converse with humans has been a central goal of artificial intelligence since the formulation of the Turing test," say co-organizers in their recent paper on Automated Turing test.

Current breakthroughs of deep learning critically depend on quantity and quality of data used to train models. This is why every human evaluator volunteer is so valuable for the success of the competition. Please join us in our efforts to build an open dataset for the development of the next generation of conversational AI.

First round of evaluation will be done remotely on 24th-30th of July, 2017 via a Telegram messaging platform. During the week competition teams members and human-volunteers will chat with bots and evaluate the quality of the dialogue systems, submitted to the competition. Winning teams will be qualified for the NIPS finals. To contribute to the conversational intelligence research and participate in evaluation of bots you should just chat to @ConvaiBot via

The competition is organised by Université de Montréal, McGill University, Carnegie Mellon University and Moscow Institute of Physics and Technology in partnership with Facebook, Flint Capital, IVADO, Maluuba and Element AI.


Flint Capital is an international venture capital fund established by IT entrepreneurs with the primary mission of discovering and supporting bold-thinking founders and their teams from Europe, Israel and US across a number of industries. Flint capital leads funding rounds at the Seed, Series A and Series B stages, focusing on companies that have the potential to disrupt their respective industries. Flint Capital has an extensive expertise in artificial intelligence and machine learning. Previously Flint Capital has supported the following AI/ML companies: WalkMe, YouAPPI, CyberX, Epistema, Flo, Findo. Flint Capital has had several significant exits amongst portfolio companies, including Lending Club and Blazemeter, with current investments in dozens of industry leading companies, including WalkMe, CyberX, Virool and Findo. Flint Capital has offices in Palo Alto, Berlin, Boston, Moscow and Tel Aviv.

Made on