A Comparison of Natural Language Understanding Services to build a chatbot in Italian

All leading IT companies have developed cloud-based platforms that allow building a chatbot in few steps and most times without knowledge about programming languages. These services are based on Natural Language Understanding (NLU) engines which deal with identifying information such as entities and intents from the sentences provided as input. In order to integrate a chatbot on an e-learning platform, we want to study the performance in intent recognition task of major NLU platforms available on the market through a deep and severe comparison, using an Italian dataset which is provided by the owner of the e-learning platform. We focused on the intent recognition task because we believe that it is the core part of an efficient chatbot, which is able to operate in a complex context with thousands of users who have different language skills. We carried out different experiments and collected performance information about F-score, error rate, response time and robustness of all selected NLU platforms.