<br /> <b>Warning</b>: Missing argument 2 for issart_title(), called in /var/www/blog.issart.ru/wp-includes/class-wp-hook.php on line 288 and defined in <b>/var/www/blog.issart.ru/wp-content/themes/seashell/functions.php</b> on line <b>289</b><br /> <br /> <b>Warning</b>: Missing argument 3 for issart_title(), called in /var/www/blog.issart.ru/wp-includes/class-wp-hook.php on line 288 and defined in <b>/var/www/blog.issart.ru/wp-content/themes/seashell/functions.php</b> on line <b>289</b><br /> December 2019 - Custom Web Development Blog

Archive - December 2019

Migration From DialogFlow to RASA: the missing part

Implementing DialogFlow chatbots is cool and convenient if you have something trivial and easy to prototype: fancy UI – easily, extracting base entities like name, surname and phone number – here is a tool if don’t want to install and deploy – cloud solution is at your service.
But what if you need to go deeper:

– Do you have a Japanese tokenizer, dear DialogFlow?
– Nope
– Transparent and customizable intent classification tool?
– Sorry, guys.
– Also I want to integrate my search index, knowledge graph and custom dialogue management policy.
– What are you talking about?

In a nutshell if you want fully controlled system, if you need custom advanced AI in your app, if you need natural language processing in your chatbot pipelines, if you want to scale your chatbot behaviour – on-premise solution is the way for a chatbot developer. And here it’s Rasa framework that really shines.

All ai chatbots developers have gladly met new update from Rasa chat bot framework – now 1.0 version is available and  it’s ten times cooler than it’s 0.9 version ancestor. Now there is single entry point for bot and NLU modules (no need to run separate NLU process), cool CLI for managing your models and Rasa X – UI for fine-tuning your bot and training it on the fly, all the stuff is at your service. This makes moving to Rasa from other cloud-hosted solutions more attractive, Maybe it’s time to migrate your DialogFlow bot to more maintainable and customizable Rasa platform.

Recently Rasa team has updated their product with migration tools that are compatible with Rasa 1.0 version. Check out this articleIt’s an official Rasa blog article, and it has very thorough description of how to move your DialogFlow agent to Rasa platform, but it lacks one issue – how to convert dialog tree into (and probably that’s the most important one). Read our approach described in this article if you also feel this gap to be painful.

But first  let’s look at some Rasa and DF data structures and how they are related, define common terms and scope of work.


What is dialog

Dialog is a chain of request-response pairs expressed in natural language to lead a user from some starting point to one of ending points accompanied by some side effects like searching information in 3rd party sources,  changing DB records, preparing artifacts etc. 

Transition from one state to another is governed by algorithm which detects the intent. The Intent is a user message category which is derived from user text input. This implies that user wants some action from a bot and expresses it implicitly with natural language phrase. Bot’s task is to decide which category should be assigned to a given natural language phrase and do some desired action.

Examples of intents can be following:

hi                                   - greet

hello                                - greet

bye                                  - end_conversation

goodbye                              - end_conversation

Suggest me some mexican restaurants  - restaurant_search

im looking for restaurants           - restaurant_search

So it’s some kind of directed graph, which may include cycles – when bot didn’t recognize human input correctly or intent or didn’t manage to perform a side effect  it can return to previous states to clarify some data issue from user’s side.

Probabilistic nature of predicting the intent in Rasa makes training bot behaviour routines scalable – instead of following complex rules for bot to stick to dialogue branching, the goal transition from one step to another is calculated in terms of probability, and the most probable intent is transitioned to.

Training data for NLU is stored in nlu.json file and  looks like this.

  "rasa_nlu_data": {

   "common_examples": [


       "intent": "greet",

       "text": "Hello "



       "intent": "inform_location",

       "text": "Show me good restraunt in Rome"



       "intent": "inform_people_num",

       "text": “I want a table for six people"



Rasa training data format elements

Detection what user wants (or NLU – natural language understanding – task) is necessary but we definitely need the bot to respond and govern the flow of conversation. So we need some more stuff to resolve.

There are several things you have to know to prepare data for training your bot. And intents, that were covered above, are only a part of the inventory list in Rasa data model.

Besides intents there are also:

  • actions
  • stories

Actions (as by definition)  are bot reactions to user intents. When detecting an intent bot chooses which action to apply for user’s response. The action may be a simple utterance, a question for clarification of user request or a side-effect – making external API  call or altering DB or whatever you need.

Intent-action mapping sequences are grouped by higher order entities called stories. A story is a description of user path from starting point to the end of user goal. Stories are written as plain list of steps with an intent (or several intents) mapped to one or more reactions.

Example of a story is the following:

## user_reccomendation_for_restraunt    <!-- name of the story -->

* greet

   - action_ask_howcanhelp

* inform_location{"location": "rome", "price": "cheap"}  <!-- user utterance, in format intent{entities} -->

   - action_on_it

   - action_ask_cuisine

* inform_cuisine{"cuisine": "spanish"}

   - action_ask_numpeople        

* inform_people_num{"people": "six"}

   - action_ack_dosearch

* confirm_search

   - action_do_dosearch

Here is name for story marked with ##. user intents marked with asterisk and corresponding actions marked with hyphens.

So the conversation here might look like the following:

User(U) - Hello
Bot(B) - Hello! How can I help you
U: - I woud like a cheap restraunt in Rome
B: - What kind of food would you like?
U: - Spanish cuisin.
B: - For how many people?
U: - For six people
B: - So we will search for cheap spanish restraunt in Rome for six people. Correct?
U: - Yes.
B: - Ok. Here are the suitable places: …

All migration stuff except this can be treated with Rasa cli commands, and it’s cool, but when it comes to a question on how to migrate a tree of dialog the blog has the following answer:

Since DialogFlow’s dialogue management is a rule-based approach, you cannot export any training data which you could use directly to train the Rasa dialogue model. The good news is that you have access to conversations history on DialogFlow and you can use it as a basis for generating training data for Rasa Core model.

So the recipe is “go to your log of bot’s conversations and write stories by hand”.  Don’t give up – there is a way how automatically or semi-automatically to solve this. Let’s move to …



So now it’s time to share on how to convert structure used by DialogFlow to structure used in Rasa

First we are going to collect intents and examples with code like this:

import glob
import json
import os

# Function to convert JSON string from file to 
def convert_to_objects(filename):
   json_file = open(filename)
  examples = open(f’filename[:-4]_usersays_en.json’)	
   return {**json.loads(json_file.read()), “examples”:  json.loads(examples.read())}

# Get all json files
intents_path = os.path.join(os.getcwd(), 'testagent', 'intents')

#Ignore filenames for user examples 
files = set(glob.glob(f'{intents_path}/*')) - set(glob.glob(f'{intents_path}/*usersays*'))
objects = list(map(convert_to_objects, files))

So now we have a list of objects representing states of dialog, that can be viewed as nodes in dialog graph. Let’s convert them to graph nodes where intent name will be a node identifier and all the rest will be node’s metadata. We will use networkx library to do all graph related stuff.

import networkx as nx
G = nx.DiGraph()

for obj in objects:
   G.add_node(obj['name'], **obj)

Now our plain python objects are transformed into graph nodes. And our nodes lack connecting edges using ‘affectedContexts’ property as identifier for connected intent . Let’s add them.

for obj in objects:
   if obj['responses']:
       for r in obj['responses']:
           for ctx in r['affectedContexts']:
               out_ctx = ctx['name']
               G.add_edge(obj['name'], out_ctx)

Now when we have a connected graph and we can unroll it into plain list of intent steps. All we have to do is to find paths from a starter intent to termination intent. Luckily networkx has a handy method all_simple_paths exactly for this task:

# Collect all possible paths from top to bottom
paths_list = list(nx.all_simple_paths(G, 'greet', 'bye'))

And finally all we have to do is to format the extracted path and save it in file. Each path is a story.

for i, paths in enumerate(paths_list):
   story_text = ''
   story_text += f'## story_{i}\n'

   for edge in paths:
       story_text += f'* {edge[0]} \n -  {edge[1]} \n'

# Store the data in file
open(‘stories.md’, ‘w’’).write('\n'.join(story_texts))

Though this approach is quite simple and doesn’t extract entities, or handling fallbacks, but you can tune this to capture all this issues if you take relevant data from nodes meta-data.

Going beyond

This approach can be extended to other sources of data. If you have any other data structures like DB dump or txt files or something else you should do the following:

  • Define what can serve as categories or intent labels in your data. Create a mapping for example texts and intents labels.
  • Define how the transition from one intent to another is done. Create a graph with intents transitions as edges between nodes as intents.
  • Define starter intent and termination intents, iterate over all possible paths by extracting nodes metadata and writing to Rasa-consumable format.

Have you ever tried migrating anything from ai chatbots to RASA? Please share your pain and insights in comments!

ISS Art Named Global Development Partner by Clutch

As a reliable software development partner, we always strive to integrate our solutions into your marketing and strategic plans. Since 2003, our  company has been helping the world’s leading companies translate even their most complex objectives into stunning realities!

We focus on machine learning, data science, IoT solutions, and can code in a number of languages like Python, Java, C++, Swift and Kotlin. We firmly believe that your challenge is our inspiration, and we’ve successfully built solutions for the most challenging projects with that  approach in mind.

In light of our contributions to the technology community, we’ve been named a top global leader in Custom Software Development as Clutch’s online research shows. We’d like to take this time to thank our customers for participating in client reviews on our behalf, to gauge our impact on their businesses.

Read More

5 use cases of AI based recommendation systems

Artificial intelligence solutions are widely used in a variety of businesses. With opportunities they provide, it becomes possible to optimize processes and bring revenues to a new level.

E-commerce is not an exception. Lots of companies are now looking for ways to cross-sell and up-sell effectively. This is where an AI based recommender system can help. 

As McKinsey reports have shown, 75% of content that Netflix users consume and 35% of products that Amazon users buy come from recommendations. After implementing a recommender system, Amazon reported a 29% increase in sales. Alibaba group managed to drive the conversion rates by 20% when it applied ML based recommendation algorithms to provide shoppers with personalized offers during the sales festival in 2016.

Actually, most online shoppers expect companies to provide them with personalized recommendations. According to Evergage, 56% of users will come back to the sites that offer recommendations again and again.

Wondering what kind of an intelligent recommendation engine to implement for your business? Or probably you are interacting with people who need to implement such a system? If any of these is the case, you definitely need to look through the possible use cases below.

Read More