Building End to End Search Engine Chatbot for the website using Amazon Lex, Google Knowledge Graph…

Last Updated on October 1, 2020 by Editorial Team

Author(s): anuragbisht

Cloud Computing, Programming

Building End to End Search Engine Chatbot for the website using Amazon Lex, Google Knowledge Graph, and CloudFront

Chabot term was coined from the original term “ChatterBot” created back in 1994, the term itself says a machine that can do human conversations. Although it's interesting to learn about chatbot the actual power lies in solving actual business world use-cases where you can automate most of the manual labor done physically by human beings.

Imagine you interacting with a smart speaker today booking a flight or ordering food for you, nobody would have thought doing these things with the same convenience a decade back. Imagine the customer support today with automated chatbots handling customers with automated replies. How much time and money would it require humans doing the same work. That’s the power of conversation AI whether it’s a voice-based system or chat-based system.

Amazon provides a powerful service, Amazon Lex that can create conversation AI chatbot, now Amazon Lex uses the same conversational engine that powers Amazon Alexa (voice-based system).

Today we will create a Search Bot using Amazon Lex that can find information for us from the web and deploy it using web-based UI with web application hosted over AWS CloudFront.

Before we dive deep into building the conversation AI chatbot, let’s first understand the important terminologies used in conversation AI space.

Important terminologies:

1. Intent: Action user wants to perform. Eg Booking a flight, we call this as Book-flight intent.
2. Utterance: To act on that intent, what all ways can the conversation take place by the user, eg. I would like to book a flight, can you help me with flight booking, etc.
3. Slot: These are the parameters in any conversation when the user is uttering to perform the intent. Eg. Book a flight at 10 am between New York and Paris. Here we have 3 slots: time, source, and destination.
4. Slot types: Each slot needs to have a data type, eg 10 am is the date, source and destination are cities.
5. Score Card: Response card generated by Bot with information in card UI format with buttons at the bottom. The card provides a context in a visual format using images. Eg. Query on the products ordered returns a photo of the product.
6. Knowledge Graph: This terminology is not specific to chatbots but is more close to entity-relationship modeling which involves the collection or network of interconnected entities.
7. Cloud Formation: Not specific to conversational AI, but this AWS service allows the creation of the whole infrastructure as a code (eg web application) that can be deployed anywhere in minutes.
8. CloudFront: Don’t confuse it with cloud formation, it’s a content delivery network service that caches the static data(images, videos, or pages) at the edge location nearest to the user making the applications load fast.

We will follow the following steps to create an end to end solution.

1. We will start by building our chatbot blueprint.
2. Then we will set up the knowledge graph service
3. Add the AWS lambda function to fulfill the response
4. Deploying website using Cloudformation stack for Bot UI

Let’s start with the first step:

1. Building the chatbot blueprint:

If you are new to AWS, then you can register for an AWS account, otherwise, AWS users can search for Amazon Lex under Machine Learning services within the AWS console.

You can either choose predefined templates, but for our use case, we will create a custom bot.

For other information, I have kept them as default.

The first step is to create an intent for specific actions users can perform. In this case, we will create WelcomeIntent for greeting users.

Now we will note all the possible utterances that user can ask to call this intent (greeting in our case)

Once you have recorded all the utterances, you will add the responses that you want your chatbot to give using the response option:

Let’s test the chatbot’s first intent. For this, you have to build the bot and use the test bot section.

Now we have a simple bot base ready, let’s go one step further adding new intent: SearchCountry. This intent allows the users to ask the chatbot to search for information about the country.

2. Setup Google Knowledge Graph service:

Now we also need a search engine that can search for information about countries for us. We will use the Google Knowledge Graph API service.

For this google has put up a well-documented section on how to start with google knowledge graph search API service.

Google Knowledge Graph Search API | Google Developers

Essentially we will obtain an API key to use the knowledge graph service API and use any programming language to send the HTTP requests with query string parameters(the same mechanism works when to search for anything on google search, observer the address URL bar).

3. Add the AWS lambda function to fulfill the response

Once we have access to the service, we will add a new intent and link that intent with the google service.

For this, we will create a new intent for user search action: SearchQuery

Now we have to identify slot(entity) in the user query, in this case, the intent is about searching the country information, hence we can conclude that country is one of the slots. We will add a slot named country and select the type as AMAZON.country.

Now we will add utterances but this time we will replace the actual query word with the slot placeholder, the country in our case.

So far we have created a mechanism to capture country value, but how will we return the search results. For that, we will use the AWS lambda function to get the relevant information by calling the knowledge graph API service.

We will head over to the search console and under computing services, we will select AWS lambda.

We will create a new AWS Lambda function using the create function button at the top right.

From the predefined templates, let's select one template programmed in python as shown below:

Next, we will fill the relevant information to create our function

Once the function is created, let's add the following code and comment existing function definition.

def search_query(intent_request):

query = get_slots(intent_request)[“Country”]
params = {
‘query’: query,
‘limit’: 10,
‘indent’: True,
‘key’: api_key,
}

# url = service_url + ‘?’ + urllib.parse.urlencode(params)
# response = json.loads(urllib.request.urlopen(url).read())

# response=json.loads(requests.get(url))
url = service_url
response=json.loads(requests.get(url,params=params).content)
try:
information=response[‘itemListElement’][0][‘result’][‘detailedDescription’][‘articleBody’]

except:
information=’Sorry could not find what you are looking for.’

return close(intent_request[‘sessionAttributes’],
‘Fulfilled’,
{‘contentType’: ‘PlainText’,
‘content’: information})

def dispatch(intent_request):
“””
Called when the user specifies an intent for this bot.
“””

logger.debug(‘dispatch userId={}, intentName={}’.format(intent_request[‘userId’], intent_request[‘currentIntent’][‘name’]))

intent_name = intent_request[‘currentIntent’][‘name’]

# Dispatch to your bot’s intent handlers
if intent_name == ‘SearchQuery’:
return search_query(intent_request)

raise Exception(‘Intent with name ‘ + intent_name + ‘ not supported’)

“”” — — Main handler — — “””

def lambda_handler(event, context):
“””
Route the incoming request based on intent.
The JSON body of the request is provided in the event slot.
“””
# By default, treat the user request as coming from the America/New_York time zone.
os.environ[‘TZ’] = ‘America/New_York’
time.tzset()
logger.debug(‘event.bot.name={}’.format(event[‘bot’][‘name’]))

return dispatch(event)

You can find the full lambda function here.

Once the function is ready, click the deploy button to deploy the lambda function.

After deployment of the lambda function, we can select that particular lambda function to fulfill the response by the chatbot.

Now let’s build the bot again and now is the time to test whether our bot can search for any country-related information or not.

4. Deploying website using Cloudformation stack for Bot UI

Now we have the Search bot ready, but we have to think about hosting the user interface of this bot, the easiest way is to use a predefined template as a reference from the amazon link.

For this activity, we will create an AWS cloud formation stack that allows the deployment of the whole web application with just a few clicks. This type of deployment methodology is termed as infrastructure as a code.

Make sure you the region selected is the same as that of our bot:

Northern Virginia: Link

Oregon: Link

Ireland: Link

Sydney: Link

You will also have to fill some details for the bot as mentioned below:

Clicking the create button will create a stack for us. Once the stack has been created and deployed, we will see the following entries as mentioned in the screenshot.

Now let’s head over to the domain name of the deployed CloudFront endpoint(it's a service that hosts static data at the nearest edge locations to the user).

Now let’s play around with our deployed search bot. Open the URL in a new tab and you will see your chatbot deployed as a full page.

Congratulations. You have a baseline working search bot ready, now you can further enhance it by using Iframe as UI and restricting with a small dialog screen than full screen. You can check out more at SnippetUrl for the code snippet. Also, you can enable the AWS Incognito service for providing a security layer on who can access web application but that’s beyond scope of this post. Let me know your thoughts in the response section if you would like to know more.

If you really enjoyed this post, stay tuned for more tutorials on AI/machine learning, data analytics, and BI.

Check me out on LinkedIn

---

Building End to End Search Engine Chatbot for the website using Amazon Lex, Google Knowledge Graph… was originally published in Towards AI — Multidisciplinary Science Journal on Medium, where people are continuing the conversation by highlighting and responding to this story.

Published via Towards AI