Introduction
Regardless of how much artificial intelligence a bot possesses, there may still be times when it needs to hand off the conversation to a human being. For example you want to build a bot that automatically replies some questions and is able to meet your customers wherever they are, but still be able to escalate issues to a human. Or if the bot couldn’t handle every situation, or there were edge cases, the bot should be able to pass off to a person who had the right authority. The bot should recognize when it needs to hand off and provide the user with a clear, smooth transition. In this exercise, you will learn how you can use a bot to initiate a conversation with a user, and then hand off context to a human agent.
First, you will learn how to use Scorables
to intercepts incoming and outgoing events/messages. With this, you will handle the user-agent communication and the specials command only available for agents. Later you will modify your bot to use the new Scorables
and add a dialog to hand off the bot conversation to a human agent.
Inside this folder you will find a solution with the code that results from completing the steps in this exercise. You can use this solution as guidance if you need additional help as you work through this exercise. Remember that for using it, you first need to complete the keys in Web.config.
For more details about the hand-off approach used in this exercise you can check this session from BUILD 2017.
This diagram outlines the components of the bot for this exercise:
Prerequisites
The following software is required for completing this exercise:
- Visual Studio 2017 Community or higher
- An Azure subscription
- An account in the LUIS Portal
- The Bot Framework Emulator (make sure it’s configured with the
en-US
Locale)
Task 1: Build the Hand Off Logic
In this task you will add the necessary behind-the-scenes logic to handle the bridged communication between two persons, one as a user and other as an agent. You will learn how to create and put scorables to intercepts incoming and outgoing events/messages.
The scorables in the Bot Builder SDK for .NET enables your bot to intercept every message sent to a conversation and apply a score to the message based on logic defined by you. To create a Scorable you create a class that implements the IScorable
interface by inheriting from the ScorableBase
abstract class. To have that Scorable applied to every message in the conversation, the bot registers that IScorable
interface as a Service
with the Conversation
‘s Container
. When a new message arrives to the Conversation
, it passes that message to each implementation of IScorable
in the Container
to get a score. The Container
then passes that message to the IScorable
with the highest score for processing. For more information about Scorables
, see this sample.
- Open the app you’ve obtained from the previous exercise. Alternatively, you can use the app from the exercise6-MoodDetection folder.NOTE: If you use the solution provided remember to replace:
- the [LuisModel(“{LUISAppID}”, “{LUISKey}”)] attribute placeholders in
RootDialog.cs
with your LUIS App Id and Programmatic API Key (as explained in exercise 3) - the TextAnalyticsApiKey in
Web.config
with your Text Analytics Key (as explained in exercise 6) - the AzureSearchAccount and AzureSearchKey in
Web.config
with your search account name and key (as explained in exercise 4)
- the [LuisModel(“{LUISAppID}”, “{LUISKey}”)] attribute placeholders in
- Create a folder
HandOff
in your project and add the following files from the assets folder.AgentExtensions.cs
Contains a simple logic to convert a normal user to an Agent and to identify an Agent. You can use this to implement your own logic going forward for managing conversations, users and agents.Provider.cs
Builds a queue with the users waiting for a human agent. Notice that this class does not persist the queue in an external storage. This is also where the conversations metadata is stored. If you wanted to store a conversation in a data store, you could either update or inherit fromProvider
with your custom implementation.CommandScorable.cs
This Scorable is reached when the message is from an Agent and only triggers its resolution when receivesagent help
,connect
orresume
messages. If the user message doesn’t match those it is not processed with this Scorable.
- Create a
RouterScorable.cs
class in theHandOff
folder using the following boilerplate code. The router will be in charge of knowing where each message needs to be sent to, either to the agent or the user.namespace HelpDeskBot.HandOff { using System; using System.Threading; using System.Threading.Tasks; using Microsoft.Bot.Builder.Dialogs.Internals; using Microsoft.Bot.Builder.Internals.Fibers; using Microsoft.Bot.Builder.Scorables.Internals; using Microsoft.Bot.Connector; public class RouterScorable : ScorableBase<IActivity, ConversationReference, double> { private readonly ConversationReference conversationReference; private readonly Provider provider; private readonly IBotData botData; public RouterScorable(IBotData botData, ConversationReference conversationReference, Provider provider) { SetField.NotNull(out this.botData, nameof(botData), botData); SetField.NotNull(out this.conversationReference, nameof(conversationReference), conversationReference); SetField.NotNull(out this.provider, nameof(provider), provider); } protected override Task DoneAsync(IActivity item, ConversationReference state, CancellationToken token) { return Task.CompletedTask; } } } - Add the
PrepareAsync
,PrepareRouteableAgentActivity
, andPrepareRouteableUserActivity
methods inRouterScorable.cs
.ThePrepareAsync
method receives the incoming message and triggers its resolution by calling some of the other methods.protected override async Task<ConversationReference> PrepareAsync(IActivity activity, CancellationToken token) { var message = activity as Activity; if (message != null && !string.IsNullOrWhiteSpace(message.Text)) { // determine if the message comes from an agent or user if (this.botData.IsAgent()) { return this.PrepareRouteableAgentActivity(message.Conversation.Id); } else { return this.PrepareRouteableUserActivity(message.Conversation.Id); } } return null; }PrepareRouteableAgentActivity
triggers the Scorable when the message is from an Agent connected with a normal user.protected ConversationReference PrepareRouteableAgentActivity(string conversationId) { var conversation = this.provider.FindByAgentId(conversationId); return conversation?.User; }PrepareRouteableUserActivity
triggers the Scorable when the message is from a normal user waiting for an Agent or connected to an Agent.protected ConversationReference PrepareRouteableUserActivity(string conversationId) { var conversation = this.provider.FindByConversationId(conversationId); if (conversation == null) { conversation = this.provider.CreateConversation(this.conversationReference); } switch (conversation.State) { case ConversationState.ConnectedToBot: return null; // continue normal flow case ConversationState.WaitingForAgent: return conversation.User; case ConversationState.ConnectedToAgent: return conversation.Agent; } return null; } - Add
HasScore
andGetScore
methods inRouterScorable.cs
.HasScore
is only evaluated whenPrepareAsync
returns a validConversationReference
andGetScore
returns the maximun score to resolve the message.protected override bool HasScore(IActivity item, ConversationReference destination) { return destination != null; } protected override double GetScore(IActivity item, ConversationReference destination) { return 1.0; } - Add a
PostAsync
method inRouterScorable.cs
. If this Scorable won the resolution of the message theConversationReference
receives the destination of the message. If the destination is the same user of the current conversation the Scorable sends a message to the user informing the status of the queue, in any other case the Scorable routes the incoming message to the destination.protected override async Task PostAsync(IActivity item, ConversationReference destination, CancellationToken token) { string textToReply; if (destination.Conversation.Id == conversationReference.Conversation.Id) { textToReply = “Connecting you to the next available human agent… please wait”; } else { textToReply = item.AsMessageActivity().Text; } ConnectorClient connector = new ConnectorClient(new Uri(destination.ServiceUrl)); var reply = destination.GetPostToUserMessage(); reply.Text = textToReply; await connector.Conversations.SendToConversationAsync(reply); }
Task 2: Update the Bot to Hand off the Conversation
In this task you will update the bot to connect to the routing Scorables and add the necessary dialogs to handle the handoff conversation flow.
- Open the LUIS Portal and edit your app to add a HandOffToHuman intent with the following utterances:
- I want to talk to an IT representative
- Contact me to a human being
- Operator
- Train and publish your app again.
- Copy
AgentLoginScorable.cs
from the assets folder to theDialogs
folder. This class manages the switching between normal users and human agents. - Open
Global.asax.cs
and add the following using statement.using HandOff; using Microsoft.Bot.Builder.Dialogs.Internals; - In
Global.asax.cs
add the registration of the newIScorable
‘s implementations to handle the communication between two users.protected void Application_Start() { GlobalConfiguration.Configure(WebApiConfig.Register); var builder = new ContainerBuilder(); // Hand Off Scorables, Provider and UserRoleResolver builder.Register(c => new RouterScorable(c.Resolve<IBotData>(), c.Resolve<ConversationReference>(), c.Resolve<Provider>())) .As<IScorable<IActivity, double>>().InstancePerLifetimeScope(); builder.Register(c => new CommandScorable(c.Resolve<IBotData>(), c.Resolve<ConversationReference>(), c.Resolve<Provider>())) .As<IScorable<IActivity, double>>().InstancePerLifetimeScope(); builder.RegisterType<Provider>() .SingleInstance(); // Bot Scorables builder.Register(c => new AgentLoginScorable(c.Resolve<IBotData>(), c.Resolve<Provider>())) .As<IScorable<IActivity, double>>() .InstancePerLifetimeScope(); builder.RegisterType<SearchScorable>() .As<IScorable<IActivity, double>>() .InstancePerLifetimeScope(); builder.RegisterType<ShowArticleDetailsScorable>() .As<IScorable<IActivity, double>>() .InstancePerLifetimeScope(); builder.Update(Microsoft.Bot.Builder.Dialogs.Conversation.Container); } - In
RootDialog.cs
add aHandOff
method to handle the HandOffToHuman intent and put the user in the queue to talk to an agent.[LuisIntent(“HandOffToHuman”)] public async Task HandOff(IDialogContext context, LuisResult result) { var conversationReference = context.Activity.ToConversationReference(); var provider = Conversation.Container.Resolve<HandOff.Provider>(); if (provider.QueueMe(conversationReference)) { var waitingPeople = provider.Pending() > 1 ? $”, there are { provider.Pending() – 1 } users waiting” : string.Empty; await context.PostAsync($”Connecting you to the next available human agent… please wait{waitingPeople}.”); } context.Done<object>(null); } - Also add the following using statement.using Autofac; using Microsoft.Bot.Builder.ConnectorEx;
- In
UserFeedbackRequestDialog.cs
update theMessageReceivedAsync
method to call the Handoff dialog created in the previous step if the user satisfaction score is below 0.5. For simplcity, you can replace the full method with the following code (two methods).public async Task MessageReceivedAsync(IDialogContext context, IAwaitable<string> result) { var response = await result; double score = await this.textAnalyticsService.Sentiment(response); if (score == double.NaN) { await context.PostAsync(“Ooops! Something went wrong while analying your answer. An IT representative agent will get in touch with you to follow up soon.”); } else { string cardText = string.Empty; string cardImageUrl = string.Empty; if (score < 0.5) { cardText = “I understand that you might be dissatisfied with my assistance. An IT representative will get in touch with you soon to help you.”; cardImageUrl = “https://raw.githubusercontent.com/GeekTrainer/help-desk-bot-lab/master/assets/botimages/head-sad-small.png”; } else { cardText = “Thanks for sharing your experience.”; cardImageUrl = “https://raw.githubusercontent.com/GeekTrainer/help-desk-bot-lab/master/assets/botimages/head-smiling-small.png”; } var msg = context.MakeMessage(); msg.Attachments = new List<Attachment> { new HeroCard { Text = cardText, Images = new List<CardImage> { new CardImage(cardImageUrl) } }.ToAttachment() }; await context.PostAsync(msg); if (score < 0.5) { var text = “Do you want me to escalate this with an IT representative?”; PromptDialog.Confirm(context, this.EscalateWithHumanAgent, text); } else { context.Done<object>(null); } } } private async Task EscalateWithHumanAgent(IDialogContext context, IAwaitable<bool> argument) { var confirmed = await argument; if (confirmed) { var conversationReference = context.Activity.ToConversationReference(); var provider = Conversation.Container.Resolve<HandOff.Provider>(); if (provider.QueueMe(conversationReference)) { var waitingPeople = provider.Pending() > 1 ? $”, there are { provider.Pending() – 1 } users waiting” : string.Empty; await context.PostAsync($”Connecting you to the next available human agent… please wait{waitingPeople}.”); } } context.Done<object>(null); } - Also add the following using statement.using Autofac; using Microsoft.Bot.Builder.ConnectorEx;
Task 3: Test the Bot from the Emulator
- Run the app clicking in the Run button and open two instances of the emulator. Type the bot URL as usual (
http://localhost:3979/api/messages
) in both. - In one emulator type
I need to reset my password, this is urgent
to create a new ticket and confirm the submission. When the bot asks for feedback, type a negative phrase likeit was useless
. You should see a new prompt asking you if you want to talk with an agent. - Confirm the prompt to send the user to the queue of users waiting.
- Now, in the second emulator type
/agent login
to take control of the agent privileges. The bot should inform you that there is one user waiting. If you typeagent help
you should see a message with the agent’s options. - Type
connect
to begin the conversation with the user. Note that in the first emulator the bot will inform the user of this connection.Agent messagesUser messages - Now you can play with the emulators and see the communication between agent and user.Agent messagesUser messages
- In order to finish the interaction type
resume
in the second emulator (the agent emulator) and the bot should inform to both participants the end of the communication.Agent messagesUser messagesNOTE: Another possible scenario is “supervised hand off”. In this case, depending on the user question, the bot might contact a human agent asking which one of the possible answers the bot has prepared is the correct one.