Solution overview

Inquiries from employees to the internal IT service desk are increasing in the Pasona Group. They developed a Q&A Bot that focuses on inquiries regarding login issues. In fact, inquiries regarding login issues are most frequent inquiries to the internal IT service desk. They developed a Q&A Bot with Bot Framework. They used Cognitive Services (LUIS) to classify inquiries to this Q&A Bot. This Q&A Bot is deployed on Microsoft Azure as Web App. Employees operate this Q&A Bot on Microsoft SharePoint as Webchat.

Key technologies

  • Bot Framework for developing a Q&A Bot.
    https://dev.botframework.com/
  • Cognitive Services (LUIS) for classifying inquiries from employees.
    https://www.luis.ai/
  • Azure Web App for deploying a Q&A Bot.
    https://azure.microsoft.com/ja-jp/services/app-service/web/
  • Azure SQL Server for storing answers.
    https://azure.microsoft.com/ja-jp/services/sql-database/
  • SharePoint for embedding a Q&A Bot as a Webchat.
    https://www.microsoft.com/ja-jp/sharepoint/

Project members

Pasona Group project members:

  • Atsushi Watanabe - Manager, Group IT Management Division
  • Noboru Nishimoto - Group IT Management Division
  • Yuko Kuroda - Senior Leader, Group IT Management Division

Microsoft DX Japan project members:

  • Hiroyuki Watanabe - Technical Evangelist
  • Ayako Omori - Technical Evangelist

Figure1:Hackfest Figure 1:Hackfest

Customer profile

Pasona Group Inc. http://www.pasonagroup.co.jp/english/company/outline_g_e.html

The Pasona Group Inc. operates nearly 50 companies in Japan and overseas, and is one of the leading companies in the staffing services field. Pasona’s mission is to “solve society’s problems,” and is engaged with ways to create employment. They strive to ensure that, regardless of age, sex, nationality, or disability, everyone can freely choose the job they want to do. Their goal is to help build a society in which everyone to work in a way that fits their lifestyle. Even beyond staffing solutions, they endeavor to build infrastructure to make it possible for working people to enjoy a richer life.

Problem statement

There is a total of over 8,000 employees working at the Pasona Group Inc., including all the group companies. The company is planning to increase the efficiency of its internal IT service desk (call center) and reduce the number of inquiries to the IT service desk.

There are currently five people handling inquiries to the internal IT service desk and these staff members deal with upwards of 2,000 inquiries each month.

Solution and steps

1st Phase: They developed a demo version of a Q&A Bot to assess the employee response and verified that the number of inquiries could actually be reduced. In this Hackfest, they developed a prototype system designed to cut down on the number inquiries related to their internal systems.

2nd Phase: They will develop an IT service desk Bot with the goal of putting an internal IT service desk system into actual operation.

Architecture diagrams Figure 2: This is the Q&A Bot architecture. Figure 2: The Q&A Bot architecture Pasona executed the 1st Phase mentioned above.There are two processes of the Q&A Bot. One responds to login issues in natural language with LUIS. The other responds to the fixed questions by menu selection. All answers are stored on Azure SQL server.

Cooperation between Q&A Bot and operators of the service desk. Figure 3: Figure 3: Cooperation between Q&A Bot and operators of service desk This is a diagram of the cooperation between Q&A Bot and operators of the service desk. For inquiries that the Q&A Bot can’t deal with, operators of the service desk respond to these by email or phone.

The Q&A Bot with LUIS Figure 4: Figure 4: The Q&A Bot with LUIS This is a diagram of the Q&A Bot with LUIS. The Q&A Bot responds to inquiries from users in natural language with LUIS. The Q&A Bot adds images and audio to the responses.

Menu selection of Q&A Bot Figure 5: Figure 5: Menu selection of Q&A Bot This is a diagram of menu selection of the Q&A Bot. The Q&A Bot responds to the fixed questions by use of the menu selection.

Technical delivery

This selection will include the following details of how the solution was implemented:

Prerequisites

LUIS Implementation Figure 6: Figure 6: LUIS design LUIS for responding to login issues from users in natural language. The Pasona Group members had to learn how to use LUIS. The Microsoft members provided LUIS design know-how gained from their past projects. They went to luis.ai and created LUIS application for this Q&A Bot. The following is a snipet from json code of LUIS.

{
  "luis_schema_version": "1.3.0",
  "name": "pBot",
  "desc": "",
  "culture": "ja-jp",
  "intents": [
    {
      "name": "Greeting"
    },
    {
      "name": "LoginIssue"
    },
    {
      "name": "LoginIssueOther"
    },
    {
      "name": "None"
    },
    {
      "name": "SystemIssue"
    }
  ],
  "entities": [
    {
      "name": "DeviceName",
      "children": [
        "iPhone",
        "PC",
        "Printer",
        "otherDevice"
      ]
    },
    {
      "name": "NetworkName",
      "children": [
        "ShareDrive",
        "VPN",
        "otherNetwork"
      ]
    },
// 中略
    {
      "text": "vpnにつなげない",
      "intent": "SystemIssue",
      "entities": [
        {
          "entity": "NetworkName::VPN",
          "startPos": 0,
          "endPos": 0
        }
      ]
    },
    {
      "text": "会議室予約を変更できない",
      "intent": "SystemIssue",
      "entities": [
        {
          "entity": "O365Name::MtgRm",
          "startPos": 0,
          "endPos": 2
        }
      ]
    },
    {
      "text": "こんにちは",
      "intent": "Greeting",
      "entities": []
    }
  ]
}

Then they edited MessegeController.cs file. They used LuisIntent attributes to separate the Q&A Bot response. The following is a snipet from C# code of LUIS. MessageController.cs

    public class MessagesController : ApiController
    {
        [LuisModel("xxxxxxxxx", "xxxxxxxx")]
        [Serializable]
        public class FAQDialog : LuisDialog<object>
        {
            string message = "";

            [LuisIntent("")]
            [LuisIntent("None")]
            [LuisIntent("Greeting")]
            public async Task Greeting(IDialogContext context,  LuisResult result)
            {
                message = "こんにちは!Q&A Bot です。質問を入力してください";
                await context.PostAsync(message);
                context.Wait(MessageReceived);
            }

            [LuisIntent("LoginIssue")]
            public async Task LoginIssue(IDialogContext context, LuisResult result)
            {
                message = "メールのログインができないのですね。";

                var deviceName = "";
                for (int i=0; i<result.Entities.Count; ++i)
                {
                    if (result.Entities[i].Type.Contains("DeviceName"))
                    {
                        deviceName = result.Entities[i].Type.Split(':')[2];
                        message = deviceName + " で " + message;
                    }
                }
                await context.PostAsync(message);

                message = "最近 Windows ログインのパスワードを変更されませんでしたか?";
                await context.PostAsync(message);
                context.Wait(MessageReceived);
            }

            [LuisIntent("LoginIssueOther")]
            public async Task LoginIssueOther(IDialogContext context, LuisResult result)
            {
                message = "ログインできないのですね。";

                var systemName = "";
                for (int i = 0; i < result.Entities.Count; ++i)
                {
                    if (result.Entities[i].Type.Contains("SystemName"))
                    {
                        systemName = result.Entities[i].Type.Split(':')[2];
                        message = systemName + " に " + message;
                    }
               }
                await context.PostAsync(message);
                context.Wait(MessageReceived);
            }
            [LuisIntent("SystemIssue")]
            public async Task SystemIssue(IDialogContext context, LuisResult result)
            {
                message = "お困りですね。";
                var systemName = "";
                var networkName = "";
                for (int i = 0; i < result.Entities.Count; ++i)
                {
                    if (result.Entities[i].Type.Contains("SystemName"))
                    {
                        systemName = result.Entities[i].Type.Split(':')[2];
                        message = systemName + " で " + message;
                    }
                    if (result.Entities[i].Type.Contains("NetworkName"))
                    {
                        networkName = result.Entities[i].Type.Split(':')[2];
                        message = networkName + " で " + message;
                    }
                }
                await context.PostAsync(message);
                context.Wait(MessageReceived);
            }
        }

Bot’s character changing Figure 7: Figure 7: Bot's character changing The Q&A Bot can change the Bot’s character for each user. This function is implemented by rich cards of the Bot Framework. The following is a snipet from C# code fo rich cards function.

CommonFunction.cs

        public IMessageActivity TalkMessage(
            IDialogContext context,             
            Chacter CharObj,                    
            string MessageNo,                   
            bool AutoPlay                       
        )
        {
            IMessageActivity resultMessage      =   context.MakeMessage();
            resultMessage.AttachmentLayout      =   AttachmentLayoutTypes.Carousel;
            resultMessage.Attachments           =   new List<Attachment>();

            MessageInfoBox          MsgBox;
            MessageInfo             MsgObj;
            AudioCard               audioCard;
            List<VideoCard>         Video;
            List<CardAction>        lstButton;

            MsgBox          =   CharObj.Messages[MessageNo];
            int seed        =   Environment.TickCount;
            int MsgPos      =   0;

            for (int i = 0; i < MsgBox.Msgs.Count; i++)
            {
                Random rnd = new Random(seed++);
                MsgPos = rnd.Next(0, MsgBox.Msgs.Count);
            }

            MsgObj          =   MsgBox.Msgs[MsgPos];
            lstButton       =   MakeActionButton(MsgObj.Action);
            Video           =   MakeVideoButton(MsgObj.Action);     

            HeroCard heroCard = new HeroCard
            {
                Title       =   "",
                Subtitle    =   CharObj.Name,
                Text        =   MsgObj.Msg,
                Images      =   new List<CardImage> { new CardImage(MsgObj.FaceType) },
                Buttons     =   lstButton
            };
            resultMessage.Attachments.Add(heroCard.ToAttachment());

            if (MsgObj.VoiceType.Length > 0)
            {
                audioCard = new AudioCard
                {
                    Title       =   "",
                    Autostart   =   MsgObj.VoiceAutoPlay,
                    Shareable   =   false,
                    
                    Media = new List<MediaUrl>
                    {
                        new MediaUrl()
                        {
                            Profile =   "ダウンロードしないでください",
                            Url     =   MsgObj.VoiceType
                        }
                    },
                };
                resultMessage.Attachments.Add(audioCard.ToAttachment());
            }
            if (Video.Count > 0)
            {
                foreach (VideoCard obj in Video)
                {
                    obj.Title = "";
                    obj.Subtitle = CharObj.Name;
                    obj.Autostart = AutoPlay;
                    obj.Buttons = lstButton;
                    obj.Image.Url = MsgObj.FaceType;
                    resultMessage.Attachments.Add(obj.ToAttachment());
                }
            }
            return resultMessage;
        }
        public bool CheckSessionTimeOut(
            IDialogContext context
        )
        {
            string LastAccessTime = context.UserData.Get<string>("LastAccessTime");
            DateTime nowtime = DateTime.Now;

            if (LastAccessTime != null)
            {
                TimeSpan Check = nowtime - DateTime.Parse(LastAccessTime);
                TimeSpan interval = new TimeSpan(1, 0, 0);
                if (Check > interval)
                {
                    context.UserData.SetValue<string>("LastAccessTime", nowtime.ToString());
                    return true;
                }
            }
            context.UserData.SetValue<string>("LastAccessTime", nowtime.ToString());
            return false;
        }

        public Chacter GetCharObject(
            IDialogContext context
        )
        {
            Chacter CharObj = new Chacter();
            string CharType = context.UserData.Get<string>("CharType");
            if (CharType == null)
            {
                CharType = "001";   
                context.UserData.SetValue<string>("CharType", CharType);
                CharObj = GlobalData.CCtl.GetChacter(CharType);
            }
            else
            {
                CharObj = GlobalData.CCtl.GetChacter(CharType);
            }
            return CharObj;
        }
        public IMessageActivity TalkQMessage(
            IDialogContext context,             
            Chacter         CharObj,            
            QuessionCard     QAnswer,         
            bool            AutoPlay            
        )
        {
            IMessageActivity resultMessage = context.MakeMessage();
            resultMessage.AttachmentLayout = AttachmentLayoutTypes.Carousel;
            resultMessage.Attachments = new List<Attachment>();
            AudioCard           audioCard;
            List<VideoCard>     Video = new List<VideoCard>();
            List<CardAction>    lstButton = new List<CardAction>();

            lstButton   =   MakeActionButton(QAnswer.Macro);
            Video       =   MakeVideoButton(QAnswer.Macro);

            HeroCard heroCard = new HeroCard
            {
                Title       =   "[質問]",
                Subtitle    =   CharObj.Name,
                Text        =   QAnswer.Quession,
                Images      =   new List<CardImage> { new CardImage(QAnswer.FaceType) },
                Buttons     =   lstButton
            };
            resultMessage.Attachments.Add(heroCard.ToAttachment());
            if (QAnswer.VoiceType.Trim().Length > 0)
            {
                audioCard = new AudioCard
                {
                    Autostart   =   QAnswer.VoiceAutoPlay,           
                    Subtitle    =   CharObj.Name,
                    Media = new List<MediaUrl>
                    {
                        new MediaUrl()
                        {
                            Url = QAnswer.VoiceType
                        }
                    },
                    Shareable = false
                };
                resultMessage.Attachments.Add(audioCard.ToAttachment());
            }
            if (Video.Count > 0)
            {
                foreach (VideoCard obj in Video)
                {
                    obj.Title       =   "";
                    obj.Subtitle    =   CharObj.Name;
                    obj.Image.Url   =   QAnswer.FaceType;
                    resultMessage.Attachments.Add(obj.ToAttachment());
                }
            }
            return resultMessage;
        }

Conclusion

Pasona Group finished implementing the Q&A Bot using the Bot Framework and LUIS. They discovered that they could develop a Q&A Bot in a short period time with the Bot Framework. They also discovered that they could develop a Q&A Bot responding to natural language with LUIS. They verified that the number of inquiries related to their internal systems could actually reduced using this Q&A Bot.

Their goal is to develop a solution for an inquiry Bot as an ISV, based on the technical knowledge they gained from this project. They will promote sales of this solution to all their group companies and business units.

The following is a comment from Pasona Group that participated Hackfest.
“We are very grateful to Microsoft for your support of developing a prototype of the Q&A Bot. We discovered that the feasibility of AI related technology in this project. At the same time, we discovered a fundamental issue of human operation and promoted redifinition of our business process. We would like to pursue the innovation of business process by technologies. Thank you very much.”
Atsushi Watanabe - Manager, Group IT Management Division