Slides from February 13, 2018 BASPUG Meeting

Thank you all who came out last night to see my presentation at the Boston Area SharePoint Users Group last night!

As promised, here are my slides from the presentation, hosted on SlideShare:

And here is a link to the blog post which contains all of the steps we demoed during the presentation: https://sharepointyankee.com/2018/01/23/creating-a-chat-bot-with-microsofts-qna-maker-and-azure-bot-service/

Advertisement

Creating Runbooks in Azure and Calling Them from SharePoint Using Webhooks and Flow

AzureFlowSharePointAutomationRunbooks are a feature of Azure Automation that allow you to execute workflows from within Azure or remotely to automate processes.

To give an example, lets say you have a script that monitors an Azure service every 5 minutes to see if it is running or not. The script, will test and see the status of an Azure App Service. If it tests the site, and does not get the HTTP 200/OK message, then it triggers an alert, creates a ticket, and now someone has to go recycle the Azure App Service. If this can happen frequently, then it is something you would look to automate.

In comes the Azure Automation Runbook. You create a PowerShell script that is hosted in Azure (a Runbook), and when your script detects that the service is not responding, it makes a call out to a URL, and the URL runs the Runbook, which restarts the Azure App Service. The monitoring script then runs again, sees that the service is back up, and the appropriate steps are taken.

This might seem like a lot of extra work, but, if you are, say, connecting in through a VPN to manage an Azure environment, it can be quite time consuming just to restart a service.

However, we are not using that as our working example in this article. That was just to give you an idea of the kinds of things that can be done using Runbooks. In this article, we will be showing you how to create a Runbook, and call it from SharePoint, using Microsoft Flow. It will not be a real exciting example either, but, it will show you how to do all this, so you can do more on your own!

Prerequisites

This article assumes the following:

  • You have an Azure subscription. If you do not, you can get one here for free to play around
  • You have SharePoint Online

Creating an Azure Automation Account

Before we can create our Runbook, we need to create an Azure Automation Account. Login into the Azure Portal, click on New > Monitoring + Management > Automation

image

Configure the following settings for your Automation Account:

  • Name: What are you going to call it?
  • Subscription: Select the subscription to use
  • Resource Group: Either create a new one, or, use an existing.
  • Location: Which Azure region should this run in? I am using East US 2… since I’m in the East US.
  • Create Azure Run As account: This is not needed for our test, but, if you’re doing anything in Azure with your runbooks, you will want to configure this. For more information, visit: https://docs.microsoft.com/en-us/azure/automation/automation-offering-get-started#authentication-planning

image

Then press Create.

It’ll take a moment while this deploys…

image

Once done… access it either by the Automation Accounts blade on the left side, or, via the Notifications link Go to resource once its done deploying.

image

And you will be brought to the landing page for your Automation Account, AutomationTest

image

Creating an Azure Automation Runbook

Now that we have our Automation Account, we need to create our runbook. From within the Automation Account, click on Runbooks under Process Automation on the left hand side.

image

Then click Add a runbook at the top of the runbooks dashboard

image

Click on Quick Create / Create a new runbook

image

Fill in the details

  • Name: Check-Website
    Give your runbook a name
  • Type: PowerShell
    You can also choose Python 2, Graphical, PowerShell Workflow, and Graphical PowerShell Workflow
  • Description: Check the status of a website
    Enter in a description for the runbook

Then click on Create

image

And viola! Your runbook has been created!

image

It doesn’t do anything yet, so, we will need to add code. Click on Edit at the top of the dashboard.

Here is where we will type out, or paste in our PowerShell code for the runbook.

image

NOTE: Do not use Write-Host, there is no “host” per-say to write to. Instead, ensure all output is written using Write-Output

Let’s add the following code to test if Google is up and running…

Function OutputStatus($type,$status) {
    Write-Output "$type | $status";
}

Function Get-HTTPSStatus($url,$type) {
    $HTTPSStatus = Invoke-WebRequest $url -Method Get –UseBasicParsing
    if ($HTTPSStatus.StatusCode -eq "200") {
        return OutputStatus -type $type -status "Success"
    } else {
        return OutputStatus -type $type -status "Error"
    }
}

Get-HTTPSStatus "http://www.google.com" "Google Website"

image

Click on Save

image

Now lets test it…click on Test pane

image

Click on Start

image

You will see a message that it is being submitted

image

You can then see that it gets queued

image

And finally, we see the status and the output displayed

image

Pretty neat!

Now, lets say we want to add some parameters to our script, so we can specify the input… and not have it statically set as just “http://www.google.com” as the site, and “Google Website” as the description. Let’s update the code with some parameters…

To get back to your code, click on Edit PowerShell Runbook in the breadcrumb navigation at the top

image

Update our code with the parameters $Site and $Description, and then Save, and then go back on over to the Test pane

image

You can now see we have two fields for Site and Description under Parameters. Fill those out…

image

And run the script again…

image

Looks good! Now… we can do this all day from within Azure… but remember way back to the start of this article, I mentioned calling this from Microsoft Flow from within SharePoint? To do that… we’re going to need to make a change to our script, as well as create a webhook.

First, lets change our script. You know how we just added parameters? Well, when calling a webhook, we’re going to be making a REST call to a URL. We cannot pass in parameters like we just did to the script. That is good for running within Azure itself… in order to pass parameters to our runbook via a webhook… we need to change the parameters. We will be passing in an object called WebhookData (or whatever else you want to call it). Which will be the JSON data sent along with the REST call. So, let’s update our code to this:

image

We will then parse out the Site and Description name/value pairs from that and pass it into our script from the $WebhookData object.

The code for the above is here:

Param (
    [object]$WebhookData
)

Function OutputStatus($type,$status) {
    Write-Output "$type | $status";
}

function Get-HTTPStatus($url,$type) {
    $HTTPStatus = Invoke-WebRequest $url -Method Get –UseBasicParsing
    if ($HTTPStatus.StatusCode -eq "200") {
        return OutputStatus -type $type -status "Success"
    } else {
        return OutputStatus -type $type -status "Error"
    }
}

if ($WebhookData -ne $null) {
    Get-HTTPStatus $WebhookData.RequestHeader.Site $WebhookData.RequestHeader.Description
} else {
    Write-Error "No data received in webhook call."
}

We need to Publish it first before creating the webhook. Go back to the code view, and click on Publish

image

It will prompt you to confirm, click Yes, and it’ll be published.

image

Now that we’ve got that straightened out… let’s move on to creating our webhook.

Creating a Runbook Webhook

From our runbook Dashboard, click on Webhook at the top of the dashboard

image

Click on Webhook – Create a new webhook

image

Then give it a name, and an expiration date, and if it should be enabled or not…

image

Now… notice the big warning sign at the top of this screen…

image

See? Now… copy and paste that URL at the bottom, and save it somewhere. There is no way to get this URL once the webhook has been created.

image

Once you have done that, click OK

Then click on Parameters and run settings and then click OK there

image

Then click Create at the bottom of the form. Until you do that, you can still get the webhook URL…

Ok… now what? Let’s call it from PowerShell, since we need to do a POST to access it.

image

We can see in the Content section of the output, we are given a JobId of 4164eb1f-57ba-41c3-a7cb-2f556652e9ad

In our runbook, if we go to Jobs under Resources

image

We can see that a job successfully ran

image

Click on it, and we can see the status, and you will see the JobId matches what we got from the call from Invoke-WebRequest

image

You will see there were errors… because we didn’t actually send any data along with it. We just called it directly. But now that we have it… we can move on to SharePoint and Flow.

Creating a Flow to Call our Webhook from SharePoint

Now that we’ve gone through the meat an potatoes of this project… let’s look at linking at all together with SharePoint and Flow.

Log into your SharePoint Online tenant… and lets create a new list.

I’ve got a basic custom list called Flowtest

image

Now… once created, in the Modern interface… click on Flow > Create a flow

image

Click on See your flows at the bottom, because we’re going to create a brandy-new one…

image

Click on + Create from blank at the top of the page

image

Click on Search at the bottom of the next screen, and search for SharePoint created… we want to add a trigger for when a new item is created in our list.

image

Select SharePoint – When an item is created

Select your SharePoint Online site from the list, or, enter in the URL, then select the list… in this case, we’re using Flowtest

image

Then click + New step > Add an action

image

Click on HTTP under Connectors

image

Choose HTTP – HTTP

image

Then fill out the details…

  • Method: POST
  • Uri: The URL we copied when we created our webhook
  • Headers
    Site:
    http://www.google.com
    Description: Google’s Website (FROM SHAREPOINT!)

image

And then click on Save Flow

Also… don’t forget to give your flow a name Smile

image

You should now see your Flow

image

Now… open a new window, and go back to your list, and create a new item…

image

And if you check back on your flows…

image

You will see one succeeded!

Clicking on it will give you the breakdown of the flow run (which is one of the more awesome features of Flow… over IFTTT IMHO FWIW YK?)

image

Now… let’s go check Azure…

If we look at the jobs for our Runbook… we’ll see a new one in there…

image

Click on it, and then click on the output

image

image

It worked!

Now… let’s make this a bit more functional. Go back to your list settings in SharePoint

image

I’ve changed the Title field to URL, and added a field called Description as a single line of text.

image

Now, let’s go back to our Flow…

And edit the HTTP step

image

Edit the values for Site and Description, and then select the corresponding Site and Description values from the Dynamic Content list that pops up to the right. See what we’re doing here?

image

Let’s run our Flow… create a new list item, passing in a URL and Description…

image

and check the status…

image

It worked! It’s a day of miracles people! While this is not a really exciting example, it shows how to use Azure Runbooks and Webhooks, and how they can be accessed remotely to do a specific task.

What sort of cool things are you doing or have you done with Flow and Runbooks, if anything?

Resources and References

Creating a Chat Bot with Microsoft’s QnA Maker and Azure Bot Service

Image

Microsoft has a service in preview, called QnA Maker. It is a free (presently) service, which provides a REST and web-based service that trains AI to respond to questions asked of it. You’ve seen these on other sites, as these have become quite common, and ease the burden on support for SharePoint, or any other service out there. Chat bots can field common questions, freeing up time and energy spent on answering common asks.

They make it very easy to use – you can point it to a URL, type in your questions and answers, or upload a document (.docx, .doc, .pdf, .xlsx, and .tsv). The tricky part, is getting this from loading your knowledge base, to actually having a chat bot on your site. This is what this blog post is for, is to show you how to do it. They do have directions, but not a full walk-through on what to do.

This post assumes that you have at the very least, an Azure Account. If not, you can start one for free.

QnA Maker Setup

First, go to http://qnamaker.ai, this is where you will get started. Click on Create New Service

image

Then sign in with a Microsoft ID, or organizational account like you would into Azure, Office365, Outlook.com, etc.

image

It will then ask you to provide permissions to QnAMaker, click Accept

image

Then you must agree to the terms and privacy statement, check the box and hit Continue

image

Enter in a name for your bot. I am choosing Testbot 9001

image

You now have 3 options for loading data into your Chat Bot…

  1. From a URL that links to a resource which has a clear Q&A format, like a FAQ
  2. From a file, in Question/Answer format (supported file types are .TSV*, .PDF, .DOC, .DOCX, and .XLSX format)
    *TSV stands for Tab Separated Values. Like a CSV, but, you know, with Tabs. You can do this in Excel.
  3. Enter them in manually

For our Testbot 9001, we will be uploading from Excel. So, let’s create a new Excel file. In the first column, type in the question, or phrase, which the chatbot AI will hit on and use to provide the answer. Use the second column to provide that answer. Here is my example file:

image

Now, save the file. And go back to the QnA Maker setup, and choose Select file… next to the FILES section

image

and select the XLSX file from your local computer you just saved

image

Then scroll to the bottom and select Create

image

You will then see a dialog window appear while the service scans your file, and imports your Questions and Answers
image

Once complete, it will take you to the Testbot 9001 Knowledgebase.

image

Select Test from the side bar, and you can test the functionality with your questions
image

Cool, huh? The testing also allows you to train the bot as well. Such as choosing the best answer based upon what you had typed
image

Or providing multiple alternative phrasings, or synonyms

image

When done, click Save and Retrain to save your changes

image

You can see back in the main Knowledgebase, that it has been updated by what we asked and what we updated the suggestions and phrasings with

image

You can also add new Q & A pairs by clicking on Add new QnA pair

image

image

When done, Save & Retrain again to save your settings.

Once you are ready… click Publish

image

It will then show you what will be published… and then when ready, click Publish

image

The deployment confirmation screen will show you the example HTTP request for using your bot through the REST API. These are also available in the Settings link on the sidebar of the Knowledgebase.

image

We will need some f this information in a few minutes, so take down the Knowledgebase ID

image

And the Subscription Key

image

and put those aside for a few minutes…

Azure Bot Service Setup

Next we’re going to use the Azure Bot Service to setup our bot. To do so, we’ll be following these instructions: https://docs.microsoft.com/en-us/bot-framework/bot-service-quickstart

Log into the Azure Portal

Click on New > AI + Cognitive Services > Web App Bot

image

Now, let’s configure the botimage

  • Name: Testbot9001
    So you can identify the soldiers in your bot army!
  • Subscription: GeoffOps
    Choose your own subscription. This one is mine. You can’t have it.
  • Resource Group: Testbot9001
    Use an existing or create a new one
  • Location: East US
    Select the data center region you would like to use
  • Pricing Tier:  FO (10K Premium Messages)
    This defaults to S1 Standard, change this to F0 Free. Or else you will pay for the service
  • App Name: Testbot9001
  • Bot template:  Question and Answer
  • App service plan/Location: Testbot9001
    You will need to create a new one, or use an existing. For this example, I created a new one also in East US
  • Azure Storage: testbot9001a325
    I am letting Azure create a new one for me. This is a data storage account, like a fileshare. You can use an existing one if you have one as well.
  • Application Insights: Off
    This is not free. So I turned it off.

Then click Create to bring life to your Azure Bot. I twill validate your choices, and then when complete, will begin deployment.

image

When deployment has completed, you can find the bot and associates Azure resources under the All Resources blade of the left, or, if you have other stuff in Azure, go to the Resource Groups blade, and click on your Resource Group we created in the Bot setup above. In my case, the Resource Group is Testbot9001

image

Ok. So, we’ve created our bot with QnA Maker, and created a Bot Service in Azure… now what? We need to connect the two.

Select your bot, it’ll be the one which has a TYPE of Web App Bot

image

On the left hand side, click on APP SERVICE SETTINGS > Application Settings

image

And then scroll down to App settings. You will see two blank settings called QnAKnowledgebaseId and QnASubscriptionKey

Now, remember those things we copied off earlier? Pate these in there.

image

And then click Save up top

ScreenClip

Now, click on Test in Web Chat on the left side, and test out your new bot!

image

Pretty neat! But… now what?

Click on Channels on the left side. You will see Web Chat setup and running by default. You can also see, you can extend this out to Teams, Skype, and more pretty easily.

image

Click on Edit, and copy the Embed Code

image

And show one of the keys at the top

image

and replace the YOUR_SECRET_HERE in the embed code

image

And you can now drop that into a site, web part, whatever you’d like. It’s just an iFrame, so the possibilities are endless.

A simple example – using Embed Code on a SharePoint page:

image

image

image

Would I implement it like this for a client? Likely not… I’d be more apt to have a sliding pop-up window, similar to Facebook chats that can be minimized and maximized as needed. This was just an example to show how to build one of these.

 

References

%d bloggers like this: