PowerShell Script to Get HTTP Headers

Quick script to grab HTTP headers from a given URL.

I named the script Get-HTTPHeaders.ps1, you can save it as whatever you’d like, or, incorporate it into your script as a function, or not even read this post… whatever you want to do. I just find it handy, and wanted to share it.

    [string] $Url

$request = [System.Net.WebRequest]::Create( $Url )
$headers = $request.GetResponse().Headers
$headers.AllKeys |
     Select-Object @{ Name = "Key"; Expression = { $_ }},
     @{ Name = "Value"; Expression = { $headers.GetValues( $_ ) } }

All you have to do is pass it a URL as a parameter. See the example below:


Creating Runbooks in Azure and Calling Them from SharePoint Using Webhooks and Flow

AzureFlowSharePointAutomationRunbooks are a feature of Azure Automation that allow you to execute workflows from within Azure or remotely to automate processes.

To give an example, lets say you have a script that monitors an Azure service every 5 minutes to see if it is running or not. The script, will test and see the status of an Azure App Service. If it tests the site, and does not get the HTTP 200/OK message, then it triggers an alert, creates a ticket, and now someone has to go recycle the Azure App Service. If this can happen frequently, then it is something you would look to automate.

In comes the Azure Automation Runbook. You create a PowerShell script that is hosted in Azure (a Runbook), and when your script detects that the service is not responding, it makes a call out to a URL, and the URL runs the Runbook, which restarts the Azure App Service. The monitoring script then runs again, sees that the service is back up, and the appropriate steps are taken.

This might seem like a lot of extra work, but, if you are, say, connecting in through a VPN to manage an Azure environment, it can be quite time consuming just to restart a service.

However, we are not using that as our working example in this article. That was just to give you an idea of the kinds of things that can be done using Runbooks. In this article, we will be showing you how to create a Runbook, and call it from SharePoint, using Microsoft Flow. It will not be a real exciting example either, but, it will show you how to do all this, so you can do more on your own!


This article assumes the following:

  • You have an Azure subscription. If you do not, you can get one here for free to play around
  • You have SharePoint Online

Creating an Azure Automation Account

Before we can create our Runbook, we need to create an Azure Automation Account. Login into the Azure Portal, click on New > Monitoring + Management > Automation


Configure the following settings for your Automation Account:

  • Name: What are you going to call it?
  • Subscription: Select the subscription to use
  • Resource Group: Either create a new one, or, use an existing.
  • Location: Which Azure region should this run in? I am using East US 2… since I’m in the East US.
  • Create Azure Run As account: This is not needed for our test, but, if you’re doing anything in Azure with your runbooks, you will want to configure this. For more information, visit: https://docs.microsoft.com/en-us/azure/automation/automation-offering-get-started#authentication-planning


Then press Create.

It’ll take a moment while this deploys…


Once done… access it either by the Automation Accounts blade on the left side, or, via the Notifications link Go to resource once its done deploying.


And you will be brought to the landing page for your Automation Account, AutomationTest


Creating an Azure Automation Runbook

Now that we have our Automation Account, we need to create our runbook. From within the Automation Account, click on Runbooks under Process Automation on the left hand side.


Then click Add a runbook at the top of the runbooks dashboard


Click on Quick Create / Create a new runbook


Fill in the details

  • Name: Check-Website
    Give your runbook a name
  • Type: PowerShell
    You can also choose Python 2, Graphical, PowerShell Workflow, and Graphical PowerShell Workflow
  • Description: Check the status of a website
    Enter in a description for the runbook

Then click on Create


And viola! Your runbook has been created!


It doesn’t do anything yet, so, we will need to add code. Click on Edit at the top of the dashboard.

Here is where we will type out, or paste in our PowerShell code for the runbook.


NOTE: Do not use Write-Host, there is no “host” per-say to write to. Instead, ensure all output is written using Write-Output

Let’s add the following code to test if Google is up and running…

Function OutputStatus($type,$status) {
    Write-Output "$type | $status";

Function Get-HTTPSStatus($url,$type) {
    $HTTPSStatus = Invoke-WebRequest $url -Method Get –UseBasicParsing
    if ($HTTPSStatus.StatusCode -eq "200") {
        return OutputStatus -type $type -status "Success"
    } else {
        return OutputStatus -type $type -status "Error"

Get-HTTPSStatus "http://www.google.com" "Google Website"


Click on Save


Now lets test it…click on Test pane


Click on Start


You will see a message that it is being submitted


You can then see that it gets queued


And finally, we see the status and the output displayed


Pretty neat!

Now, lets say we want to add some parameters to our script, so we can specify the input… and not have it statically set as just “http://www.google.com” as the site, and “Google Website” as the description. Let’s update the code with some parameters…

To get back to your code, click on Edit PowerShell Runbook in the breadcrumb navigation at the top


Update our code with the parameters $Site and $Description, and then Save, and then go back on over to the Test pane


You can now see we have two fields for Site and Description under Parameters. Fill those out…


And run the script again…


Looks good! Now… we can do this all day from within Azure… but remember way back to the start of this article, I mentioned calling this from Microsoft Flow from within SharePoint? To do that… we’re going to need to make a change to our script, as well as create a webhook.

First, lets change our script. You know how we just added parameters? Well, when calling a webhook, we’re going to be making a REST call to a URL. We cannot pass in parameters like we just did to the script. That is good for running within Azure itself… in order to pass parameters to our runbook via a webhook… we need to change the parameters. We will be passing in an object called WebhookData (or whatever else you want to call it). Which will be the JSON data sent along with the REST call. So, let’s update our code to this:


We will then parse out the Site and Description name/value pairs from that and pass it into our script from the $WebhookData object.

The code for the above is here:

Param (

Function OutputStatus($type,$status) {
    Write-Output "$type | $status";

function Get-HTTPStatus($url,$type) {
    $HTTPStatus = Invoke-WebRequest $url -Method Get –UseBasicParsing
    if ($HTTPStatus.StatusCode -eq "200") {
        return OutputStatus -type $type -status "Success"
    } else {
        return OutputStatus -type $type -status "Error"

if ($WebhookData -ne $null) {
    Get-HTTPStatus $WebhookData.RequestHeader.Site $WebhookData.RequestHeader.Description
} else {
    Write-Error "No data received in webhook call."

We need to Publish it first before creating the webhook. Go back to the code view, and click on Publish


It will prompt you to confirm, click Yes, and it’ll be published.


Now that we’ve got that straightened out… let’s move on to creating our webhook.

Creating a Runbook Webhook

From our runbook Dashboard, click on Webhook at the top of the dashboard


Click on Webhook – Create a new webhook


Then give it a name, and an expiration date, and if it should be enabled or not…


Now… notice the big warning sign at the top of this screen…


See? Now… copy and paste that URL at the bottom, and save it somewhere. There is no way to get this URL once the webhook has been created.


Once you have done that, click OK

Then click on Parameters and run settings and then click OK there


Then click Create at the bottom of the form. Until you do that, you can still get the webhook URL…

Ok… now what? Let’s call it from PowerShell, since we need to do a POST to access it.


We can see in the Content section of the output, we are given a JobId of 4164eb1f-57ba-41c3-a7cb-2f556652e9ad

In our runbook, if we go to Jobs under Resources


We can see that a job successfully ran


Click on it, and we can see the status, and you will see the JobId matches what we got from the call from Invoke-WebRequest


You will see there were errors… because we didn’t actually send any data along with it. We just called it directly. But now that we have it… we can move on to SharePoint and Flow.

Creating a Flow to Call our Webhook from SharePoint

Now that we’ve gone through the meat an potatoes of this project… let’s look at linking at all together with SharePoint and Flow.

Log into your SharePoint Online tenant… and lets create a new list.

I’ve got a basic custom list called Flowtest


Now… once created, in the Modern interface… click on Flow > Create a flow


Click on See your flows at the bottom, because we’re going to create a brandy-new one…


Click on + Create from blank at the top of the page


Click on Search at the bottom of the next screen, and search for SharePoint created… we want to add a trigger for when a new item is created in our list.


Select SharePoint – When an item is created

Select your SharePoint Online site from the list, or, enter in the URL, then select the list… in this case, we’re using Flowtest


Then click + New step > Add an action


Click on HTTP under Connectors


Choose HTTP – HTTP


Then fill out the details…

  • Method: POST
  • Uri: The URL we copied when we created our webhook
  • Headers
    Description: Google’s Website (FROM SHAREPOINT!)


And then click on Save Flow

Also… don’t forget to give your flow a name Smile


You should now see your Flow


Now… open a new window, and go back to your list, and create a new item…


And if you check back on your flows…


You will see one succeeded!

Clicking on it will give you the breakdown of the flow run (which is one of the more awesome features of Flow… over IFTTT IMHO FWIW YK?)


Now… let’s go check Azure…

If we look at the jobs for our Runbook… we’ll see a new one in there…


Click on it, and then click on the output



It worked!

Now… let’s make this a bit more functional. Go back to your list settings in SharePoint


I’ve changed the Title field to URL, and added a field called Description as a single line of text.


Now, let’s go back to our Flow…

And edit the HTTP step


Edit the values for Site and Description, and then select the corresponding Site and Description values from the Dynamic Content list that pops up to the right. See what we’re doing here?


Let’s run our Flow… create a new list item, passing in a URL and Description…


and check the status…


It worked! It’s a day of miracles people! While this is not a really exciting example, it shows how to use Azure Runbooks and Webhooks, and how they can be accessed remotely to do a specific task.

What sort of cool things are you doing or have you done with Flow and Runbooks, if anything?

Resources and References

Connecting to SharePoint Online using the PnP PowerShell Library and NOT Having to Log In Every. Single. Time…

imageBefore you can do anything with the SharePoint Patterns & Practices PowerShell library, you need to first connect to SharePoint Online. Sounds pretty basic, right? You need to establish who you are, and maintain your access during your session with the site you are working with.

Now, the Documentation does show you how to do this:


Connect-PnPOnline –Url https://geoff365.sharepoint.com –Credentials (Get-Credential)

When you do this… you are prompted for credentials… Every. Single. Time.


This is good for production, however, if you are developing a script, you may run this tens or hundreds of times… and, it gets old pretty fast. So, here is what I do. In my script, I set variables for the username and password (alternatively, you could pass these as parameters, and pass them along using a batch file).


Then, I convert the password into a secure string, and create a PSCredential object with the username and secure password.


I can then connect to SharePoint Online using the Connect-PnPOnline command (as shown above), wrapped in a try/catch block, and not be prompted for credentials!


Here’s the full script:

#region Imports
Import-Module SharePointPnPPowerShellOnline -WarningAction SilentlyContinue
#endregion Imports

#region Variables
$Username = "admin@geoff365.onmicrosoft.com"
$Password = "ThisIsNotMyRealPassword!"
$SiteCollection = "https://geoff365.sharepoint.com/sites/powershellplayground"
#endregion Variables

#region Credentials
[SecureString]$SecurePass = ConvertTo-SecureString $Password -AsPlainText -Force
[System.Management.Automation.PSCredential]$PSCredentials = New-Object System.Management.Automation.PSCredential($Username, $SecurePass)
#endregion Credentials

#region ConnectPnPOnline
try {
    Connect-PnPOnline -Url $SiteCollection -Credentials $PSCredentials
    if (-not (Get-PnPContext)) {
        Write-Host "Error connecting to SharePoint Online, unable to establish context" -foregroundcolor black -backgroundcolor Red
} catch {
    Write-Host "Error connecting to SharePoint Online: $_.Exception.Message" -foregroundcolor black -backgroundcolor Red
#endregion ConnectPnPOnline

Creating New Service Application Proxy Groups and Associating Services and Sites

Sometimes the need arises to create separate Service Application Proxy groups in SharePoint. Starting with SharePoint 2010, you’ve been able to do this. In SharePoint 2007, you would have created different Shared Service Providers. Your needs might be, that you are exposing web applications to a different group of users, and need separate applications such as Search and the User Profile Service. This also allows you to run those service applications under different accounts, if you needed to for security reasons.

First, let’s create the proxy group we want to use. And lets give it a name.. in the example, I’ll be using "Redacted"… because all my screenshots have had the real service application group identity redacted 🙂 But you can name this anything you’d like. Load up the SharePoint Version Management Console… and call the New-SPServiceApplicationProxyGroup PowerShell cmdlet.

New-SPServiceApplicationProxyGroup "Redacted"

Once you have your new Service Application Proxy Group created, you can then change the web application subscriptions to the proxy groups. To do so, go into Central Administration > Application Management > Web Applications > Manage web applications and select one of your sites. In the ribbon, then select Service Connections under the Management group.


You can always verify this by then going into Central Administration > Application Management > Service Applications > Configure service application associations


Select the Web Applications view, and then you should see your sites, and their associated applications with their Application Proxy Groups.


When you create a new service application, by default, it is going to get tossed into the default group. There is no way in the UI presently to allow you to change associations in the UI once you have created your new proxy group, so, what you need to do is to handle this in PowerShell. The best way to get the IDs for your service applications is to use Get-SPServiceApplication, and then only display the two columns you need, DisplayName (so you know what ones you are looking for), and the Id.

Get-SPServiceApplication | select-object DisplayName,Id


Once you have the ID’s that you need for your Service Applications, you now need to add them as members to the new proxy group you created earlier. You can do this by using the Add-SPServiceApplicationProxyGroupMember PowerShell cmdlet, like so:

Add-SPServiceApplicationProxyGroupMember "Redacted" -Member "f166672c-24b5-4f1a-bd2d-e8436d966abb"

This will add the "Secure Store Service – Redacted" Service Application to my new proxy group Redacted.

If for some reason, one of the service applications do not want to move out of the default group after adding them into the new group, you can remove them with the Remove-SPServiceApplicationProxyGroupMember PowerShell cmdlet. Just an FYI – the default group is referenced as "", so, if you needed to now remove the above service from the default group, you will need to address it as:

Remove-SPServiceApplicationProxyGroupMember "" -Member "f166672c-24b5-4f1a-bd2d-e8436d966abb"

SharePoint Patterns and Practices Cmdlet Reference Guide Now Available

Finally here! Microsoft has released a reference guide for the over 200 Patterns and Practices Cmdlets!


Check it out here: http://aka.ms/sppnp-powershell

Speaking at SPTechCon San Francisco in March 2013

imageI have the privilege to announce that I will be presenting again in lovely San Francisco at The SharePoint Technology Conference on March 3-6th, 2013.

Information on the event and the sessions I will be presenting is below. I hope to see you there! San Francisco is a beautiful place, and I cannot wait to get back down (and over… diagonally really) there.


Event Description

SPTechCon: The SharePoint Technology Conference – is the largest independent SharePoint conference in the world – for IT professionals, business managers, analysts, and developers. Choose from more than 90 Classes and Workshops!!! Learn from the brightest minds in the SharePoint universe – more than 40 Expert Speakers and Microsoft MVPs! There’s a shortage of SharePoint experts! Develop your skills, and improve your own professional standing! "I would recommend SPTechCon to SharePoint admins and developers. By far the best Tech Event I have attended." Venki Oruganti , Software Developer – Pitney Bowes


Planning and Configuring Extranets in SharePoint 2010

Monday, March 4 – 11:15 AM – 12:30 PM

Most companies, large or small, require contact and collaboration with external entities, whether they are vendors, clients, or contractors. SharePoint gives us the ability to open up portals for collaboration with these external entities, and this session will show you how to accomplish this using SharePoint 2010.
We will review what is required to make SharePoint “open” to the external world and discuss scenarios regarding security and privacy. We will also walk through configuring Forms-Based Authentication, Claims-Based Authentication, as well as using Business Connectivity Services in SharePoint 2010 to authenticate and manage our external users. Once completing this session, you should have a firm grasp on how to configure an extranet environment using SharePoint 2010, as well as what should be considered during the planning of your extranet scenarios.

Level: Advanced
Topic Area: IT Pro Essentials

Automating Your Enterprise Application Deployments With PowerShell

Tuesday, March 5 – 4:00 PM – 5:15 PM

In enterprise application deployments to SharePoint, there are generally farms or Web applications that host specific applications for specific groups, rather than having just an all-in-one deployment. Oftentimes, you do not have access to these other environments, and will need to pass the deployment baton off to the system administrators of those farms.
This class will walk you through how you can deploy your applications—without needing to have the administrator who will be deploying the application—have to do much more than type a few keys into the SharePoint Administration Console and press enter, and having full configuration and deployment of your custom SharePoint applications to other environments.

Level: Advanced
Topic Area: IT Pro Essentials

Extracting Solution Packages (WSPs) from SharePoint using PowerShell

I just stumbled across this link from Kirk Evanshttp://blogs.msdn.com/b/kaevans/archive/2011/12/05/extract-a-wsp-solution-from-sharepoint-2010.aspx

Basically, using 3 lines of PowerShell (which can be turned really into one if needed), to extract a SharePoint Solution Package (WSP) from the Farm Solutions collection.

From his post:

$farm = Get-SPFarm

$file = $farm.Solutions.Item("extendeddiagnosticproviders.wsp").SolutionFile


As awesome as it is easy.

PowerShell script to list all Webs and Site Templates in use within a Site Collection

And one more quick post today, this PowerShell script will iterate through all Webs within a Site Collection, print out their Title, URL, and WebTemplate (Site Template) name.

   1: $site = Get-SPSite "http://yoursite"

   2: foreach ($web in $site.AllWebs) { 

   3:     $web | Select-Object -Property Title,Url,WebTemplate 

   4: }

   5: $site.Dispose()

And again, before you use this – please read this: https://gvaro.wordpress.com/2011/03/26/test-out-your-powershell-scripts-first-in-a-non-production-environment-first/ (and also read the comment by Anders Rask)

LotD: Using PowerShell to Approve List Items

The Link of the Day (LotD) goes out to Eric Shupps for this quick PowerShell snippet http://www.binarywave.com/blogs/eshupps/Lists/Posts/Post.aspx?List=89cbe813%2D99f7%2D4257%2Da23a%2D5fefc377336b&ID=241&Web=c7893495%2Dbe3b%2D4d73%2D9875%2D28b039760651. This PowerShell code will loop through a list of your choice and approve list items. This is a huge timesaver for me right at the moment. Thank you Shupps!

I do have some slight changes to the code. There is no need to open the site collection object (line 1), we can just use Get-SPWeb to open the web we need, and, also disposing of the web object at the end, so we don’t leave it sitting around eating up memory on the server.

Normally I do not create posts like this, but, I think I had something to add to it, and, I know exactly where to find it later on, as I am sure I will need it.

   1: $web = Get-SPWeb "http://site"

   2: $list = $web.Lists["Posts"]

   3: $items = $list.Items 


   5: foreach ($item in $items)

   6: {

   7:     $item["_ModerationStatus"] = 0

   8:     $item.Update()

   9: } 

  10: $web.Dispose()

And before you use this – please read this: https://gvaro.wordpress.com/2011/03/26/test-out-your-powershell-scripts-first-in-a-non-production-environment-first/ (and also read the comment by Anders Rask)


You may be wondering why I applied such bad grammar to the title of this post, it came from a comment on this: https://www.nothingbutsharepoint.com/sites/itpro/Pages/Seven-Virtues-for-the-SharePoint-IT-Pro.aspx

PowerShell is a beast. Sure, it’s hard to learn the syntax, and there are 600+ commands that come along with SharePoint 2010. But, it also gives you direct access to the API for SharePoint. Never, ever, ever, ever, ever, ever, ever, ever, ever, ever, ever, ever, ever, ever, ever, under any circumstances what-so-ever, run un-tested PowerShell code in a production environment. ever. ever. ever. ever. ever. ever. ever. ever. ever. ever. ever. ever. ever. ever. ever.

What may have worked for one person, in which they have posted it on their blog, mailing list, or company-wide fax – it should be tested first, in a non-production environment.

It may work fine, sure, but, it may also grind everything to a halt, and end your career.

In closing, re-read the above, and make it your mantra. No go fourth, SharePoint admins (and devs – yes you too), and prosper.

%d bloggers like this: