Even more on Correlation IDs within SharePoint 2010

I briefly talked about log correlation IDs in two previous posts (More on Correlation IDs in SharePoint 2010 and The SharePoint 2010 Developer Dashboard), and am also including a bit from yet another post (Diagnostic Logging Enhancements in SharePoint 2010). When you get to an error page in SharePoint 2010, you are now presented with a Correlation ID. Basically, this is just expanding upon Wictor Wilén’s post here: Working with SharePoint 2010 Correlation ID in PowerShell and code. I basically just want to flesh these out a bit, and ways to get at the errors you are looking for.

Using our Boston Area SharePoint Users Group website as an example – we don’t have search configured yet on the new site, however, we have the search box available currently (we’ll fix it, I swear!). Running a search produces the following error message:



Straight, and to the point. While about as useless as your regular everyday SharePoint error, you get a pretty icon, but, more importantly, you get a correlation ID. Now, let’s see what we get using the Get-SPLogEvent PowerShell command mentioned in Wictor’s article…

So, I am running the following command –

   1: Get-SPLogEvent | ?{$_.Correlation -eq "41e7b6f1-ac66-4a16-87c0-18a85ad13f21"} | ft Category, Message -Autosize

And it’ll take a few moments, as it has a LOT of log entries to run through…


So, let’s take a step back. Now, this is NOT SUPPORTED BY MICROSOFT, at least as far as I know. So, don’t do it. ever. really. They’ll find you. This may change however once the RTM copy hits the shelves, and it is available at Target stores world-wide. Once it is released, then we’ll be able to find out all of what is and what is not supported.

But let’s say, for the sake of argument, I want to find those log entries fast, and, as luck would have it, I am also the resident SQL DBA, so, I have access to the SharePoint databases on the farm’s SQL server.

Take a look at the WSS_Logging database. Specifically, at the ULSTraceLog view. Notice in there, that there is a column named CorrelationId? Well kids, that means we can do a direct SQL search for it, in addition to the handy PowerShell Get-SPLogEvent command, as well as the code-based approach Wictor outlined in his post!

   1: SELECT * 
   2: FROM WSS_Logging.dbo.ULSTraceLog
   3: WHERE CorrelationId = '41e7b6f1-ac66-4a16-87c0-18a85ad13f21'

This, as you may notice if you also try, this will take a few moments, depending on the size of your logs, but, what I have noticed, is that it runs far quicker than the Get-SPLogEvent PowerShell command.


So, in short, if you need a completely unsupported way to quickly get to your logs by using your CorrelationId, there is a faster way than viewing and searching through the ULS text logs on the file system, or using the Get-SPLogEvent PowerShell cmdlet.

I have not run actual timing tests on these, these are just from personal experience. Maybe one day I will get around to that…


More on Correlation IDs in SharePoint 2010

A while back, I did a post on the new developer dashboard in SharePoint 2010, I’ve also discussed this topic in several presentations on What’s New for Developers in SharePoint 2010, and I’ve talked about the wonderful new feature of correlation IDs that now appear in the developer dashboard, and other error messages, that help with tracking down errors within your logs.
Wictor Wilén made a great post on Working with SharePoint 2010 Correlation ID in PowerShell and code. This article focuses on using the Get-SPLogEvent PowerShell cmdlet to track down logs using that correlation ID, and using it in code. A good read!

Microsoft.BusinessData.Runtime.ExceededLimitException: Database Connector has throttled the response.

If you’ve created a BCS list before (such as defined by the excellent tutorial put together by Laura Rogers for connecting to a SQL data source using a SQL account – http://www.sharepoint911.com/blogs/laura/Lists/Posts/Post.aspx?ID=90), and received this error on your Read List view page:

Unable to display this Web Part. To troubleshoot the problem, open this Web page in a Microsoft SharePoint Foundation-compatible HTML editor such as Microsoft SharePoint Designer. If the problem persists, contact your Web server administrator.

Correlation ID:117e242f-0985-42e9-9972-b1a879ce7e3b

After investigating the ULS Logs, located in %SYSTEM DIR%\Program Files\Common Files\Microsoft Shared\web server extensions\14\LOGS, and searching for the correlation id mentioned above. You may find the following error:

Error while executing web part: Microsoft.BusinessData.Runtime.ExceededLimitException: Database Connector has throttled the response. The response from database contains more than ‘2000’ rows. The maximum number of rows that can be read through Database Connector is ‘2000’. The limit can be changed via the ‘Set-SPBusinessDataCatalogThrottleConfig’ cmdlet.     at Microsoft.SharePoint.BusinessData.SystemSpecific.Db.ThrottledIDataReader.Read()     at Microsoft.SharePoint.BusinessData.SystemSpecific.Db.DbEntityInstanceEnumerator.MoveNext()     at Microsoft.SharePoint.BusinessData.Runtime.EntityInstanceEnumeratorBase.MoveNext()     at Microsoft.SharePoint.SPListDataSource.GetFilteredEntityInstancesInternal(XmlDocument xdQueryView, Boolean fFormatDates, Boolean fUTCToLocal, String firstRowId, Bo…

Thanks to the wonders of the interwebs, the solution to this is fairly simple, and the error explanation is extremely helpful. Do a search for this: Set-SPBusinessDataCatalogThrottleConfig, which will lead you to a great article on the Microsoft Business Connectivity Services Team Blog by Adam Outcalt here: http://blogs.msdn.com/b/bcs/archive/2010/02/16/bcs-powershell-introduction-and-throttle-management.aspx which will give you very great instructions on modifying the BCS throttling configuration.

Be careful with this however, the throttling is there for a reason. Not just to give you another option to configure.

The SharePoint 2010 Developer Dashboard

Another one of the great new features (for me, and for a lot of other administrators and developers), is the new Developer Dashboard within SharePoint 2010. The Developer Dashboard gives you plenty of output regarding the page loading process. This enables you to see how long each artifact of the page takes to load, any SQL queries, their stack traces, and IO stats, any warnings or assertions that were made, and more. This is extremely useful to use when debugging web parts, or other code that is called during the page loading process. This also gives administrators a unique insight into any bottlenecks in code running in the page created by their developers, offering them a second set of eyes on the performance of their code.

The Developer Dashboard provides insight only available from custom applications before, or, 3rd party tools such as Idera’s SharePoint Performance Manager.

The Developer Dashboard has 3 different states, On, OnDemand, or Off. When the Dashboard is set to On, it will always be shown on every page. When it is set to OnDemand, you have the ability to show and hide the Dashboard. When it is set to Off, it is, as you may have guessed, not available.

The Developer Dashboard is not enabled by default in SharePoint, at least Beta 1, or the current public Beta 2, I am not sure what the future holds for this specific configuration in the RTM version, which is still a ways away. We’ll cover the settings for this later in this post.


Enabling and Disabling the Developer Dashboard

There are three different methods you can use to change the setting for the Developer Dashboard, programmatically via code, using STSADM, or by using PowerShell.

1.) Programmatically

I have created a console application in Visual Studio 2010 Beta 2, to turn the Developer Dashboard On as shown below.

   1: using System;
   2: using System.Text;
   3: using Microsoft.SharePoint;
   4: using Microsoft.SharePoint.Administration;
   6: namespace SPDeveloperDashboard
   7: {
   8:     class Program
   9:     {
  10:         static void Main(string[] args)
  11:         {
  12:             try {
  13:                 SPWebService ws = SPWebService.ContentService;
  14:                 ws.DeveloperDashboardSettings.DisplayLevel = SPDeveloperDashboardLevel.On;
  15:                 ws.DeveloperDashboardSettings.Update();
  16:             } 
  17:             catch (Exception ex)
  18:             { 
  19:                 Console.WriteLine(ex.ToString());
  20:             }
  21:         }
  22:     }
  23: }

You can also set the Developer Dashboard to run OnDemand, as stated above, by using the following code snippet

   1: SPWebService ws = SPWebService.ContentService;
   2: ws.DeveloperDashboardSettings.DisplayLevel = SPDeveloperDashboardLevel.OnDemand;
   3: ws.DeveloperDashboardSettings.Update();

And of course, it can be turned off completely, by using the following code snippet

   1: SPWebService ws = SPWebService.ContentService;
   2: ws.DeveloperDashboardSettings.DisplayLevel = SPDeveloperDashboardLevel.Off;
   3: ws.DeveloperDashboardSettings.Update();

It is very easy to do programmatically, however, there are easier methods for tweaking this option “On Demand”.


As stated above, the STSADM command line utility can be used to change these settings as well. To do so, run the following command

   1: stsadm -o setproperty -pn developer-dashboard -pv COMMAND

whereby replacing COMMAND above with On, OnDemand, or Off.

3.) PowerShell

And my new favorite love in SharePoint for administration, PowerShell. You can also do this via PowerShell, by issuing the following commands from the SharePoint 2010 Management Shell (replacing COMMAND below with On, OnDemand, or Off).

   1: [Microsoft.SharePoint.Administration.SPWebService]::ContentService.DeveloperDashboardSettings.DisplayLevel = 'COMMAND';
   2: [Microsoft.SharePoint.Administration.SPWebService]::ContentService.DeveloperDashboardSettings.Update();

There are a few ways to skin a cat with PowerShell of course, and a few other blogs, such as Bill Baer’s, and John W. Powell’s on MSDN show you alternative methods.


The Developer Dashboard

Now that we have a firm grasp on enabling and disabling the Developer Dashboard, let’s have a look at it. With the Dashboard set to On, it will always be displayed. If we have it on and go to a page within our site, we can see it is enabled at the bottom of the screen.


If we have the Developer Dashboard set to OnDemand, we see an icon appear in what I like to refer as the HUD (Heads Up Display) in SharePoint


Clicking this icon, will either enable or disable the Developer Dashboard. This is a very useful function, as sometimes you may not want to show it all the time, but can enable it as the need arises.

Now that we have it there, let’s see what we see…

The first section we see on the right, displays each operation being performed, and the total time each operation took to run, as well as the total execution time.


The next section we see, starting at the top left of the dashboard, shows us some overall information, such as the total execution time, the current user, the current page checkout level, current operations being performed, as well as the log correlation id of this load, to help you identify related log entries within the ULS logs (very cool!)

And another cool thing, each of these has a tooltip, so if you hold your mouse over, you get a description of the item you are viewing

The next piece of information we see, is if any Asserts or Critical Events occurred during this operation.

And next we come to Database Queries, which gives us a list of all of the queries run during this page load operation

Each of these are also links as well,. that when clicked, gives you a pop-up window showing you the text of the query performed, the callstack, as well as the IO statistics

We also get to to see the number of Service Calls ( none were performed during this demonstration to show – sorry 😦 ), as well as SPRequest Allocations (SPWeb and SPSite objects)

which if we look at the tooltip for this section, we see the following
which goes back to something we developers learned a LONG time ago, is that we need to limit the number of these we run, as they eat up memory, and also need to be disposed of properly when done, it even provides a useful tip!

And last, but certainly not least, we get to see the WebPart Events Offsets, which shows us the timing offset of the event within a web part since it’s parent event had been instantiated, i.e. SPWebPartManager OnLoad.


This is a little more useful to look at and understand when there is more than 1 web part on the page, to see what I am talking about above. For each of the parent events (OnLoad, OnPreRender, etc.) that are triggered from the SPWebPartManager, you can easily see how long from when these events were started, to when they finally completed, for instance, my 17666.12ms load from my external ULS Logs List View Web Part below is taking a LONG time to run.



Additional Settings

In addition to being able to view this information, you can also set some other settings, other than the display level for the Developer Dashboard.

RequiredPermissions – By setting required permissions, using SPBasePermissions, you can restrict just who can view the Developer Dashboard. By default this is set to AddAndCustomizePages

TraceEnabled – This is a really cool property. This allows you to either display or hide a link at the bottom of the developer dashboard to display verbose tracing information. If we set this to true…

   1: SPWebService.ContentService.DeveloperDashboardSettings.DisplayLevel = SPDeveloperDashboardLevel.On;
   2: SPWebService.ContentService.DeveloperDashboardSettings.TraceEnabled = true;
   3: SPWebService.ContentService.DeveloperDashboardSettings.Update();

We now see a new link, “Show or hide additional tracing information…”, at the bottom of the Developer Dashboard


Although I have not quite yet been able to get this to work in Beta 2 as of yet. It was working in Beta 1. If anyone does have any luck, please let me know! This basically just dynamically enabled or disabled the regular ASP.NET in-page tracing information.

MaximumCriticalEventsToTrack – Sets the maximum number of critical events or assets that will be shown in a single transaction. Anything surpassing this limit is ignored. Setting this to 0 disables it altogether.

MaximumSQLQueriesToTrack – Just like the above property, however, anything above this settings will still be counted, however the call stack and query text will not be captured or visible.


Updated 1.9.10 – Wictor Wilén wrote up a great post dealing with the Developer Dashboard, and what I perceive as a new development best practice for SharePoint, making use of the SPMonitoredScope object, you can then insert your own web part calls into the Developer Dashboard, utilizing this object. More information can be found in his blog post here: http://www.wictorwilen.se/Post/Improve-your-SharePoint-2010-applications-with-monitoring-using-SPMonitoredScope.aspx?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+WictorWilen+%28Wictor+Wil%C3%A9n%29


%d bloggers like this: