SharePoint Search: Partial Page Exclusion

Out of the box, SharePoint provides a mechanism for excluding publishing pages from the search index by any number of criteria, but what if you want to exclude only parts of a page?  This becomes useful when you have content on numerous pages that contains common keywords.  For example, say you have a search result webpart in your page layout to display your latest press release: “Product A just announced”.  When someone performs a search for “Product A” they will get back every page in your site, instead of just the press release and related product pages.  To prevent this, I needed a way to keep SharePoint from indexing that content when it’s performing a crawl.

Web Control

I decided that a System.Web.UI.WebControls.Panel control would be a good model to build my control on.  It allows you to easily drop it in the page layout using SharePoint Designer, and you can put other html and controls within it.  I didn’t want to inherit from the Panel control though, because it adds unwanted ‘div’ tags to the rendered output.  The key to the Panel control’s behavior are the following two attributes on the class: [ParseChildren(false), PersistChildren(true)].  These attributes allow the content within the control to persist as controls and not properties of this control. 

User Agent

The second part of the equation is knowing when to show or hide the contents of the web control.  SharePoint gives us a way to identify that it’s performing a crawl through the UserAgent property of the http request by adding “ms search” to it. 

Code

Putting this all together we come up with the following class:

    [ParseChildren(false), PersistChildren(true)]
    public class SearchCrawlExclusionControl : WebControl
    {
        private string userAgentToExclude;

        public string UserAgentToExclude
        {
            get
            {
                return (string.IsNullOrEmpty(userAgentToExclude)) ? "ms search" : userAgentToExclude;
            }
            set
            {
                userAgentToExclude = value;
            }
        }

        protected override void CreateChildControls()
        {
            string userAgent = this.Context.Request.UserAgent;

            this.Visible = (!string.IsNullOrEmpty(userAgent)) ? !userAgent.ToLower().Contains(UserAgentToExclude) : true;

            base.CreateChildControls();
        }

    }

Using It

After adding the register tag to the page layout, we can wrap all the content we want to exclude with our control:

<SearchUtil:SearchCrawlExclusionControl ID="SearchCrawlExclusionControl1" runat="server">

    <div>Some Content To Exclude</div>

</SearchUtil:SearchCrawlExclusionControl>

Comments

[Casey Liss]

Very slick chunk of code, Scott. It's always nice to get a lot of functionality from just a few lines of code. :)