SharePoint Advanced Search Properties Don't Work - the skinny on Created By, Modified By, Author and more

If you've ever done much work with the Advanced Search web part it won't take you long until you discover that most of the default property searches simply don't work. As of writing this post there were dozens of very long forum threads describing the problem with no comprehensive fix in sight by Microsoft or anyone else. Perhaps until now...?

What's not Broken?

Despite the claims of many, the Size and (Created/Modified)Date properties are not completely broken. They simply require the correct format for input. It's also worth noting that the explicit value is required when using property searches. No wildcards or partials!

• Size - takes a value in bytes, so that 1000000 = 1Mb.
• Dates - require a xx/xx/xxx or xx/xx/xxxx format. The order of the days/months will depend on your regional settings. If you're still getting no results, then check the metadata mappings and XSLT references below.

Although uncovering the correct format was pain, it was nothing compared with what followed.

What is Broken and Why?

As many have discovered, the Created By, Last Modified By, Created Date, Last Modified Date and Author properties are all affected to some degree.
The root cause for all this is down to improper mapping of crawled properties to their managed properties within Search Administration and keeps going all the way through to the XSLT for the Advanced Search and Search Core Results web parts.
The reasons behind these poor relationships become obvious shortly after you begin looking for a solution. To be frank, it also becomes obvious why most people gave up trying!

Thanks to...

Much thanks goes Anne Stenberg and her 6-part series entitled - Mystery Solved - Crawled Properties in SharePoint.
In this series Anne patently and painstakingly goes through every last property in each defined category, and providing a description for many. I'm not entirely sure where she came by all this information but it proved invaluable when it came to identifying and testing the result of many changes to come in my metadata property mapping.

Please explain!

Using a combination of Anne's tables, the U2U CAML Query Builder feature, the ever useful SharePoint Manager, and the XSLT within the search web parts - it quickly becomes obvious that it's going to take more than a packet of off-the-shelf headache tablets to get through this.
Without going into too much detail - ignorance being bliss - let's take a look at something as simple as Author.

• We have a visible Author column whose internal name is _Author.
• A hidden Created By column whose internal name is Author.
• And a managed property called Author that seems to want to hedge its bets by trying to cover all these bases as well as a few more.

But that that's just the beginning - Created By and Modified By searches will invariably return zero results and also have their fair share of possible mappings and hidden values. What the heck is Write anyway?? Apparently just another value for Modified Date...but more on that later. I'm sure anyone's who's that interested can do their own research. I won't bore everyone alse any further.

What's the fix already!?

OK, OK. Keep your propeller hat on.
After days of stuffing around, tweaking mappings, modifying web part properties and performing a full crawl each time(!) I have finally found - I think - a solution. At least, a number of searches - using Author, Created By and Last Modified By properties with the AND operator - all returned correct results.
It's also worth noting that this solution is not Office-centric and will work with any document type.

First, the Metadata

This assumes a good knowledge of Central Administration. If you require detailed steps they can be fond elsewhere.
You can use all or some of the settings shown below but the only ones that really matter are the Mappings themselves. After you've added the crawled properties, be sure to click each one and check the "Include values for this property in the search index" checkbox, otherwise it won't get added to the index! In all cases I went with the default "Inlcude values from all crawled properties mapped" option.
Also note that there are often TWO properties with exactly the same name - e.g. Office:4(Text). Picking the right one is essential and I have provided the Property Set IDs below where this is relevant.
And, remember, what follows is in no way Gospel - it's just what worked for me.
NB: Don't forget to run a Full Crawl after making these changes.

Property Name	Type	May be deleted	Use in scopes	Mappings
Author	Text	No	Yes	_Author(Text), ows__Author(Text)
Created	Date and Time	Yes	No	Office:12(Date and Time), Basic:15(Date and Time)
CreatedBy	Text	Yes	No	Office:4(Text), ows_Created_x0020_By(Text
LastModifiedTime	Date and Time	No	Yes	Basic:14(Date and Time), Basic:16(Date and Time), ows_Modified(Date and Time)
ModifiedBy	Text	Yes	Yes	Office:8(Text)

• Office:12(Date and Time) - f29f85e0-4ff9-1068-ab91-08002b27b3d9
• Basic:15(Date and Time) - b725f130-47ef-101a-a5f1-02608c9eebac
• Office:4(Text) - f29f85e0-4ff9-1068-ab91-08002b27b3d9
• Office:8(Text) - f29f85e0-4ff9-1068-ab91-08002b27b3d9

Advanced Search XSLT

The following go in PropertyDefs. There are many default values here, I'm just providing the full block. You'll then need to add the same 'Name' references to each ResultType in the order you prefer.
<propertydef name="Author" datatype="text" displayname="Author">
<propertydef name="Size" datatype="integer" displayname="Size">
<propertydef name="Keywords" datatype="text" displayname="Keywords">
<propertydef name="CreatedBy" datatype="text" displayname="Created By">
<propertydef name="Created" datatype="datetime" displayname="Created Date">
<propertydef name="ModifiedBy" datatype="text" displayname="Last Modified By">
<propertydef name="LastModifiedTime" datatype="datetime" displayname="Last Modified Date">

Search Core Results XSLT

Unless you're trying to provide custom results using some of the values described above you won't need to make any changes here. Quite frankly it's a little daunting but great things can be done - such as displaying Size, Author and a custom link to open the containing folder for each result. I'll probably leave this for another post as it's a topic in itself.

In conclusion...

So, hopefully, if you've done everything right and performed a full crawl, you should now be able to search using one or all of the properties we've discussed here.
One thing you may find still doesn't work is the "Does not equal" operator. You might also that it's not described anywhere in the web part code but is managed by a separate core JavaScript file. I'm just not willing to look into this right now - or the reasons why "Contains" and "Doesn't contain" aren't available for partial search term querying. If anyone else has any ideas - performance notwithstanding - feel free to drop me a line.
I look forward to any further insight and feedback others might have and hope that all my hard work isn't undone with the next upgrade!

Deleting unused SharePoint Content Types

It happens to every site collection admin at some stage. For whatever reason you're required to replace a defunct content type in your document library with a new one. Adding t he new one is easy. But then you try to delete the old one and receive the terrifically informative "Content Type is still in use" error.

You've tried everything:

• Updated any files you could find with the new content type.
• Checked every file twice to make sure you didn't miss any.
• Managed Checked Out files to locate all those tricky 'hidden' docs that haven't been checked in yet and only exist in some funky temp/draft state! (You'll either need to take ownership of these as site collection admin, or email a list to the original authors to get them to check them in or delete then. I can't believe there is no option to delete these en masse!)
• Run custom CAML queries to make REALLY sure you didn't miss any (using the fabulous U2U SahrePoint CAML Query feature - www.u2u.be/res/Software.aspx.
• Written a console app just to make ABSOLUTELY sure you didn't miss any.
• Emptied the recycling bin - both of them!

But still the error persists.

Just when I thought I'd exhausted all options it occurred to me that perhaps versions were the culprit. Looking at the version history for a few suspect documents confirmed that they were.

Running the following SQL query will find them all.

SELECT *
FROM AllUserData
WHERE (tp_DirName LIKE '%sitename/Shared Documents%')
AND ((tp_ContentType = 'myDodgyCT'))
ORDER BY tp_DirName

This should be self-explanatory but tp_dirName is just using a relative path from the domain to the library. And ContentType is the explicit name of the content type to search on.

From there I just exported the results to Excel, filtered out duplicates and was left with a workable list of documents. Simply publish a final version of each (if required), then go to the Version History and delete all previous versions.

You could do all this in one step by turning off versioning for the library and then turning it on again. But this would delete versions for ALL documents. A good trick to remember when you're site quota is reached.

Good luck!

Import Publishing Sub Site as Site Collection

Any farm administrator will eventually feel the need to restructure when they watch in horrid fascination as their once insignficant sub sites grow to monster sites. Current MOSS limitations such as the 15GB limit for the stsadm import/export operations mean that unless you're using a terrifically expesnive 3rd party migration app you need to act fast before things get out of hand

The obvious solution would be to export the site(s) in question and then reimport them to an empty site collection. Yes, that would seem obvious, but it's not quite as straightforward as you might assume. What really compounds the matter is if you happen to have enabled the publishing feature on your source site.

After many days of disappointment and incoherent error messages I finally returned to Gary Lapointe's brilliant blog - SharePoint Automation. Gary has developed dozens of custom stsadm commands in order to fill the cavernous and aching void left by the default offerings. One thread in particular, Subsite to Site Collection, deals with this pesky issue and also includes his own hair-pulling frustration with the out-of-the-box limitations to achieve this. BTW, the solutions he provides are not strictly supported - but they work where nothing else will and are very easy to use.

In my case I simply deleted every failed test and started from scracth using the gl-convertsubsitetositecollection operation. All I had to do was provide the source site and destination URL and the command did the rest.

stsadm -o gl-convertsubsitetositecollection -sourceurl http://mydomain.com/source -targeturl http://mydomain/destination -nofilecompression -owneremail me@home.com -ownerlogin "mydomain\me" -nositetemplate 

This exports the source site, creates the destination site collection and site (based on source site template), and activates any features. The source site was a Team Site with publsihing and several custom features enabled. The only thing I did was create the managed path but this command even has a flag for that!

The only thing it doesn't appear to do (yet) is allow you to specify a new content database. I've already put my request in for this and hope to see it soon. If this helps someone, please thank Gary - not me! :)

Renaming SharePoint Site Collections (the Secret of Managed Paths)

Everyone knows, or at least should know, that SharPoint sites (formally referred to as Webs) can be easily renamed with either stsadm -o renameweb or through the GUI.

But what about renaming site collections?

Well, the short answer to that question is no. Site collections cannot be renamed, at least not those on managed paths.

e.g. domain1.com/sites/sitecollection_name

Hosted site collections using host headers can be renamed using the stsadm -o renamesite command but it is limited to hosted sites only.

e.g. domain1.com -> domain2.com

Yes, another massive limitation of the current SharePoint version. But the real culprit - and what makes this all the more confounding - is managed paths. If you've never had any real experience with these little beasties then I envy you. But get ready to put your hard hat on because we're going in.

You can create a managed path for a site collection using one of two options - "Explicit exclusion" or "Wildcard inclusion" - but not both! The limitations of this will become obvious soon.

Let's say you want the following structure for your site collections:

domain1.com/groups domain1.com/groups/team1

- where both groups and team 1 are site collections.

This cannot be achieved conventionally. The reason is because to create a site collection at the managed path of /groups it must be set to "Explicit exclusion" - meaning you cannot create site collections below this path.

And if you create the /groups path to use "Wildcard inclusion" then you can only create sites below this path. Make sense?

i.e. domain1/groups/my_sitecollection

This means that site collections can have no distinct hierarchy. Not that this matters given that they are treated as separate entities and current version web parts are unable to query across site collections anwyay!

So is there a workaround to both these limitations? I'm glad you asked. ;)

The following refers to explicitly renaming a single site collection which uses "Explicit exclusion" only. Depending on your needs you can use "Wildcard inclusion" for ensuing site collections.

To rename a site collection you need to:

1. Backup your exsiting site collection using stsadm -o backup.
2. Create a managed path of newpath "Explicit eclusion" using the new name you want.
3. Restore the site collection to the new URL using stsadm -o restore.

To create a virtual URL hierarchy you can:

1. Create a managed path of newpath/newsite "Explicit exclusion".
2. Create a new site collection using this path.

Yes, it's true! MOSS will allow you to create a managed path in this form, despite the explicit exclusion on the original /newpath path.

Great isn't it? Now, whether this is intended or flawed behaviour, or a good idea or not is another matter entirely. ;)

Either way - good luck! And may version 14 resolve these and many other issues we're plagued with.

Page Settings: More Bad Paths - MOSS Restore Issues continued

It happens when you restore WCM (publishing) sites to another location. The Page Layout source URL is not updated to reflect the new location.

There are several painful options I tried before finally coming across this solution:

http://stsadm.blogspot.com/2007/08/fix-publishing-pages-page-layout-url.html

There’s also a standalone executable and source code available here (the script often requires tweaking and recompiling to work, so having the source is a terrific help):

http://blog.thekid.me.uk/archive/2007/08/20/fixing-page-layout-urls-after-importing-a-publishing-site-in-sharepoint.aspx

One or the other usually provides the fix required.

UPDATE

if you're still having problems, there are a few things to watch for.

• Firstly, checkout-out pages (and quite possibly drafts) will not be affected by these commands, so make sure you approve and check in all pages first.
• If you're still getting the mysterious 'Unknown User' error on some pages chances are that the person who originally created your master/layout pages (or even site template!) no longer exists in the current site/collection. In this case you will have to temporarily add that user back, run the command again, and hopefully all will be good.

Good luck and please add a comment if you've found anything else or are still having problems.

SharePoint 2007 - Disaster. Recovery?

Nowhere on any Microsoft website will you find a clear and complete guide on what is obviously one of the most fundamental and critical aspects of any system – disaster recovery.

In order to save others much pain and suffering I have outlined my lessons learnt below:

• It really, really helps if the environments you are working between match as closely as possible.
  • This includes - but is by no means limited to - ensuring that the root site is of the same type in both locations.
• Farm-level backup does not include the following:
  • Items in the Global Assembly Cache (any custom DLLs)
  • Web config files
  • IIS web app settings
  • Customisations and some elements in the 12 hive

These items should be backed up separately and applied to the new location first.

The following is an approach to migrating a farm (single- or multi-server) from one environment to the other. It is provided here free of charge and should be used as a guide only. I have attempted to list and present these items in some form of chronological order but no doubt everyone's needs and tastes here will differ.

1. Web Apps
   • Recreate these on the destination server(s) first. You don’t want to copy anything here because these would usually be using different host headers, URLs etc.
2. Custom DLLs in GAC
   • These can just be copied across. I keep a separate folder of all custom DLLs on another box (dev) and just copy these to C:\Windows\assembly on the target server(s) – all WFE (web front ends).
   • These also need to be registered as Safe Controls in your web.config.
   • You can usually determine what your custom items are from the web.config files.
3. 12 Hive
   • It’s quite possible that developers have made changes here. Usually they add any custom logos, images and page copies in here.
   • The good news is that the 12 hive from one default installation to the next is identical.
   • Modifications will usually have been restricted to the TEMPLATE folder so you can start by just copying that.
4. Web.config(s)
   • You will have a web.config file for each web app on each WFE. Ignore the confusing existence of web.config files in the 12 hive CONFIG directory. These don’t appear to get used at all.
   • What no one bothers telling you is WHICH web.config file(s) need to be changed! There are dozens and it’s bluddy confusing. The ones you actually need to affect changes in are again on the WFEs only, within the relevant IIS folder – e.g. C:\Inetpub\wwwroot\wss\VirtualDirectories\www.yourwebapp.com80
   • Easiest method is to copy an existing one to the new location. Make a copy of the destination file first.
   • If you have a diff tool (there are dozens but I like the one that ships free with TortoiseSVN) you can compare them to see the differences for yourself.

There you have it. Four fairly simple steps (once you know what you need!).

Once all that is in place you should be able to do a Farm or site collection restore on the server and hopefully everything will just work for you. In our case we also had to reconfigure the following because MOSS seems to use hard-coded site roots in some areas:

• Content Query Web Parts (and others based on it) – sometimes but not in all cases these need to be reconfigured
• Send To links – this field only accepts an absolute URL so any values you’ve applied here need updating
• Publishing Layout Pages – inheritance paths for these sometimes still point to the old server.
  • You can check this by trying to edit the Page Settings for any publishing page.
  • If you get an error then you are affected by this bug. It can be resolved using one of two methods described in another blog entitled Page Settings: Bad Paths - MOSS Restore Issues continued.

Once you’ve got the new environment up and running then using Content Deployment seems to be the easiest way to get new content across. The main issue I ran into here was that we wanted to use the same GUIDs but Content Deployment doesn’t like this.

Seems pretty strange when you consider that any custom solution, workflow, site definition etc., all use GUIDs natively to determine what content is being referenced!

Managing those slippery MOSS Web part Pages

What bugs me most about SharePoint is that there is almost always an application ASPX page that can provide exactly the functionality you're looking for.

Take Web Part pages for example, they have their own maintenance page but the only time you'll ever get to see the link for it is when one or more web parts on a page is causing problems. When this happens you get the usual generic error message we've all come to know and love, and a link to the holy Web Part Maintenance Page.

But what about when there is no error and you just want to manage the web parts for a given page - Close, Reset or Delete one or more? Fear not. Like many hard learnt solutions in MOSS, simply knowing what ASPX page to call (and with what parameters).

Simply add /_layouts/spcontnt.aspx?&url=[url] after your domain/server root with the appropriate path to the page you want to manage.

e.g. http://mysite/_layouts/spcontnt.aspx?&url=/subsite/Pages/mypage.aspx

Happy managing!