SharePoint 2007 Utility #3 - SSPC - Utility to move Managed properties and Crawled properties.

Posted on 1/15/2007 @ 5:51 PM in #SharePoint by | Feedback | 9554 views

Recently, Angus Logan from Microsoft Australia pinged me with a rather interesting SharePoint 2007 related problem that Tourism Western Australia (www.westernaustralia.com) was running into. See, they have SharePoint 2007 on their intranet. In their SharePoint 2007 installation, they had search configured to search the Australian Tourism Data warehouse – a HUGE database using BDC (Business Data Catalog) (read more). So far so good – until of course they need to create an Extranet, and now they need to move all the Managed properties and Crawl properties over the Extranet SSP (Shared Service Provider).
This scenario is actually fairly common. Say for instance, you have a serious BDC catalog – configured on a development server. And now you wish to move it over to a production server. You’d probably run into the same situation. While you could backup/restore your DB, you know that approach suffers from the “copy way more than required” syndrome.
Now if you’ve been using SharePoint 2007 a bit, you’d notice that moving all this stuff over manually is a royal pain in the behind. So with the help of Angus, I wrote a neat little command line utility that lets you do four things.

- Accept relevant inputs from the user
- Export Managed Properties
- Export Crawl Properties and relevant categories
- Import Managed properties and map relevant crawl properties.


Now, I’ve already applied for a workspace at codeplex, so hopefully they will allow me to share the code and the executable with you guys. But in here I’m gonna explain how did I achieve the above 4.


#1 – Accepting relevant inputs from the user.

Accepting inputs is always interesting. What I did was, I created a strongly typed business object called “ProgramInputs” as shown below:

With this setup, accepting inputs and working on them becomes rather easy as shown below:

if (programInputs.IsValid)
{
    switch (programInputs.CurrentOperation)
    {
        case ProgramOperation.ManagedProperties:
            ManagedProperties.HelperClass.ExportXML(programInputs);
            break;
        case ProgramOperation.CrawlProperties:
            CrawlProperties.HelperClass.ExportXML(programInputs);
            break;
        case ProgramOperation.CreateManagedPropertyAndMapCrawlProperty: // abbreviated to MAMC
            MAMC.HelperClass.ImportXML(programInputs);
            break;
        case ProgramOperation.Invalid: // ignore
            break;
        default:                       // ignore, something is horribly wrong.
            break;
    }
}
else if (args.Length != 0)
{
    if (args[0] != "-help")
    {
        Trace.WriteLine("Invalid inputs specified. You must specify atleast a URL, Filename and requested operation", true);
        Trace.WriteLine("For help, type \"sspc -help\"", true);
    }
}


As you can see from the above, the enumeration tells me “What is the user trying to do”. The inputs are self validating, so it keeps my main logic really clean.
Let me next dive into the 3 possible valid inputs and dissect how I made those work.


#2: Exporting Managed Properties

Exporting managed properties is horribly simple. Here is the code –

public static void ExportXML(ProgramInputs inputs)
{
    Schema _schema = new Schema(SearchContext.GetContext(new SPSite(inputs.SSPURL)));

    BO.ManagedProperties propertyCollection = new BO.ManagedProperties();
    foreach (ManagedProperty theProperty in _schema.AllManagedProperties)
    {
        propertyCollection.AddProperty(new BO.ManagedProperty(theProperty));
    }

    propertyCollection.Save(inputs.FileName);
}


Again, good architecture keeps me focused on my main problem here. BO stands for “Business Object”. Essentially, the BO.ManagedProperties is a business object that inherits from XmlDocument, and with some clever Xml Serialization, it is able to achieve conversion of BO from XML using just one line of code –
propertyCollection.Save(inputs.FileName);
Now, some may argue that XmlSerialization is bad. I’ll probably stand in favor with you. But the point here isn’t a philosophical discussion of XmlSerialization being clunky – it’s about manageable code. Now I can go change the definition of my business object, and the serialization; just WORKS. No code change required except focusing on what I was trying to get done – changing the structure of my object and what is persisted. That’s about it.

The exported XML looks like this –

<ManagedProperties>
<ManagedProperty ManagedType= .. Name= .. />
<ManagedProperty .../>
...
</ManagedProperties>


#3 Export Crawl Properties and relevant categories:

Similar to the above, Crawl properties and relevant categories can be exported as follows –

public static void ExportXML(ProgramInputs inputs)
{
    Schema _schema = new Schema(SearchContext.GetContext(new SPSite(inputs.SSPURL)));

    BO.Categories categoriesCollection = new BO.Categories();

    foreach (Category category in _schema.AllCategories)
    {
        categoriesCollection.AddCategory(new BO.Category(category));
    }

    categoriesCollection.Save(inputs.FileName);
}

As you can see – again the code is shocking simple, and terse. Sure the categoriesCollection is a tad bit more complex – it’s got a double hierarchy going on. But since I used the XmlDocument based business object and some clever XmlSerialization, my code doesn’t get complex as the structure of my document gets complex. It lets me focus on what I am really trying to get done. The XML looks like this –


<Categories>
<Category Name=..>
<CrawledProperties>
<CrawledProperty VariantType=.. PropSet=.. Name=.. />
<CrawledProperty... />
</CrawledProperties>
</Category>
</Categories>


#4 Import Managed properties and map relevant crawl properties.

Finally, let us talk about importing managed properties and mapping relevant crawl properties. This is the case where we are going from XML to SharePoint. Just like above, I created a business object based on XmlDocument, and I am deserializing an input XML structure into a strongly typed object. This can be seen as below:

XmlSerializer xser = new XmlSerializer(typeof(BO.ManagedProperties));
BO.ManagedProperties workingData = null;
using (FileStream fs = new FileStream(inputs.FileName, FileMode.Open))
{
    workingData = xser.Deserialize(fs) as BO.ManagedProperties;
    fs.Close();
}

The input XML looks a bit like this:

<?xml version=""1.0"" encoding=""utf-8"" ?>
<ManagedProperties>
<ManagedProperty Name=""AAA123123"" Type=""Text"">
<CrawledProperties>
<CrawledProperty Category=""Business Data"" Name=""BDCAWDW.Reseller.FirstOrderYear"" VariantType=""Integer"" />
<CrawledProperty Category=""Business Data"" Name=""BDCAWDW.Reseller.Phone"" VariantType=""Text"" />
</CrawledProperties>
</ManagedProperty>
</ManagedProperties>

With the data imported, I can then start importing Managed Properties and start mapping the relevant crawl properties.
To begin with, get a hold of the schema.

Schema schema = new Schema(SearchContext.GetContext(new SPSite(inputs.SSPURL)));


Then start iterating over all the managed properties and begin importing them.

foreach (BO.ManagedProperty inputManagedProperty in workingData)
{
    // Add the Datatype handler
    ManagedProperty managedProperty = 
        schema.AllManagedProperties.Create(
            inputManagedProperty.Name, 
            GetManagedDataType(inputManagedProperty.Type)
            );

    // Get the mappings
    MappingCollection mappingCollection = managedProperty.GetMappings();

    foreach (BO.CrawledProperty inputCrawledProperty in inputManagedProperty.CrawledProperties)
    {
        string propID = GetPropSetFromCrawlProperty(inputCrawledProperty.Name, schema);
        if (propID != "")
        {
            Mapping mapping = new Mapping(
                new Guid(propID), 
                inputCrawledProperty.Name, 
                GetIntDatType(inputCrawledProperty.VariantType), 
                managedProperty.PID);

            mappingCollection.Add(mapping);
            Console.WriteLine("Added Mapping to collection.");
        }
        else
        {
            Trace.WriteLine("Didn't find the Prop ID for " + inputCrawledProperty.Name, true);
        }
    }

    // Set the Mappings
    managedProperty.SetMappings(mappingCollection);
}


Yeah that’s it. No seriously, that’s all there is to it.


Now I can -

Export Managed properties using the following command line:
     sspc -o ManagedProperties -file output.xml -url http://sahilvirtual:7000/ssp/admin
Export Crawl properties using the following command line:
    sspc -o CrawlProperties -file output.xml -url http://sahilvirtual:7000/ssp/admin
Import Managed properties and map relevant crawl properties using the following command line:
   sspc -o CreateManagedPropertyAndMapCrawlProperty -file input.xml -url http://sahilvirtual:7000/ssp/admin
And of course, if you’re confused about how to use this tool, just type:
  sspc -help
or just type
  sspc
.. typing for help gives you the following online help:

=================================================
Shared Services Provider Property Creation - Help
=================================================
Usage: sspc -o <operationname> <parameters>
*********** Possible Operations are: ***********
- ManagedProperties - to export ManagedProperties (Sharepoint to XML)
- CrawlProperties - to export CrawlProperties     (Sharepoint to XML)
- CreateManagedPropertyAndMapCrawlProperty - to Create Managed Properties and Mapped Crawl Properties (XML to Sharepoint)
*********** Required Parameters are: ***********
-url
http://yourserver/ssp/admin
-file InputOrOutput.xml
********** XML File structures are: ***********
For ManagedProperties:
<ManagedProperties>
  <ManagedProperty ManagedType= .. Name= .. />
  <ManagedProperty .../>
  ...
</ManagedProperties>
For CrawlProperties:
<Categories>
  <Category Name=..>
    <CrawledProperties>
      <CrawledProperty VariantType=.. PropSet=.. Name=.. />
      <CrawledProperty... />
    </CrawledProperties>
  </Category>
</Categories>

For CreateManagedPropertyAndMapCrawlProperty:

<?xml version=""1.0"" encoding=""utf-8"" ?>
<ManagedProperties>
  <ManagedProperty Name=""AAA123123"" Type=""Text"">
     <CrawledProperties>
         <CrawledProperty Category=""Business Data"" Name=""BDCAWDW.Reseller.FirstOrderYear"" VariantType=""Integer"" />
         <CrawledProperty Category=""Business Data"" Name=""BDCAWDW.Reseller.Phone"" VariantType=""Text"" />
     </CrawledProperties>
  </ManagedProperty>
</ManagedProperties>

*********** Sample usage: ***********

sspc -o ManagedProperties -file output.xml -url http://sahilvirtual:7000/ssp/admin
sspc -o CrawlProperties -file output.xml -url http://sahilvirtual:7000/ssp/admin

sspc -o CreateManagedPropertyAndMapCrawlProperty -file input.xml -url http://sahilvirtual:7000/ssp/admin

You can find the full code here -

http://www.codeplex.com/SSSPPC/

Sound off but keep it civil:

Older comments..


On 7/7/2007 6:32:58 AM Dragan Panjkov said ..
Sahil, what about posting build f that utility to codeplex?


On 7/10/2007 5:04:28 AM Søren Nielsen said ..
Hi Sahil

Excellent tool/post!!

I've taken the liberty to improve and extend it quite a bit. Please find my updated version here: http://soerennielsen.wordpress.com/2007/07/10/tool-for-deployment-of-ssp-search-settings/.

I believe that my code changes (they are extensive) should be merged with the code on the codeplex site. Let me know your thoughts on this.


On 7/10/2007 8:36:08 AM Sahil Malik said ..
Soren -

Excellent.


I just sent you an email, please be in touch :)

Sahil


On 1/30/2008 8:47:31 AM Prashanth said ..
Sahil,

Can we export crawl rules using this tool ? I could export Content Sources, Managed Properties and Scopes but could not find an option for crawl rules.


Thanks for a great tool. You made my work much more easier.

Prashanth


On 12/24/2008 6:46:07 AM Rao said ..
Hi Sahil,

I am using SSPC.exe for exporting / importing crawl propertites. At the time of Exporting, I can easily export to OutPut.xml file but if I am importing this into another SSP then it is throwing error:

Importing CrawledProperties


Creating crawled property:SAPDocuments:DocumentNumber, in category:SapDocuments


** ERR:Catastrophic Error Occurred.


** ERR:Error details as follows.

** ERR:System.InvalidOperationException: The propset fb031e8a-7c54-455d-8220-278


56c32cf4e is not contained within this category SapDocuments


at Microsoft.Office.Server.Search.Administration.Category.CreateCrawledProper


ty(String propertyName, Boolean isNameEnum, Guid propset, Int32 variantType)


at Winsmarts.SSPC.CrawlProperties.HelperClass.ImportXML(ProgramInputs inputs)

One more thing is that this crawl property is index at the time of exporting, I would like to import properties with included in index checkbox checked.

Please help me.

Regards


Rao


On 10/12/2009 1:18:00 PM Mike said ..
Hi, I am getting a similar error to Rao. Can you provide any guidance?

Cheers


Mike


On 10/6/2011 5:06:39 AM Mark Stokes said ..
Great tool,

Thanks guys. I am using this to migrate properties from a SharePoint 2007 environment (using your tool) and then using the XML file as a configuration file to a PowerShell cmdlet that I have developed to import them into SharePoint 2010.

The ony frustrating bit (that I am going to fix in code myself prior to use) is the inclusion of the Crawled Property Category in the Managed Properties export XML.

I know this is included in the crawled properties export XML, but I don't want to import them all... just the ones that are required for my managed properties. So, rather than importing them all, or cross referencing the XML files, it would be nice to have the category as a attribute of the managed properties export.

Great tool though.

Mark Stokes


SharePoint Studio Ltd.