Controlling request retension periods with Initial loading in FIM 2010

In the FIM 2010 deployments involving the FIM Portal we have a period of initial loading of data from the various data sources we are integrating and we often have to load in excess of 150000+ objects into the Portal. This can be a bit tricky as you need to try and balance speed with the amount of sets, MPR's, workflows and sync rules that are applied to the data when it enters the portal.

Within the portal every change made to any object will be processed via the FIM Portal's workflow engine and these requests can very quickly become unmanageable during the initial load process.

We recently had a project where we had to import 70,000+ objects into the FIM Portal and the initial load generated 640,000 requests even though we limited the MPR's and workflows that applied during initial loading. This firstly makes it hard to find a specific request, and also slows the performance of the web components down when performing a full search of requests (even causing time out issues in some cases).

In order to limit this from occurring we generally reduce the time that FIM Portal keeps completed request in the Portal before archiving them. The default retention period that requests are retained is set to 30 days. This means that if you generate 200,000 request this week, 700,000 the next week you would have to live with an additional 900,000 requests until the request reach an age of 30 days. You can see how this can become a problem during initial loading as it would:

1. Slow down portal request searches for this time
2. Put an additional storage load on you SQL database ( we had to extend the storage to over 2TB at one client)
3. Put a strain of the index file ( we had to rebuild the index file numerous times)

We usually reduce the retention period to 2-5 days during initial loading, as this gives us the ability to still deal with requests in a easy manner, but have the flexibility to to get rid excessive completed requests quickly.
This should be done prior to starting the initial load, as doing the change after the fact does not change the retention age on completed requests, but only those generated after the change. It should also be noted that these requests are archived and not deleted, but fall outside the scope of search-able requests in the FIM Portal, and can be retrieved via SQL queries or reporting tools.

A well planned initial load can save you many headaches and will contribute to a successful deployment.
For more information on FIM best practises navigate to the following link:

http://technet.microsoft.com/en-us/library/ff608274(WS.10).aspx