A look at why using Multi-value tables are not always the best approach

I recently had a case at a client where they were unable to get a full import of their user information from Oracle into FIM 2010 with the full import failing after 36 hours without importing all the necessary information. The institution has a 250000+ user base and complex infrastructure.  The Oracle MA utilized a multi-value table for allocation of some RBAC components within their AD.

The Oracle MA had the following tables configured:

1.       Main User information table (250000+ items, 18 fields, 790mb)

2.       Multi-value table (250000+ items, 4 fields,  690mb)

3.       Delta table

I followed the usual troubleshooting steps ensuring that:

1.       The Oracle tables were correctly built and indexed.

2.       The Oracle client version was correct.

3.       The FIM Server was installed, configured and patched correctly.

4.       All possible bottlenecks were identified and eliminated between servers in question.

With all these resolved, we did not get a significant increase in speed, so I did a test of importing the user information excluding the multi-value table from the configuration and the import completed in less than 40 minutes. It was in fact the multi-value table causing the import performance degradation. Although it was correctly indexed, contained only 4 fields, it was almost the same size as the main user table.

We had three options:

1.       Reduce the complexity of the data in the main and multi-value tables (was not possible in this instance).

2.       Consolidate the data into a single view and pass the multi-value processing to FIM rules extension.

3.       Write a custom management agent to incorporate multi-values directly (Due to the work involved in achieving this it was not really an option).

I then asked the client to consolidate the data into a single view writing the multi-values delimited to a single value field. I was then able to still do the import in less than 40 minutes and use a rules extension to extract the multi-values from the single value table to an array which I then flowed to the relevant multi-value attributes in the metaverse. The total process of full import and sync then completed in less than 3 hours.

I am a big advocate of using multi-value tables where appropriate, but there certain scenarios where using multi-value tables are just not feasible and other approaches may be needed.

When evaluating whether or not to use a multi-value table it is important remember that the size and data complexity if the table will affect performance. My own rule of thumb is once the multi-value table exceeds 33-50% (based on complexity) of size of the main table there will be a performance impact on the import of data into the connector space.

FIM 2010 R2 RTM now available on MSDN

As of the 1st of June 2012 FIM 2010 R2 RTM has been released on MSDN And TechNet. What is very notable is that BHOLD is also available.

The media can be downloaded via the following links below:
FIM 2010 R2 RTM: https://msdn.microsoft.com/en-US/subscriptions/securedownloads/#FileId=49037
BHOLD: https://msdn.microsoft.com/en-US/subscriptions/securedownloads/#FileId=49036

For additional information on FIM 2010 R2 please refer to the release  notes

Controlling request retension periods with Initial loading in FIM 2010

In the FIM 2010 deployments involving the FIM Portal we have a period of initial loading of data from the various data sources we are integrating and we often have to load in excess of 150000+ objects into the Portal. This can be a bit tricky as you need to try and balance speed with the amount of sets, MPR's, workflows and sync rules that are applied to the data when it enters the portal.

Within the portal every change made to any object will be processed via the FIM Portal's workflow engine and these requests can very quickly become unmanageable during the initial load process.

We recently had a project where we had to import 70,000+ objects into the FIM Portal and the initial load generated 640,000 requests even though we limited the MPR's and workflows that applied during initial loading. This firstly makes it hard to find a specific request, and also slows the performance of the web components down when performing a full search of requests (even causing time out issues in some cases).

In order to limit this from occurring we generally reduce the time that FIM Portal keeps completed request in the Portal before archiving them. The default retention period that requests are retained is set to 30 days. This means that if you generate 200,000 request this week, 700,000 the next week you would have to live with an additional 900,000 requests until the request reach an age of 30 days. You can see how this can become a problem during initial loading as it would:

1. Slow down portal request searches for this time
2. Put an additional storage load on you SQL database ( we had to extend the storage to over 2TB at one client)
3. Put a strain of the index file ( we had to rebuild the index file numerous times)

We usually reduce the retention period to 2-5 days during initial loading, as this gives us the ability to still deal with requests in a easy manner, but have the flexibility to to get rid excessive completed requests quickly.
This should be done prior to starting the initial load, as doing the change after the fact does not change the retention age on completed requests, but only those generated after the change. It should also be noted that these requests are archived and not deleted, but fall outside the scope of search-able requests in the FIM Portal, and can be retrieved via SQL queries or reporting tools.

A well planned initial load can save you many headaches and will contribute to a successful deployment.
For more information on FIM best practises navigate to the following link:

http://technet.microsoft.com/en-us/library/ff608274(WS.10).aspx

Populating a RCDC dropdownlist with object masterdata information in FIM 2010 Portal

I have been playing around with the idea of populating master data (i.e. Department, Location,etc) into the FIM Portal and having this data presented in a dropdownlist. Presenting this information in a identitypicker, hyperlink, or listview is documented will enough so I will focus solely on the dropdownlist in this post.

Firstly I built a class called Department, added the relevant Attributes and binded everything up nicely. Next, I imported the departments into the Portal and tried to populate the dropdownlist. Here i ran into my first snag, as I could not populate this using the normal datasources provided in the RCDC datasources as per the Microsoft RCDC reference guide http://technet.microsoft.com/en-us/library/ee534918(WS.10).aspx#BKMK_DataSources.

I had a look at some of the posts on the FIM Forums, and saw an undocumented data source mentioned in the  http://social.technet.microsoft.com/Forums/en-HK/ilm2/thread/ae6e6eb4-e84b-44a7-8fd2-a9538cee4670 forum. I did some investigation and found that this data source was able to search objects and could easily populate my dropdownlist by creating a data source  in my RCDC configuration as below:

<my:ObjectDataSource my:TypeName="PrimaryResourceObjectDataSource" my:Name="object" my:Parameters=""/> 
<my:ObjectDataSource my:TypeName="PrimaryResourceDeltaDataSource" my:Name="delta"/>
<my:ObjectDataSource my:TypeName="PrimaryResourceRightsDataSource" my:Name="rights"/>
<my:ObjectDataSource my:TypeName="SchemaDataSource" my:Name="schema"/>
<my:ObjectDataSource my:TypeName="DomainDataSource" my:Name="domain"/>
<my:ObjectDataSource my:TypeName="TimeZoneDataSource" my:Name="timezone"/>
<my:ObjectDataSource my:TypeName="UocSearchDataSource" my:Name="search"/>

Now i was able to my create my dropdownlist object bound to department as below:

<my:Control my:Name="department" my:TypeName="UocDropDownList" my:Caption="{Binding Source=schema, Path=department.DisplayName}" my:Description="{Binding Source=schema, Path=department.Description}"  my:RightsLevel="{Binding Source=rights, Path=department}">
<my:Properties>
<my:Property my:Name="Required" my:Value="{Binding Source=schema, Path=department.Required}"/>
<my:Property my:Name="Columns" my:Value="40"/>
<my:Property my:Name="ItemSource" my:Value="{Binding Source=search, Path=Department}"/>
<my:Property my:Name="SelectedValue" my:Value="{Binding Source=object, Path=department, Mode=TwoWay}"/>
</my:Properties>
</my:Control>

There was however only one problem which I still had to resolve; being that although the values in the dropdownlist items were displayed as text, selecting an item stored the reference value of the selected item in the department attribute value.I looked at various options, and finally found the solution.

Enter the reference binding! Thanks to Almero (PuttyQ) for pointing me to a post by UNIFYBob http://social.technet.microsoft.com/wiki/contents/articles/3945.aspx which gave me the final pieces to the puzzle. By creating a reference (departmentRef in this case) and binding this value to the person, I was able to pass the reference correctly. So I amended my RCDC control to look a follows:

<my:Control my:Name="departmentRef" my:TypeName="UocDropDownList" my:Caption="{Binding Source=schema, Path=departmentRef.DisplayName}" my:Description="{Binding Source=schema, Path=departmentRef.Description}" my:RightsLevel="{Binding Source=rights, Path=departmentRef}">
<my:Properties>
<my:Property my:Name="Required" my:Value="{Binding Source=schema, Path=departmentRef.Required}"/>
<my:Property my:Name="Columns" my:Value="40"/>
<my:Property my:Name="ItemSource" my:Value="{Binding Source=search, Path=Department}"/>
<my:Property my:Name="SelectedValue" my:Value="{Binding Source=object, Path=departmentRef, Mode=TwoWay}"/>
</my:Properties>
</my:Control>

Now I was able to build some Portal or SyncEngine rules to push this information to my connected data sources.

"The breakpoint will not currently be hit. No symbols have been loaded for this document." when debugging FIM 2010 Code in VS2008/2010

I have been running into situations where I tried to debug some FIM MA and MV extension code in VS2008/2010 and hit the "The breakpoint will not currently be hit. No symbols have been loaded for this document." problem even though I was:

• Correctly attaching to the "miiserver.exe" process

• Setting breakpoint that would be hit
• Manually forcing a Debugger.Launch()
• Not running the code in a separate process
• Running VS2008/2010 in Administrator context.

When i started looking at the problem more closely I remembered a similar problem with MIIS pre- SP1 that occurred once framework 2.0 was installed and the MIIServer.exe.config was not forces to run in Framework 1.1. So I had two choices, either force the framework or change my compiling framework to 3.0-3.5 which is natively supported by FIM 2010.

So my initial VS advanced compiling configuration looked as follows:

Once I changed the settings to the settings below, and recompiled the code I was able to debug my code with no issues.

"The Forefront Identity Manager Service has not started yet." errors when trying to do a password reset you via the FIM 2010 Password Portal

When trying to do a password reset from the FIM Password Portal you get the following error: "The Forefront Identity Manager Service has not started yet."

This error is mostly caused by the Forefront Identity Manager Password Reset Client Service (FIMPasswordReset) service not being started. This can be easily remedied by starting the service. However, if you have verified that the service is running on the workstation you are trying to do the reset from, there could be a problem with the IE zone and protected mode setting for the Local Intranet Zone.

Please note that you need to make sure that the site is running under a "Local Intranet" zone and that Protected mode is switched OFF.

A configuration as shown below will cause the error as the ActiveX component cannot check the service status if incorrectly configured, and generates the error in question.

Applying the defaults zone settings to the Local Intranet Zone should allow the components to run as expected

Error: The WinRM client received an HTTP status code of 456 from the remote WS-Management service

When connect to an Outlook Live Powershell session you get:

[ps.outlook.com] Connecting to remote server failed with the following error message : The WinRM client received an HTTP status code of 456 f
rom the remote WS-Management service. For more information, see the about_Remote_Troubleshooting Help topic.
    + CategoryInfo          : OpenError: (System.Manageme....RemoteRunspace:RemoteRunspace) [], PSRemotingTransportException
    + FullyQualifiedErrorId : PSSessionOpenFailed
Import-PSSession : Cannot validate argument on parameter 'Session'. The argument is null. Supply a non-null argument and try the command again.
At line:3 char:17
+ Import-PSSession <<<<  $Session
    + CategoryInfo          : InvalidData: (:) [Import-PSSession], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.ImportPSSessionCommand

or

When trying to run the Outlook Live MA may get an stopped-extensible-extension-error with the following stack trace:

"Microsoft.MetadirectoryServices.ExtensibleExtensionException: Connecting to remote server failed with the following error message : The WinRM client received an HTTP status code of 456 from the remote WS-Management service. For more information, see the about_Remote_Troubleshooting Help topic.

 at Microsoft.Exchange.XmaConnector.PSDataProvider.ReportError(Exception e, ScorecardCounter scorecard)
 at Microsoft.Exchange.XmaConnector.PSDataProvider.InvokeCmdlet(PSCommand cmd)
 at Microsoft.Exchange.XmaConnector.PSDataProvider.ReportScorecard()
 at Microsoft.Exchange.XmaConnector.XmaExportExLabs.ReportScorecard()
 at Microsoft.Exchange.XmaConnector.MAExtension.IlmMAExtension.EndExport()
Microsoft Identity Integration Server 3.3.1139.2"

This error occurs when the account that you are connecting with is blocked for sign-in as can be seen below.



You will need to contact the Live@Edu support services in order to resolve this issue