To Integrate or to Integrate Badly

When someone chooses to integrate there is always the wrong way and the right way to go about this. Just because you chose to integrate does not mean you did a fantastic job about it. 

One such example I came across recently was a nameless organisation decided to integrate their systems, they provided a very good level of integration and it was all working fine, then they got over zealous, a contractor they hired thought hey I can do integration also, the company not knowing any wiser said OK sure let’s do it. 

The integration was a simple one, I’d like to change my surname, and it should update all the systems, because I am married now and have taken my husband’s name by choice. 

The old way of doing this was very manual and very cumbersome and took a long time to implement.

We already provided integration of other user profile details, which were updated in the various systems in the organisation. The contractor thought hey I can do it too… 

The surname request update had a fancy front end, when the user submitted this, their request was sent as an email to someone in HR, the first problem. 
 
They would manually make the changes required in the system… The second problem.

What is worse, is that the email got it wrong, it said update my first name to this name, and not my surname, so the person in HR went and updated their first name to their new surname…  This person was outsourced from the organisation. The third problem.

This had flow on effects, that had not been thought of, when someone changes their name, their AD name needs to update, their email address needs to change, and a whole bunch of other systems need to be informed. 

To get around this they just deleted the user and made a new one as if they had just joined the organisation, the flow on was very very bad… Major problem goes with out saying.

All of their access gone, all of their email gone, their permissions to all the internal systems gone, this would have happened even if they had changed their surname. 

The loss of productivity for this person was about two weeks’ worth, the person was not a low level admin, they were senior and on a good rate. 

Take this against the decision to use a contractor who said yeah we can do it,  probably $1000 for the efforts, then add the down time of the one employee, $20000 in loss of productivity and in ability to work, you are not only paying for them to work but they can’t work and produce income for the organisation. 

Now multiply this by a low estimate of 5 people who may have been effected by this across an organisation of 7000 people…  

You have a net loss to the organisation of: $100,000  plus your low cost of implementing this in the first place.  

Take this and compare against doing it correctly in the first place, taking the time to understand the impact, going and updating the various systems, in near real time, and ensuring that the user was good to go within 15 minutes. 

A cost estimate for this would be somewhere between  $20,000 to $60,000. Had more than 5 people requested this change, think of the cost to the organisation.

It becomes very clear that you need to integrate, and you need to do it properly. You need to have the right people working with you who understand this and are able to see these issues well before they happen. 

Never believe someone who says yeah we can do that, no problem, it is easy, we will just email someone, or we will add a manual step, or we will just leverage the existing process or method, or we will just delete them and put them back… these things make me cringe… don’t do it… ever! 

Talk to the professionals, they have done it all before. 

Business Process and Integration

The corporate landscape of today includes a wide variety of heterogeneous business applications.

Each designed to solve a specific and specialised business problem. These applications are often not designed to work together seamlessly. This creates a major challenge when information needs to be extracted or combined for various purposes across the enterprise. The question then arises, “How do organisations extract relevant information and deliver it in the right format, to a destination either inside or outside of the organisation, across functional domains, at the right time?” This business challenge makes Business Integration a high priority.

The number one organisation I recommend to solve this problem is Connexion Technology I am proud to announce a partnership between myself and Connexion Technology, one that will see this organisation move to the forefront of this space, and help countless organisations.

Accessing the SharePoint user profile service from BizTalk

I have a client scenario where weird 3^rd party system 1, sends me an update of some information, from time to time, about a user, this information needs to be instantly replicated into SharePoint.

I could use BCS but it’s not instant…  It’s coming to me via BizTalk, so I have it in near real time, so 

I can update SharePoint can’t I??… well yes it seems you can…

There is the web service that SharePoint quite nicely exposes:

http://%7bserver%7d/_vti_bin/userprofileservice.asmx

It has many methods one of these is: ModifyUserPropertyByAccountName

It’s a one way send in BizTalk. 

When you add a reference, you need to add a generated item, that consumes a WCF Service.

You will get 2 schema s an orchestration and a port binding, the schema s are useful, the orchestration you can choose to use it or not, it contains a bunch of multi-part message types,  and a massive port for every single method.

The port is useful, because it contains the operations, you will need to use when you create the port, so one port can have many operations to the same web service, they give you the binding for this port, which is very nice, I suggest you use it.

Here is where all the nice stuff ends, and the really interesting stuff begins. If you use this, it does not work. 

You may get:

Error details: System.ServiceModel.FaultException: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><soap:Fault><faultcode>soap:Server</faultcode><faultstring>Server was unable to process request. ---&gt; Invalid String Value: Input must match string data type.</faultstring><detail /></soap:Fault></soap:Body></soap:Envelope>

This made no sense to me, as I was sending a string, and updating a string…..

If you look at the schema for the message:

value is defined as xs:anyType, which means you can put anything in here, which is correct, I want to, a string, a datetime, a int. So the schema is flexible…

The wsdl is very vague:

      <s:complexType name="ValueData">

        <s:sequence>

          <s:element minOccurs="0" maxOccurs="1" name="Value" />

        </s:sequence>

      </s:complexType>

The value is does not specify xs:anytype however because it does not specify a type BizTalk interprets this, bad sharepoint.

However it does not go far enough… and hence it is a real let down when you have gotten past the authentication issue to find this. See my other post to fix this.

If you call the method from .net code, it works… however to find out why it works, you look at the xml that this call generates:

<ValueData>

   <Value xsi:type="xsd:dateTime">2012-06-25T10:01:17.486123+10:00</Value>   </ValueData>

The “value”  has an uppercase V, BizTalk puts is with a lower case V.  so that’s the first fix.

The next problem: xsi:type ??

It’s not even in the schema, it’s not in the wsdl.  I try and try and try to get it in to the schema, it’s not going to happen.

I managed to add an attribute to the value, called type, however it’s coming up as:

<ns0:Value ns0:type

Now the values are being set in SharePoint, all to NULL because it cannot interpret the type.

Now I know what I need to make the message look like, I have BizTalk pipelines… I can touch up the message before I send it to SharePoint.

So I go and touch up the message and the thing works perfectly. !!

I can now communicate from BizTalk to SharePoint, to update the user profiles.

My port looked like this:

Accessing an authenticated web service using BizTalk and impersonation

I need to access the SharePoint user profile service.

SharePoint quite nicely exposes:  http://{Server}/_vti_bin/userprofileservice.asmx

It is authenticated, quite rightly so, however the user I am accessing it with has permissions, however it still asks for username and password.

I was getting this error:

The adapter failed to transmit message going to send port "WcfSendPort_UserProfileServiceSoapOneWay" with URL "http://{Server}/_vti_bin/userprofileservice.asmx". It will be retransmitted after the retry interval specified for this Send Port. Details:"System.ServiceModel.FaultException: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><soap:Fault><faultcode>soap:Server</faultcode><faultstring>Server was unable to process request. ---&gt; System.ServiceModel.EndpointNotFoundException</faultstring><detail /></soap:Fault></soap:Body></soap:Envelope>

Server was unable to process request. System.ServiceModel.EndpointNotFoundException

Not much help to me…

After some searching I found I needed to use impersonation.

The main reason was from .net I can simply do:

WSService.Credentials = System.Net.CredentialCache.DefaultCredentials;

Now the current user is impersonated and if they have permissions it works. GREAT.

I’m in BizTalk, and I’m on the WCF port…

The port is using wcf-basichttp as the protocol, and it should be able to just I have no idea how.

Well the solution I found is rather simple.

Use a wcf-custom port.  

Setup the basic-httpbinding.

First of all set the transport to client credential type ntlm 

 

Then flip over to Behavior

It will be blank.  Right click end point behavior: 

Select add extension.

Add client credentials and configure like so:

 After doing this my error went away.

Publishing an InfoPath form to Sharepoint 2010 for WebBrowser

Publishing an InfoPath form to SharePoint 2010 is not that difficult, what is difficult is getting it published when it has code behind for the use in the Web Browser.

I recently battled with this, and I must say it’s better than it used to be, making InfoPath VERY powerful, particularly if you can use code behind, and even call your own .net dll’s from the form.

For most they have got somewhere, but have not got it fully working.. if you are one of these people read on. 

Do you want to activate it in the site collection, but can't upload it?

First you need to do this:

1. Open SharePoint 2010 Central Administration.
2. Click General Application Settings.
3. On the General Application Settings page under InfoPath Forms Services, click Upload form template.

You GET: ERROR: This form template has not been published. Open the form in the InfoPath Designer and publish the form to SharePoint using the Administrator-approved form template method in the Publishing Wizard.

Do the following:

Important: Ensure that the security and trust is set to full trust, with a certificate.

1. Publish the form to SharePoint from InfoPath.

2. Note down where you published it: http://msite/template.xsn

3. Go to your web browser and go to this URL, download the PUBLISHED template.xsn

4. SAVE IT locally.

OR

4a. Better way: Publish it to a network location say c:\MyFormTemplates, you do not need it in SharePoint YET.

5. Now upload the {template}.xsn, (Rename if you like) to SharePoint admin.

You should get: The form template has been successfully uploaded to the farm. To make the form template available in a site collection, activate the form template from the Manage Form Templates page or from the feature activation page in the site collection.

6. Now you can activate it in your site collection... etc...

Go to your site where you want to use the form.

Site settings / site collection administration

Site collection features

Look for your Infopath Form Template, (it's sorted alphabetically)

Press Activate

---

THE KEY to enabling the form for the browser is in how you set up your form library...

Under Site/{FormLibraryName}/form library settings/ advanced settings.

SAY Allow management of content types = YES

Then hit ok.

Now go down to content types, and defined your content type, say add from existing site content types, which you just activated... You do remember the NAME of the form you uploaded in admin before??

Make sure that it is the default content type, and everything will be good. 

Grant permissions to your users, and have them press new document, and up comes your form in the browser...

You can then put your workflow on the form library, or hook BizTalk up to the form Library, and you have full end to end Form and Workflow.

BizTalk Server 2010 Cookbook

Steef-Jan Wiggers new book: BizTalk Server 2010 Cookbook

The book includes little over 50 recipes for BizTalk developers and administrators. It will demonstrate the out-of-the box capabilities BizTalk combined with capabilities from community- and Microsoft related BizTalk tooling.

You will learn more when you visit the Packt website: http://www.packtpub.com/biztalk-server-2010-for-developers-and-administrators-cookbook/book

Windows Server AppFabric Cookbook

Rick G. Garibay's new book: Windows Server AppFabric Cookbook

The book includes over 60 recipes ranging from fundamental to advanced techniques for
making the most of WCF 4+ and WF 4+ on the Microsoft platform with Windows Server AppFabric.

You can learn more by visiting this link: http://www.packtpub.com/microsoft-windows-server-appfabric-cookbook/book

BizTalk FILE Receive / Send with DFS network share

When using BizTalk to send and receive from a file share is usually pretty simple. However when you combine this with a DFS network file share, it gets a little bit more complex.

We found our receive location was not starting, and simply failing to work.

So we did mange to get this to work here is how:
You firstly need to have the right share, as some users will use \\machinesname1\share$ and others something else.

The correct share to use is the DFS defined share, so when the DFS storage moves the share is maintained, and not lost. Always find out what the main share name is for the DFS network.
Next you need to set the correct network file share permissions on the network folder. When connecting to a network file share using BizTalk File adapter just specifying "Modify" rights is not sufficient. In simple configuration terms, you would need to have a minimum of "Full Control" or the BizTalk Host Service Account.
Some system admins don’t like this, you need to make sure you have the permissions ONLY set on the folder you are actually reading/writing to.

For the FILE Receive Adapter
After you provide "Modify" rights, Click on "Advanced" button and on the Advanced Security Dialog, edit the permission entries for the BizTalk Service Account
Here you will notice that the "Delete Sub Folders and Files" option is not given by default. This is exactly the missing link for the receive adapter.
Simply set this property and you do not need to set "Full Control"

We had to set list access all the way up the file tree for us to even see the folder.
EG: Our share was: \\dfssharename\saa$\Folder1\Folder2\Folder3\Folder4\IN

We needed to set list permissions on Folder1, Folder2, Folder3 and Folder 4
Then full modify/delete rights, with the setting Delete Sub Folders and Files.
Only then did it work.

For the FILE Send Adapter
If you are NOT USING the "use temporary file while writing" attribute on your send port, only the following permissions are required on the permissions entry dialog
o Create Files / Write Data

If you are using the "use temporary file while writing" attribute on your send port, the following permissions are required on the permissions entry dialog
o Create Files / Write Data
o Delete Files
o Delete Sub Folders and Files
o Read Permissions

Could not create SSOSQL. To fix the problem, reinstall SSO or try 'regasm SSOSQL.dll' from a Visual Studio command prompt.Error Code: 0x80131700

You have installed BizTalk?

You have installed SQL server?

You have installed .NET4.0 ?

Everything is fine.

Then you reboot, or you restart SQL Server.

Enterprise Single Sign-on Does not want to start…. ???

It errors with:

Could not create SSOSQL. To fix the problem, reinstall SSO or try 'regasm SSOSQL.dll' from a Visual Studio command prompt.Error Code: 0x80131700

OR

Windows could not start the Enterprise Single Sign-On Service service on Local Computer.
Error 0x80131700: 0x80131700

OR you are getting:

Creation of Adapter FILE Configuration Store Entries failed.
An Error occurred while attempting to access the SSO database.

The problem is .NET 4 does something to break SSO.

This issue occurs after installing .NET Framework 4.0. The registration of the assembly used by ENTSSO to access SQL Server (SSOSQL.dll) does not specify the correct version of the .NET Framework. When .NET Framework 4.0 is installed, the assembly will try to use the newer framework and then fail to load

DO NOT do what it says in the event log it will NOT work fully.

The fix is: http://support.microsoft.com/kb/2252691

However you need to BE SURE you do it correctly.

Is your environment 64bit? Most are these days. Did you install 64bit SSO?

Is it in:

64bit: C:\Program Files\Common Files\Enterprise Single Sign-On
32Bit: C:\Program Files (x86)\Common Files\Enterprise Single Sign-On

If you have 64bit, follow the 64BIT instructions ONLY

1. Open a command window
2. Go to C:\Windows\Microsoft.NET\Framework64\v2.0.50727
3. Type each of the following and hit ENTER

32bit: regasm “C:\Program Files\Common Files\Enterprise Single Sign-On\win32\ssosql.dll”
64bit: regasm “C:\Program Files\Common Files\Enterprise Single Sign-On\ssosql.dll”

Note On a 64-bit server, regasm will need to be run for both the 32-bit and 64-bit versions of ssosql.dll.

Do both of these steps and you day will be brighter.

Dynamic SMTP Failure: Unknown Error Description

I was sending an email dynamically, from an orchestration.

I set up a nice payload message with all of these nicely distinguished fields, that contained the to/ from, subject and body of the email, so that other parts of my system could send an email, just by generating this message.

My message assignment shape had:

EmailSendMessage = EmailReceiveMessage;
EmailSendMessage(SMTP.EmailBodyText) = EmailReceiveMessage.Body;
EmailSendMessage(SMTP.CC) = EmailReceiveMessage.CopyTo;
EmailSendMessage(SMTP.From) = EmailReceiveMessage.From;
EmailSendMessage(SMTP.Subject) = EmailReceiveMessage.Subject;
EmailSendMessage(SMTP.MessagePartsAttachments) = 0;

EmailSendPort(Microsoft.XLANGs.BaseTypes.Address) = "mailto:" + EmailReceiveMessage.SendTo;


The message itself was not sent, just the properties of this message, so the message I was constructing as receive message didn’t matter, so I made it the same type of the EmailReceiveMessage. It compiled, and I deployed it.

First Try:


Event Type: Error
Event Source: BizTalk Server 2009
Event Category: (1)
Event ID: 5754
Date: 13/12/2011
Time: 1:16:31 PM
User: N/A
Computer: [computer]
Description:
A message sent to adapter "SMTP" on send port "XXX.Email.Orchestrations_1.0.0.0_XXX.Email.Orchestrations.SendEmail_EmailSendPort_43e93d0db20c465a" with URI "mailto:email@address.com" is suspended.
Error details: Unknown Error Description
MessageId: {B66F52BA-DFF0-4274-B4B2-3B1F51E862E0}
InstanceID: {C7A4CA9E-E606-4D04-9001-D34974B4D971}

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.


I then checked and you needed to have the body text charset:

EmailSendMessage(SMTP.EmailBodyTextCharset) = "UTF-8";


Second try, same error…

Several modifications later I get the message: Unknown Error Description

Which I must say is not a great deal of use when you are trying to figure out what’s wrong.

I then decided I’d make a different message type for sending, and used a transform. I used my send message and copied and pasted the schema, I changed the target name space of course.

SAME ERROR…..

I was annoyed, and tried a bunch of things, then EUREAKA! …

The send schema which I copied and pasted has the same properties promoted as distinguished fields.

This was BAD it seems, the instant I removed these properties from being promoted, and changing nothing else… everything worked….

BAD BAD BAD... schema properties, who would have thought…

DO NOT HAVE PROMOTED PROPERTIES ON YOUR SCHEMA WHEN SENDING TO THE SMTP ADAPTER.