BizTalk360 Review and summary

I am going to provide a review and summary of BizTalk 360. www.BizTalk360.com

I am approaching this from the point of view, I have not used it and I want to use it, and get it to do what it says on the box, monitor BizTalk.

Let’s start with actually getting it working…..

First blocker, the install, for some reason, even though I am an admin on the box, and I can install BizTalk fully, I need to go to a command prompt and run this as an administrator. I can install a BizTalk MSI which creates a website without needing to be an administrator; however this seems to have problems here.

Now we have it installed, we click the link on BizTalk 360, after a short pause we are greeted with this screen, nothing else to tell us why or what to do, just this screen.  Now the user of whom we selected does not have access to the management database, we selected the isolatedhost user, that can talk to this database…. For some reason it’s not allowing us to? With no detail… great.

We change the user to another user, recycle the app pool, and reload the site….

I don’t quite know what to make of this… so we poke around, and configure the environment.  Now our BizTalk is installed on the machine we are running this on, as it kind of needed it to have all the BizTalk Bits, so it should know where the management database is, as it’s stored in the registry. The default SQL instance and the standard name for the BizTalk Management database needs to be entered here. We press Test Connection, it never returns, we waited for about 10 minutes, and it never came back, just a circle spinning endlessly.

 

Read the documentation, they want us to use mixed mode authentication and use the BizTalk360 user it tried to covertly create for us, well bad news BizTalk 360, this is in violation of our security policy and not allowed, and this is not uncommon, most organisations have this as their policy. Strike 1.

Refresh the web page after also putting the license key in…. our Licence type is unknown… not sure what this means.

I notice a license icon next to the list, it brings up request new license, so I try and generate a request. Press the button…

Even though the user has permissions…..

I tweak some more, fudge it more….

I can then press the generate license request.  It wants to send some highly confidential information, as part of this request, the domain name, the admin group name, the server name, this I am simply not allowed to give to an external party, and I am unable to go any further with this.

company:WWW|adminGroup:{Domain}\{Removed} BizTalk Server Administrators|mgmtSqlInstance: {Removed Server Name} |mgmtDbName:BizTalkMgmtDb|BizTalkVersion:3.10.229|EnvironmentType:Non-Production

I did have a license key however, and it still comes up unknown….

More tweaking it around…. I get to the dashboard…. Again I’m running as a user who is now the db owner of all of the BizTalk Databases, highly unorthodox, but it is the only way I could get this thing to even get me this far. Then I am blocked again….

Why this occurs? I have no clue; it’s just not going very well for this product.

The fact it’s actually querying the message box does not sit well with me, this database is used by every host instance, every orchestration, almost every second, and it’s a very intense usage database, that you don’t just run queries in a production environment, which are not supported by the product group.

I finally got it to work… well kind of, after giving my app pool user, ownership, which is VERY risky, but this is a sandbox, and I want to make it work… Then I find I need yet another BizTalk Database to be given access… The DTA Database… again, not something you should really be querying.

 

Continue to click down on the list of things you can do, BAM Views… same problem, my App Pool user does not have access, and why would it?

Looking further, at the advanced event viewer, which basically looks at the event log.

The filter I did, gave 100% BizTalk 360 errors… What’s even stranger is that the errors are about sending emails, of which I have not turned on, and thus not configured to send.

 

I bypass this, configure it all up.... Next test, all configured, alarms set, and monitoring setup.

I am monitoring a send port, and a receive location for x Number of transactions per hour.

I am also monitoring orchestrations that stop in a particular application.

I am sitting on the monitoring dashboard, which refreshes every 60 seconds.

My environment is saying it’s healthy. I filter for errors.

I stop the orchestrations I am monitoring, I wait….

One refresh, still healthy, two refreshes, still healthy a manual refresh still healthy.

BizTalk 360 is not looking too healthy.  It never tells me it’s not running….. It never tells me later that no transactions went down the port…. BizTalk 360 FAIL, it just does not tell me....

On the whole, BizTalk 360 is trying to be the BizTalk admin console, with insufficient permissions, and bypassing proper administrator permissions, allowing ANYONE to get on here, even people who would otherwise not have permissions to BizTalk. It bypasses all the security of BizTalk Groups, the BizTalk Administrators group, the BizTalk Operators Group.

It bypasses all BAM related permissions for specific users, allowing the user to see things in BAM they would not ordinarily be allowed to see.

It does not monitor, of which one of its key features is, I received not one alert, and not one dashboard said things were bad. It always said my environment was healthy, several hours after I made it quite unhealthy.

It could only monitor things like orchestration running, and send and receive port transactions. We have MANY processes that are ESB processes, which use direct ports, and send messages to the message box, and are routed to other subscribers internally. None of this can be monitored.

It uses queries against the message box, management database, DTA database, BAM database and ESB database. A direct query of these databases is highly unorthodox, and very dangerous. I have several queries that can provide much of the detail, but I am NOT ALLOWED to, and do not run these on production, it’s downright dangerous to run un-supported queries on a production environment, particularly those running with their own permissions.

It tries to be the BizTalk Admin Console, of what the BizTalk Admin console is designed to be and ships free with the product, and can run on an operators desktop.

It tries to be the BAM portal, which comes free with BizTalk, which was built for BAM, and bypasses all permissions of this portal, in a very bad way given the data here can be highly confidential.

It tries to monitor BizTalk, which did not seem to work for me, but it’s running queries, multiple queries against the BizTalk databases, every 60 seconds, these are putting additional load on your BizTalk environment, and possibly blocking queries from BizTalk itself, highly dangerous.

The majority of things it does, come with the product, they actually make it harder to do than in the BizTalk Admin console, which has its own permission set, you can bypass this by granting a user super user access to specific applications only, which is nice, but it’s a bypass of the permission set.

Would I use and thus recommend BizTalk 360?

Do not get me wrong, I looked at this tool, and configured everything and after many problems got it working, and set up all the features, and put it through its paces, I actually wanted it to work, if what’s written on packaging, and in all the hype around this tool.

So would I use it? Absolutely not.

Why Not?

I get better and more precise monitoring from BAM, I get better alerting from BAM. I can see not just that there was a problem with system X which uses port Y, I can see what it is, I can see the number of transactions, the volume, or lack of volume, which transaction is effected, and why. I can do this for ESB to ESB transactions as well. I can better engage the business of whom is the ultimate owner of this and give them such great detail that they are often astounded by the level of knowledge we have about what’s going on.

If I use the System Centre Management pack for BizTalk, I can hook into just about every alert and event that gets raised in BizTalk. There are MANY perfmon counters that expose pretty much most of what’s happening in BizTalk, and you can configure monitoring of this in a much more supported fashion than what is exposed in BizTalk 360.

This puts me in a more supported situation, and 100% of this is free, in that it ships with the product, is supported and works, why would I use something else that does not work, and is not supported, in an organisations production environment that runs its core operations and controls a greater than million or billion dollar business…. I am not going to take a chance like this ever.

There is no way I could even convince the business to do so with risk factors like this.

BAM Portal Not Installing

You installed BizTalk Server

You want to config BAM Portal see this error:

Failed to validate BAM Portal Web site (BAMPortal)

   Additional information:

      The BAM Portal website Default Web Site is not valid.

      (Microsoft.Biztalk.Bam.CfgExtHelper.PortalHelper)

      Exception from HRESULT : 0x80005008(System.DirectoryServives)

Basically its not allowing you to select the default website, its giving a red cross. 

Simple Solution:

1. Open the Server Manager MMC snap-in (right-click on Computer and select the Manage option).
2. Expand the features node.
3. Select the Web Server node.
4. Look through the role services.  Check for IIS 6 Management Compatibility and see if it says it is installed.
5. If not installed, click Add Role Services on right of screen.  Select the IIS 6 Management Compatibility and install.
6. Go back to BizTalk configuration, the BAM Portal feature should now install without any issues.

WCF SQL operations

When adding WCF sql ports to BizTalk, you can add a whole host of different types of operation with SQL and it’s incredibly powerful. 

The problem is with all these options, the wizard rarely makes a binding file for you, it will make the schemas, but the binding file contains that extra little bit of magic…

The key to all of this in the SOAP action header you specify on the port.

There are your basic ones:

TableOp/Insert/dbo/Employee

TableOp/Update/dbo/Employee

If you read this, it will do a table operation and insert or update employee record(s) in the employee table.

Then there is of course stored procedures for both send and receive ports. 

TypedProcedure/dbo/StoredProcedureName

Simple enough, however maybe you have a few operations you want to do, you want to go for a more composite operation. 

Your Soap action can be CompositeOperation

The message you send can do a whole host of things, for example, insert into a table, and when done, run a stored procedure.

You have your insert schema, and your execute stored procedure schema, that the wizard generates, now bundle this into the following structure and all of a sudden a whole world of opportunities opens up.

<xs:element name="Request">

    <xs:complexType>

      <xs:sequence>

        <xs:element ref="ns0:Insert" />

        <xs:element ref="ns1:CompareEmployee" />

      </xs:sequence>

    </xs:complexType>

  </xs:element>

  <xs:element name="RequestResponse">

   <xs:complexType>

    <xs:sequence>

     <xs:element ref="ns0:InsertResponse" />

     <xs:element ref="ns1:CompareEmployeeResponse"/>

    </xs:sequence>

   </xs:complexType>

  </xs:element>

Then there are the Generic operations, vastly undocumented, for example a SQL reader, that can execute SQL against the database and return the results of the query…

Your Soap Action header would be: GenericOp/ExecuteReader

The schema looks like this:

  <ExecuteReader xmlns="http://schemas.microsoft.com/Sql/2008/05/GenericTableOp/">

    <Query>[PL/SQL STATEMENT1];[PL/SQL STATEMENT2];…</Query>

  </ExecuteReader>

Have a look at some of the more unknown options available, and you start to see the power in all of this. 

Operation	Soap Action  Header
ExecuteNonQuery Request	GenericOp/ExecuteNonQuery
ExecuteNonQuery Response	GenericOp/ExecuteNonQuery/response
ExecuteReader Request	GenericOp/ExecuteReader
ExecuteReader Response	GenericOp/ExecuteReader/response
ExecuteScalar Request	GenericOp/ExecuteScalar
ExecuteScalar Response	GenericOp/ExecuteScalar/response

Refer to this link for more: http://msdn.microsoft.com/en-us/library/dd788372%28v=bts.10%29.aspx

Accessing the SharePoint user profile service from BizTalk

I have a client scenario where weird 3^rd party system 1, sends me an update of some information, from time to time, about a user, this information needs to be instantly replicated into SharePoint.

I could use BCS but it’s not instant…  It’s coming to me via BizTalk, so I have it in near real time, so 

I can update SharePoint can’t I??… well yes it seems you can…

There is the web service that SharePoint quite nicely exposes:

http://%7bserver%7d/_vti_bin/userprofileservice.asmx

It has many methods one of these is: ModifyUserPropertyByAccountName

It’s a one way send in BizTalk. 

When you add a reference, you need to add a generated item, that consumes a WCF Service.

You will get 2 schema s an orchestration and a port binding, the schema s are useful, the orchestration you can choose to use it or not, it contains a bunch of multi-part message types,  and a massive port for every single method.

The port is useful, because it contains the operations, you will need to use when you create the port, so one port can have many operations to the same web service, they give you the binding for this port, which is very nice, I suggest you use it.

Here is where all the nice stuff ends, and the really interesting stuff begins. If you use this, it does not work. 

You may get:

Error details: System.ServiceModel.FaultException: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><soap:Fault><faultcode>soap:Server</faultcode><faultstring>Server was unable to process request. ---&gt; Invalid String Value: Input must match string data type.</faultstring><detail /></soap:Fault></soap:Body></soap:Envelope>

This made no sense to me, as I was sending a string, and updating a string…..

If you look at the schema for the message:

value is defined as xs:anyType, which means you can put anything in here, which is correct, I want to, a string, a datetime, a int. So the schema is flexible…

The wsdl is very vague:

      <s:complexType name="ValueData">

        <s:sequence>

          <s:element minOccurs="0" maxOccurs="1" name="Value" />

        </s:sequence>

      </s:complexType>

The value is does not specify xs:anytype however because it does not specify a type BizTalk interprets this, bad sharepoint.

However it does not go far enough… and hence it is a real let down when you have gotten past the authentication issue to find this. See my other post to fix this.

If you call the method from .net code, it works… however to find out why it works, you look at the xml that this call generates:

<ValueData>

   <Value xsi:type="xsd:dateTime">2012-06-25T10:01:17.486123+10:00</Value>   </ValueData>

The “value”  has an uppercase V, BizTalk puts is with a lower case V.  so that’s the first fix.

The next problem: xsi:type ??

It’s not even in the schema, it’s not in the wsdl.  I try and try and try to get it in to the schema, it’s not going to happen.

I managed to add an attribute to the value, called type, however it’s coming up as:

<ns0:Value ns0:type

Now the values are being set in SharePoint, all to NULL because it cannot interpret the type.

Now I know what I need to make the message look like, I have BizTalk pipelines… I can touch up the message before I send it to SharePoint.

So I go and touch up the message and the thing works perfectly. !!

I can now communicate from BizTalk to SharePoint, to update the user profiles.

My port looked like this:

Accessing an authenticated web service using BizTalk and impersonation

I need to access the SharePoint user profile service.

SharePoint quite nicely exposes:  http://{Server}/_vti_bin/userprofileservice.asmx

It is authenticated, quite rightly so, however the user I am accessing it with has permissions, however it still asks for username and password.

I was getting this error:

The adapter failed to transmit message going to send port "WcfSendPort_UserProfileServiceSoapOneWay" with URL "http://{Server}/_vti_bin/userprofileservice.asmx". It will be retransmitted after the retry interval specified for this Send Port. Details:"System.ServiceModel.FaultException: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><soap:Fault><faultcode>soap:Server</faultcode><faultstring>Server was unable to process request. ---&gt; System.ServiceModel.EndpointNotFoundException</faultstring><detail /></soap:Fault></soap:Body></soap:Envelope>

Server was unable to process request. System.ServiceModel.EndpointNotFoundException

Not much help to me…

After some searching I found I needed to use impersonation.

The main reason was from .net I can simply do:

WSService.Credentials = System.Net.CredentialCache.DefaultCredentials;

Now the current user is impersonated and if they have permissions it works. GREAT.

I’m in BizTalk, and I’m on the WCF port…

The port is using wcf-basichttp as the protocol, and it should be able to just I have no idea how.

Well the solution I found is rather simple.

Use a wcf-custom port.  

Setup the basic-httpbinding.

First of all set the transport to client credential type ntlm 

 

Then flip over to Behavior

It will be blank.  Right click end point behavior: 

Select add extension.

Add client credentials and configure like so:

 After doing this my error went away.

BizTalk Server 2010 Cookbook

Steef-Jan Wiggers new book: BizTalk Server 2010 Cookbook

The book includes little over 50 recipes for BizTalk developers and administrators. It will demonstrate the out-of-the box capabilities BizTalk combined with capabilities from community- and Microsoft related BizTalk tooling.

You will learn more when you visit the Packt website: http://www.packtpub.com/biztalk-server-2010-for-developers-and-administrators-cookbook/book

Exception: System.EnterpriseServices.TransactionProxyException

I get an exception when configuring BizTalk 2010.

BizTalk application server.
Separate SQL server for the databases.

Exception: System.EnterpriseServices.TransactionProxyException

Looking at http://support.microsoft.com/kb/293799

When a System.EnterpriseServices.TransactionProxyException exception is triggered during a transaction completion, it cannot be caught from other application domains. Instead, you receive a System.Runtime.Serialization.SerializationException exception that resembles the following:

Unhandled Exception: System.Runtime.Serialization.SerializationException: Type 'System.EnterpriseSer vices.TransactionProxyException' in Assembly 'System.EnterpriseServices, Version=2.0.0.0, Culture=ne utral, PublicKeyToken=b03f5f7f11d50a3a' is not marked as serializable.

Gave me a hint to the problem, however was not the solution…… or any where near to it be careful following this I did not.

I saw that the databases were created during the configuration, and then deleted; only leaving the management database in a weird state. So the communication to the database server was fine.

I did some more investigation and found the dtctester tool… great for testing if my dtc was set up correctly, as this is one thing that BizTalk leverages extensively.

I ran the 'dtctester' tool, and it came up with the following:

Executed: dtctester.exe
DSN: PMS
User Name: *******
Password: ********
tablename= #dtc16032
Creating Temp Table for Testing: #dtc16032
Warning: No Columns in Result Set From Executing: 'create table #dtc16032 (ival
int)'
Initializing DTC
Beginning DTC Transaction
Enlisting Connection in Transaction
Error:
SQLSTATE=25S12,Native error=-2147168242,msg='[Microsoft][ODBC SQL Server Driver]
Distributed transaction error'
Error:
SQLSTATE=24000,Native error=0,msg=[Microsoft][ODBC SQL Server Driver]Invalid cur
sor state
Typical Errors in DTC Output When
a. Firewall Has Ports Closed
-OR-
b. Bad WINS/DNS entries
-OR-
c. Misconfigured network
-OR-
d. Misconfigured SQL Server machine that has multiple netcards.
Aborting DTC Transaction
Releasing DTC Interface Pointers
Successfully Released pTransaction Pointer.

Ok, so my problem was DTC….

I found a great response to a similar problem that told me to look at a couple of DTC issues. (http://dbaspot.com/sqlserver-server/215054-msdtc-doesnt-seem-work.html)

http://support.microsoft.com/kb/817064
http://support.microsoft.com/kb/301600

If you are running Win2K3 SP1, you will need to reset the DTC security parameters:

http://support.microsoft.com/kb/899191

Since you are beginning the transaction on a remote host, that host will also need to properly configure its DTC services.

Finally, if the communication must transit any firewalls, then you will need to restrict RPC ports on both Client and Server for Internet Ports, and then authorize these Ports in the firewall ACLs.

http://support.microsoft.com/kb/300083
http://support.microsoft.com/kb/250367/EN-US

I checked the firewall, no problem, I checked the other things, and made a change to the DTC security not via the method mentioned but via the component services.

Everyone should know by now to change the security settings under component services for the dtc on the sql server and the BizTalk server.

Go to the Component services/My Computer/Distributed Transaction Controller/LocalDTC

Properties/security

Ensure that everything in here is checked, (Remote admin is optional)

I usually select no Authentication required, and ensure that the network service account is running the service.

I did this, thinking I’ve found the problem…..

Nope same problem re occurs…

I find out that the SQL server is clustered, so I go to the cluster manager, and look at the dtc settings, and open the localdtc from there, the same thing, all the settings are correct because I had already changed them.

Then I notice, CLUSTER DTC, and expand that and mange that…. The cluster has its OWN instance of DTC, of course, it does not use the local DTC at all.

I look at the security settings of the cluster DTC, and change those to match, and wolla… it works…. Nice to know if the SQL server is clustered…

Always look at DTC, if your BizTalk does not wish to install to a remote SQL server.

Exam 70-595 BizTalk 2010 Exam Released

After much pushing, and some prodding of the people who look after exams at Microsoft. Several members of the community got involved with Microsoft to create a BizTalk 2010 Exam.

It has now been released... have a look here

The more interesting thing to note, this exam will be quite different from exams on this topic from previous exams as the community created it, the community they knows in depth about the product.

Just look at the main areas:

Configuring a Messaging Architecture (20 percent)
Developing BizTalk Artifacts (20 percent)
Integrating Web Services and Windows Communication Foundation (WCF) Services (14 percent)
Implementing Extended Capabilities (13 percent)
Deploying, Tracking, and Supporting a BizTalk Solution (16 percent)

With a large focus on Architecture and actual development, without forgetting about deployment and support for BizTalk Solutions, an area that was sadly lacking any real content however significantly needed.

I have passed previous exams on BizTalk, and was really pushing for an exam that was not like other ones and set the mould for new exams.

Go on, get out there and have a go... here