Welcome!

Silverlight Authors: Automic Blog, Michael Kopp, Jyoti Bansal, Kaazing Blog, Steven Mandel

Related Topics: Silverlight, Microsoft Cloud

Silverlight: Article

Identify Performance Bottlenecks in Your BizTalk Environment - Part 3

Orchestration and calling external services

In my last two articles I wrote about how to Use BizTalk Performance Counters and how to Analyze Adapter and Pipeline Performance. In this final article I focus on Orchestration and calling external services.

Step 4: Analyzing Orchestration
Orchestrations can be as simple as reading a file from a file system, transforming it and writing it out to a different file. They can also be much more complex such as calling external web services depending on certain conditions in the incoming messages, taking the response of these services and calling other services or writing a transformed version of the response to a file or the database. The following screenshot shows a rather simple Orchestration taken from one of the examples that ships with BizTalk:

Orchestration Example showing a message flow including a call to  an external service

Orchestration Example showing a message flow including a call to an external service

The process starts by receiving a file from the receiving port ReceivePO. The message then gets transformed using a mapping definition that maps individual fields from the original message to the destination message. The following screenshot shows such a mapping definition:

Business Mapping for Message Transformation

Business Mapping for Message Transformation

The Orchestration goes on by sending a message to port POWebServicePort. The response of this Web Service call is transformed again using another mapping definition. The final transformed message is then sent to the SendInvoicePort which in my case means writing it to the file system.

What about performance?
In BizTalk message processing there are different components involved. We already learned how to analyze Adapters and Pipelines in my previous blog entry and what we can do there to analyze whether these components have a negative impact on message processing performance.

The actual logic behind the Orchestration Definition is compiled into an assembly, loaded into BizTalk, and gets executed when messages are processed by that Orchestration Definition. Here is a sample PurePath that shows the execution of compiled orchestration code:

PurePath of Orchestration Logic Execution

PurePath of Orchestration Logic Execution

What can we observe here?
Applying the Transformation takes 444ms in this scenario
The SendMessage method puts the transformed message back into the MessageBox on SQL Server. The actual interaction with the MessageBox happens in the Commit method which takes 541ms in this scenario.

I ran some more load against my BizTalk environment analyzing all the PurePath’s that trace the logic of my orchestration. With increasing load the bottleneck in my environment clearly is the MessageBox, indicated by the Commit method that takes up to 99% of the transaction execution time.

Commit taking 99% of execution time with huge transaction time  variance

Commit taking 99% of execution time with huge transaction time variance

Not only is Commit taking 99% of the time for the slowest transaction, it also has a huge variance in execution time – meaning, it doesn’t scale in my environment. If you look at the PurePath duration in the screenshot above we see execution times ranging from 1.1 to 6.9 seconds. Analyzing the methods that were executed in these transactions gives me the following information:

Non Scaling Methods in my BizTalk Environment

Non Scaling Methods in my BizTalk Environment

I added min and max execution time of the methods involved in these transactions. Not surprisingly do we see a huge range from several milliseconds to 6.9 seconds for the Commit method. The execute method on top is a transaction I haven’t yet shown you. It is thread that pulls out information from the MessageBox to further process the message.

Problem: In my scenario the MessageBox is clearly my bottleneck in my environment. Putting messages into the box and getting them out again doesn’t scale with the load I want to put on it.

What about performance counters?

Performance counters are always a great source of information. BizTalk provides a set of counters for the Orchestration Engine as well as for the MessageBox. Check out the two previous links for a description on these counters.

HINT: BizTalk MessageBox Viewer Tool

Jean-Pierre Auconie is the creator of the BizTalk MessageBox Viewer. Check out his blog and the download site for this tool. This tool runs a set of tests and queries against the MessageBox database. The results will be displayed in the tool and can also be exported as HTML reports including links. The following screenshot shows the output in the tool on my installation:

Analysis Output in BizTalk MessageBox Viewer Tool

Analysis Output in BizTalk MessageBox Viewer Tool

The tool gives an overview of what might be wrong in your installation. In my case it seems that I ran into a “classical” BizTalk MessageBox issue: My Sql Agent Jobs don’t run on my machine to clear up the database. Also check out this MSDN forum thread that talks about the same scenario somebody else ran into.

Step 5: Analyzing External Service Calls (SendPorts)
The last thing I want to show you is when my BizTalk Orchestration makes a call to the external Web Service. The following image shows a PurePath that shows the Orchestration engine making the call to the web service. The PurePath shows the web service client proxy as well as the execution of the web service on the server that implements the web service:

Distributed PurePath showing a Web Service call from BizTalk to  the Web Service implementation

Distributed PurePath showing a Web Service call from BizTalk to the Web Service implementation

The 2nd call in this trace is the call to the generated web service proxy within my BizTalk Host Instance. The web service is executed synchronously in the ASPX implementation hosted in IIS. The interesting thing here is that I have huge execution differences of this web service call. The method that varies greatly in execution time is the client side call to the proxy that is doing all the marshalling of data.

Final Words
This is the final article in my series on BizTalk. The key take away is to make sure you monitor your BizTalk performance counters. Use the BizTalk MessageBox Viewer to run checks against your individual MessageBox databases. In case you are interested in in-depth tracing of messages through BizTalk using dynaTrace you should check out Managing BizTalk Performance and The Business Process It Automates, our Community Downloads to get access to the BizTalk Fast Pack (registered users only), and a White Paper I wrote about Performance Management in Enterprise .NET Systems.

Related reading:

  1. Identify Performance Bottlenecks in your BizTalk Environment – Part I Microsoft BizTalk enables companies to integrate and automate their business...
  2. Identify Performance Bottlenecks in your BizTalk Environment – Part II In Part I of this blog series I gave a...
  3. Performance Analysis: Identify GC bottlenecks in distributed heterogeneous environments Garbage Collection can have a major impact on application performance....
  4. Performance Analysis: How to identify synchronization issues under load? Synchronization is a necessary mechanism to control access to shared...
  5. Performance Analysis: How to identify “bad” methods messing up the GC Whenever the Garbage Collector kicks in to free up memory...

More Stories By Andreas Grabner

Andreas Grabner has been helping companies improve their application performance for 15+ years. He is a regular contributor within Web Performance and DevOps communities and a prolific speaker at user groups and conferences around the world. Reach him at @grabnerandi

@ThingsExpo Stories
In his session at @ThingsExpo, Greg Gorman is the Director, IoT Developer Ecosystem, Watson IoT, will provide a short tutorial on Node-RED, a Node.js-based programming tool for wiring together hardware devices, APIs and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using a wide range of nodes in the palette that can be deployed to its runtime in a single-click. There is a large library of contributed nodes that help so...
SYS-CON Events announced today that mruby Forum will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. mruby is the lightweight implementation of the Ruby language. We introduce mruby and the mruby IoT framework that enhances development productivity. For more information, visit http://forum.mruby.org/.
SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
Mobile device usage has increased exponentially during the past several years, as consumers rely on handhelds for everything from news and weather to banking and purchases. What can we expect in the next few years? The way in which we interact with our devices will fundamentally change, as businesses leverage Artificial Intelligence. We already see this taking shape as businesses leverage AI for cost savings and customer responsiveness. This trend will continue, as AI is used for more sophistica...
SYS-CON Events announced today that SourceForge has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SourceForge is the largest, most trusted destination for Open Source Software development, collaboration, discovery and download on the web serving over 32 million viewers, 150 million downloads and over 460,000 active development projects each and every month.
SYS-CON Events announced today that Daiya Industry will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Daiya Industry specializes in orthotic support systems and assistive devices with pneumatic artificial muscles in order to contribute to an extended healthy life expectancy. For more information, please visit https://www.daiyak...
SYS-CON Events announced today that Nihon Micron will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Nihon Micron Co., Ltd. strives for technological innovation to establish high-density, high-precision processing technology for providing printed circuit board and metal mount RFID tags used for communication devices. For more inf...
SYS-CON Events announced today that Massive Networks, that helps your business operate seamlessly with fast, reliable, and secure internet and network solutions, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. As a premier telecommunications provider, Massive Networks is headquartered out of Louisville, Colorado. With years of experience under their belt, their team of...
SYS-CON Events announced today that Suzuki Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Suzuki Inc. is a semiconductor-related business, including sales of consuming parts, parts repair, and maintenance for semiconductor manufacturing machines, etc. It is also a health care business providing experimental research for...
Elon Musk is among the notable industry figures who worries about the power of AI to destroy rather than help society. Mark Zuckerberg, on the other hand, embraces all that is going on. AI is most powerful when deployed across the vast networks being built for Internets of Things in the manufacturing, transportation and logistics, retail, healthcare, government and other sectors. Is AI transforming IoT for the good or the bad? Do we need to worry about its potential destructive power? Or will we...
SYS-CON Events announced today that B2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. B2Cloud specializes in IoT devices for preventive and predictive maintenance in any kind of equipment retrieving data like Energy consumption, working time, temperature, humidity, pressure, etc.
What is the best strategy for selecting the right offshore company for your business? In his session at 21st Cloud Expo, Alan Winters, U.S. Head of Business Development at MobiDev, will discuss the things to look for - positive and negative - in evaluating your options. He will also discuss how to maximize productivity with your offshore developers. Before you start your search, clearly understand your business needs and how that impacts software choices.
SYS-CON Events announced today that SIGMA Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. uLaser flow inspection device from the Japanese top share to Global Standard! Then, make the best use of data to flip to next page. For more information, visit http://www.sigma-k.co.jp/en/.
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp em...
Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the conn...
There is huge complexity in implementing a successful digital business that requires efficient on-premise and cloud back-end infrastructure, IT and Internet of Things (IoT) data, analytics, Machine Learning, Artificial Intelligence (AI) and Digital Applications. In the data center alone, there are physical and virtual infrastructures, multiple operating systems, multiple applications and new and emerging business and technological paradigms such as cloud computing and XaaS. And then there are pe...
Real IoT production deployments running at scale are collecting sensor data from hundreds / thousands / millions of devices. The goal is to take business-critical actions on the real-time data and find insights from stored datasets. In his session at @ThingsExpo, John Walicki, Watson IoT Developer Advocate at IBM Cloud, will provide a fast-paced developer journey that follows the IoT sensor data from generation, to edge gateway, to edge analytics, to encryption, to the IBM Bluemix cloud, to Wa...
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, will discuss how data centers of the future will be managed, how th...