Communication between TIBCO EMS and FTL Messaging Services

27 Feb, 2020 | 5 minutes read
| , ,

In this blog, we will describe how we can exchange message distribution between these quite used TIBCO provided service busses for storing and processing units of data for integration purposes. The reason for this communication came up as a proof-of-concept for one of our clients to migrate from an on-premise BusinessWorks 5.x to a BusinessWorks ContainerEdition cloud solution gradually, to investigate and evaluate the approach. The canonical messaging format and the Publish-Subscribe architecture in place should roughly be kept the same with minimal changes in the existing code and configuration. Initially, we have used this kind of scenario for the sole purpose of ensuring our clients if in doubt, that the migration from the on-premise to cloud-based services will be performed successfully and with minimal costs.

Prerequisites

We converted the BusinessWorks 5.x code into BWCE using our standard methodology and checklists. For the migration part read more here:

TIBCO Business Works 5.x to 6.x Migration

So we will create a simple process in TIBCO BWCE where we will publish a JMS message through a topic, which will then be exported to the appropriate endpoint in FTL and receive the message. Afterward, the BWCE process will send back a message through an FTL endpoint (imported to EMS queue) to the other side as a confirmation that the message is received and processed.

Configuring EMS and FTL

First, we need to create an Application on FTL server, where we can set all the necessary predefined values for message distribution like stores, durables, transports and the most likely format for structuring the incoming and outgoing messages and create the corresponding queue and topic on EMS server for sending and receiving messages.

FTL server configuration

We also need to create an FTL Format for structuring the message, which will appear later in the FTL activities when we are specifying the format.

FTL Format

The name of the format is ‘Example Format’ which contains two string fields. The field with the ‘_text’ name will be used for actual content and the second field ‘filteringProperty’ will be used for filtering the incoming or outgoing message, according to the value specified in the content matcher on FTL Subscriber.

EMS server configuration

From EMS point of view, we need to define queue and topic for import, export, and populate corresponding configuration files with appropriate data.

queue: import.test.queue

topic:  export.test.topic

tibemsd.conf

In this file we are enabling FTL transport, configuring appropriate FTL application, server, library path, authentication, logging, etc.

transports.conf

In this file, we are creating corresponding transports for message distribution.

Export configuration

In the Endpoint sections, we are using the same Endpoints specified in the FTL configuration and other than that, we are using additional property named ‘export_constant’ which depending on the values defined in the configuration itself will appear in the exported messages as an additional field.

Import configuration

Since we will use the same FTL Format for import (FTL-EMS) approach, we need to define content-matcher in ‘import_match_string’ configuration property, which will filter the imported messages from FTL (in this case I’m using raw string “false”)

After creating this transport, we need to add them to the desired topics(export) and queues (import) from the administration tool.

Implementation

There are several ways that we can distribute messages back and forth but we will stick to the simplest one, which requires fewer changes in the existing code.

Export (EMS to FTL)

First, we create a simple publisher process, which will export messages from EMS to FTL and configure JMSSendMessage activity with export.test.topic for exporting message, Text for Message Type, and add simple string value “Test Message” in the input of the activity. In the implementation itself, we assume that connections are accordingly configured and both servers are up and running.

  • Initial starter process that sends a JMS message:

Then we need to create a simple FTL Subscriber process, which will receive and parse the incoming message from EMS. We need to configure the FTL Subscriber with corresponding endpoint, durable, format, and content matcher which are previously defined in appropriate transport.

  • FTL Subscriber process

And since JMSSendMessage activity in JMS Publisher process is using “Text” for Message Type we need to set explicit name for one of the data types in FTL Format to parse the exported message, and in this case, we must name it “_text ” because in every other case FTL Subscriber will not recognize the incoming message.

Now we are ready to start the runtime on debug mode and test the flow.

Import (FTL to EMS)

On the other hand, for import message from FTL to EMS we need to use a custom XSD schema for JMSSubscriber activity, with appropriate data fields (_text and filteringProperty) because we cannot convert the message from FTL to EMS if we use the same configuration as we did in the previous approach.

  • In this case for JMS Subscriber for Message Type we will use Map value:

JMS Subscriber process:

  • In the FTL publisher process we are configuring FTL Publisher activity with “ExampleFormat” for the format, FTL_PUB_EndPoint for Endpoint and “false” in filteringProperty element since we are using that value in corresponding transport for filtering purposes.

FTL Publisher process:

After complete setup, this flow is ready to be tested in debug mode.

Advantages

  • Easy to configure on the EMS server-side because we are only using few properties.
  • We can use the same transports for multiple queues and topics.
  • Fast, secure and reliable migration to cloud-based services without creating a mediation code.
  • Minor code changes, and easy for testing and configuring against the existing code. This part may vary depending on the complexity of the on-premise messaging model.
  • Securing higher throughput and lower latency among message distribution than a mediation code.

Disadvantages

  • The export and import are limited with only one FTL App.
  • When we are importing messages from FTL to EMS, we must use Map for Message Type in JMS Receiver activity which will require custom XSD schema and potential code change.
  • If we need to filter messages in case of import we need to define custom ”import_match_string” property in transport configuration where FTL will create a single EMS publisher for specific EMS subscribers in the case where condition in content-matcher is fulfilled.
  • Having all transport configuration (which is sensitive data for business requirements) in only one file (transport.conf ) can be risky, and for utilization and security, it is better only a few members of the project to have access for updating and modifying this file on demand from the developers.

Conclusion

In both cases (import and export) we have described the most basic approach for message distribution between both messaging systems.

The drawback, in this case, is that in real life scenario there might be a situation where a lot of transports need to be created and configured for different interfaces, and we need to be careful with the values specified in the properties itself because in runtime there might be unpredictable results (back and forth) if we are not taking care of the values in the properties.

Nevertheless, the approach does not require a lot of code changes but requires us to create an appropriate transport configuration in transport.conf file according to the business logic and traffic requirements, and then reference those transports in corresponding queues and topics. It is also a quick and efficient way to evaluate the migration efforts and the new cloud platform, without affecting the functionalities of the existing on-premise applications.