In recent years the Complex Event Processing (CEP) applications market has been moving more and more toward Event Stream Processing (ESP). ESP is focused on processing continuous streams of events (like a stock market feed or IoT sensors data). Working more like a database turned upside down where queries are stored and stream data is run through to get results.
With its product BusinessEvents, TIBCO has been present in the CEP market since 2004, making the company one of the oldest and at the same time one of the strongest commercial vendors in this ever growing market. BusinessEvents (BE) is Inference rule system oriented CEP (comes with a Rule Engine), meaning that the action is triggered when certain states are reached. Additionally BE has a closely integrated data grid (ActiveSpaces). Starting from 2009, BE also supports query-based features with its extension Event Stream Processing. Without going into further details I can state that BE covers most of CEP functional features (like: filtering, event modeling, transformation, pattern detection, context awareness, machine-learning capabilities, integrated IDE…) making it one of the most complete CEP platforms in terms of features.
In 2013 TIBCO went a step further in building its CEP capabilities by acquiring StreamBase Systems. StreamBase is a software company founded in 2003 and its main CEP product is StreamBase built on top of Aurora MIT project. StreamBase is one of the major players in the ESP market for years, being number one CEP provider in the Capital Markets for over a decade.
TIBCO considers StreamBase CEP platform as well as LiveView (second StreamBase product) as key components in its Fast Data platform. Thus, being able to integrate the StreamBase products in his existing BE CEP architecture very smoothly:
Figure 1. Tibco Event Processing in the Fast Data architecture
Therefore now TIBCO isn’t offering two same CEP platforms (although they do have overlapping features), but it is able to provide architecture that is integrating both. If offered as separate products they can cover different CEP architectures: rule based and stateful event processing architecture (BusinessEvents) vs event stream processing architecture (StreamBase).
Tibco StreamBase as a computing platform is designed specifically to meet the performance requirements of high volume, real time streaming applications. To accomplish this, StreamBase processes the inbound data (stream) while it is in flight as it streams through the server, continuously looking for patterns, and being able to act in real-time. The platform has three main parts (Figure 2):
- StreamBase Studio
- StreamBase Server
- Adapters and Handlers
Figure 2. Tibco StreamBase components
TIBCO StreamBase Studio
It is a powerful Eclipse-based integrated development environment (IDE), which enables developers, quantitative analysts and others to collaborate in designing and implementing rich applications for processing real time data. One of the main features of StreamBase Studio is the unique graphical dataflow language – EventFlow based on a stream-relational model. Sample of the language and the IDE is shown on Figure 3. EventFlow language is the key component for Agile Development using StreamBase studio. Event based applications can be developed and deployed 2-5 times faster than with traditional programing languages.
The language is not designed for end users but for people with capabilities in software engineering space. But the code can be separated in different modules (components) that can be developed/designed by team members with different skills (data scientists, business analysts, developers, quantitative analysts etc.). All with the goal to support smooth creation of very complex and large applications.
Figure 3. Tibco StreamBase Studio 10.0 IDE
StreamBase Studio provides some unique features for a visual design tool, like:
- Very rich in options Feed Simulation for testing the flow (Generating sample event streams)
- Visually managing applications in source control (Merge/Diff can be done in the visual editor)
- Application features can be extended with Java, and debug can be done seamlessly between the visual debugger and the Java code
Besides graphical, the Studio supports text-based application development using StreamSQL query language stored in text files with an .ssql extension. In the beginning StreamSQL had all features as EventFlow, but recent features in EventFlow have not been added to StreamSQL.
Trial version of the studio can be downloaded here.
Online documentation can be found here, as well as in the studio itself, currently the online doc is not provided for the latest release of the IDE (10.0)
There are 10 built in and well documented demo applications that come with the studio. Also there are 150 sample applications (some of them with as much as 20 different flows) separated in 12 different categories, covering almost all features of the language. All this makes the TIBCO StreamBase Studio very approachable for learning.
StreamBase Studio offers tools for all stages of the Development Lifecycle: Discovery/Specification, Architecture, Development, Testing and Deployment. StreamBase Server
Is the deployment server of the TIBCO StreamBase event processing platform. This is an ultra-low latency high-throughput application server optimized for processing real time streaming event data. It enables deployments with minimum tuning effort versus production feeds. Main features:
- Performance
- Low-latency, in scale of microseconds
- High throughput, as much as millions of messages per second
- Scalability, multi-threaded and clustered server. Enables scaling horizontally as well as vertically
- High Availability and Fault Tolerance
- Management and Monitoring Tools
- Security and Entitlements Integration
- Continues Deployment
- Able to use High Performance Hardware
- Multicore (24, 32 and more cores support)
- Large memory (10s of gigabytes)
- 64-bit Linux, Windows, Solaris deployment
- Hardware acceleration (GPU, Solace, Tervela)
Adapters and Handlers
These refer to the StreamBase third party integrations and pre built connectivity. Making a collection of over 150 adapters and visualization options to a variety of real-time data feeds and messaging systems, high-capacity data stores, real-time interactive dashboards, and user front-ends. Additionally StreamBase offers extensive APIs for building custom Adapters. StreamBase can be used not only for connecting to different real time data but also to be integrated with preexisting analytics and run them vs the data streams.
Some of StreamBase adapters are for connecting to: Apache Kafka, Apache Hadoop Distributed File System, Apache Cassandra, IBM MQ Series, Informatica Ultra Messaging, JMS, MQTT, Rabbit MQ, TIBCO RV, UDP, Wall Street Systems, Microsoft Excel. A list of all StreamBase embedded adapters can be found here.
In conclusion TIBCO StreamBase offers an amazing visual development experience, with great support for simulation event streams, back-testing, debugging, ect. It has been mostly used for algorithmic trading, but can be great for fraud detection, sensor data analytics, and monitoring of different types.
Author
