In recent years the Complex Event Processing (CEP) applications market has been moving more and more toward Event Stream Processing (ESP). ESP is focused on processing continuous streams of events (like a stock market feed or IoT sensor data). Working more like a database turned upside down where queries are stored and stream data is run through to get results.
With its product BusinessEvents, TIBCO has been present in the CEP market since 2004, making the company one of the oldest and at the same time one of the strongest commercial vendors in this ever-growing market. BusinessEvents (BE) is an Inference rule system-oriented CEP (comes with a Rule Engine), meaning that the action is triggered when certain states are reached. Additionally BE has a closely integrated data grid (ActiveSpaces). Starting from 2009, BE also supports query-based features with its extension Event Stream Processing. Without going into further details I can state that BE covers most of CEP’s functional features (like filtering, event modeling, transformation, pattern detection, context awareness, machine-learning capabilities, and integrated IDE…) making it one of the most complete CEP platforms in terms of features.
In 2013 TIBCO went a step further in building its CEP capabilities by acquiring StreamBase Systems. StreamBase is a software company founded in 2003 and its main CEP product is StreamBase built on top of the Aurora MIT project. StreamBase has been one of the major players in the ESP market for years, being the number one CEP provider in the Capital Markets for over a decade.
TIBCO considers the StreamBase CEP platform as well as LiveView (second StreamBase product) as key components in its Fast Data platform. Thus, being able to integrate the StreamBase products in his existing BE CEP architecture very smoothly:
Therefore now TIBCO isn’t offering two same CEP platforms (although they do have overlapping features), but it is able to provide architecture that integrates both. If offered as separate products they can cover different CEP architectures: rule-based and stateful event processing architecture (BusinessEvents) vs event stream processing architecture (StreamBase).
Tibco StreamBase as a computing platform is designed specifically to meet the performance requirements of high-volume, real-time streaming applications. To accomplish this, StreamBase processes the inbound data (stream) while it is in flight as it streams through the server, continuously looking for patterns, and being able to act in real-time. The platform has three main parts (Figure 2):
- StreamBase Studio
- StreamBase Server
- Adapters and Handlers
TIBCO StreamBase Studio
It is a powerful Eclipse-based integrated development environment (IDE), which enables developers, quantitative analysts, and others to collaborate in designing and implementing rich applications for processing real-time data. One of the main features of StreamBase Studio is the unique graphical dataflow language – EventFlow based on a stream-relational model. A sample of the language and the IDE is shown in Figure 3. EventFlow language is the key component for Agile Development using StreamBase studio. Event-based applications can be developed and deployed 2-5 times faster than with traditional programming languages.
The language is not designed for end users but for people with capabilities in the software engineering space. The code can be separated into different modules (components) that can be developed/designed by team members with different skills (data scientists, business analysts, developers, quantitative analysts, etc.). All with the goal of supporting the smooth creation of very complex and large applications.
StreamBase Studio provides some unique features for a visual design tool, like:
- Very rich in options Feed Simulation for testing the flow (Generating sample event streams)
- Visually managing applications in source control (Merge/Diff can be done in the visual editor)
- Application features can be extended with Java, and debugging can be done seamlessly between the visual debugger and the Java code
Besides graphical, the Studio supports text-based application development using StreamSQL query language stored in text files with a .ssql extension. In the beginning, StreamSQL had all the features of EventFlow, but recent features in EventFlow have not been added to StreamSQL.
The trial version of the studio can be downloaded here.
Online documentation can be found here, as well as in the studio itself, currently the online doc is not provided for the latest release of the IDE (10.0)
There are 10 built-in and well-documented demo applications that come with the studio. Also, there are 150 sample applications (some of them with as many as 20 different flows) separated into 12 different categories, covering almost all features of the language. All this makes the TIBCO StreamBase Studio very approachable for learning.
StreamBase Studio offers tools for all stages of the Development Lifecycle: Discovery/Specification, Architecture, Development, Testing and Deployment. StreamBase Server.
Is the deployment server of the TIBCO StreamBase event processing platform. This is an ultra-low latency high-throughput application server optimized for processing real-time streaming event data. It enables deployments with minimum tuning effort versus production feeds. Main features:
- Performance
- Low latency, in the scale of microseconds
- High throughput, as much as millions of messages per second
- Scalability, multi-threaded, and clustered server. Enables scaling horizontally as well as vertically
- High Availability and Fault Tolerance
- Management and Monitoring Tools
- Security and Entitlements Integration
- Continues Deployment
- Able to use High-Performance Hardware
- Multicore (24, 32 and more cores support)
- Large memory (10s of gigabytes)
- 64-bit Linux, Windows, Solaris deployment
- Hardware acceleration (GPU, Solace, Tervela)
Adapters and Handlers
These refer to the StreamBase third-party integrations and pre-built connectivity. Making a collection of over 150 adapters and visualization options for a variety of real-time data feeds and messaging systems, high-capacity data stores, real-time interactive dashboards, and user front-ends. Additionally, StreamBase offers extensive APIs for building custom Adapters. StreamBase can be used not only for connecting to different real-time data but also to be integrated with preexisting analytics and run them vs the data streams.
Some StreamBase adapters are for connecting to Apache Kafka, Apache Hadoop Distributed File System, Apache Cassandra, IBM MQ Series, Informatica Ultra Messaging, JMS, MQTT, Rabbit MQ, TIBCO RV, UDP, Wall Street Systems, Microsoft Excel. A list of all StreamBase embedded adapters can be found here.
In conclusion, TIBCO StreamBase offers an amazing visual development experience, with great support for simulation event streams, back-testing, debugging, etc. It has been mostly used for algorithmic trading but can be great for fraud detection, sensor data analytics, and monitoring of different types.
