Introduction
Event based system design is a viable solution for many technical use cases that require decoupling of individual components. Especially in GUI frameworks, components communicate with each other by means of sending and processing(receiving) events. Modeling system interaction in an event driven way decouples the different components and allows to introduce new behaviour into a system at runtime. It is not the goal of this article to explain the pattern in detail, nor will it give references to scenarios where eventing is applicable. Instead, in this article I will try to compare existing eventing solutions with respect to their offered features and performance.
My personal motivation to writing this article is that I created an event bus system that I recently released as an open source project. I created the MBassador library since none of the available solutions offered the functionality I needed and I felt that I wanted to give it a try. Since I was interested in how my solution compares to the other available libraries I started writing some test scenarios that I ran with every of the listed libraries using a general adapter for each.
The Candidates
Google Guava Event Bus
The Google guava library contains many very useful and powerful general purpose datastructures, most of them with concurrency support built-in. I really like the library a lot and have used the cache implementations frequently. Google Guava was my first choice until I discovered that the event bus uses strong references. Additionally, its feature list is quite small.
http://code.google.com/p/guava-libraries/
Tested version: 14.0-rc1
Simple Bus
As the name suggests, a very simple implementation of an event bus. The good thing is, that it uses weak references to the subscribed listeners such that one can simply forget about what was stuffed in. Event publication is very fast. It is the only bus that supports cancelation of event delivery (vetoing).
http://code.google.com/p/simpleeventbus/
Tested version: 1.2
EventBus
EventBus comes with a lot of features and has been around the longest (I guess). It is used in various projects and I expect it to be quite mature. Due to the number of features it offers it is quite slow.
Tested version: 1.4
MBassador
I proudly present you my first open source project. MBassador was designed with performance and ease of use in mind. It works well in concurrent environments and has generally a very high throughput. It uses weak references to the subscribed objects. It does support customzation using custom implementations of subscriptions and different dispatch strategies. Feature wise it roughly compares with EventBus but highly exceeds it in performance.
https://github.com/bennidi/mbassador
Tested version: 1.1.1
A brief feature comparison
The following table is a currently incomplete list of features offered by the different libraries. I only included the main features that I consider fundamental for an event bus solution. More exotic features offered by some of the products where excluded since the primary intend of this article is to compare the performance of the different products for the most common use cases.
|
Event Bus |
Listener declaration |
Synch. dispatch |
Asynch dispatch |
Filtering |
Event type hierarchy |
Multimode |
Reference Type |
Handler Priorities |
|
Google Guava |
Via annotations |
Yes |
Via specialized class only |
No |
? |
No |
Strong |
No |
|
SimpleBus |
Via annotations |
No |
Yes |
No |
? |
No |
Weak |
No |
|
EventBus |
Via annotations |
Yes |
Yes |
Static? |
Configurable |
? |
Both |
Yes |
|
MBassador |
Via annotations |
Yes |
Yes |
Static |
Yes |
Yes |
Weak |
Yes |
Note: I can not guarantee that this feature list is completely correct since I did not inspect all the solutions in much detail. As per the date of writing this article, the documentation for the EventBus was not available online. I am happy to include corrections and additions to this list, so feel free to comment.
Listener declaration = How are listeners defined? Is it non-invasive or does it affect the listeners class hierarchy?
Synchronous dispatch = Is synchronous event dispatching supported (publication method blocks until event is received be every handler). This is the most common mode of operation.
Asynchronous dispatch = Is asynchronous event dispatching supported (publication method returns immediately and event delivery is run asynchronously in a different thread).
Filtering = Does a mechanism for event filtering exist, such that handlers are not notified of every event that matches their parameter type?
Event type hierarchy = Is the type hierarchy of the published event considered during event delivery? Implementations that do support type hierarchies for events will deliver the event E to all handlers that accept E or one of its super types
Multimode = Is it possible to use the same bus to dispatch events either synchronously or asynchronously?
Reference Type = What kind of references are used to store subscribed listeners. Strong references require the client code to take care of proper deregistration of subscribed listeners or else a memory leak occurs.
Handler Priorities = Can the execution order of handlers be influenced by an ordering criteria?
The Test scenarios
For the performance tests I created various scenarios that isolate the different functionalities of the tested libraries. Scenarios are implemented as Runnable that take an implementation of the adapter interface. The adapter is the common interface I designed to make the basic functionalities of each library available in a standardized way and is a very thin wrapper around the method invocations of the actual event bus component. All tests are run single threaded and multi threaded – each table shows how many threads are used and the workload per thread. For each single-threaded test a multi-threaded one with the same overall workload is run to evaluate how concurrent access affects the components performance
Note: Since some of the event bus implementations use weak references and others not, a strong reference to each listener will be created before it is subscribed to the bus. Of course this impacts the overall performance but it does so equally for all tested implementations. Furthermore, the SimpleBus needed some extra treatment since it publishes events asynchronously and all tests of SimpleBus therefore need a bit of extra code to wait until event delivery is really finished (all handlers have actually been invoked)
The code of the performance comparison can be found here https://github.com/bennidi/eventbus-performance
Test machine: Samsung X22 Laptop (2.2 Ghz Intel Dual Core, 3 Gb ram)
Scenario 1: Subscribing/Unsubscribing listeners
This scenario tests the performance of subscribing and unsubscribing new listeners. No messages are published to the listeners. Different classes of listeners with different handler definitions are used. Not every subscribed listener is subsequently unsubscribed such that with each round the number of registered listeners grow. Each loop subscribes three different listeners and unsubscribes under certain conditions.
|
Subscriptions per Thread |
6000 |
|
Number of Threads |
1 |
|
Implementation |
Duration(ms) |
|
Google Guava |
453 |
|
SimpleBus |
1377 |
|
EventBus |
1866 |
|
MBassador |
112 |
|
Subscriptions per Thread |
18000 |
|
Number of Threads |
1 |
|
Implementation |
Duration(ms) |
|
Google Guava |
595 |
|
SimpleBus |
6734 |
|
EventBus |
12911 |
|
MBassador |
191 |
|
Subscriptions per Thread |
300 |
|
|
|
Number of Threads |
20 |
|
|
|
Implementation |
Avg(ms) |
Min(ms) |
Max(ms) |
|
Google Guava |
360 |
251 |
407 |
|
SimpleBus |
1705 |
1559 |
1777 |
|
EventBus |
2166 |
1966 |
2238 |
|
MBassador |
96 |
58 |
118 |
|
Subscriptions per Thread |
900 |
|
|
|
Number of Threads |
20 |
|
|
|
Implementation |
Avg(ms) |
Min(ms) |
Max(ms) |
|
Google Guava |
601 |
480 |
642 |
|
SimpleBus |
5789 |
4772 |
6140 |
|
EventBus |
13729 |
11897 |
14582 |
|
MBassador |
189 |
118 |
216 |
Scenario 2: Publishing events
This scenario tests the performance of event publication. Each test is setup with a number of listeners that are subscribed to the bus (not part of the measured performance). After the test is set up event publications will be issued to the bus. Since the message type hierarchy is not considered by all implementations, only the event class that is highest in the hierarchy will be used.
|
Events per Thread |
2000 |
|
Number of Threads |
1 |
|
Number of listeners |
6000 |
|
Implementation |
Duration(ms) |
|
Google Guava |
1322 |
|
SimpleBus |
17603 |
|
EventBus |
3078 |
|
MBassador |
228 |
|
Events per Thread |
100 |
|
|
|
Number of Threads |
20 |
|
|
|
Number of listeners |
6000 |
|
|
|
Avg(ms) |
Min(ms) |
Max(ms) |
|
|
Google Guava |
1038 |
883 |
1168 |
|
SimpleBus |
28334 |
27772 |
37753 |
|
EventBus |
6446 |
5931 |
6727 |
|
MBassador |
261 |
51 |
382 |
Scenario 3: Mixed usage
This scenario is designed as I would expect a lot of use cases to look like. Listeners are subscribed and/or unsubscribed while events get published.
|
Events per Thread |
4000 |
|
|
|
Number of Threads |
1 |
|
|
|
Subscriptions per Thread |
6000 |
|
|
|
Duration(ms) |
|
|
|
|
Google Guava |
2535 |
|
|
|
SimpleBus |
162259 |
|
|
|
EventBus |
4893 |
|
|
|
MBassador |
622 |
|
|
|
Events per Thread |
200 |
|
|
|
Number of Threads |
20 |
|
|
|
Subscriptions per Thread |
300 |
|
|
|
Implementation |
Avg(ms) |
Min(ms) |
Max(ms) |
|
Google Guava |
2257 |
2054 |
2387 |
|
SimpleBus |
31367 |
26477 |
33569 |
|
EventBus |
4510 |
4136 |
4670 |
|
MBassador |
761 |
626 |
831 |
|
Events per Thread |
600 |
|
|
|
Number of Threads |
20 |
|
|
|
Subscriptions per Thread |
900 |
|
|
|
Implementation |
Avg(ms) |
Min(ms) |
Max(ms) |
|
Google Guava |
12575 |
11905 |
12970 |
|
SimpleBus |
101030 |
87618 |
103493 |
|
EventBus |
34814 |
32038 |
38350 |
|
MBassador |
2497 |
2231 |
2660 |
|
Events per Thread |
1000 |
|
|
|
Number of Threads |
50 |
|
|
|
Subscriptions per Thread |
1500 |
|
|
|
Avg(ms) |
Min(ms) |
Max(ms) |
|
|
Google Guava |
396176 |
359027 |
403041 |
|
SimpleBus |
> 15 min. |
|
|
|
EventBus |
OutOfMemoryException |
|
|
|
MBassador |
32525 |
22746 |
34087 |
Results
The shown performance characteristics of the compared implementations indicate that
-
Listener subscription is an expensive operation for all implementations but MBassador and Guava
-
Concurrent access does generally slow down the bus performance because of higher contention/synchronization.
-
SimpleBus is by far the slowest implementation
-
MBassador is by far the fastest implementation in all scenarios. It also offers the best scaling characteristics meaning that higher concurrency rates do not slow down the bus performance as much as the others. This is because MBassador relies on a custom data structure with very fast write operations that do not block readers and at the same time do not copy existing data structures (most other implementations use CopyOnWriteArrayList).
To sum it up, MBassador is not only astonishingly fast but also offers a rich set of features that only EventBus can compare with. Google Guava's event bus implementation improved a lot from version 13 to 14 but is still clearly behind MBassador considering both performance and features. Especially in high concurrency/throughput scenarios MBassador is the clear winner when compared to Guavas.
I would go as far and say that MBassador is the best (Intra-JVM) event bus implementation that is currently available for the Java language. MBassador was the only implementation that did not produce exceptions during test execution (all other implementations threw OutOfMemory or other kinds of exceptions every once in a while). It is also quite low on resources (best memory footprint of all implementations). So, use it, file issues, request features!
I will keep on working on the code adding more unit tests and features. I am currently working on an integration component for the spring environment that supports conditional event dispatch triggered by transaction events (e.g. commit, rollback). For those who have been using EJB3 this will sound familiar and is quite a nice feature. A base implementation with proof of concept already exists and will be released soon. That's it for now. Any feedback is more than welcome.
Recent Comments