Marko Topolnik on foojay.io - Friends of OpenJDK

Billion Events Per Second, Millisecond Latency: Giga Streaming Analytics

Thu, 25 Mar 2021 17:04:38 +0000

This post is a part of a series:

Part 1 (Intro and high-throughput streaming benchmark)
Part 2 (batch workload benchmark)
Part 3 (low-latency benchmark)
Part 4 (concurrent GC with green threads)
Part 5 (you are here)

We’re preparing a scientific paper on Hazelcast Jet, describing its architecture based on symmetric, data-local, non-blocking distributed event processing. As a part of this effort, we implemented the vendor-neutral NEXMark benchmark suite, consisting of 8 streaming queries that aim to capture typical kinds of questions you’re likely to ask about your real-time data.

Sub-10 ms Latency in Java: Concurrent GC with Green Threads

Wed, 05 Aug 2020 17:44:00 +0000

This post is a part of a series:

Part 1 (Intro and high-throughput streaming benchmark)
Part 2 (batch workload benchmark)
Part 3 (low-latency benchmark)
Part 4 (you are here)
Part 5 (billion events per second)

In Part 3 we showed that a modern JVM running live stream aggregation can achieve a 99.99% latency lower than 10 milliseconds. The focus of that post was comparing the different GC options available for the JVM. In order to maintain a level playing field, we kept to the default settings as much possible.

Performance of Modern Java on Heavy Workloads: Low-Latency Rematch

Tue, 23 Jun 2020 17:48:00 +0000

This post is a part of a series:

Part 1 (Intro and high-throughput streaming benchmark)
Part 2 (batch workload benchmark)
Part 3 (you are here)
Part 4 (concurrent GC with green threads)
Part 5 (billion events per second)

This is a followup on Part 1 of the blog post series we started earlier this month, analyzing the performance of modern JVMs on workloads that are relevant to the use case of real-time stream processing.

Performance of Modern Java on Data-Heavy Workloads: Batch Processing

Tue, 09 Jun 2020 14:45:00 +0000

This post is a part of a series:

Part 1 (Intro and high-throughput streaming benchmark)
Part 2 (you are here)
Part 3 (low-latency benchmark)
Part 4 (concurrent GC with green threads)
Part 5 (billion events per second)

Batch Pipeline Benchmark

A batch pipeline processes a finite amount of stored data. There are no running results, we need the output of the aggregate function applied to the entire dataset. This changes our performance requirements: the key factor in streaming, latency, doesn’t exist here since we are not processing data in real time. The only metric that matters is the total run time of the pipeline.

Performance of Modern Java on Workloads: Real-Time Streaming

Tue, 09 Jun 2020 01:45:00 +0000

This post is a part of a series:

Part 1 (you are here)
Part 2 (batch workload benchmark)
Part 3 (low-latency benchmark)
Part 4 (concurrent GC with green threads)
Part 5 (billion events per second)

The Java runtime has been evolving more rapidly in recent years and, after 15 years, we finally got a new default garbage collector: the G1. Two more GCs are on their way to production and are available as experimental features: Oracle’s ZGC and OpenJDK’s Shenandoah.