JDK Flight Recorder on foojay.io - Friends of OpenJDK

Micrometer & Prometheus in Spring Boot: Kafka Burger Orders

Fri, 14 Nov 2025 10:13:03 +0000

👨‍💻 GitHub: https://github.com/vinny59200/dukeburger

🔵⚪⚪⚪⚪⚪⚪⚪⚪⚪⚪⚪

This guide shows how to use Micrometer and Prometheus in Spring Boot to track a custom metric for a Kafka-driven Burger Orders app. You’ll post a burger order to a REST endpoint, publish it to Kafka, consume the topic, and increment a counter for all “DukeBurger” orders. Copy the snippets, run, and you’ll see your metric on /actuator/prometheus.

Remotely Recording a JFR Log from a Container (Without Using the Command Line)

Tue, 07 May 2024 04:32:29 +0000

Java Flight Recorder (JFR) is the go-to technology for recording and viewing JVM and system metrics.

JFR records JFR logs which reveal much about the running application, the health of the JVM and the stability of the system. JFR logs are easily obtained by simply going into the command line / terminal and entering a few commands.

But what if you don’t have access to the command line or terminal directly on the system where the JVM is running, like when the JVM is running in a container?

Do you trust profilers? I once did too!

Wed, 01 Mar 2023 13:14:06 +0000

Profilers are great tools in your toolbox, like debuggers, when solving problems with your Java application (I’ve been on a podcast on this topic recently).

I’ll tell you some of their problems and a technique to cope with them in this article.

There are many open-source profilers, most notably JFR/JMC, and async-profiler, that help you to find and fix performance problems.

But they are just software themselves, interwoven with a reasonably large project, the OpenJDK (or OpenJ9, for that matter), and thus suffer from the same problems as the typical problems of the applications they are used to profile:

JFR Timestamps and System.nanoTime Foojay.io

Wed, 08 Feb 2023 15:33:34 +0000

Did you ever wonder whether JFR timestamps use the same time source as System.nanoTime?

This is important when you have miscellaneous logging besides JFR events; otherwise, you would not be able to match JFR events and your logging properly.

We assume here that you use System.nanoTime and not less-suited timing information from System.currentTimeMillis().

The journey into this started with a question on the JDK Mission Control slack channel, which led me into a rabbit hole:

JFR Event Collection

Thu, 08 Dec 2022 10:08:52 +0000

Ever wondered what all the JDK Flight Recorder events are, in which JDK versions they are supported, and what examples of an event looks like?

Wonder no more, I created the JFR Event Collection website which contains all this and more.

This site gives you an up-to-date collection of all OpenJDK JFR events for every JDK since 11, giving you the following additional information:

configuration properties
fields with their types and description
examples from a renaissance benchmark run
with which GC this event appears
additional descriptions collected by JFR users

The idea for this website came during the development of my prototypical JFR UI:

Java Profiling Overview

Mon, 14 Nov 2022 09:14:25 +0000

When I ask Java developers whether they do profile, the answer is usually “no”.

The few that profiled before usually used VisualVM as a student and maybe JProfiler or YourKit years ago at work.

One of the reasons for this is a lack of available information and thus knowledge for everyday Java developers.

This is a pity as profiling should be a part of the tool belt for every experienced developer (not just for Java). The problem is that most of the open-source profilers are targeted to the OpenJDK developers (or their colleagues), even if they won’t admit it. This can be seen in the lack of entry-level material on this topic and even the little that is out there is distributed across multiple conference websites, blogs, YouTube channels, and Twitter accounts.

Thinking About Massive Throughput? Meet Virtual Threads!

Thu, 28 Apr 2022 07:41:25 +0000

The goal of this article is to examine known facts about an upcoming Java threading model extension.

No, no-worries, the current Java threading model remains but behind the curtains something good is already knocking on the virtual door.

Yes, we are talking about JEP-425: Virtual Threads.

Knocking on current concurrency limits

Let’s first look at the current Java threading model. It provides an implementation of the Thread class. A Thread can be considered a Java concurrency unit which can execute so-called Runnable tasks. The instance of a Thread class is also an object but there is a bit more happening behind the scenes.

Monitoring Spring Boot Applications (Part 1)

Wed, 12 Jan 2022 16:34:53 +0000

In part 1 of this series, we will learn about JDK Flight Recorder and how we can use it to monitor a Spring Boot application.

Introduction

Application monitoring is one thing that we as developers cannot overlook. It doesn’t matter if you run a monolith or microservices application because application monitoring is a must for production workloads to know what’s happening with your application. If you are using Spring Boot, there are different ways to monitor an application. The most popular way to monitor a Spring Boot application is to enable the spring-boot-actuator module to achieve the desired result.

Contributing to OpenJDK Mission Control

Mon, 18 Oct 2021 08:33:12 +0000

Since this month is Hacktoberfest , I thought it would be a good idea to talk a bit about how to contribute to the OpenJDK Mission Control project.

Some of the content of this article will be applicable to any of the OpenJDK projects, especially the Skara (OpenJDK on Git) bits.

The OpenJDK Mission Control Project

The OpenJDK Mission Control project is the observability tools suite for OpenJDK. It contains a JMX Console, a JFR visualizer and analyzer, a heap waste analysis tool, and many other little useful tools and utilities. Since it is all open source, pretty much anyone can contribute to the project.

On The Costs of Hidden Logging

Mon, 11 Oct 2021 08:05:33 +0000

A while ago I received a customer escalation ticket regarding performance degradation when using Datadog Continuous Profiler for Java . The degradation was observable as an increased CPU usage as well as unexpected latency.

The Beginning: Unusual Customer Escalation

To bootstrap the troubleshooting, I usually try to isolate the area which might be causing the regression.

The profiler is packaged as a Java agent and for ease of use it is bundled together with the Datadog Java tracer agent. The profiler itself uses JDK Flight Recorder (JFR) under the hood.

Announcement: JMC 8.0.1 Has Been Released!

Fri, 25 Jun 2021 18:10:26 +0000

The 8.0.1-ga tag was just set in the jmc8 repository on GitHub. This is a patch update release, and will therefore not include any new features.

The 8.0.1 release contains the following fixes:

Jira Issue	Summary
7188	JMC fails to dump file and gets stuck when flightrecording is attempted on jmxremote connection
7172	Fix spell mistake in secure store class
6920	UI improvements
6336	Remove Triple DES Cipher in Secure store
6398	Better JNDI Usage

It is up to individual vendors to release binaries of JMC 8.0.1, and some vendors already have binary builds available, for example Oracle .

OpenJDK and the Future of Production Profiling

Thu, 24 Jun 2021 08:46:28 +0000

Some thoughts on the future of continuous production profiling on the OpenJDK platform.

A long time ago, the JRockit Runtime Analyzer (JRA) was introduced into the JRockit JVM as a means of being able to figure out what was going on in the JVM. It was mainly there to find out how customers were using the JVM, so that the JVM could be optimized for actual real-world production work-loads. The JRA output the data as XML, since customers insisted on the data being human readable so that they could see exactly what they would be sending us. Later LAT (LATency Analyzer) was introduced, since after the introduction of the JRockit low latency garbage collector (a.k.a the DetGC), some customers complained about the GC not keeping its latency promises. More often than not, it turned out it was other kinds of thread stalls causing the latencies, so LAT was introduced so that the JRockit team could figure out where the problematic code was. Since there could be a considerable amount of data in LAT, a binary format was introduced for the events recorded.

Towards Continuous Performance Regression Testing

Thu, 25 Feb 2021 09:44:43 +0000

Functional unit and integration tests are a standard tool of any software development organization, helping not only to ensure correctness of newly implemented code, but also to identify regressions — bugs in existing functionality introduced by a code change. The situation looks different though when it comes to regressions related to non-functional requirements, in particular performance-related ones: How to detect increased response times in a web application? How to identify decreased throughput?

Improved JFR Allocation Profiling in JDK 16

Wed, 17 Feb 2021 09:46:33 +0000

JFR (JDK Flight Recorder) has been providing the support for on-the fly allocation profiling for a while with the help of ObjectAllocationInNewTLAB and ObjectAllocationOutsideTLAB events.

Short excursion - what is TLAB?

TLAB stands for Thread Local Allocation Buffer and it is a region inside Eden, which is exclusively assigned to a thread. Each thread has its own TLAB. Thanks to that, as long as objects are allocated in TLABs, there is no need for any type of synchronization. Allocation inside TLAB is a simple pointer bump (that’s why it’s sometimes called pointer bump allocation).

The SolarWinds Hack for Java Developers

Tue, 09 Feb 2021 07:39:53 +0000

In December 2020, an advanced persistent threat attacked many companies by injecting malicious code into a vendor application that belonged to SolarWinds. This technique is called a “supply chain attack,” because instead of targeting the victim directly, the attacker damaged something higher up the supply chain and simply waited. The US government uses SolarWinds, so by attacking SolarWinds in a way that would infiltrate the US government, the attacker effectively got their target.

How & When to Use JDK Flight Recorder in Production

Tue, 15 Dec 2020 12:44:28 +0000

While it would certainly be useful to record the whole lifetime, this is unpractical, even airplane Flight Data Recorders (and Cockpit Voice Recorders) only keep recent history. Instead, it’s possible to aim at specific time frames where a recording could be useful:

At startup, the JVM does a lot of things, so does the application, it generally initializes a lot of long lived objects, generally services, threads, etc.
Continuously at runtime , this is likely a sliding time window in which one can access what’s happened in last X <time unit>(this is either limited by age or by size). It this case the dump could be done when required.
At shutdown whether the JVM was killed or crashed. It this case the JFR files are an alternative to heap dumps for the autopsy.

For more details on this topic, see Continuous Production Profiling and Diagnostics , by Marcus Hirt.

Using Java Flight Recorder and Mission Control (Part 3)

Thu, 19 Nov 2020 09:13:02 +0000

Continuing from part 1 and part 2 , while from JDK 14 events can be consumed on the fly, previous JDK versions (from JDK 11) offer a public API useful enough to control Flight Recorder programmatically or to read events from a JFR file.

Configuration c = Configuration.getConfiguration("profile"); [1]
Recording r = new Recording(c);
r.setName("monitor jvm");
r.enable("jdk.*"); [2]
r.setMaxAge(java.time.Duration.ofMinutes(4)); [3]
r.start(); [4]

// to be profiled

r.stop(); [5]
r.dump(Files.createFile("/var/log/jfr/app-initiated.jfr")); [6]

As shown above, choose the JFR configuration.
Choose which events the recording should be interested in. Another signature accepts classes, it’s unlikely to be helpful for JDK events, but it may get interesting for custom events, your classes.
Eventually set recording constraints, like the maximum age of the records.
Hit record.
When the recording session is over, stop JFR.
Then store the results in the location of your choosing.

The above snippet creates a continuous profiling session with a 4 minute window.

Continuous Production Profiling and Diagnostics

Mon, 09 Nov 2020 15:57:38 +0000

I’ve gotten a lot of questions about continuous production profiling lately. Why would anyone want to profile in production, or, if production profiling seems reasonable, why the heck leave it on continuously? I thought I’d take a few moments and share my take on the problem and the success I’ve seen the past years applying continuous production profiling in systems in the real world.

Trigger warning: this blog will not contain code samples.

Using Java Flight Recorder and Mission Control (Part 2)

Thu, 05 Nov 2020 16:50:59 +0000

Now, continuing from part 1 , to exploit the recording by analyzing it, we have a tool named jfr that ships with the JDK. On Linux the alternative jdk management may not be aware of jfr, which means you may need to use the full path to this executable.

The first interesting thing to do is to get an overview of the recording, the summary sub-command displays an histogram of the events, shown below.

Using Java Flight Recorder and Mission Control (Part 1)

Mon, 02 Nov 2020 09:49:11 +0000

Java Flight Recorder is the profiler you can use in production, continuously. Flight Recorder has been available before in the JDK, e.g., it shipped as part of the JDK 8, but to use it, it required that you set specific commercial VM flags to unlock Flight Recorder, this is not anymore necessary from Java 11 onwards.

|—|———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————-| | | This page assumes JDK 11, that means there is no need to set -XX:+UnlockCommercialFeatures, -XX:+FlightRecorder flags. Or, more particular to JFR, the use of -XX:+UnlockDiagnosticVMOptions and -XX:+DebugNonSafepoints source |

JmFrX: A Bridge From JMX to Java Flight Recorder

Thu, 03 Sep 2020 09:24:29 +0000

I’m excited to share news about an open-source utility I’ve been working on lately: JmFrX , a tool for capturing JMX data with Java Flight Recorder.

When using JMX (Java Management Extensions ), the Java platform’s standard for monitoring and managing applications, JmFrX allows you to periodically record the attributes from any JMX MBean into Java Flight Recorder (JFR) files, which you then can analyse using JDK Mission Control (JMC).

Monitoring REST APIs with Custom Java Flight Recorder Events

Wed, 26 Aug 2020 08:36:57 +0000

The Java Flight Recorder (JFR) is an invaluable tool for gaining deep insights into the performance characteristics of Java applications. Open-sourced in JDK 11, JFR provides a low-overhead framework for collecting events from Java applications, the JVM and the operating system.

In this blog post, we’re going to explore how custom, application-specific JFR events can be used to monitor a REST API, allowing to track request counts, identify long-running requests and more. We’ll also discuss how the JFR Event Streaming API , new in Java 14, can be used to export live events, making them available for monitoring and alerting via tools such as Prometheus and Grafana.

A Closer Look at Java Flight Recorder Streaming

Mon, 17 Aug 2020 11:04:00 +0000

By Marcus Hirt and JP Bempel

Since JDK 14, there is a new Java Flight Recorder (JFR) kid on the block – JFR streaming. 🙂 This blog post, written together with JP Bempel, will discuss some of the things that you can do with JFR streaming, as well as some of the things you may want to avoid.

An Introduction to JFR Streaming

In the most recent version of the JDK, a new JFR-related feature was introduced – JFR streaming. It is a feature allowing a developer to subscribe to select JFR data and to decide what to do with that data in the host process. JFR events can also be consumed from a separate process by pointing to the file repo of a separate JVM process – the mechanism is the same.