Jonathan Ellis on foojay.io - Friends of OpenJDK

New Java Benchmark for Coding LLMs puts GPT-5 at the top

Mon, 18 Aug 2025 08:06:40 +0000

The Brokk Power Ranking is a new open-source coding benchmark , featuring 93 tasks from large, real-world codebases. You can check out the current Power Ranking here .

SWE-bench is the closest thing we have to a standard, objective benchmark for LLM coding performance, but it has a bunch of issues, the largest of which are that it’s Python-only, and it’s old enough that almost certainly some labs are now training to the test. (Epoch AI has a great writeup on the more subtle problems with SWE-bench if you want to go deeper.)

Brokk: AI for Large (Java) Codebases

Fri, 30 May 2025 04:39:44 +0000

There are two reasons that AI makes mistakes writing code:

The LLM just isn’t smart enough to tackle the problem effectively, and it simply gets the answer wrong.
The AI doesn’t know enough about the relationships and dependencies of the code it’s editing, so (best case) it hallucinates APIs that don’t exist or (worse, because more subtle) it solves the problem in an awkward way that increases technical debt.

The first category is small, but it does still exist; e.g., concurrent data structures remain challenging for today’s models to get right the first time. AI labs will continue to make progress on this.

Indexing all of Wikipedia, on a laptop

Wed, 29 May 2024 15:54:32 +0000

In November, Cohere released a dataset containing all of Wikipedia , chunked and embedded to vectors with their multilingual-v3 model .

Computing this many embeddings yourself would cost in the neighborhood of $5000, so the public release of this dataset makes creating a semantic, vector-based index of Wikipedia practical for an individual for the first time.

Here’s what we’re building:

You can try searching the completed index on a public demo instance here .

Released: JVector 1.0

Mon, 02 Oct 2023 12:36:20 +0000

JVector is a pure Java embedded vector search engine that powers DataStax Astra and is being added to Apache Cassandra .

Vector search is a critical part of today’s generative AI applications , allowing developers to quickly retrieve the most relevant context to give the large language model enough information to answer accurately and without hallucinating, but innovation in this space has mostly happened outside the Java ecosystem. JVector gives enterprises an easy way to capitalize on their investment in the powerful Java platform, and gives Java developers a state-of-the-art solution that is easy to embed in their applications.

Why Pulsar Beats Kafka for a Scalable, Distributed Data Architecture

Thu, 03 Mar 2022 14:10:45 +0000

The leading open source event streaming platforms are Apache Kafka and Apache Pulsar. For enterprise architects and application developers, choosing the right event streaming approach is critical, as these technologies will help their apps scale up around data to support operations in production.

Everyone wants results faster. We want applications that know what we want, even before we know ourselves. We want systems that constantly check for fraud or security issues to protect our data. We want applications that are smart enough to react and change plans when faced with the unexpected. And we want those services to be continuously available.

The Critical Role Streaming Plays in a Data Stack

Fri, 18 Feb 2022 17:20:13 +0000

Why Apache Pulsar is the right choice for multi-datacenter, geo-distributed deployments.

Building applications today requires companies and teams to be familiar with data flows (including streaming data flows) and how to transform them into compelling user experiences. Data fuels companies’ ability to understand their customers, move faster than competitors, and provide compelling products that the market wants. According to research by DataStax and Clearpath Strategies, almost all companies (96%) now have a formal strategy for how they will work with data, but only 38% think they’re doing a good job using data in their applications.

Four Reasons Why Apache Pulsar is Essential to the Modern Data Stack

Tue, 01 Feb 2022 22:14:58 +0000

Messaging has been on DataStax’s radar for several years. A significant motivator for this is the increasing popularity of microservices-based architectures. Briefly, microservices architectures use a message bus to decouple communication between services and to simplify replay, error handling, and load spikes.

With Apache Cassandra™ and DataStax Astra Cassandra-as-a-service, developers and architects have a database ecosystem that is:

Based on open source
Well-suited for hybrid- and multi-cloud deployments
Available in a cloud-native, consumption-priced service

There is no current messaging solution that satisfies these requirements, so we’re building one. We started by evaluating the most popular option, Apache Kafka. We found that it came up short in four areas:

Apache Cassandra 4.0: Taming Tail Latencies with Java 16 ZGC

Tue, 22 Jun 2021 07:45:40 +0000

With Apache Cassandra 4.0, you not only get the direct improvements to performance added by the Apache Cassandra committers, you also unlock the ability to take advantage of seven years of improvements in the JVM itself. This article focuses on improvements in Java garbage collection that Cassandra 4.0 coupled with Java 16 offers over Cassandra 3.11 on Java 8.

Like so many others in the Apache Cassandra community, I’m extremely excited to see that the 4.0 release is finally here. There are many, many improvements to Cassandra 4.0 . One enhancement that is more important than it might look is the addition of support for Java versions 9 and up. This was not trivial, because Java 9 made changes to some internal APIs that the most performance-oriented Java projects like Cassandra relied on (you can read more about this here ).