Raphael De Lio on foojay.io - Friends of OpenJDK

Semantic Caching with SpringBoot & Redis

Thu, 07 Aug 2025 06:58:07 +0000

TL;DR: You’re building a semantic caching system using Spring AI and Redis to improve LLM application performance.

Unlike traditional caching that requires exact query matches, semantic caching understands the meaning behind queries and can return cached responses for semantically similar questions.

It works by storing query-response pairs as vector embeddings in Redis, allowing your application to retrieve cached answers for similar questions without calling the expensive LLM, reducing both latency and costs.

Agent Memory with Spring AI & Redis

Wed, 23 Jul 2025 16:34:04 +0000

You’re building an AI agent with memory using Spring AI and Redis. Unlike traditional chatbots that forget previous interactions, memory-enabled agents can recall past conversations and facts. It works by storing two types of memory in Redis: short-term (conversation history) and long-term (facts and experiences as vectors), allowing agents to provide personalized, context-aware responses.

LLMs respond to each message in isolation, treating every interaction as if it’s the first time they’ve spoken with a user. They lack the ability to remember previous conversations, preferences, or important facts.

How I Improved Zero-Shot Classification in Deep Java Library (DJL) OSS: Enhancing Machine Learning Model Translators

Wed, 09 Jul 2025 10:51:16 +0000

Did you know the Deep Java Library (DJL) powers Spring AI and Redis OM Spring? DJL helps you run machine learning models right inside your Java applications.

Check them out:

Spring AI with DJL: https://docs.spring.io/spring-ai/reference/api/embeddings/onnx.html

Semantic Search with SpringBoot & Redis: https://foojay.io/today/semantic-search-with-spring-boot-redis/

TL;DR:

You’re doing zero-shot classification in a Java app using DJL.
DJL didn’t handle some models well — like DeBERTa. It missed support for token_type_ids, assumed wrong label positions, and oversimplified the softmax implementation.
It was fixed by reading the model config files and adjusting DJL’s translator logic.
Now DJL gives correct results across different models — just like the Transformers library does in Python.
The fix is merged and will probably be released with version 0.34.0.

What’s Zero-Shot Classification (and Why It Matters)

Zero-shot classification is a machine learning technique that allows models to classify text into categories they haven’t explicitly seen during training. Unlike traditional classification models that can only predict classes they were trained on, zero-shot classifiers can generalize to new, unseen categories.

How to send prompts in bulk with Spring AI and Virtual Threads

Wed, 21 May 2025 09:18:45 +0000

TL;DR: You’re building an AI-powered app that needs to send lots of prompts to OpenAI.

Instead of sending them one by one, you want to do it in bulk — efficiently and safely.

This is how you can use Spring AI with Java Virtual Threads to process hundreds of prompts in parallel.

When calling LLM APIs like OpenAI, you’re dealing with a high-latency, network-bound task. Normally, doing that in a loop slows you down and blocks threads. But with Spring AI and Java 21 Virtual Threads, you can fire off hundreds of requests in parallel without killing your app.

Semantic Search with Spring Boot & Redis

Mon, 05 May 2025 15:35:55 +0000

TL;DR: You’re building a semantic search app using Spring Boot and Redis. Instead of matching exact words, semantic search finds meaning using Vector Similarity Search (VSS). It works by turning movie synopses into vectors with embedding models, storing them in Redis (as a vector database), and finding the closest matches to user queries.

A traditional searching system works by matching the words a user types with the words stored in a database or document collection. It usually looks for exact or partial matches without understanding the meaning behind the words.

Sliding Window Counter Rate Limiter (Redis & Java)

Tue, 25 Feb 2025 21:21:11 +0000

This content is also available on YouTube. Check it out!

The Sliding Window Counter offers a more efficient way to handle rate limiting compared to the Sliding Window Log. While the Sliding Window Log keeps an exact log of timestamps for each request, allowing precise tracking over a rolling time period, this precision comes at the cost of higher memory usage and computational overhead as the number of requests increases.

Sliding Window Log Rate Limiter (Redis & Java)

Tue, 04 Feb 2025 12:21:56 +0000

This article is also available on YouTube. Check it out!

The Sliding Window Log is a more precise way to handle rate limiting. Instead of splitting time into fixed intervals like the Fixed Window Counter , it keeps a log of timestamps for each request. This allows it to track requests over a rolling time period, like the last second or minute, without abrupt resets at the end of an interval.

Token Bucket Rate Limiter (Redis & Java) - Raphael De Lio

Tue, 28 Jan 2025 07:07:08 +0000

This article is also available on YouTube!

The Token Bucket algorithm is a flexible and efficient rate-limiting mechanism.

It works by filling a bucket with tokens at a fixed rate (e.g., one token per second).

Each request consumes a token, and if no tokens are available, the request is rejected.

The bucket has a maximum capacity, so it can handle bursts of traffic as long as the burst doesn’t exceed the number of tokens in the bucket.

Fixed Window Counter Rate Limiter (Redis & Java)

Mon, 20 Jan 2025 16:34:18 +0000

This article is also available on YouTube! The Fixed Window Counter is the simplest and most straightforward rate-limiting algorithm. It divides time into fixed intervals (e.g., seconds, minutes, or hours) and counts the number of requests within each interval. If the count exceeds a predefined threshold, the requests are rejected until the next interval begins. Looking for a more precise algorithm? Take a look at the Sliding Window Log implementation .

Rate limiting with Redis: An essential guide

Mon, 13 Jan 2025 18:23:26 +0000

This article is also available on YouTube! Rate limiting — it’s something you’ve likely encountered, even if you haven’t directly implemented one. For example, have you ever been greeted by a “429 Too Many Requests” error? That’s a rate limiter in action, protecting a resource from overload. Or maybe you’ve used a service with explicit request quotas based on your payment tier — same concept, just more transparent. Rate limiting isn’t just about setting limits; it serves a variety of purposes. Take Figma, for instance. Their rate limiter, built with Redis, saved them from a spam attack where bad actors sent massive document invitations to random email addresses. Without it, Figma could have faced skyrocketing email delivery costs and damaged reputation. Or look at Stripe: as their platform grew, they realized they couldn’t just throw more infrastructure at the problem. They needed a smarter solution to prevent resource monopolization by misconfigured scripts or bad actors. These stories show just how versatile rate limiting is. It prevents abuse, ensures fair access, manages load, cuts costs, and even protects against downtime. But here’s the kicker: the hard part isn’t knowing why you need a rate limiter. The real challenge is building one that’s both efficient and tailored to your needs.