Fawaz Ghali on foojay.io - Friends of OpenJDK

Building Real-Time Applications to Process Wikimedia Streams

Fri, 15 Sep 2023 14:33:16 +0000

In this tutorial, developers, solution architects, and data engineers can learn how to build high-performance, scalable, and fault-tolerant applications that react to real-time data using Kafka and Hazelcast.

We will be using Wikimedia as a real-time data source. Wikimedia provides various streams and APIs (Application Programming Interfaces) to access real-time data about edits and changes made to their projects.

For example, this source provides a continuous stream of updates on recent changes, such as new edits or additions to Wikipedia articles. Developers and solution architects often use such streams to monitor and analyze the activity on Wikimedia projects in real-time or to build applications that rely on this data, like this tutorial.

Real-time Stream Processing with Hazelcast and Redpanda

Tue, 25 Jul 2023 09:33:27 +0000

In this tutorial, we explore the powerful combination of Hazelcast and Redpanda to build high-performance, scalable, and fault-tolerant applications that react to real-time data.

Redpanda is a streaming data platform designed to handle high-throughput, real-time data streams. Compatible with Kafka APIs, Redpanda provides a highly performant and scalable alternative to Apache Kafka. Redpanda’s unique architecture enables it to handle millions of messages per second while ensuring low latency, fault tolerance, and seamless scalability.

Boosting Similarity Search with Real-time Stream Processing

Tue, 30 May 2023 12:42:35 +0000

The goal of similarity search and vector databases is to find similar results to the search query for unstructured data, such as text, images and videos. The unstructured data first is vectorised, and stored in a vector format. There are publicly available tools to create vectors from unstructured data, similarly, there are vector databases to store and perform similarity searches. This is important because of the rising popularity of Large Language Models (LLMs) and their combination with vector databases.

Enriching Kafka Applications with Contextual Data

Thu, 18 May 2023 15:11:22 +0000

Developing high-performance large-stream processing applications is a challenging task.

Choosing the right tool(s) is crucial to get the job done; as developers, we tend to focus on performance, simplicity, and cost.

However, the cost becomes relatively high if we end up with two or more tools to do the same task.

Simply put, you need to multiply development time, deployment time, and maintenance costs by the number of tools.

Kafka

Kafka is great for event streaming architectures, continuous data integration (ETL), and messaging systems of record (database).

How to Build and Deploy a Real-time Cloud-based Logging System

Fri, 24 Mar 2023 15:04:51 +0000

Logs and traces generated by applications are valuable sources of information that can help detect issues and improve performance.

However, they are often treated separately from other data, even though they are no different from the data an application works with.

In this tutorial, we will explore a different approach: treating logs and traces as part of a scalable cloud storage repository that can be analysed with the same techniques used for big data.

Failover Client with the Hazelcast Viridian Serverless

Fri, 24 Mar 2023 14:55:42 +0000

Failover is an important feature of systems that rely on near-constant availability.

In Hazelcast, a failover client automatically redirects its traffic to a secondary cluster when the client cannot connect to the primary cluster.

Consider using a failover client with WAN replication as part of your disaster recovery strategy.

In this tutorial, you’ll update the code in a Java client to automatically connect to a secondary, failover cluster if it cannot connect to its original, primary cluster.

How to Get Started with the Hazelcast Viridian Serverless

Fri, 10 Mar 2023 10:28:45 +0000

In this tutorial, you’ll learn how to build real-time streaming applications with the Hazelcast Viridian Serverless using SQL.

You’ll also learn how to build a materialized view from streaming data and contextual data in Hazelcast.

Streaming data is data that is continuously generated in small sizes.

Streaming data includes a wide variety of sources such as retail purchases, financial trades, or telemetry from connected vehicles.

This data isn’t usually useful by itself because of its size.

How to Get Started with the Hazelcast Viridian Serverless

Wed, 22 Feb 2023 08:13:32 +0000

Hazelcast Serverless means that Hazelcast manages your Cloud infrastructure for you.

Each Viridian Serverless cluster is an independent deployment of Hazelcast Platform in a Kubernetes container.

This design guarantees resource isolation, prevents resource stealing, and provides isolated network access.

Viridian Serverless clusters come in two types:1) Development: Capped storage.

You can store at most 1 GiB of data; and 2) Production: Uncapped storage.

The cluster scales as you add or remove data. In this tutorial, you’ll learn how to connect a client to a cluster and use SQL to query data in the cluster.

Announcing Real-time Stream Processing Unconference

Tue, 07 Feb 2023 07:20:06 +0000

I’m happy to share Hazelcast will be kicking off 2023 by hosting #RTSPUnconf to connect with community members and industry experts on the Future of Real-Time Stream Processing.

Interested in joining our Stream Processing Fundamentals workshop in-person at CodeNode in London? Connect with other developers and receive a digital badge.

Sign-up here: https://hazelcast.com/lp/unconference/

Why Attend:

Gain valuable insight into the future of real-time stream processing from industry experts and peers
Network and connect with like-minded professionals in the community through networking opportunities
To learn and enhance your skills in building real-time stream processing applications
Have an amazing time with new and old friends from the community!

Are you interested in speaking or joining the Real-Time Stream Processing Roundtable?

Real-time Stream Processing with Hazelcast and StreamNative

Wed, 01 Feb 2023 09:09:04 +0000

Introduction

One of the most useful features of real-time stream processing is to combine the strengths and advantages of various technologies to provide a unique developer experience and an efficient way of processing data in real time at scale.

Hazelcast is a real-time distributed computation and storage platform for consistently low latency queries, aggregation and stateful computation against real-time event streams and traditional data sources.
Apache Pulsar is a real-time multitenant geo-replicated distributed pub-sub messaging and streaming platform for real-time workloads handling millions of events per hour.

However, real-time stream processing is not an easy task, especially when combining multiple live streams with large volumes of data stored in external data storages to provide context and instant results.