DataStax on foojay.io - Friends of OpenJDK

Foojay Podcast #56: Vectors in Java Code, Databases, and LLMs

Mon, 08 Jul 2024 07:33:28 +0000

In this Foojay podcast, we enter the world of mathematics by discussing Vectors and how they are crucial for AI and machine learning.

As ChatGPT explains: “A Vector is a mathematical structure that holds numerical values. Vectors are fundamental to the field of Artificial Intelligence, as they allow mathematical operations to be performed efficiently and form the basis of many machine learning algorithms.”

OK, but… how are these vectors crucial for the whole Artificial Intelligence evolution?

Indexing all of Wikipedia, on a laptop

Wed, 29 May 2024 15:54:32 +0000

In November, Cohere released a dataset containing all of Wikipedia , chunked and embedded to vectors with their multilingual-v3 model .

Computing this many embeddings yourself would cost in the neighborhood of $5000, so the public release of this dataset makes creating a semantic, vector-based index of Wikipedia practical for an individual for the first time.

Here’s what we’re building:

You can try searching the completed index on a public demo instance here .

Unified Event-Driven Architecture for the Cloud Native Enterprise

Thu, 06 Oct 2022 19:58:06 +0000

CTOs and enterprise architects have long recognized the importance of event-driven architectures (EDA). While once considered purely a technology concern, the foresight of organizations that have invested in EDA has become readily apparent as the world has shifted around us. In the past decade, we’ve witnessed changes in nearly every aspect of our technological worlds, and the vast majority of those have been affected in some way by a move toward event-driven, real-time processing.

Reclaiming Persistent Volumes in Kubernetes

Wed, 28 Sep 2022 09:55:19 +0000

Kubernetes is a widely used open-source container management platform for running stateless, containerized applications at scale. In recent years, Kubernetes has been extended to also support stateful workloads, including databases and key-value stores.{#43de}

There are three important API resources when it comes to managing stateful applications in Kubernetes:{#8e8a}

STSs schedule stateful pods, which can claim PVs through PVCs and mount them as volumes. Once a PV is claimed by an STS replica, Kubernetes will make sure that the volume stays with the replica, even if the pod gets rescheduled.{#e855}

How the world caught up with Apache Cassandra

Thu, 15 Sep 2022 16:50:50 +0000

The O’Reilly book, Cassandra: The Definitive Guide, features a quote from Ray Kurzweil, the noted inventor and futurist: {#speakable-summary}

“An invention has to make sense in the world in which it is finished, not the world in which it is started.”

This quote has a prophetic ring to it, especially considering my co-author Eben Hewitt included it in the 2010 first edition of this book we wrote, back when Apache Cassandra, the open-source, distributed, and highly scalable NoSQL database, was just on its 0.7 release.

Baeldung Series Part 2: Build a Dashboard With Cassandra, Astra and CQL – Mapping Event Data

Tue, 13 Sep 2022 17:46:02 +0000

1. Introduction

In our previous article , we looked at augmenting our dashboard to store and display individual events from the Avengers using DataStax Astra , a serverless DBaaS powered by Apache Cassandra using Stargate to offer additional APIs for working with it.

In this article, we will be making use of the exact same data in a different way. We are going to allow the user to select which of the Avengers to display, the time period of interest, and then display these events on an interactive map. Unlike in the previous article, this will allow the user to see the data interacting with each other in both geography and time.

Multi-cluster Cassandra with Google Kubernetes Engine (Pt. 2)

Thu, 08 Sep 2022 17:20:08 +0000

This is the second in a series of posts examining patterns for using K8ssandra to create Cassandra clusters with different deployment topologies.

In the first article in this series, we looked at how you could create a Cassandra cluster with two datacenters in a single cloud region, using separate Kubernetes namespaces in order to isolate workloads. For example, you might want to create a secondary Cassandra datacenter to isolate a read-heavy analytics workload from the datacenter supporting your main application.

Running your Database on OpenShift and CodeReady Containers

Tue, 06 Sep 2022 09:43:57 +0000

Let’s take an introductory run-through of setting up your database on OpenShift, using your own hardware and RedHat’s CodeReady Containers.

CodeReady Containers is a great way to run OpenShift K8s locally, ideal for development and testing. The steps in this blog post will require a machine, laptop or desktop, of decent capability; preferably quad CPUs and 16GB+ RAM.

Download and Install RedHat’s CodeReady Containers

Download and install RedHat’s CodeReady Containers as described in Red Hat OpenShift 4 on your laptop: Introducing Red Hat CodeReady Containers

Deploy a Multi-Datacenter Cassandra Cluster in Kubernetes

Wed, 10 Aug 2022 08:27:59 +0000

The Get Started examples on the K8ssandra site are primarily concerned with spinning up a single Apache Cassandra datacenter in a single Kubernetes cluster.

However, there are many situations that can benefit from other deployment options.

In this series of articles, we’ll examine different deployment patterns and show how to implement them using K8ssandra.

Flexible topologies with Cassandra

From its earliest days, Cassandra has included the ability to assign nodes to datacenters and racks. A rack was originally conceived as mapping to a single rack of servers connected to shared resources, like power, network, and cooling.

Five Data Models for IoT: Managing the Latest IoT Events Based on a State in Apache Cassandra

Thu, 28 Jul 2022 17:36:28 +0000

Apache Cassandra is a rock-solid choice for managing IoT and time series data at scale. The most popular use case of storing, querying and analyzing time series generated by IoT devices in Cassandra is well-understood and documented. In general, a time series is stored and queried based on its source IoT device. However, there exists another class of IoT applications that require quick access to the most recent data generated by a collection of IoT devices based on a known state. The question that such applications need to answer is: Which IoT devices or sensors are currently reporting a specific state? In this blog post, we focus on this question and provide five possible data modeling solutions to efficiently answer it in Cassandra.

NoSQL Use Cases: When to Use a Non-Relational Database

Thu, 28 Jul 2022 07:32:07 +0000

For decades, many companies have relied on relational databases to store, protect, and access their data.{#d8cf}

SQL databases, in particular, worked well for a long time and still do for many use cases. But, today, there is a wide range of situations where SQL databases can no longer satisfy the needs of modern enterprises, especially those that have made the move to the cloud.{#d8cf}

Increasingly, these companies are turning to NoSQL databases to meet their goals.{#d8cf}

Kubernetes and the Data Centre: Is Full Scale Migration Possible?

Thu, 21 Jul 2022 13:48:13 +0000

Data centres used to be exactly that - the centres of our data worlds, where all information, files and records would be kept and controlled.

Today, that is no longer the case. More companies are moving to cloud services, whether this is dialling down on their traditional on-premises environments or never setting them up in the first place.

This is not to say that data centres will be wholly abandoned for cloud, but the mix will continue to evolve and be very different to today. Gartner predicts that by 2025, 85% of infrastructure strategies will take a more hybrid approach, integrating on-premises, colocation, cloud and edge delivery options, compared with 20% in 2020.

Why a Cloud-Native Database Must Run on K8s

Tue, 19 Jul 2022 17:44:09 +0000

We’ve been talking about migrating workloads to the cloud for a long time, but a look at the application portfolios of many IT organizations demonstrates that there’s still a lot of work to be done. In many cases, challenges with persisting and moving data in clouds continue to be the key limiting factor slowing cloud adoption, despite the fact that databases in the cloud have been available for years.

For this reason, there has been a surge of recent interest in data infrastructure that is designed to take maximum advantage of the benefits that cloud computing provides. A cloud-native database is one that achieves the goals of scalability, elasticity, resiliency, observability and automation; the K8ssandra project is a great example. It packages Apache Cassandra and supporting tools into a production-ready Kubernetes deployment.

The End of the Beginning for Apache Cassandra

Tue, 12 Jul 2022 17:43:39 +0000

Image: Pixabay

Editor’s note: This story originally ran on July 27, 2021, the day that Apache Cassandra 4.0 was released.{#4bbb}

Today is a big day for those of us in the Apache Cassandra community. After a long uphill climb, Apache Cassandra 4.0 has finally shipped . I say finally, because it has at times seemed like an elusive goal. I’ve been involved in the Cassandra project for almost 10 years now and I have seen a lot of ups and downs.{#d6a4}

Available Now - gRPC for Apache Cassandra

Thu, 07 Jul 2022 07:38:47 +0000

Build microservices easily with the NoSQL standard database

If you are like most developers, you are embracing applications built using microservices and a NoSQL database. There are many good reasons: faster time to market, lower total cost of ownership, better performance, less downtime, and easy scalability.

Creating cloud-native applications is hard because microservices are often written in different languages, database drivers have a maintenance burden, or familiar HTTP APIs lack the performance needed.

K8ssandra ramps up security features to match Kubernetes’ best practices

Thu, 23 Jun 2022 19:54:58 +0000

New security features are coming to the open-source data platform: K8ssandra. The goal? To align even more with the security best practices of Kubernetes. Here’s an introduction to the platform’s security mission and an update on current initiatives.{#e3b8}

The security defaults applied by K8ssandra are about to get even more aligned withKubernetes’ security practices . In an upcoming release of K8ssandra, pod and container security configurations give users full customization capabilities and default values out of the box.{#c373}

Data Modeling in Cassandra and Astra DB

Thu, 16 Jun 2022 19:22:05 +0000

What does it take to build an efficient and sound data model for Apache Cassandra ® and DataStax Astra DB ? Where would one start? Are there any data modeling rules to follow? Can it be done consistently time and time again? The answers to these and many other questions can be found in the Cassandra data modeling methodology.{#3856}

In this post, we present a high-level overview of the data modeling methodology for Cassandra and Astra DB , and share over half a dozen complete data modeling examples from various real-life domains. We apply the methodology to create Cassandra and Astra DB data models for IoT, messaging data, digital library, investment portfolio, time series, shopping cart, and order management. We even provide our datasets and queries for you to try.{#4bd9}

Tombstones and Ghost Data Don’t Have to Be Scary!

Thu, 09 Jun 2022 20:36:25 +0000

Working with tombstones can be one of the more frustrating aspects of Apache Cassandra®. Problems with tombstones can happen when a large number of deletes occur within a short period of time, resulting in slower queries and disk bloat. This article will discuss tombstones, including the purpose they serve, and strategies/methods on how to avoid them.{#2508}

Tombstones in Apache Cassandra ® are written markers that are created whenever a value is deleted. Their purpose is to indicate that any prior values have been deleted. When a read occurs, it pulls data from the sorted string table (SSTable) files on a node that may contain the data. All prior values for the requested key are returned, ordered by write-time.{#d711}

Managing Distributed Applications in Kubernetes Using Cilium and Istio with Helm and Operator for Deployment

Tue, 07 Jun 2022 14:51:03 +0000

This post will show you the benefits of managing your distributed applications with Kubernetes in cross-cloud, multi-cloud, and hybrid cloud scenarios using Cilium and Istio with Helm and Operator for deployment.{#6864}

In our recent post on The New Stack , we showed you how you can leverage Kubernetes (K8s) and Apache Cassandra TM to manage distributed applications at scale, with thousands of nodes across both on-premises and in the cloud. In that example, we used K8ssandra and Google Cloud Platform (GCP) to illustrate some of the challenges you might expect to encounter as you grow into a multi-cloud environment, upgrade to another K8s version, or begin working with different distributions and complimentary tooling. In this post, we’ll explore a few alternative approaches to using K8s to help you more easily manage distributed applications.{#fec3}

5 More Reasons to Choose Apache Pulsar Over Apache Kafka

Thu, 05 May 2022 18:11:20 +0000

Author’s note: I originally published this blog post in 2019, while I was CEO of Kesque, a real-time messaging service built on Apache Pulsar , the cloud-native distributed messaging and streaming platform. It’s a follow-up to an earlier post, “ 7 Reasons to Choose Apache Pulsar over Apache Kafka .” A lot of big changes have happened since these two posts went live, including Kesque’s acquisition , in January 2021, by DataStax. The reasons to choose Pulsar, however, haven’t changed.

SQL vs. NoSQL: Pros & Cons

Thu, 21 Apr 2022 09:28:50 +0000

What’s the best way to store, protect and access your data?{#abb7}

This is a fundamental, yet critical decision.{#abb7}

After all, data is the cornerstone of success for just about every modern organization.{#abb7}

For most companies, the choice comes down to SQL and NoSQL databases. Each has unique strengths and weaknesses.{#abb7}

SQL databases have been a proven option since the 1970s. They are made up of highly structured tables, consisting of rows and columns, related to one other through common attributes. Every column is required to have a value for its corresponding row.

NoSQL (“not only SQL” or “non-SQL”) databases came along later to break the relational table straitjacket, with the ability to store and access all data types, structured and unstructured, together.

They’re extremely flexible and easy for developers to work with and modify. Learn more about SQL and NoSQL databases and their basic differences .{#3ad0}

Full Stream Ahead: Astra Streaming, Powered by Apache Pulsar

Thu, 24 Mar 2022 09:22:15 +0000

With a serverless architecture, multi-region geo replication, and Apache Cassandra’s legendary performance, DataStax Astra DB makes it easy for developers and enterprises to start small with their applications and grow them to infinite scale without constant performance tuning and optimization exercises.

However, cloud app developers also face significant challenge`s beyond data at rest. Modern data apps require high-scale streaming technologies that can deliver the reactive engagement at the point of interaction that end users have come to expect. That’s why today, we’re excited to announce that these capabilities have been added to DataStax Astra.

Reaper 3.0 for Apache Cassandra is available

Fri, 18 Mar 2022 14:55:43 +0000

The K8ssandra team is pleased to announce the release of Reaper 3.1 . Let’s dive into the features and improvements that 3.0 recently introduced (along with some notable removals) and how the newest update to 3.1 builds on that.

Starting with 3.1.0, Reaper can now compile and run with jdk11. Note that jdk8 is still supported at runtime.

Over the years, we regularly discussed dropping support for Postgres and H2 with the The Last Pickle (TLP) team, now part of DataStax , the organization leading the open-source development of Reaper. Despite our lack of expertise in Postgres, the effort required to maintain support for these storage backends was moderate as long as Reaper’s architecture was simple. However, complexity grew with more deployment options, culminating with the addition of the sidecar mode.

Have you upgraded to Cassandra 4.0 yet?

Fri, 11 Mar 2022 08:35:20 +0000

Alongside stability, Apache Cassandra 4.0 stands out for its ability to scale operations faster, its new auditing capabilities, and the way it embraces privacy by design. In this article, we’ll cover the new features and walk you through the steps required to upgrade to Cassandra 4.0.

The general availability of Apache Cassandra 4.0 marks the most stable release in the project’s history.

Even with the 4.0 beta 1 release back in July 2020, Project Management Committee (PMC) members were confident the latest version of the distributed NoSQL database would be ready to go to work with Apache Cassandra Committer; PMC member Sankalp Kohli at the time indicated that “users will be able to use the release knowing it is production-ready on day one.”

The Serverless Database You Really Want

Thu, 03 Mar 2022 20:43:05 +0000

The dreaded part of every site reliability engineer’s (SRE) job eventually: capacity planning. You know, the dance between all the stakeholders when deploying your applications. Did engineering really simulate the right load and do we understand how the application scales? Did product managers accurately estimate the amount of usage? Did we make architectural decisions that will keep us from meeting our SLA goals? And then the question that everyone will have to answer eventually: how much is this going to cost? This forces SREs to assume the roles of engineer, accountant and fortune teller.

Save Hours on your Setup of K8ssandra with the New Config Builder

Thu, 24 Feb 2022 19:36:47 +0000

Setting up K8ssandra in your workflow just got a whole lot easier. With the new Config Builder you can be running Apache Cassandra® on Kubernetes in a matter of minutes.

The purpose of K8ssandra is to make it easy to run Apache Cassandra® on Kubernetes. We recently took another big step in that direction by releasing the K8ssandra config builder .

Even if you’ve created thousands of nodes or integrated K8ssandra in your stack, you’ll probably want to give the config builder a try. Walk through the interactive wizard defining the shape of your cluster, resource requirements, and toggle features to fit your needs. This is great for smoothing out the on-ramp for a production ready environment of K8ssandra.

Why we decided to build a K8ssandra Operator - Part 4

Tue, 22 Feb 2022 17:04:28 +0000

In the first , second , and third posts in this series, we’ve shared conversations with K8ssandra core team members on our journey to build a Kubernetes operator for K8ssandra. We’ve discussed the virtues of the Helm package manager versus Kubernetes operators for deploying and managing infrastructure in Kubernetes and some of our implementation choices for the operator.

In this final post of the series, we pick up from the previous post with a discussion of how we decided to structure our projects in GitHub, how we test the K8ssandra operator, and our hopes for how the operator will expand the K8ssandra developer community.

Cassandra Database Migration to Kubernetes with Zero Downtime

Tue, 15 Feb 2022 15:22:15 +0000

K8ssandra is a cloud-native distribution of the Apache Cassandra® database that runs on Kubernetes, with a suite of tools to ease and automate operational tasks. In this post, we’ll walk you through a database migration from a Cassandra cluster running in AWS EC2 to a K8ssandra cluster running in Kubernetes on AWS EKS, with zero downtime.

As an Apache Cassandra user, your expectation should be that migrating to K8ssandra would happen without downtime. To make that happen with “classic” clusters running on virtual machines or bare metal instances, you will use the datacenter (DC) switch technique which is commonly used in the Cassandra community to transfer clusters to different hardware or environments. The good news is that it’s not very different for clusters running in Kubernetes as most Container Network Interfaces (CNI) will provide routable pod IPs.

Building Reactive Java Applications with Spring Framework

Tue, 15 Feb 2022 14:09:28 +0000

In one of our many free tutorials on DataStax Developers YouTube channel , we walked you through how to build a reactive implementation of Spring PetClinic in Apache Cassandra® using Spring WebFlux. The full series is available on YouTube .{#08a7}

If you’re a Java developer who uses the Spring ecosystem, you’ve probably seen the Spring Pet Clinic . In this workshop, we will walk you through a new reactive implementation of the Pet Clinic backend that uses Spring WebFlux and Apache Cassandra ® (via DataStax Astra DB ).{#47b5}

SQL vs. NoSQL: Pros & Cons

Wed, 09 Feb 2022 09:26:22 +0000

What’s the best way to store, protect and access your data? This is a fundamental, yet critical decision. After all, data is the cornerstone of success for just about every modern organization. For most companies, the choice comes down to SQL and NoSQL databases. Each has unique strengths and weaknesses.

SQL databases have been a proven option since the 1970s. They are made up of highly structured tables, consisting of rows and columns, related to one other through common attributes. Every column is required to have a value for its corresponding row. NoSQL (“not only SQL” or “non-SQL”) databases came along later to break the relational table straitjacket, with the ability to store and access all data types, structured and unstructured, together. They’re extremely flexible and easy for developers to work with and modify. Learn more about SQL and NoSQL databases and their basic differences .

Deploying to Multiple Kubernetes Clusters with the K8ssandra Operator

Fri, 04 Feb 2022 22:10:56 +0000

We built the new K8ssandra Operator to simplify deploying multiple Apache Cassandra data centers in different regions and across multiple Kubernetes (K8s) clusters. We’re not at the finish line just yet, but we’ve hit the first major milestone. Now, it’s easier than ever to run Apache Cassandra® across multiple K8s clusters in multiple regions with the K8ssandraCluster!

The K8ssandraCluster is a new custom resource for K8ssandra that covers all the bases necessary for installing a production-ready, multi-cluster K8ssandra deployment. Head over to the DataStax Tech blog to learn more about how to specify your remote clusters with the K8ssandraCluster, its deployment architecture, and what’s coming next in our continued development of the K8ssandra operator.

Backing up K8ssandra with MinIO

Thu, 03 Feb 2022 22:50:09 +0000

K8ssandra includes Medusa for Apache Cassandra® to handle backup and restore for your Cassandra nodes. Recently Medusa was upgraded to introduce support for all S3 compatible backends, including MinIO , the popular k8s-native object storage suite. Let’s see how to set up K8ssandra and MinIO to backup Cassandra in just a few steps.

Deploy MinIO

Similar to K8ssandra, MinIO can be simply deployed through Helm.

First, add the MinIO repository to your local list:

Four Reasons Why Apache Pulsar is Essential to the Modern Data Stack

Tue, 01 Feb 2022 22:14:58 +0000

Messaging has been on DataStax’s radar for several years. A significant motivator for this is the increasing popularity of microservices-based architectures. Briefly, microservices architectures use a message bus to decouple communication between services and to simplify replay, error handling, and load spikes.

With Apache Cassandra™ and DataStax Astra Cassandra-as-a-service, developers and architects have a database ecosystem that is:

Based on open source
Well-suited for hybrid- and multi-cloud deployments
Available in a cloud-native, consumption-priced service

There is no current messaging solution that satisfies these requirements, so we’re building one. We started by evaluating the most popular option, Apache Kafka. We found that it came up short in four areas:

Unboxing K8ssandra: The Data Layer For Your Kubernetes-Powered Applications

Tue, 01 Feb 2022 17:51:35 +0000

A Complimentary Live Webinar, Sponsored by DataStax

Kubernetes made it easy to deploy and scale out your cloud-native applications. With K8ssandra , you can now scale application data with the same simplicity and high availability. Join us as we unbox K8ssandra a cloud native data layer for Kubernetes and explore how you can deploy it alongside your applications.

Install k8ssandra
Authenticate with Stargate
Query your data via a convenient API (REST, document, or graphql)

Not familiar with Cassandra? Astra DB is a great (free) place to learn without any of the infrastructure setup, or management headaches.

Taking Your Database Beyond a Single Kubernetes Cluster

Wed, 19 Jan 2022 10:31:47 +0000

By Christopher Bradford and Ty Morton {#31db}

Global applications need a data layer that is as distributed as the users they serve. Apache Cassandra has risen to this challenge, handling data needs for the likes of Apple, Netflix and Sony. Traditionally, managing data layers for a distributed application was handled with dedicated teams to manage the deployment and operations of thousands of nodes — both on-premises and in the cloud.

Kubernetes and Apache Cassandra: What Works (and What Doesn’t)

Fri, 14 Jan 2022 16:30:50 +0000

“I need it now and I need it reliable”

– ANYONE WHO HASN’T DEPLOYED APPLICATION INFRASTRUCTURE

If you’re on the receiving end of this statement, we understand you here in the K8ssandra community . Although we do have reason for hope. Recent surveys have shown that Kubernetes (K8s) is growing in popularity, not only because it’s powerful technology, but because it actually delivers on reducing the toil of deployment.

How to Put a Database in Kubernetes?

Thu, 13 Jan 2022 09:18:38 +0000

Learn the key steps of deploying databases and stateful workloads in Kubernetes and meet the cloud-native technologies, like K8ssandra, that can streamline Apache Cassandra for K8s.{#95cc}

The idea of running a stateful workload in Kubernetes (K8s) can be intimidating, especially if you haven’t done it before. How do you deploy a database? Where is the actual storage? How is the storage mapped to the database or the application using it?{#6cde}

Requirements for Running K8ssandra for Development

Thu, 06 Jan 2022 12:15:27 +0000

The K8ssandra Quick start is a excellent guide for doing a full installation of K8ssandra on a dev laptop and trying out the various components of the K8ssandra stack. While this is a great way to get your first hands-on experience with K8ssandra, let’s state the obvious: running K8ssandra locally on a dev laptop is not aimed at performance. In this blog post, we will start Apache Cassandra® locally then explain how to run benchmarks to help evaluate what level of performance (especially throughput) you can expect from a dev laptop deployment.

Why K8ssandra?

Thu, 06 Jan 2022 12:10:09 +0000

The problem: scalable data persistence for cloud-native applications

The past decade and a half has brought tremendous change in how we in the software industry think about building and delivering internet applications. With the launch of Amazon Web Services (AWS) in 2006, companies of any size could access cloud computing infrastructure. The burst of creativity that followed gave birth to concepts like cloud-native in 2010, and methodologies such as the twelve-factor app in 2011.

New Survey Finds Data on Kubernetes Is No Longer a Pipe Dream

Thu, 30 Dec 2021 15:52:00 +0000

For people that work in infrastructure and application development, the pace of change is quick. Finish one project and it’s on to the next. Each iteration requires an evaluation asking if the right technology is being used and if it provides a new advantage. Kubernetes has been on the fast track of continuous evaluation. New projects and methodologies are continuously emerging and it can be hard to keep up. Then there is the question of running stateful services. The Data on Kubernetes community has released a report titled “Data on Kubernetes 2021” to give us a snapshot of where our industry sits with stateful workloads. Over 500 executives and tech leaders were asked some very direct and insightful questions about how they use Kubernetes. It turns out that there were a lot of surprising finds. Some that I would have never predicted. Let’s dig into some of the highlights that stood out to me.

The future of cloud-native databases begins with Apache Cassandra 4.0

Wed, 29 Dec 2021 07:40:00 +0000

“Reliability at massive scale is one of the biggest challenges we face at Amazon.com, one of the largest e-commerce operations in the world; even the slightest outage has significant financial consequences and impacts customer trust.”

This was the first line of the highly impactful paper titled “Dynamo: Amazon’s Highly Available Key-value Store.” Published in 2007, it was written at a time when the status quo of database systems was not working for the massive explosion of internet-based applications. A team of computer engineers and scientists at Amazon completely re-thought the idea of data storage in terms of what would be needed for the future, with a firm footing in the computer science of the past. They were trying to solve an immediate problem but they had unwittingly sparked a huge revolution with distributed databases and the eventual collision with cloud-native applications.

Build a Status Dashboard Using Spring Boot and Astra DB

Fri, 17 Dec 2021 09:28:19 +0000

In this article, we are going to build “Tony Stark’s Avengers Status Dashboard”, used by The Avengers to monitor the status of the members of the team.

This will be built using DataStax Astra , a DBaaS powered by Apache Cassandra using Stargate to offer additional APIs for working with it.

On top of this, we will be using a Spring Boot application to render the dashboard and show what’s going on.

7 Reasons to Choose Apache Pulsar over Apache Kafka

Wed, 15 Dec 2021 10:03:13 +0000

I wrote an earlier version of this article in 2019, while I was CEO of Kesque, a real-time messaging service built on Apache Pulsar , the cloud-native distributed messaging and streaming platform. A lot of big changes have happened in the interim; perhaps the most significant of these is the fact that the company I founded in early 2019 was acquired , in January, by DataStax. One thing that hasn’t changed, however, is the rationale behind our choice of Apache Pulsar.

Fast JMS for Apache Pulsar: Blazing Performance

Fri, 10 Dec 2021 08:54:44 +0000

DataStax recently announced the availability of Fast JMS for Apache Pulsar , a JMS 2.0 API. By combining the industry-standard Java Messaging Service (JMS) API with the cloud-native and horizontally scalable Apache Pulsar™ streaming platform, DataStax is providing a powerful way to modernize your JMS infrastructure, improve performance, and reduce costs. Fast JMS is open source and is included in DataStax’s Luna Streaming Enterprise support of Apache Pulsar.

What is JMS?

Java Message Service (JMS), or Jakarta Messaging as it is now known, is a standard messaging API that is part of the Java Enterprise Edition (now Jakarta EE ) ecosystem. It defines how to create, send, receive, and read messages between loosely coupled producers and consumers.

Adelphi: Apache Cassandra Testing Goes Cloud Native

Thu, 09 Dec 2021 15:55:43 +0000

|———————————————————————————————————————–| | adelphi (ancient greek ἀδελφός) - Subs.: brother or sister, son of the same mother. Adj.: double, twin, in pairs. |

What is Adelphi?

Adelphi is an open-source QA tool for Apache Cassandra™, it’s packaged as a Helm chart and it simplifies the tasks of running data integrity and performance tests on Kubernetes.

Built for simplicity, Adelphi is an opinionated tool; it runs an Argo workflow of well-defined steps and prefers convention over configuration, but it exposes some knobs that you can override.

Why a Serverless Data API Might be Your Next Database

Wed, 01 Dec 2021 14:53:44 +0000

App development stacks have been improving so rapidly and effectively that today there are a number of easy, straightforward paths to push code to production, on the cloud platform of your choice. But what use are applications without the data that users interact with? Persistent data is such an indispensable piece of the IT puzzle that it’s perhaps the reason the other pieces even exist.

Enter cloud and internet scale requirements, essentially mandating that back-end services must be independently scalable / modular subsystems to succeed. Traditionally, this requirement has been difficult in the extreme for stateful systems. No doubt, database as-a-service (DBaaS) has made provisioning, operations, and security easier. But as anyone who has tried to run databases on Kubernetes will tell you: auto scaling databases, especially ones that are easy for developers to use, remain out of reach for mere mortals.

Enterprise-Level Apache Cassandra Sink Connector for Apache Pulsar

Wed, 24 Nov 2021 19:16:16 +0000

When DataStax started investing in streaming with Apache Pulsar ™, we knew that one of the first things people would want to do was connect existing enterprise data sources to Apache Cassandra™ using Pulsar.

Apache Pulsar has a powerful framework called Pulsar IO to enable this kind of use case, and at DataStax we already had a best-in-class Kafka Connect Sink that enables you to store structured data coming from one or more Kafka topics into DataStax Enterprise, Apache Cassandra, and Astra.

K8ssandra: Production-Ready Platform for Cassandra on Kubernetes

Wed, 24 Nov 2021 09:45:12 +0000

K8ssandra , an open-source distribution of Apache Cassandra for Kubernetes, aims to provide a “production-ready platform”, and this includes automation for operational tasks such as repairs, backups, and monitoring. Cassandra is a distributed NoSQL database designed for global scale and fault tolerance for the most demanding applications on the planet, written in Java.

K8ssandra is deployed using Helm and supports stateful workloads out of the box, which enables database administrators (DBAs) and site reliability engineers (SREs) to set up and operate Cassandra clusters using best practices in a Kubernetes environment.

Connect Stateful Workloads Across Kubernetes Clusters

Fri, 23 Jul 2021 08:44:32 +0000

One of the biggest selling points of Apache Cassandra™ is its shared-nothing architecture, making it an ideal choice for deployments that span multiple physical datacenters. So when our Cassandra as-a-service single-region offering reached maturity, we naturally started looking into offering it cross-region and cross-cloud.

One of the biggest challenges in providing a solution that spans multiple regions and clouds is correctly configuring the network so that Cassandra nodes in different data centers can communicate with each other successfully, even as individual nodes are added, replaced, or removed. From the start of the cloud journey at DataStax, we selected Kubernetes as our orchestration platform, so our search for a networking solution started there. While we’ve benefited immensely from the ecosystem and have our share of war stories, this time we chose to forge our own path, landing on ad-hoc overlay virtual application networks (how’s that for a buzzword soup?).

The Search for a Cloud-Native Database

Fri, 16 Jul 2021 07:40:06 +0000

The concept of “cloud-native” has come to stand for a collection of best practices for application logic and infrastructure, including databases. However, many of the databases supporting our applications have been around for decades, before the cloud or cloud-native was a thing. The data gravity associated with these legacy solutions has limited our ability to move applications and workloads.

As we move to the cloud, how do we evolve our data storage approach? Do we need a cloud-native database? What would it even mean for a database to be cloud-native? Let’s take a look at these questions.

Best Practices for Data Pipeline Error Handling in Apache NiFi

Thu, 15 Jul 2021 05:25:32 +0000

Learn actionable strategies for error management modeling in Apache NiFi data pipelines, and understand the benefits of planning for error handling.

According to a McKinsey report, “the best analytics are worth nothing with bad data”. We as data engineers and developers know this simply as “garbage in, garbage out”. Today, with the success of the cloud, data sources are many and varied. Data pipelines help us to consolidate data from these different sources and work on it. However, we must ensure that the data used is of good quality. As data engineers, we mold data into the right shape, size, and type with high attention to detail.

A Case for Databases on Kubernetes from a Former Skeptic

Tue, 13 Jul 2021 09:12:18 +0000

Kubernetes is everywhere. Transactional apps, video streaming services and machine learning workloads are finding a home on this ever-growing platform. But what about databases?

If you had asked me this question five years ago, the answer would have been a resounding “No! " — based on my experience in development and operations. In the following years, as more resources emerged for stateful applications, my answer would have changed to “Maybe,” but always with a qualifier: “It’s fine for development or test environments…” or “If the rest of your tooling is Kubernetes-based, and you have extensive experience…”

Kubernetes Is The Best Technology For Running A Cloud-Native Database

Wed, 30 Jun 2021 09:13:19 +0000

For this reason, there has been a surge of recent interest in data infrastructure that is designed to take maximum advantage of the benefits that cloud computing provides. A cloud-native database is one that achieves the goals of scalability, elasticity, resiliency, observability, and automation; the K8ssandra project is a great example. It packages Apache Cassandra and supporting tools into a production-ready Kubernetes deployment.

Introduction to New AutoScale Feature Available in Payara Server

Wed, 23 Jun 2021 07:47:09 +0000

Running your application sometimes requires multiple instances to handle the requests of the users. Within the Payara Platform, the Domain Data Grid helps you configure your environment to run your application in a cluster. Besides the setup of a cluster itself, many applications can benefit from an environment that scales dynamically.

The number of instances must increase when the application usage peaks and should decrease when activity is low to reduce the resource usage and the corresponding costs associated with it. The introduction of the AutoScale feature in Payara Server Community 5.2021.4 helps us achieve this dynamic scalability.