Storage in distributed systems, such as those run by Kubernetes, is always a challengeView in browser »
The New Stack Update

ISSUE 247: Strong Consistency for a Distributed World

Talk Talk Talk

“Value streams are everything in the software delivery lifecycle (SDLC), from idea to production, that are needed to deliver software products or services to customers.”

Add It Up
Istio has a big service mesh lead, but only among a segment of early adopters. Among those already using a service mesh in production, 63% have adopted Istio, which is more than twice as many as Linkerd, according to our analysis of the Cloud Native Computing Foundation’s (CNCF) survey earlier this year. Besides HashiCorp’s Consul, no other option broke double digits.
What's Happening

Kubernetes is certainly the platform of choice for cloud native computing, but the widely adopted container orchestrator still remains difficult to configure and manage. Tools and platforms for configuration, policy and audit management, security, monitoring and a range of other operations-related tasks are emerging. They are helping to make Kubernetes clusters simpler to manage while requiring fewer manual processes.

Recorded for KubeCon + CloudNativeCon North America, this The New Stack Makers podcast features Eric Sorenson, technical product manager for Relay at IT infrastructure management software provider Puppet, and Dave Lindquist, general manager and vice president engineering for hybrid cloud management at Red Hat, discuss the state of Kubernetes cluster configuration management. TNS correspondent B. Cameron Gain hosted the episode.

Why K8 Cluster Management Is Not Expected to Become Boring Anytime Soon

Strong Consistency for a Distributed World

Storage in distributed systems, such as those run by Kubernetes, is always a challenge. Everyone looks to improve performance, making sure that data can be speedily distributed across all the different nodes in a system. This is indeed a challenge. But a bigger challenge is one of consistency. Consistency means that when a bit of data is written to a system, it becomes immediately available across the entire system. When a database is sharded across multiple nodes, perhaps even across different geographic regions, it is extremely difficult to keep all the copies of the data in sync. Many real-time operations focus on improving performance while settling for “eventual consistency.”

Nonetheless achieving real-time consistency has been an issue with many cloud native companies, as discussed in our latest TNS Makers podcast.

As Ticketmaster’s Jackie Fong told podcast host B. Cameron Gain, customers “need to buy tickets — if that particular seat is available this second, it may not be available half a second later,” said Fong. “So our data has to be instantaneous. And that’s a big struggle between a cloud-based solution versus an on-prem solution.”

Amazon Web Services Chief Technology Officer Werner Vogels also discussed this issue during this keynote at the company’s re:Invent annual user conference, held virtually this year. With the company’s Simple Storage Service, as Vogels explained in the keynote, the service can experience delays in showing when a new file has been placed into a storage bucket. This delay can be problematic for operations that require strict consistency, or the ability to consult the file as soon as it lands in the bucket. The small delay took place not only with basic GET, PUT, and LIST operations, but also with object tags, ACLs, and metadata.

As a result, many users  “were building [the] complexity into their applications to achieve strong consistency. We knew if we did that work for them, life would be a lot easier for them,” Vogels said.

So the company undertook work to bring stronger consistency to S3, creating new protocols to boost cache coherence and distributed systems communication.

As a result, Vogels announced, S3 has been revamped to offer stricter guarantees about read-write consistency. It now offers "strong read-after-write consistency" (built-in "strong consistency") for all storage requests. Users can update objects hundreds of times per second if they want, and there is no impact on performance. This is great news not only for operators of data lakes but even real-time applications as well.

Ortelius Battles Microservices Sprawl with Configuration Mapping, Version Control

Ortelius, the open source microservice management platform, was created to track the many changes that today’s microservices go through. With Ortelius, a change to a microservice creates a new version, and then that new version creates a new version of every application that consumes it. Ortelius could enable the sharing of microservices not only across applications and deployments, but across companies and organizations in much the same way that open source software packages are shared today.

And the List Goes On: Even More Problems with GitOps

GitOps is an emerging way to manage the actual state of systems. But while GitOps as an idea is great, we are not even close to having that idea be useful in a practical sense. In this third and final installment, CodeFresh’s Viktor Farcic takes a look at some more of the challenges with deploying GitOps to automate the push of new code into production, including the lack of established patterns and incompatibility with current continuous integration systems.

How to Write a Better README Mission Statement

Whether you’re a developer working on a side project or an engineer creating an open source project as part of your job, your goal is similar: To solve a problem that needs solving and to build a community of contributors that extends beyond the original individual or team who started the project. In this contributed post, marketing guru Emily Omier instructs us how to write a perfect README to explain the hows and whys of your project.

Party On

AWS re:Invent: Teresa Carlson, vice president or worldwide public sector at AWS, on her team's initiative to reduce world hunger.

Annie Hancock, head of creative and customer experiences, AWS Global Events Team, speaks during this week's AWS re:Invent.

The White Buffalo jams out at AWS re:Invent.

On The Road
DataStax Pancake Breakfast // Jan. 12, 2021 // VIRTUAL 9 AM PT

Jan. 12, 2021 // VIRTUAL 9 AM PT

DataStax Pancake Breakfast

It’s not the World Wide Waffle, it’s pancakes and data management time at the helm with the Kubernetes and Apache Cassandra crew! Join us for a short stack with The New Stack as we get to the bottom of the batter and discuss who manages data in Kubernetes, anyways? Who are these people, what are their biggest challenges, what tools are available, and what do the next five years look like? Pass the syrup, Cassandra! Register now!

Copyright © 2020 The New Stack, All rights reserved.

Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list

Email Marketing Powered by Mailchimp