Karmada Security Audit

TL;DR

Shielder, together with OSTIF and CNCF, performed a Security Audit on the Karmada project.

The audit resulted in six (6) findings ranging from high to informational severity. Most of them have been addressed by the Karmada core team, while two of them are marked for a future iteration.

Today, we are publishing the full report in our dedicated repository.

Introduction

In September 2024, Shielder was hired to perform a Security Audit of Karmada, an open, multi-cloud, multi-cluster Kubernetes orchestration and management system. The audit has been sponsored by the CNCF and facilitated by the Open Source Technology Improvement Fund (OSTIF).

Karmada aims to provide a unified control plane for multi-cluster applications. To do so, it extends the Kubernetes API to support management of resources across multiple clusters, exposing a single interface (the Karmada API) for operations. It is mainly written in Go.

The Karmada source code is available at https://github.com/karmada-io/karmada, and the website provides documentation of the project at https://karmada.io/docs/.

Context and Scope

The aim of the audit was to assess the overall security posture of Karmada, from its design to its implementation, also including the documentation and provided examples.

For the assessment, the Shielder team has modeled the following attackers:

• Unauthenticated attacker: an attacker with no access to valid credentials to authenticate against neither the Karmada control plane nor its member clusters.
• Compromised cluster: an attacker that has compromised one of the member clusters, with the goal to move horizontally or vertically in the federation.
• Malicious operator: an attacker that owns valid credentials for either the control plane or one of the member clusters.

In this context, the goals were to assess if the Karmada project:

• Designs its multi-cluster federation in a way that does not introduce paths for vertical or horizontal movements between clusters.
• Correctly implements Golang security by design principles when handling user-controlled input.
• Employs the correct segregation/sandboxing mechanisms for network or local resources.
• Provides documentation that can be followed by users of the tool without introducing insecure defaults or additional risks in their Kubernetes federation.

The scope of this audit is the Karmada version v.1.11.0 released on August 31, 2024.

Findings Summary and Recommendations

The overall security posture of the Karmada project is mature from a code point-of-view.

The project correctly re-uses battle-tested and standard APIs from the Kubernetes project, and mostly follows best practices in terms of security. The Shielder team was able to identify 1 (one) high, 1 (one) medium, 2 (two) low findings plus 2 (two) informational issues.

¹ The issue is caused by a third-party dependency (gopher-lua). The maintainer has been contacted, and the issue will be fixed in a future iteration. ² The issue does not have a direct security impact, but it is a good practice to harden the security configuration of containers. It will be fixed in a future iteration.

If you are a developer or cloud engineer using Karmada, the recommendation is to, whenever possible, update it to the latest release or to prefer the Push Mode of deployment to make sure that member clusters are not capable of directly contacting the Karmada control plane.

The full details can be read in the report.

Conclusions

When assessing libraries, frameworks or more in general tools that are by-design highly flexible and customizable, it is crucial to perform effective threat modeling to understand where the most interesting attack surfaces lie.

In the context of multi-cloud, multi-cluster environments, it is important to assume that one of the clusters might be compromised (or operated by a malicious actor), and to design the system in a way that is resilient to that.

For Karmada, the main threats identified were caused by an insecure design of the Pull Mode, which allowed a member cluster to compromise other clusters in the federation, by a lack of input sanitization in the CRD archive extraction, and a DoS when custom Lua scripts are passed to the control plane, which might be used to stall or crash the control plane.

We would like to thank the Karmada core maintainers and community - notably Kevin Wang, Hongcain Ren and Zhuang Zhang - for being extremely collaborative in triaging and addressing the findings.

It was a pleasure for our team to work with OSTIF, the CNCF, and the Karmada core team.

Pitch 🗣

Did you know OSTIF helps sensitive open-source projects in securing funds to perform security audits? They will also help you in scoping the assessment, finding a trusted partner to perform the analysis, and ensuring full transparency along the way.

P.S. if you need help in threat modeling and auditing your kubernetes-powered projects –> get in touch with us!