Cloud · Kubernetes · Security

Kubeflow Pipelines CVE-2026-77596: close the shared-upload boundary

Patch Kubeflow Pipelines CVE-2026-77596, audit shared versions, pin recurring runs and verify the 2.18.0 authorization fix.

TOPIC HUBCloud, DevOps & Kubernetes
Industrial pipes and control valves representing authorization gates in a shared pipeline
N

Kubeflow Pipelines CVE-2026-77596 is a critical authorization bypass in the KFP API server's multipart pipeline-upload endpoints. In multi-user mode, Kubeflow Pipelines 2.17.3 and earlier can skip Kubernetes SubjectAccessReview when the target namespace resolves to an empty value—the representation used for shared pipelines. A low-privilege tenant can therefore create a shared pipeline or add an attacker-controlled version. Upgrade the server and matching manifests to 2.18.0, then audit existing shared versions and recurring runs that follow “latest.”

Answer first

This issue matters to platform teams running multi-tenant Kubeflow because it crosses the intended tenant-to-shared-catalog boundary. The upload bypass is reproduced and confirmed. A later path—an unpinned recurring run selecting a malicious latest version and executing it under another namespace's service account—is credible but conditional; the advisory authors did not execute that complete victim-schedule chain end to end. The reviewed primary sources do not establish active exploitation in the wild as of November 10, 2026.

Search intent: incident response and upgrade guidance for Kubeflow Pipelines CVE-2026-77596. Primary keyword: Kubeflow Pipelines CVE-2026-77596. Related entities include KFP API server, multi-user isolation, shared pipelines, multipart upload, SubjectAccessReview, Kubernetes namespaces, recurring runs, service accounts and version pinning.

What the vulnerability is

The canonical Kubeflow Pipelines security advisory assigns CVE-2026-77596 a critical CVSS score of 9.6. It affects ghcr.io/kubeflow/kfp-api-server through 2.17.3 and is patched in 2.18.0. The vulnerable paths are:

  • POST /apis/v2beta1/pipelines/upload
  • POST /apis/v2beta1/pipelines/upload_version

In multi-user mode, writes should authorize the caller for the relevant namespace using Kubernetes SubjectAccessReview. The defect appears when the request namespace resolves to an empty string. That value denotes a shared, cluster-wide pipeline, yet the vulnerable handlers omit the authorization review instead of checking a protected administrative namespace. The condition is reachable by default because REQUIRE_NAMESPACE_FOR_PIPELINES defaults to false and the shipped manifests do not enable it.

If the KFP API server is reachable without the expected authenticating proxy, an unauthenticated caller may also reach the vulnerable route. That is a deployment-dependent exposure, not a claim that every Kubeflow installation is public.

Evidence: confirmed, conditional and unknown

StatementEvidenceOperational meaning
A low-privilege tenant can upload a shared pipeline or add a version without the expected authorization check.Confirmed by the maintainer's live upload reproduction.Treat shared-catalog integrity as requiring review.
A crafted pipeline version can change a container image, command or arguments.Confirmed capability of the uploaded specification.An untrusted version can carry an arbitrary workload definition.
An unpinned recurring run can select the malicious latest version and execute under a victim namespace's service account.Conditional path inspected by the advisory authors; not performed end to end.Prioritize schedules that follow latest, but do not call every cluster compromised.
Active exploitation has occurred in the wild.Not established by the reviewed primary sources.A public proof of the upload bypass is not evidence of a confirmed breach.

The distinction prevents two dangerous errors: understating a proven cross-tenant authorization failure, or overstating a conditional execution chain as a completed compromise.

A control valve is an architectural metaphor: shared pipeline writes need an explicit authorization gate.
A control valve is an architectural metaphor: shared pipeline writes need an explicit authorization gate. Open for a larger view

Who should act first

  1. Multi-user Kubeflow Pipelines deployments on 2.17.3 or earlier.
  2. Clusters where kfp-api-server is reachable outside the normal authentication proxy.
  3. Organizations using shared pipelines across Profiles or namespaces.
  4. Teams with recurring runs that resolve the latest shared version.
  5. Platforms whose pipeline service accounts have broad data, secret or workload permissions.

Kubeflow's multi-user isolation documentation explains that Profiles and Kubernetes namespaces are the isolation building blocks, while pipeline definitions have historically been shared. The defect weakens authorization around that shared control-plane object.

Upgrade and containment plan

  1. Inventory the running server. Record the kfp-api-server image tag and digest, confirm whether multi-user mode is enabled, and map every ingress path.
  2. Upgrade to KFP 2.18.0 or later. Use the official 2.18.0 release and deploy matching server components and manifests together. Updating a client SDK does not fix the server.
  3. Verify negative authorization. From a low-privilege test Profile, exercise the shared-upload routes without a namespace. The request must no longer bypass authorization. Test outside production first and retain API-server and Kubernetes authorization logs.
  4. Review existing shared versions. The upgrade does not delete versions uploaded before the patch. Compare image references, commands, arguments, owners, creation times and expected provenance.
  5. Inspect consumers. Find recurring runs that resolve latest. Pause them or pin them to a reviewed immutable version. Review recent runs and Pods for unexpected images, commands, environment variables, volumes, service accounts and network destinations.
  6. Rotate credentials only where evidence warrants it. If a suspicious workload accessed secrets or external credentials, preserve evidence and rotate the affected material. A vulnerable version alone does not prove that every credential was exposed.

What 2.18.0 changes

KFP 2.18.0 no longer treats an empty namespace as “no authorization needed.” Shared writes are authorized against the KFP system namespace, and the project added regression cases for the affected authorization paths. A tenant can manage objects in its authorized namespace; writes to shared control-plane state require an administrative authorization decision.

Follow the official installation documentation for the deployment method used by your cluster. Confirm the running image digest after rollout instead of relying only on the applied manifest.

Interim hardening

ControlBenefitCost or limitation
Set REQUIRE_NAMESPACE_FOR_PIPELINES=true.Rejects namespace-less requests and closes the default empty-namespace route.May break workflows that intentionally use shared pipelines; test compatibility.
Restrict direct access to kfp-api-server.Prevents callers from bypassing the platform's authenticating proxy.Does not repair authorization for authenticated tenants.
Pause or pin recurring runs that follow latest.Reduces automatic selection of an unreviewed shared version.Requires an explicit promotion process.
Monitor shared upload and version-creation events.Improves detection of control-plane changes.Detection cannot replace the patch and depends on log retention.

Do not grant every tenant access to the KFP system namespace as a shortcut. That converts a bypass into intentional privilege expansion. Network isolation alone is also insufficient when the attacker is an authenticated low-privilege tenant.

Safer shared-pipeline architecture

Separate authoring, review, promotion and execution:

  1. Tenant namespaces produce candidate pipeline packages.
  2. A controlled promotion service validates provenance, image digests, policy and ownership.
  3. The service creates an immutable shared version only after approval.
  4. Schedules reference the reviewed version, not a moving latest alias.
  5. Runtime service accounts receive only the permissions required by that pipeline.

This design keeps capability and authority separate. A shared catalog is a supply-chain boundary, not merely a convenient storage list.

For related controls, see the Kyverno 1.19.2 security update guide and the secure ingestion quarantine pattern. The topics hub links this incident response to wider cloud, DevOps and software-engineering guidance.

Best practices

  • Pin production schedules to reviewed pipeline versions and container image digests.
  • Keep the API server behind the supported authentication path.
  • Separate tenant authoring from shared-catalog promotion.
  • Record version provenance, approver, image digests and policy results.
  • Use narrowly scoped Kubernetes service accounts.
  • Test authorization denials after every control-plane upgrade.
  • Preserve audit evidence before deleting suspicious definitions.

Anti-patterns

  • Assuming namespace isolation automatically protects shared control-plane objects.
  • Following latest from unattended schedules without a promotion gate.
  • Updating only the SDK while leaving the vulnerable server image.
  • Treating public reproduction as proof that a specific environment was breached.
  • Granting broad system-namespace access to suppress authorization errors.

Limits

Authentication topology, shared-pipeline usage, recurring-run configuration, service-account privileges, network policy and available logs change the practical risk. No public evidence of active exploitation was found in the reviewed primary sources, but absence of evidence is not evidence that exploitation is impossible. Conversely, a vulnerable version is not proof of compromise.

Executive checklist

  • Upgrade KFP server and matching manifests to 2.18.0 or later.
  • Block direct unauthenticated access to the API server.
  • Review shared pipeline versions created during the exposure period.
  • Pin or pause recurring runs that follow latest.
  • Investigate unexpected images, commands, service accounts and outbound connections.
  • Document what is confirmed, conditional and unknown.

References

  1. Kubeflow Pipelines advisory GHSA-rjr3-43j3-4q8g / CVE-2026-77596
  2. Kubeflow Pipelines 2.18.0 release
  3. Kubeflow Pipelines multi-user isolation
  4. Kubeflow Pipelines installation
  5. Nenad Stojković — industrial pipes and valves, CC BY 2.0, Wikimedia Commons

Prepared by: Noor Yasser

Official references

These references document the tools discussed. Examples and design decisions are illustrative and should be adapted to the project and its versions.

Prepared by: Noor Yasser

FROM DECISION TO DELIVERY

Working through a similar engineering challenge?

I help teams turn architecture decisions into a clear scope and dependable, reviewable implementation.

Book a 30-minute callRelated serviceBackend engineering & API integrationsRelevant projectLogistics at scale