Skip to content

Kubernetes agent as a free Amazon EKS add-on on AWS Marketplace - #5007

Merged
simlarsen merged 8 commits into
masterfrom
claude/cool-bassi-729295
Oct 18, 2026
Merged

simlarsen merged 8 commits into
masterfrom
claude/cool-bassi-729295

Conversation

@simlarsen

Copy link
Copy Markdown
Contributor

Makes the free Kubernetes agent installable as an Amazon EKS add-on through AWS Marketplace, so clusters that send to OneUptime Cloud (or a self-hosted OneUptime) can add it from the EKS console's Get more add-ons, next to Datadog, Dynatrace and New Relic. Nothing is submitted to AWS here: the AWS-side steps are below.

What changes

  • Chart (HelmChart/Public/kubernetes-agent): new value oneuptime.existingSecretName, so the ingestion key can come from a Secret the operator creates (key api-key). Every workload then reads it from there and the chart renders no Secret of its own; setting it together with oneuptime.apiKey is refused. EKS shows add-on configuration values in plain text and refuses configuration schemas that take keys, so the add-on needs this. The common labels write app.kubernetes.io/managed-by: Helm instead of reading .Release.Service, which EKS rejects in an add-on.
  • Add-on edition: build.py --marketplace aws-eks-addon builds the agent chart for AWS's add-on rules. Every image comes from the add-on product's Marketplace ECR (amd64 and arm64), with the release's tags. Defaults: OneUptime Cloud, the key from the Secret oneuptime-ingestion-key, the oneuptime-agent namespace, the Helm install's resource names, and a cluster name that EKS fills in at install (${AWS_EKS_CLUSTER_NAME}). It leaves out the profiler and the debug sidecar. It ships aws_mp_configuration_schema.json: a draft-07 schema of the few values a buyer may set, none of them a secret. The chart's own schema is open at its root for the k8version value EKS renders add-ons with: unopened, AWS's own helm template command fails on the chart.
  • Release automation (HelmChart/Marketplace/automation): aws_catalog.py builds the add-on version (AddDeliveryOptions with EksAddOnDeliveryOptionDetails, Limited), checked against AWS's own field rules, and adds go-public for after testing. release_plan.py and ci_setup.py cover the new product.
  • Workflow (marketplace-release.yml): aws-eks-addon-push (marketplace-push) and aws-eks-addon-publish (marketplace-publish, waits for approval), in the same skip-what-is-there style as the AWS jobs. The weekly run adds no add-on version while AWS is still applying one or one is still in testing; a run by hand ships whatever. release.yml is untouched.
  • Images (from scanning every add-on image with trivy 0.75.0, HIGH and CRITICAL, both architectures):
    • kube-state-metrics v2.13.0 to v2.20.0: 1 critical and 49 high, down to 9 high.
    • OpenCost 1.121.0 to 1.121.3: 36 high, down to 7 high.
    • Prometheus v2.55.1 to v3.15.0: 4 critical and 104 high, down to 8 high. The bundled config passes promtool 3.15, and in a KinD run it scrapes cAdvisor and OpenCost with the cost series present.
  • Docs: "Install as an Amazon EKS add-on" (with aws eks create-addon and configuration-value examples) and "Keep the ingestion key in a Secret" on the Kubernetes agent page, and a card on the cloud marketplaces page. Both are in all 17 docs languages, line for line.
  • Runbook: SELF_HOSTED_RELEASING.md covers the product, its setup, releasing, testing in ap-northeast-2, going public and the listing text.

Testing

  • python3 -m unittest discover -s HelmChart/Marketplace -p 'test_*.py': 136 tests.
  • bash HelmChart/Tests/run.sh lint marketplace with Helm 3.19.0, the version AWS validates with: 149 marketplace checks, including AWS's exact helm lint and helm template --set k8version=<v> --kube-version <v> --include-crds --no-hooks for EKS 1.34 to 1.37. The unmodified chart fails that command (additional properties 'k8version' not allowed).
  • helm unittest HelmChart/Public/kubernetes-agent: 346 tests, including the new existing-secret_test.yaml.
  • Tests/Ops: MarketplaceRelease and the other suites that read the workflows or the agent chart.
  • Docs: the translation, content-rule, facts and search suites for both pages (10,125 tests), plus npm run docs:check-anchors.

On the AWS side (maintainer)

  1. Create the product: in the Management Portal, Server products > Create server product > Container, pricing Free. The listing text is in the runbook. Initial repository kubernetes-agent. Submit, which makes it Limited.
  2. Its ID into editions.json: run aws_catalog.py find-product, set aws-eks-addon.productId, then run aws_catalog.py add-repositories --edition aws-eks-addon.
  3. The release role: run ci_setup.py aws --apply again, so it may add versions to the new product.
  4. EKS add-on onboarding: check that Request changes > Add new version offers an Amazon EKS add-on delivery option. If it does not, ask Seller Operations to enable it for the product: name kubernetes-agent, type monitoring, namespace oneuptime-agent, Kubernetes 1.34 to 1.37, no IAM permissions. Confirm the seller alias EKS puts before the name: the docs assume oneuptime_kubernetes-agent (sellerAlias in editions.json).
  5. First version: set "ready": true for aws-eks-addon, run Marketplace Release by hand with marketplaces aws-eks-addon and dry run off, then approve the publish job.
  6. Testing: after 5 to 10 business days the version shows as Limited. Test it in ap-northeast-2 on one EKS cluster per named Kubernetes version (amd64 and arm64 nodes), and keep the clusters until AWS approves.
  7. Go public: run aws_catalog.py go-public --version X.Y.Z. The first time, also submit Update visibility for the product.

No IAM is needed: the agent calls no AWS API, and the product is free.

Not done here

  • The collector image: otel/opentelemetry-collector-contrib:0.96.0 has 4 critical findings, all fixed upstream, and AWS's scan would block the add-on on them. Moving the chart off 0.96.0 is a follow-up of its own. Collector 0.161.0, which the other agents run, has none, but its kubeletstats receiver no longer emits k8s.node.cpu.utilization, k8s.pod.cpu.utilization or container.cpu.utilization. OneUptime's ingest, dashboards and monitors read those names, so that change keeps them.
  • Not verifiable until the first submission: the seller alias, the cluster-name override, and AWS's review of the privileged eBPF DaemonSet (eBPF is on by default; groundcover and Upwind run eBPF sensors as EKS add-ons).

oneuptime.existingSecretName names a Secret the operator created, holding
the key under api-key. Every workload that sends to OneUptime then reads it
from there, and the chart renders no Secret of its own; setting it together
with oneuptime.apiKey is refused. The Amazon EKS add-on needs this: EKS shows
an add-on's configuration values in plain text and refuses configuration
schemas that take keys.

The common labels write managed-by out instead of reading .Release.Service,
which EKS refuses in an add-on, and two comments no longer name lookup.
build.py --marketplace aws-eks-addon builds the agent chart for the AWS
Marketplace EKS add-on catalog: every image it can run from the add-on
product's Marketplace ECR repositories (third-party ones at the chart's
pins, OneUptime's at the release), OneUptime Cloud's address, the key from
the Secret oneuptime-ingestion-key, the Helm install's resource names, a
cluster name EKS replaces at install, no profiler or debug sidecar, and
aws_mp_configuration_schema.json: a draft-07 schema of the few values a
buyer may set, none of them a secret. The chart's own schema is open at its
root for the k8version value EKS renders add-ons with.

mirror-images.sh copies the add-on's images like the AWS edition's:
linux/amd64 and linux/arm64 only, nothing an earlier run copied.
…add-on

aws-eks-addon is a fourth edition the Marketplace Release workflow can
ship, signed in with the same AWS role. aws_catalog.py builds its version as
one EKS add-on delivery option (AddDeliveryOptions with
EksAddOnDeliveryOptionDetails, Limited, the cluster's name filled in by EKS)
and checks it against AWS's own limits before sending it. The weekly run
adds no add-on version while AWS is still applying one or a version is in
testing; a run by hand ships whatever. go-public makes a tested version
public, by hand. ci_setup.py lets the release role add versions to the
add-on product once editions.json has its ID.
The Marketplace Release workflow ships aws-eks-addon like the AWS edition:
aws-eks-addon-push (marketplace-push) asks the add-on product whether it has
the release, and on the weekly run whether a version is still being
published or tested, then copies the images and pushes the chart, each
skipped when an earlier run did it; aws-eks-addon-publish
(marketplace-publish, approved by a maintainer) adds the version for
testing. The edition action takes the released agent chart too, and
build.py refuses one from before the add-on, which the marketplace suite
then skips.

The marketplace suite checks the add-on as AWS does: its helm lint and
helm template for every Kubernetes version it names, no hook, lookup, CRD
or Release object but .Name and .Namespace, every image from its registry
and named in its version, the key only read from the buyer's Secret, the
configuration examples and a Region change rendering.
What the add-on is and how it meets AWS's add-on rules, the one-time setup
on the AWS side (the free product, its ID, repositories, the release role,
the add-on delivery option and the seller alias), how a version is released,
tested in ap-northeast-2 on every Kubernetes version it names and made
public, its listing content, and what has not met AWS yet. The chart README's
section on the Secret moves below the AI agent's.
The Kubernetes agent page says how to add the agent from the EKS console or
with aws eks create-addon: the Secret with the ingestion key first, the
add-on's defaults (OneUptime Cloud, the EKS cluster's name, the
oneuptime-agent namespace), the configuration values it takes, with an
update-addon example, and what needs the Helm install instead. It also says
how a Helm install keeps the key in a Secret (oneuptime.existingSecretName),
and the cloud marketplaces page links to the add-on. In every docs
language, line for line, with the EKS console's labels in English.
…thout known critical CVEs

AWS Marketplace scans every image of the EKS add-on. Scanned with trivy
0.75.0 (HIGH and CRITICAL, linux/amd64 and linux/arm64):

- kube-state-metrics v2.13.0 -> v2.20.0: 1 critical and 49 high -> 9 high.
  The same flags; /healthz and / are still served on the probed ports.
- OpenCost 1.121.0 -> 1.121.3: 36 high -> 7 high (a patch release, the one
  OpenCost's own chart runs).
- Prometheus v2.55.1 -> v3.15.0: 4 critical and 104 high -> 8 high.
  promtool 3.15 accepts the bundled config, the flags and health paths are
  unchanged, OpenCost keys its range offsets on the Prometheus version, and
  the cost poller's max_over_time peak query reads the same on 3.x.

What is left is Go stdlib and golang.org/x issues newer than each upstream
build.
@simlarsen

Copy link
Copy Markdown
Contributor Author

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@simlarsen
simlarsen merged commit d67b40d into master Oct 18, 2026
7 of 64 checks passed
@simlarsen
simlarsen deleted the claude/cool-bassi-729295 branch October 18, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants