Introducing Smart Alerting

Most teams find out about a cost spike one of two ways. Someone in finance forwards last month's invoice with a question mark next to a line item. Or an engineer notices a namespace looks busier than usual and starts digging around. Both happen after the money is already spent.

Today we're launching Smart Alerting inside Kubeadapt. It watches every cluster, namespace, and team you track, learns what normal spending looks like for each one, and opens an incident the moment something drifts away from it. No dashboard to check every morning, no threshold to guess at.

Channels, policies, and rules

The model behind Smart Alerting has three pieces, and they're deliberately separate so you can reuse each one instead of rebuilding it for every alert.

A channel is where an alert goes: Slack, email, or a signed webhook. A policy decides how it gets delivered, immediately, bundled into a daily digest, or kept in-app only, plus a daily cap and quiet hours so one bad afternoon doesn't turn into fifty notifications. A rule ties a scope, an organization, a cluster, a namespace, a team, a department, to one or more alert types and a policy.

You attach one policy to as many rules as you want. Change the delivery behavior once and every rule using it updates. And Kubeadapt won't let two rules cover the exact same scope with conflicting settings; it catches that at creation time, not three weeks later when someone asks why they got paged twice.

Cost Spike doesn't ask you to guess a threshold

This is the one alert type we'd call automatic without qualifying it. Most alerting tools ask you to type in a number: notify me if spend crosses $X. That number is almost always wrong within a month. A Black Friday traffic spike looks like an incident to a static threshold and a normal Tuesday to a team that planned for it.

Cost Spike learns each scope's own baseline instead, a rolling seven-day median, and adjusts as spending patterns shift. A brand-new namespace needs about a week of history before detection turns on. Until then it stays quiet rather than firing false alarms on day one.

Two more alert types for the cases a rule already covers well

Not everything needs to be adaptive. Two alert types handle situations where a fixed condition is exactly what you want:

Budget Threshold opens an incident when spend crosses a percentage of a budget you set: monthly, quarterly, or aligned to your fiscal calendar, with notifications at 50%, 80%, 90%, and 100%. You choose whether that budget counts loaded cost (CPU, RAM, GPU, and shared overhead) or workload cost alone.

New Expensive Workload flags a newly deployed workload once it crosses a monthly cost floor you set, with a minimum age so short-lived jobs don't trip it, and exclusions for Jobs, CronJobs, Argo, Spark, Ray, and Kubeflow-owned workloads that are supposed to be expensive and temporary.

A third type, Unused Resources, is on the roadmap for v1.1. We'd rather ship idle-resource detection once it's been tested against enough real clusters than ship a version that buries you in false positives.

Every incident explains itself

An incident firing is only half the job. The other half is telling you whether it matters and what caused it.

When Smart Alerting opens an incident, it rates how unusual the change is against history: unprecedented, rare, normal, or common. It tells you whether the increase is concentrated in one workload or spread across many. It classifies whether a node-level change or a workload-level change drove it. And it shows how much of the dollar delta it can actually explain versus what's still unaccounted for, so you know when to trust the summary and when to look closer yourself.

Each contributing workload gets tagged with a likely cause: a CPU or memory request change, a replica count change, a new image, an autoscaler bound that moved, an OOM burst, a crashloop. That's the difference between "spend went up 14%" and "the checkout service's replica count doubled at 2 AM and nobody scaled it back down."

Ask it to explain the incident in plain language

If the structured breakdown isn't enough context on its own, every incident has an "Explain this incident" button. It streams a plain-language write-up of what happened, why it matters, and what to do next, built from the same findings above, with a confidence rating (high, medium, or low) and an indicator for how much observation history backed the analysis. It's metered per workspace: if the budget for it runs out during a billing period, you'll see that stated plainly instead of the feature quietly going away.

Delivery you can actually verify

An alert that fires and disappears into a channel nobody's watching is worse than no alert at all. Every notification attempt gets logged: delivered, failed and retrying, suppressed because the rule was snoozed or muted, deferred into a digest, or sent in-app only, each with the exact rendered message you can open and review after the fact. If a Slack webhook starts failing, the channel shows unhealthy before your next incident tries to route through it and vanishes without a trace.

Getting started

Wire up a Slack webhook, an email address, or a signed webhook as your first channel. Group your workloads into rules by scope. Cost Spike starts building its baseline the moment a rule goes live, and you'll see automatic detection kick in within about a week.

Try Kubeadapt or read the setup guide in the docs to turn Smart Alerting on for your own clusters.