Skip to content
All releases
feat
(8mo ago)

Shipped by

  • Ugurcan CaykaraUgurcan Caykara

Smart Alerting: Catch Cost Issues Before They Hit the Bill

Rules, channels and notification policies that watch your spend: cost-spike detection fitted to each scope's history, budget alerts and new-workload alerts.

Update, : Corrected to match what ships. Cost-spike detection is now described as it works today. Quiet hours, timed mutes, unused-resources alerts and notify-only optimization alerts were removed from this entry because they are not available.

What shipped

Cluster spend drifts quietly. A bad rollout doubles CPU requests, a forgotten job runs all weekend, a new workload lands at $900/month. Usually you find out when the invoice arrives.

Smart Alerting watches your spend and notifies you when it moves. You set it up with three building blocks: channels, policies and rules.

The building blocks

Rules: what to watch

A step-by-step editor: name the rule, pick a scope, choose alert types, attach a notification policy, review. One rule per scope, with any number of alert types on it. Scopes go from broad to narrow:

Organization > Cluster > Namespace > Team > Department

Team and Department scopes use your cost attribution, so you can alert on "Team Payments crossed 90% of budget" without adding a single label. You can mute a rule when you already know about something.

Channels: where alerts land

ChannelUse it forConfig
SlackTeam channelsSlack incoming webhook URL
EmailPeople and mailing listsOne address or a list
WebhookAny HTTP integrationJSON payload, HMAC-SHA256 signed when you set a signing secret
In-appThe Alerts pageBuilt in, nothing to configure

Send a test to any channel before you rely on it. Each channel shows its delivery health, and failed deliveries are retried.

Notification policies: how and when

Policies control how alerts go out, and you can reuse one policy across many rules:

  • Delivery mode: immediate, daily digest (sent at midnight UTC) or in-app only
  • Daily cap: alerts over the cap roll into a digest instead of flooding the channel

Edit the policy once and every rule attached to it follows.

How detection works

Cost spike. Kubeadapt fits a normal range to each scope's own daily cost history, including weekday patterns, and compares each day's spend with it. It opens an incident only when spend stays outside that range and the excess clears a minimum dollar amount. You don't set a threshold. A new scope needs about two weeks of daily cost history before detection starts.

Incident explanations. Every incident shows how unusual the change is, whether it is concentrated in one workload, whether nodes or workloads drove it, and each contributing workload's likely cause. With Kubeadapt AI turned on, Explain this incident writes this up in plain language; that entry lists exactly what data the model sees.

Noise control. Daily caps keep channels calm. New-workload alerts skip workloads owned by operator custom resources (Argo, Spark, Ray, Kubeflow) by default.

Alert types

  • Cost spike: spend outside the scope's normal range, as described above
  • Budget threshold: monthly, quarterly or annual budgets, with notifications at 50, 80, 90 and 100%; count fully loaded cost or workload cost only
  • New expensive workload: a newly deployed workload that crosses a monthly cost you set ($750/month by default) after a minimum age

Incidents

Every alert that fires becomes an incident. Filter by status, time range or type. Each incident shows the cost change per day, how long it has been firing and the top contributing workload, plus a log of every notification sent for it. Work them with Acknowledge, Snooze and Resolve. States move through Pending, Firing, Acknowledged, Snoozed and Resolved.

A firing incident reads like this:

Cost spike on Team Payments: checkout-worker CPU requests went from 80 to 220 cores, +$620/day against its normal range.

Getting started

Go to Alerts. Connect a channel, create a notification policy, then add your first rule; the built-in guide walks you through all three. For details, see Smart Alerting.