Rules

A Rule defines what to watch, where to watch it, and which Policy routes its alerts. This page lists every field and shows how one Rule can cover several alert types.

What a Rule is

A Rule has a scope (where to look), one or more alert types (what to look for), a Policy (where to send the alert) and a state (whether it is running). One Rule produces zero or more incidents over its lifetime.

A Rule does not name a channel. Channel routing lives on the Policy; see Policies.

The fields

A Rule has the following user-editable fields:

• Name: a short label, shown everywhere, such as "Production cost spike" or "Idle resources, staging".
• Scope: an organization, cluster, namespace, team or department. The scope is fixed when you create the rule; to retarget, create a new rule.
• Alert types: turn on one or more of cost spike, budget threshold, unused resources and new expensive workload. Each type you turn on has its own settings. At least one type must be on.
• Policy: the Policy that routes this rule's incidents to channels.
• Notify on resolve: whether to send a notification when the condition clears.

One rule, several types

One Rule can fire on more than one condition without duplicating the scope or the routing. For example, a single "Production cluster, finance" Rule can turn on both cost spike and budget threshold, each with its own settings, both routed to the same Slack channel through one Policy.

The settings for each type are independent: turning off cost spike does not clear the budget threshold settings.

Per-type conditions

Cost spike

Cost spike has no settings. The expected range and the minimum dollar amount come from the scope's spending history, so there is nothing to tune. The rule editor shows a banner explaining this when you turn the type on. See How cost-spike actually works.

Budget threshold

Five fields:

• Budget amount ($): the period total you've allocated to this scope.
• Period: monthly, quarterly, or fiscal year. Monthly resets on the 1st, quarterly on calendar quarter, fiscal on your org's fiscal year start.
• Cost mode: loaded (CPU + RAM + GPU + overhead) or workload_only (direct workload cost, no shared overhead). Match this to how you decided the budget.
• Notify at thresholds: pick any subset of 50%, 80%, 90%, 100%. At least one required. The rule fires once when each selected threshold is first crossed in the period.
• Rollover: currently fresh (a clean reset at the start of each period). The field is stored for future extension.

Period-to-date cost is recomputed on every snapshot, so threshold crossings show up within minutes of crossing.

Unused resources

Three fields:

• Minimum idle percentage (min_idle_pct): only flag a resource if its idle fraction exceeds this. Default 80%.
• Schedule: pinned to weekly. Stored as a field for future extension; today the rule always runs weekly.
• Top-N cap (top_n_cap): the digest lists at most this many idle items, sorted by cost descending. Default 20.

The digest delivery means you won't get pinged the moment a workload goes idle — you'll see it in the next weekly summary. Use this type to keep weekly cleanup on the team's calendar, not as a real-time alert.

New expensive workload

Four fields:

• Minimum cost ($/month): only flag workloads projected to cost at least this much per month. Default $750.
• Minimum age (hours): wait this long after the workload first appears before evaluating. Default 2 hours. Lets short-lived workloads finish without firing, and lets cost projections stabilize.
• Auto-exclude CRD-owned workloads: skip anything whose top-level owner is a custom resource (operator-managed). Recommended on. Catches Argo Workflows, Spark Operator, Ray, Kubeflow training jobs.
• Exclude owner kinds: explicit list of owner Kinds that should never trigger this rule. Defaults cover Job, CronJob, and common ML operators. Add your own platform's Kinds as needed.

Rule state

A Rule is always in one of four states. The state decides whether evaluation happens and whether incidents fire.

Muting takes a duration (4 hours, 24 hours, 7 days, or forever-until-reactivated). Disable is the right choice when you know you won't need the rule for weeks but want to keep its history.

Previewing a rule before saving

The rule editor previews a draft rule against the last 30 days of data and shows:

• Notification volume: low, medium, or high bucket based on how many times the rule would have fired in the preview window.
• Warning: flags noisy rules (noisy), rules that would have fired so often they should be blocked (block_noisy), budgets that look implausibly small (tiny_budget), or near-duplicates of existing rules (duplicate_rule).

The preview is a guide, not a forecast. A change in your cluster's behavior tomorrow won't be in the preview window today.

Editing a rule

Most fields can be changed in place. Two cannot:

• Scope: fixed at creation. Retargeting requires a new rule.
• The settings each alert type offers: you can turn a type on or off, but its settings are fixed by the type.

Changing the policy, the conditions of an enabled type, or the rule's state takes effect on the next scheduled evaluation.

Deleting a rule

Deleting a rule removes it from the rule list and stops further evaluation. Existing incidents are preserved in the incidents archive so post-mortems remain accurate. To suppress the incident history too, delete the linked policy first.

What's not here

• No multi-scope rules. One rule, one scope. Use a Policy to route multiple rules to the same channels instead.
• No conditional logic between types. Enabled types fire independently — there's no "fire cost_spike only if budget_threshold also fires" wiring.
• No custom alert types. The four types in the rule editor are the four types.

Next steps

• Policies: route a rule's incidents to one or more channels with a delivery mode and a daily cap.
• Channels: configure the actual destinations.