Config Changes Not Applying

You published a policy update, but agents still show the old config or report errors. This guide helps you diagnose why.

Quick Diagnostic Flow

Published policy → Agent shows "Reconciling"? → Wait 10-30 seconds
                                    ↓
                            Agent shows "Error"? → Check validation
                                    ↓
                            Agent shows "Applied" but old behavior? → Check drift
                                    ↓
                            Agent not in target selector? → Check labels

1. Check the Agent's Target Selector Match

The most common reason: the agent doesn't match the policy's target selector.

Verify in the Dashboard

1. Go to Fleet → Overview
2. Click on the agent's name to open the detail view
3. Check the Labels section
4. Compare with your policy's matchLabels / matchExpressions

Example Mismatch

Policy selector:

{"matchLabels": {"env": "production"}}

Agent labels:

{"env": "staging", "host.os": "linux"}

→ No match. The agent won't receive this policy.

Fix

Either:

• Update the policy selector to include the agent's labels
• Or add the expected label to the agent's resource attributes

2. Check Config Validation Errors

Before applying any config, the Supervisor runs a validation check:

otelcol --config /tmp/staged-config.yaml validate

If this fails, the config is rejected and the agent reports Config Error.

Read the Validation Error

Windows:

PS >Get-Content 'C:\ProgramData\CollectorCtrlSupervisor\supervisor.log' -Tail 30 | Select-String 'validation|error|failed'

Linux:

$sudo journalctl -u collectorctrl-supervisor -n 30 --no-pager | grep -i 'validation\|error\|failed'

Common Validation Errors

3. Check for Config Drift

If the agent shows Applied but the collector behavior hasn't changed, the Supervisor may be fighting a manual edit.

How Drift Detection Works

1. Supervisor receives new config from server
2. Writes it to the local config file
3. Continuously monitors the file hash
4. If someone edits the file manually → hash changes → Supervisor overwrites it
5. This creates a loop that appears as constant "Reconciling"

Check for Drift

1. Go to Fleet → Overview
2. Look at the Remote Config column
3. If it cycles between "Applied" and "Reconciling", drift is occurring

Fix

Stop manually editing config files on agent machines. All config changes should go through CollectorCtrl policies.

If you need node-specific tweaks, use the Override Config feature:

1. Click on the agent in the Fleet Overview
2. Go to the Config tab
3. Edit the Override Config
4. Click Save

The Supervisor will merge the override with the policy and apply the result.

4. Check the Config Apply on the Agent

When the effective configuration changes, the Supervisor writes it to disk and performs a fast, supervised restart of the collector process. If the new configuration is broken, self-healing logic detects the crash loop and falls back to a known-good state.

Check the Supervisor Logs

$sudo journalctl -u collectorctrl-supervisor -n 50 --no-pager

Look for apply errors or repeated collector restarts:

[OpAMP] Received remote config
[Config] Writing effective config
[Agent] Config is changed. Signal to restart the agent.

If the agent is stuck in a crash loop, fix the policy YAML in the console and republish — the Supervisor will apply the corrected config automatically.

Validate the Config Manually

Run the collector's built-in validator against the pushed config to catch YAML or pipeline errors:

$otelcol validate --config /etc/otelcol/config.yaml

On Windows: otelcol.exe validate --config "C:\Program Files\otelcol\config.yaml"

5. Check Policy Priority Conflicts

An agent can match multiple policies. CollectorCtrl resolves this by priority — higher priority wins.

Check for Overlapping Policies

1. Go to Fleet → Policies
2. Check the Priority column for each policy
3. If two policies target the same agent, the higher-numbered one takes precedence

Example Conflict

If your new policy has lower priority than an existing one, it will be overridden.

Fix

Increase your policy's priority number so it takes precedence.

6. Check the Effective Config

The Effective Config is what actually runs on the collector. It's the result of:

Policy Template + Override Config = Effective Config

View the Effective Config

1. Click on an agent in the Fleet Overview
2. Go to the Config tab
3. View the Effective Config panel

Compare this with what you expected. If the merge didn't produce what you wanted, check:

• YAML list overwrite behavior (lists are replaced, not merged)
• Indentation in the override config
• Matching keys between policy and override

7. Policy Not Published

Sounds obvious, but it's easy to miss: did you click Publish?

• Saving a policy as a draft does not push it to agents
• Only clicking Publish triggers the OpAMP broadcast
• Check the policy status — it should show "Active", not "Draft"

Summary Table

Next Steps

• Read about Fleet Policies
• Read about config merge logic
• Agent won't connect?