Custom Binary Builder

CollectorCtrl features an integrated compilation interface powered by the OpenTelemetry Collector Builder (OCB). This tool allows infrastructure teams to assemble, compile, and distribute custom OTel Collector binaries directly from the administration dashboard, without requiring local Go environments or command-line compilation scripts.

---

1. The Need for Custom Builders

Using pre-compiled, upstream distributions (like OTel Core or Contrib) often introduces drawbacks in enterprise environments:

• Binary Bloat: Upstream distributions compile in hundreds of community plugins. This increases the memory footprint of your collector daemons on thousands of virtual machines.
• Component Deficits: Custom vendor distributions may compile out competing exporters or processors (e.g. excluding a S3 cold archiving exporter).
• Compliance & Security: Minimizing the number of active components inside the binary reduces the attack surface and simplifies container vulnerability scans (CVE audits).

---

2. Using the Compilation Interface

The Custom Builder dashboard provides a guided workflow to configure, build, and register custom binaries.

Step-by-Step Compilation Workflow

1. Target Architecture Selection: Define the system targets for compilation:

   • Target OS: windows, linux, or darwin.
   • CPU Architecture: amd64 or arm64.

2. Component Checklist Selection: Browse and check components categorized into standard OpenTelemetry blocks:

   • Receivers: otlp, hostmetrics, windowsperfcounters, prometheus, etc.
   • Processors: batch, memory_limiter, resourcedetection, filter, etc.
   • Exporters: otlp, otlphttp, awss3, prometheus, clickhouse, etc.
   • Extensions: health_check, pprof, zpages, etc.

3. Triggering the Build Engine: When you click Build Collector, the CollectorCtrl backend initiates the compiler execution pipeline:

   • It dynamically generates the builder manifest YAML (builder-config.yaml) containing module repository paths:

     dist:
       name: otelcol-custom
       description: Custom Enterprise OTel Collector
       version: 0.1.0
       output_path: /tmp/otelcol-builds
     receivers:
       - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v0.95.0
     exporters:
       - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/awss3exporter v0.95.0
     

   • Spawns the official ocb binary under the hood, passing the generated builder configuration file.
   • Compiles the custom binary dynamically on the host (requiring go to be installed on the Management Server host).

---

3. Real-Time Status & Package Syncing

Build Console Stream

During compilation, the builder UI streams the compiler's output logs in real-time. You can monitor each stage of the build process:

• Go modules downloading and dependency resolution.
• Code generation.
• Binary compilation and link-editing stages.
• Compilation success or error status output.

Automated Package Registry Sync

Once compilation finishes successfully:

1. The server moves the compiled binary file to the COLLECTORCTRL_PACKAGES_DIR directory.
2. It automatically registers the new binary inside the Package Management database.
3. Calculates the cryptographic SHA-256 hash.
4. Makes the custom package immediately available for deployment via Fleet Orchestration upgrade policies.