Plugin Auto-Fetching from Catalogs#

This tutorial explains how scafctl automatically fetches plugin binaries from remote catalogs at runtime. You can declare plugin dependencies in your solution, and scafctl will resolve, download, cache, and load them without a prior build step.

Overview#

The plugin auto-fetch flow:

Solution declares         Catalog chain            Plugin cache         Provider
plugin dependencies  →  resolves version     →   checks cache     →   registration
                        (local → remote)         (cache hit/miss)     (gRPC plugin)

1. Declare plugin dependencies in your solution’s bundle.plugins section
2. Resolve — scafctl looks up the plugin version in the catalog chain (local first, then remote OCI registries)
3. Cache — if the binary is already cached locally, it’s reused; otherwise it’s fetched and written to the cache
4. Load — the cached binary is launched as a gRPC plugin and its providers are registered

Declaring Plugin Dependencies#

Add a bundle.plugins section to your solution:

apiVersion: scafctl.io/v1
kind: Solution
metadata:
  name: my-solution
  version: 1.0.0
spec:
  resolvers:
    data:
      resolve:
        with:
          - provider: custom-provider
            inputs:
              query: "SELECT * FROM table"
  bundle:
    plugins:
      - name: custom-provider
        kind: provider
        version: "^1.0.0"

Fields#

Version constraints follow semver conventions:

• ^1.5.0 — any 1.x.y where x ≥ 5
• ~1.5.0 — any 1.5.x
• >=2.0.0 — 2.0.0 or higher
• 1.2.3 — exact match

Pre-Fetching Plugins#

Use scafctl plugins install to download plugin binaries before running a solution:

# Install plugins for a solution
scafctl plugins install -f solution.yaml

# Install for a specific platform (useful in CI)
scafctl plugins install -f solution.yaml --platform linux/amd64

# Use a custom cache directory
scafctl plugins install -f solution.yaml --cache-dir /tmp/plugins

# Install plugins for a solution
scafctl plugins install -f solution.yaml

# Install for a specific platform (useful in CI)
scafctl plugins install -f solution.yaml --platform linux/amd64

# Use a custom cache directory
scafctl plugins install -f solution.yaml --cache-dir /tmp/plugins

This is useful for:

• CI/CD: Pre-fetch plugins in a setup step, then run solutions offline
• Air-gapped environments: Fetch once on a connected machine, copy the cache
• Reproducibility: Pin versions with a lock file, then install from locks

Listing Cached Plugins#

View what’s in your local plugin cache:

# Table view (default)
scafctl plugins list

# JSON output
scafctl plugins list -o json

# YAML output
scafctl plugins list -o yaml

# Table view (default)
scafctl plugins list

# JSON output
scafctl plugins list -o json

# YAML output
scafctl plugins list -o yaml

Note: By default, plugins list shows only the latest cached version of each plugin (per platform). Pass --all-versions (or its alias --all) to see every cached version:

scafctl plugins list --all-versions

Lock Files for Reproducibility#

When you package a solution with scafctl package solution, plugin versions are pinned in a lock file (.scafctl.lock.yaml). The lock file records, for every plugin:

• Exact resolved version
• Content digest (sha256)
• Source catalog alias (resolvedFrom) and the machine-independent canonical origin (resolvedCanonical) it was fetched from

The resolvedCanonical origin is what makes the pin portable across machines and catalog aliases; plugins declared with an explicit source block additionally record that source block. See How the origin is recorded in the lock file .

When running with a lock file, scafctl uses the pinned versions exactly. Without a lock file, scafctl resolves from catalogs and requires the catalog to provide a digest. If no digest is available, the fetch fails:

plugin my-plugin@1.0.0: no digest available for verification;
Run 'scafctl package solution' to generate a lock file with pinned digests

This mandatory digest verification prevents supply chain attacks where a compromised catalog or man-in-the-middle attacker could serve a malicious binary. Always use lock files for production deployments.

How the lock file is consulted – exact pins, constraint ranges, or best-effort – is controlled by the lock mode. See Plugin Lock Modes for the --lock-mode flag and the source-based defaults.

Signature Verification#

Beyond digest verification, scafctl supports Sigstore/cosign keyless signature verification for plugin binaries. This provides cryptographic proof that a plugin was built by a trusted identity.

Configuring Signature Verification#

Add the plugins.signatures section to your config:

# ~/.config/scafctl/config.yaml
plugins:
  signatures:
    mode: "warn"          # or "enforce"
    trustedIssuers:
      - "https://token.actions.githubusercontent.com"
    trustedIdentities:
      - "https://github.com/oakwood-commons/*"

Verification Modes#

How Verification Works#

When a plugin binary is fetched from a catalog with signature verification enabled:

1. The OCI artifact digest is resolved from the catalog
2. scafctl queries the Rekor transparency log for a cosign signature
3. The signing certificate is validated against Fulcio CA roots
4. The certificate’s OIDC issuer and identity are matched against the policy
5. On success, the verification result is logged (issuer, identity, timestamp)
6. On failure, the mode determines the outcome (warn or fail)

Build Tag Requirement#

Signature verification requires the cosign build tag:

# Build scafctl with cosign signature verification support
go build -tags cosign -o scafctl ./cmd/scafctl/scafctl.go

Without the build tag, the stub verifier logs a warning (warn mode) or returns an error (enforce mode) indicating that cosign support is not compiled in.

Embedder Policy Override#

Embedders can enforce a signature policy programmatically, overriding user config:

opts := &scafctl.RootOptions{
    PluginSignaturePolicy: &plugin.SignaturePolicy{
        Mode:              plugin.SignatureModeEnforce,
        TrustedIssuers:    []string{"https://token.actions.githubusercontent.com"},
        TrustedIdentities: []string{"https://github.com/my-org/*"},
    },
}

CI/CD Recommendation#

For production CI/CD pipelines, combine signature enforcement with strict mode:

# Enforce both explicit plugin declarations and valid signatures
scafctl run solution -f solution.yaml --strict

With plugins.signatures.mode: "enforce" in the CI config, this ensures all plugins are declared, version-pinned, and cryptographically signed.

Catalog Chain#

Plugins are resolved through a catalog chain that tries sources in order:

1. Local catalog — $XDG_DATA_HOME/scafctl/catalog/
2. Remote OCI catalogs — configured in ~/.config/scafctl/config.yaml

Configuring Remote Catalogs#

Add OCI registries to your config:

# ~/.config/scafctl/config.yaml
catalogs:
  - name: company-registry
    type: oci
    url: registry.company.com/scafctl
  - name: community
    type: oci
    url: ghcr.io/scafctl-community

The chain stops at the first catalog that has the requested artifact.

Pinning a plugin to a specific registry#

By default a plugin is resolved by its short name through the catalog chain above – the first catalog that has an artifact of that name wins. To bind a plugin dependency to an explicit OCI registry instead, add a source block:

bundle:
  plugins:
    - name: exec                          # solution-local alias (the provider handle)
      kind: provider
      version: "^1.5.0"                    # semver constraint or "latest"
      source:
        registry: ghcr.io/oakwood-commons/providers   # OCI registry + namespace
        artifact: exec                                 # artifact leaf (defaults to name)

Field reference:

Behavior:

• Without source the dependency is local/short-name: resolved by name through the catalog chain (local catalog first, then remote catalogs in order).

• With source the dependency is sourced: fetched from the named registry, bypassing chain order. This is useful when the same short name exists in multiple catalogs, or to pin a provider to a trusted origin.

• name is only the solution-local alias you reference from a resolver or action’s provider: field. It can differ from source.artifact – when the two differ the dependency is “aliased”.

• source.registry must resolve to a configured catalog. scafctl will not fetch from an arbitrary registry: if the registry is not listed under catalogs, preparation fails with:

  provider registry "ghcr.io/oakwood-commons/providers" is not a configured
  catalog; add it to the catalogs configuration or reference a configured registry

  Add the registry to your config to authorize it:

  # ~/.config/scafctl/config.yaml
  catalogs:
    - name: oakwood
      type: oci
      url: ghcr.io/oakwood-commons/providers

For official first-party providers, the default registry (ghcr.io/oakwood-commons/providers/<name>) is already known, so a short name declaration is enough – an explicit source is only needed to override the origin. Use the list_official_providers MCP tool to see each official provider’s default catalog reference.

How the origin is recorded in the lock file#

Every plugin – short-name and sourced – records its resolved origin in the lock file, so later runs fetch the exact same artifact from the exact same place regardless of catalog chain order or local catalog aliases. The fields common to all lock entries are:

A plugin declared with an explicit source block additionally stores a source block in its lock entry:

Do not confuse the two: resolvedCanonical is present on every entry and is how scafctl records where a plugin was fetched from, while the source block is present only on entries whose bundle.plugins declaration used source.

The solution-local alias (name in bundle.plugins) is deliberately not stored – the lock records the resolved (canonical origin, artifact leaf, kind) identity instead. Two aliases that point at the same origin+artifact therefore dedupe to a single lock entry, and renaming the alias in the solution does not churn the lock.

A sourced lock entry is matched back to its dependency by the (source.registry, artifact leaf, kind) tuple; a short-name entry is matched by (artifact leaf, kind) against entries with no source block. Either way, because the canonical origin is stored, a teammate on a different machine who has the same registry configured under a different catalog alias still replays the identical pin. On the next build the pinned version is replayed as long as it still satisfies the current constraint; if you tighten or bump the constraint past the pin, that one entry is re-resolved and re-pinned while everything else stays frozen.

This is what makes a plugin deterministic: the origin (resolvedCanonical, plus source.registry when sourced), the artifact (name), the exact version, and the content digest are all frozen in the lock, so scafctl run solution pulls byte-identical plugins on every machine. Always commit the lock file for production deployments. See Plugin Lock Modes for how strictly those pins are enforced.

Plugin Cache#

Downloaded binaries are stored in a content-addressed cache:

$XDG_CACHE_HOME/scafctl/plugins/
└── custom-provider/
    └── 1.5.3/
        └── darwin-arm64/
            └── custom-provider    # executable binary

Cache Structure#

• <name>/<version>/<os>-<arch>/<name> — platform-safe directory layout
• Digest verification on cache reads (when lock file provides a digest)
• Atomic writes (temp file + rename) prevent corruption
• Cache is shared across all solutions

Managing the Cache#

# List cached plugins
scafctl plugins list

# Cache is at $XDG_CACHE_HOME/scafctl/plugins/
# To clear the entire cache:
rm -rf ~/.cache/scafctl/plugins/

# List cached plugins
scafctl plugins list

# Cache is at $env:LOCALAPPDATA\scafctl\plugins\ (Windows)
# To clear the entire cache:
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\scafctl\plugins\"

Schema Caching (Offline Access)#

In addition to caching plugin binaries, scafctl caches provider descriptor schemas locally. This means that once a plugin provider’s schema has been fetched via get_provider_schema or get_provider_output_shape in the MCP server, subsequent schema lookups can be served from disk without spawning the plugin process or requiring network access.

How It Works#

1. When get_provider_schema or get_provider_output_shape successfully resolves a plugin provider, the full descriptor (input schema, output schemas, capabilities, examples) is persisted to disk.
2. On subsequent requests, if the plugin binary is unavailable (offline, not yet installed, network error), the cached descriptor is returned with "source": "cached" in the response.
3. Cache entries expire after 24 hours by default. After expiry, scafctl attempts a fresh fetch and updates the cache.
4. Running scafctl plugins install automatically invalidates cached descriptors for the installed plugins so the next schema request returns up-to-date data.

Cache Location#

$XDG_CACHE_HOME/scafctl/provider-schemas/
|-- exec.json
|-- git.json
|-- github.json
|-- directory.json
|-- ...

Managing the Schema Cache#

# Schema cache is at $XDG_CACHE_HOME/scafctl/provider-schemas/
# To clear all cached schemas (forces re-fetch on next access):
rm -rf ~/.cache/scafctl/provider-schemas/

# To invalidate a single provider:
rm ~/.cache/scafctl/provider-schemas/exec.json

# Schema cache is at $env:LOCALAPPDATA\cache\scafctl\provider-schemas\
# To clear all cached schemas:
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\cache\scafctl\provider-schemas\"

# To invalidate a single provider:
Remove-Item "$env:LOCALAPPDATA\cache\scafctl\provider-schemas\exec.json"

Use Cases#

• CI/CD: Pre-fetch plugin schemas in a warm-up step, then run MCP sessions offline
• Air-gapped environments: Copy the provider-schemas/ directory to disconnected machines
• Faster MCP startup: Cached schemas avoid spawning plugin processes for schema-only queries

Multi-Platform Support#

Plugin artifacts can include platform-specific binaries. The AnnotationPlatform annotation on catalog artifacts identifies the target platform:

dev.scafctl.plugin.platform: linux/amd64

When fetching, scafctl:

1. Lists all artifacts for the plugin version
2. Matches the dev.scafctl.plugin.platform annotation against the current (or requested) platform
3. Falls back to a direct fetch if no platform annotation exists (single-platform plugin)

Specifying a Target Platform#

# Fetch for a different platform
scafctl plugins install -f solution.yaml --platform linux/amd64

# Fetch for a different platform
scafctl plugins install -f solution.yaml --platform linux/amd64

This is useful for cross-platform CI where you build on one architecture but deploy on another.

Runtime Auto-Fetch (During Solution Execution)#

When you run a solution that declares plugin dependencies:

scafctl run solution -f solution.yaml

scafctl run solution -f solution.yaml

The prepare phase automatically:

1. Reads bundle.plugins from the solution
2. Checks the lock file for pinned versions
3. Fetches any missing plugins from the catalog chain
4. Caches the binaries locally
5. Loads the plugins and registers their providers
6. Cleans up plugin processes on exit

No explicit plugins install step is needed – but pre-fetching is recommended for predictability.

Official Providers#

scafctl ships a built-in registry of official providers distributed as external plugins from ghcr.io/oakwood-commons/providers/<name>:

Official providers are plugins, not built-ins. That distinction matters for how they are resolved.

Solution execution requires explicit declaration#

During solution execution – scafctl run solution, scafctl run resolver, and scafctl render solution – every non-built-in provider a solution references must be declared in bundle.plugins, including official providers. There is no silent auto-fetch on this path.

If a solution references an official provider (for example exec) without declaring it, preparation fails before any network fetch is attempted:

providers [exec] are not builtins and are not declared in bundle.plugins;
add them to bundle.plugins

Declaring the provider fixes it:

bundle:
  plugins:
    - name: exec
      kind: provider
      version: "^1.0.0"

This keeps runs deterministic and auditable: the complete set of external plugins a solution depends on is always visible in the solution and pinned in its lock file. See Plugin Lock Modes for how those pins are applied.

run provider <name> auto-resolves#

The direct provider-invocation command, scafctl run provider <name>, is the one place that still fetches an official provider on demand. It is a convenience for ad-hoc use and is not part of solution execution:

# Fetches the exec provider plugin on first use, then runs it
scafctl run provider exec command='echo hello' -o json

# Fetches the env provider plugin
scafctl run provider env name=HOME -o json

# Fetches the exec provider plugin on first use, then runs it
scafctl run provider exec command='echo hello' -o json

# Fetches the env provider plugin
scafctl run provider env name=HOME -o json

The binary is fetched on first use and cached for subsequent calls. Dynamic help (--help) also triggers the fetch so you can view the provider’s schema.

API server pre-loading#

When you run the API server (scafctl serve), official providers are pre-loaded into the shared plugin pool at startup as a warm-up optimization. This does not change request-time behavior: a solution submitted to the server must still declare its providers in bundle.plugins, exactly like the CLI, or preparation fails with the same undeclared-provider error.

Disabling official-provider resolution#

To turn off the run provider convenience fetch and the API startup pre-load (for air-gapped or restricted environments), disable official providers in your config:

# ~/.config/scafctl/config.yaml
settings:
  disableOfficialProviders: true

--strict and official auth handlers#

Unlike providers, official auth handlers (fetched when a solution references them via the identity provider) are still auto-resolved during solution execution. The --strict flag on run and render disables that auth-handler auto-resolution, requiring every auth handler to be declared in bundle.plugins as well:

# Also require official auth handlers to be declared explicitly
scafctl run solution -f ./solution.yaml --strict
scafctl run resolver -f ./solution.yaml --strict

# Also require official auth handlers to be declared explicitly
scafctl run solution -f ./solution.yaml --strict
scafctl run resolver -f ./solution.yaml --strict

Use --strict in CI so that neither providers nor auth handlers rely on implicit resolution.

Example: End-to-End Workflow#

# 1. Develop your solution with plugin dependencies
cat > solution.yaml << 'EOF'
apiVersion: scafctl.io/v1
kind: Solution
metadata:
  name: data-pipeline
  version: 1.0.0
spec:
  resolvers:
    data:
      resolve:
        with:
          - provider: my-db-provider
            inputs:
              connection: "postgres://localhost/db"
  bundle:
    plugins:
      - name: my-db-provider
        kind: provider
        version: "^2.0.0"
EOF

# 2. Build to create a lock file (pins plugin versions)
scafctl package solution -f solution.yaml --version 1.0.0

# 3. Pre-fetch plugins (optional but recommended)
scafctl plugins install -f solution.yaml

# 4. Run the solution (plugins loaded from cache)
scafctl run solution -f solution.yaml

# 5. Check what's cached
scafctl plugins list

# 1. Develop your solution with plugin dependencies
@'
apiVersion: scafctl.io/v1
kind: Solution
metadata:
  name: data-pipeline
  version: 1.0.0
spec:
  resolvers:
    data:
      resolve:
        with:
          - provider: my-db-provider
            inputs:
              connection: "postgres://localhost/db"
  bundle:
    plugins:
      - name: my-db-provider
        kind: provider
        version: "^2.0.0"
'@ | Set-Content solution.yaml

# 2. Build to create a lock file (pins plugin versions)
scafctl package solution -f solution.yaml --version 1.0.0

# 3. Pre-fetch plugins (optional but recommended)
scafctl plugins install -f solution.yaml

# 4. Run the solution (plugins loaded from cache)
scafctl run solution -f solution.yaml

# 5. Check what's cached
scafctl plugins list

Troubleshooting#

Plugin not found in any catalog#

Error: plugin my-plugin (provider): resolving version: ...not found in any catalog

• Verify the plugin is published to a configured catalog
• Check scafctl catalog list --kind provider to see available providers
• Ensure your config has the correct remote catalog URL

Version constraint not satisfied#

Error: resolved version 3.0.0 does not satisfy constraint ^1.0.0

• The catalog’s latest version doesn’t match your constraint
• Update the constraint in your solution, or publish a compatible version

Cache corruption#

If a cached binary seems corrupt:

# Remove the specific plugin from cache
rm -rf ~/.cache/scafctl/plugins/<plugin-name>/<version>/

# Re-fetch
scafctl plugins install -f solution.yaml

# Remove the specific plugin from cache
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\scafctl\plugins\<plugin-name>\<version>\"

# Re-fetch
scafctl plugins install -f solution.yaml