Plugin Auto-Fetching from Catalogs#

This tutorial explains how scafctl automatically fetches plugin binaries from remote catalogs at runtime. You can declare plugin dependencies in your solution, and scafctl will resolve, download, cache, and load them without a prior build step.

Overview#

The plugin auto-fetch flow:

Solution declares         Catalog chain            Plugin cache         Provider
plugin dependencies  →  resolves version     →   checks cache     →   registration
                        (local → remote)         (cache hit/miss)     (gRPC plugin)

Declare plugin dependencies in your solution’s bundle.plugins section
Resolve — scafctl looks up the plugin version in the catalog chain (local first, then remote OCI registries)
Cache — if the binary is already cached locally, it’s reused; otherwise it’s fetched and written to the cache
Load — the cached binary is launched as a gRPC plugin and its providers are registered

Declaring Plugin Dependencies#

Add a bundle.plugins section to your solution:

apiVersion: scafctl.io/v1
kind: Solution
metadata:
  name: my-solution
  version: 1.0.0
spec:
  resolvers:
    data:
      resolve:
        with:
          - provider: custom-provider
            inputs:
              query: "SELECT * FROM table"
  bundle:
    plugins:
      - name: custom-provider
        kind: provider
        version: "^1.0.0"

Fields#

Field	Description	Example
`name`	Plugin catalog reference	`aws-provider`
`kind`	Plugin type: `provider` or `auth-handler`	`provider`
`version`	Semver constraint	`^1.5.0`, `>=2.0.0`, `1.2.3`

Version constraints follow semver conventions:

^1.5.0 — any 1.x.y where x ≥ 5
~1.5.0 — any 1.5.x
>=2.0.0 — 2.0.0 or higher
1.2.3 — exact match

Pre-Fetching Plugins#

Use scafctl plugins install to download plugin binaries before running a solution:

Bash

# Install plugins for a solution
scafctl plugins install -f solution.yaml

# Install for a specific platform (useful in CI)
scafctl plugins install -f solution.yaml --platform linux/amd64

# Use a custom cache directory
scafctl plugins install -f solution.yaml --cache-dir /tmp/plugins

PowerShell

# Install plugins for a solution
scafctl plugins install -f solution.yaml

# Install for a specific platform (useful in CI)
scafctl plugins install -f solution.yaml --platform linux/amd64

# Use a custom cache directory
scafctl plugins install -f solution.yaml --cache-dir /tmp/plugins

This is useful for:

CI/CD: Pre-fetch plugins in a setup step, then run solutions offline
Air-gapped environments: Fetch once on a connected machine, copy the cache
Reproducibility: Pin versions with a lock file, then install from locks

Listing Cached Plugins#

View what’s in your local plugin cache:

Bash

# Table view (default)
scafctl plugins list

# JSON output
scafctl plugins list -o json

# YAML output
scafctl plugins list -o yaml

PowerShell

# Table view (default)
scafctl plugins list

# JSON output
scafctl plugins list -o json

# YAML output
scafctl plugins list -o yaml

Lock Files for Reproducibility#

When you package a solution with scafctl package solution, plugin versions are pinned in a lock file (.scafctl.lock.yaml). The lock file records:

Exact resolved version
Content digest (sha256)
Source catalog

When running with a lock file, scafctl uses the pinned versions exactly. Without a lock file, scafctl resolves from catalogs and requires the catalog to provide a digest. If no digest is available, the fetch fails:

plugin my-plugin@1.0.0: no digest available for verification;
Run 'scafctl package solution' to generate a lock file with pinned digests

This mandatory digest verification prevents supply chain attacks where a compromised catalog or man-in-the-middle attacker could serve a malicious binary. Always use lock files for production deployments.

Signature Verification#

Beyond digest verification, scafctl supports Sigstore/cosign keyless signature verification for plugin binaries. This provides cryptographic proof that a plugin was built by a trusted identity.

Configuring Signature Verification#

Add the plugins.signatures section to your config:

# ~/.config/scafctl/config.yaml
plugins:
  signatures:
    mode: "warn"          # or "enforce"
    trustedIssuers:
      - "https://token.actions.githubusercontent.com"
    trustedIdentities:
      - "https://github.com/oakwood-commons/*"

Verification Modes#

Mode	Behavior
`off` (default)	No signature check; digest verification only
`warn`	Verify signature; log a warning on failure but continue execution
`enforce`	Verify signature; fail with an error on missing or invalid signature

How Verification Works#

When a plugin binary is fetched from a catalog with signature verification enabled:

The OCI artifact digest is resolved from the catalog
scafctl queries the Rekor transparency log for a cosign signature
The signing certificate is validated against Fulcio CA roots
The certificate’s OIDC issuer and identity are matched against the policy
On success, the verification result is logged (issuer, identity, timestamp)
On failure, the mode determines the outcome (warn or fail)

Build Tag Requirement#

Signature verification requires the cosign build tag:

Bash

# Build scafctl with cosign signature verification support
go build -tags cosign -o scafctl ./cmd/scafctl/scafctl.go

Without the build tag, the stub verifier logs a warning (warn mode) or returns an error (enforce mode) indicating that cosign support is not compiled in.

Embedder Policy Override#

Embedders can enforce a signature policy programmatically, overriding user config:

opts := &scafctl.RootOptions{
    PluginSignaturePolicy: &plugin.SignaturePolicy{
        Mode:              plugin.SignatureModeEnforce,
        TrustedIssuers:    []string{"https://token.actions.githubusercontent.com"},
        TrustedIdentities: []string{"https://github.com/my-org/*"},
    },
}

CI/CD Recommendation#

For production CI/CD pipelines, combine signature enforcement with strict mode:

Bash

# Enforce both explicit plugin declarations and valid signatures
scafctl run solution -f solution.yaml --strict

With plugins.signatures.mode: "enforce" in the CI config, this ensures all plugins are declared, version-pinned, and cryptographically signed.

Catalog Chain#

Plugins are resolved through a catalog chain that tries sources in order:

Local catalog — $XDG_DATA_HOME/scafctl/catalog/
Remote OCI catalogs — configured in ~/.config/scafctl/config.yaml

Configuring Remote Catalogs#

Add OCI registries to your config:

# ~/.config/scafctl/config.yaml
catalogs:
  - name: company-registry
    type: oci
    url: registry.company.com/scafctl
  - name: community
    type: oci
    url: ghcr.io/scafctl-community

The chain stops at the first catalog that has the requested artifact.

Plugin Cache#

Downloaded binaries are stored in a content-addressed cache:

$XDG_CACHE_HOME/scafctl/plugins/
└── custom-provider/
    └── 1.5.3/
        └── darwin-arm64/
            └── custom-provider    # executable binary

Cache Structure#

<name>/<version>/<os>-<arch>/<name> — platform-safe directory layout
Digest verification on cache reads (when lock file provides a digest)
Atomic writes (temp file + rename) prevent corruption
Cache is shared across all solutions

Managing the Cache#

Bash

# List cached plugins
scafctl plugins list

# Cache is at $XDG_CACHE_HOME/scafctl/plugins/
# To clear the entire cache:
rm -rf ~/.cache/scafctl/plugins/

PowerShell

# List cached plugins
scafctl plugins list

# Cache is at $env:LOCALAPPDATA\scafctl\plugins\ (Windows)
# To clear the entire cache:
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\scafctl\plugins\"

Schema Caching (Offline Access)#

In addition to caching plugin binaries, scafctl caches provider descriptor schemas locally. This means that once a plugin provider’s schema has been fetched via get_provider_schema or get_provider_output_shape in the MCP server, subsequent schema lookups can be served from disk without spawning the plugin process or requiring network access.

How It Works#

When get_provider_schema or get_provider_output_shape successfully resolves a plugin provider, the full descriptor (input schema, output schemas, capabilities, examples) is persisted to disk.
On subsequent requests, if the plugin binary is unavailable (offline, not yet installed, network error), the cached descriptor is returned with "source": "cached" in the response.
Cache entries expire after 24 hours by default. After expiry, scafctl attempts a fresh fetch and updates the cache.
Running scafctl plugins install automatically invalidates cached descriptors for the installed plugins so the next schema request returns up-to-date data.

Cache Location#

$XDG_CACHE_HOME/scafctl/provider-schemas/
|-- exec.json
|-- git.json
|-- github.json
|-- directory.json
|-- ...

Platform	Default Path
Linux	`~/.cache/scafctl/provider-schemas/`
macOS	`~/.cache/scafctl/provider-schemas/`
Windows	`%LOCALAPPDATA%\cache\scafctl\provider-schemas\`

Managing the Schema Cache#

Bash

# Schema cache is at $XDG_CACHE_HOME/scafctl/provider-schemas/
# To clear all cached schemas (forces re-fetch on next access):
rm -rf ~/.cache/scafctl/provider-schemas/

# To invalidate a single provider:
rm ~/.cache/scafctl/provider-schemas/exec.json

PowerShell

# Schema cache is at $env:LOCALAPPDATA\cache\scafctl\provider-schemas\
# To clear all cached schemas:
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\cache\scafctl\provider-schemas\"

# To invalidate a single provider:
Remove-Item "$env:LOCALAPPDATA\cache\scafctl\provider-schemas\exec.json"

Use Cases#

CI/CD: Pre-fetch plugin schemas in a warm-up step, then run MCP sessions offline
Air-gapped environments: Copy the provider-schemas/ directory to disconnected machines
Faster MCP startup: Cached schemas avoid spawning plugin processes for schema-only queries

Multi-Platform Support#

Plugin artifacts can include platform-specific binaries. The AnnotationPlatform annotation on catalog artifacts identifies the target platform:

dev.scafctl.plugin.platform: linux/amd64

When fetching, scafctl:

Lists all artifacts for the plugin version
Matches the dev.scafctl.plugin.platform annotation against the current (or requested) platform
Falls back to a direct fetch if no platform annotation exists (single-platform plugin)

Specifying a Target Platform#

Bash

# Fetch for a different platform
scafctl plugins install -f solution.yaml --platform linux/amd64

PowerShell

# Fetch for a different platform
scafctl plugins install -f solution.yaml --platform linux/amd64

This is useful for cross-platform CI where you build on one architecture but deploy on another.

Runtime Auto-Fetch (During Solution Execution)#

When you run a solution that declares plugin dependencies:

Bash

scafctl run solution -f solution.yaml

PowerShell

scafctl run solution -f solution.yaml

The prepare phase automatically:

Reads bundle.plugins from the solution
Checks the lock file for pinned versions
Fetches any missing plugins from the catalog chain
Caches the binaries locally
Loads the plugins and registers their providers
Cleans up plugin processes on exit

No explicit plugins install step is needed – but pre-fetching is recommended for predictability.

Official Provider Auto-Resolution#

scafctl includes a built-in registry of 10 official providers distributed as external plugins. These providers are automatically resolved from ghcr.io/oakwood-commons/providers/<name> when referenced in a solution – even without a bundle.plugins declaration.

Official Providers#

Provider	Description
`directory`	File system directory operations
`env`	Environment variable lookup
`exec`	Shell command execution
`git`	Git repository operations
`github`	GitHub API interactions
`hcl`	HCL/Terraform file parsing
`identity`	Identity token provider
`metadata`	Solution and system metadata
`secret`	Secret retrieval
`sleep`	Delay execution

How It Works#

When a solution references provider: exec (or any official provider):

scafctl checks the local registry – provider not found as a built-in
scafctl checks the official provider list – match found
The plugin binary is fetched from ghcr.io/oakwood-commons/providers/exec
The binary is cached locally for future use
The provider is registered and execution continues

A warning is logged when auto-resolution occurs:

WARN  auto-resolved official provider "exec" from ghcr.io/oakwood-commons/providers/exec

`run provider` Auto-Resolution#

The run provider command also auto-resolves official providers. You can invoke any official provider directly without installing it first:

Bash

# Auto-resolves the exec provider plugin and runs it
scafctl run provider exec command='echo hello' -o json

# Auto-resolves the env provider plugin
scafctl run provider env name=HOME -o json

PowerShell

# Auto-resolves the exec provider plugin and runs it
scafctl run provider exec command='echo hello' -o json

# Auto-resolves the env provider plugin
scafctl run provider env name=HOME -o json

The provider binary is fetched on first use and cached for subsequent calls. Dynamic help (--help) also triggers auto-resolution so you can view the provider’s schema.

When to Declare `bundle.plugins`#

Auto-resolution is convenient for local development, but you should declare plugins explicitly for:

CI/CD pipelines – use --strict to catch undeclared dependencies
Air-gapped environments – pre-fetch with scafctl plugins install
Version pinning – specify exact version constraints
Reproducible builds – combine with lock files

bundle:
  plugins:
    - name: static
      kind: provider
      version: "^0.1.0"
    - name: parameter
      kind: provider
      version: "latest"

Strict Mode#

The --strict flag disables auto-resolution and requires all providers to be either built-in or declared in bundle.plugins:

Bash

# Fails if any provider would be auto-resolved
scafctl run solution -f solution.yaml --strict

# Also works with run resolver
scafctl run resolver -f solution.yaml --strict

PowerShell

# Fails if any provider would be auto-resolved
scafctl run solution -f solution.yaml --strict

# Also works with run resolver
scafctl run resolver -f solution.yaml --strict

Use --strict in CI to ensure all dependencies are declared and reproducible.

Disabling Auto-Resolution#

For air-gapped or restricted environments, disable official provider auto-resolution entirely:

# ~/.config/scafctl/config.yaml
settings:
  disableOfficialProviders: true

With this setting, solutions must declare all non-built-in providers in bundle.plugins.

Example: End-to-End Workflow#

Bash

# 1. Develop your solution with plugin dependencies
cat > solution.yaml << 'EOF'
apiVersion: scafctl.io/v1
kind: Solution
metadata:
  name: data-pipeline
  version: 1.0.0
spec:
  resolvers:
    data:
      resolve:
        with:
          - provider: my-db-provider
            inputs:
              connection: "postgres://localhost/db"
  bundle:
    plugins:
      - name: my-db-provider
        kind: provider
        version: "^2.0.0"
EOF

# 2. Build to create a lock file (pins plugin versions)
scafctl package solution -f solution.yaml --version 1.0.0

# 3. Pre-fetch plugins (optional but recommended)
scafctl plugins install -f solution.yaml

# 4. Run the solution (plugins loaded from cache)
scafctl run solution -f solution.yaml

# 5. Check what's cached
scafctl plugins list

PowerShell

# 1. Develop your solution with plugin dependencies
@'
apiVersion: scafctl.io/v1
kind: Solution
metadata:
  name: data-pipeline
  version: 1.0.0
spec:
  resolvers:
    data:
      resolve:
        with:
          - provider: my-db-provider
            inputs:
              connection: "postgres://localhost/db"
  bundle:
    plugins:
      - name: my-db-provider
        kind: provider
        version: "^2.0.0"
'@ | Set-Content solution.yaml

# 2. Build to create a lock file (pins plugin versions)
scafctl package solution -f solution.yaml --version 1.0.0

# 3. Pre-fetch plugins (optional but recommended)
scafctl plugins install -f solution.yaml

# 4. Run the solution (plugins loaded from cache)
scafctl run solution -f solution.yaml

# 5. Check what's cached
scafctl plugins list

Troubleshooting#

Plugin not found in any catalog#

Error: plugin my-plugin (provider): resolving version: ...not found in any catalog

Verify the plugin is published to a configured catalog
Check scafctl catalog list --kind provider to see available providers
Ensure your config has the correct remote catalog URL

Version constraint not satisfied#

Error: resolved version 3.0.0 does not satisfy constraint ^1.0.0

The catalog’s latest version doesn’t match your constraint
Update the constraint in your solution, or publish a compatible version

Cache corruption#

If a cached binary seems corrupt:

Bash

# Remove the specific plugin from cache
rm -rf ~/.cache/scafctl/plugins/<plugin-name>/<version>/

# Re-fetch
scafctl plugins install -f solution.yaml

PowerShell

# Remove the specific plugin from cache
Remove-Item -Recurse -Force "$env:LOCALAPPDATA\scafctl\plugins\<plugin-name>\<version>\"

# Re-fetch
scafctl plugins install -f solution.yaml

Plugin Auto-Fetching from Catalogs#

Overview#

Declaring Plugin Dependencies#

Fields#

Pre-Fetching Plugins#

Listing Cached Plugins#

Lock Files for Reproducibility#

Signature Verification#

Configuring Signature Verification#

Verification Modes#

How Verification Works#

Build Tag Requirement#

Embedder Policy Override#

CI/CD Recommendation#

Catalog Chain#

Configuring Remote Catalogs#

Plugin Cache#

Cache Structure#

Managing the Cache#

Schema Caching (Offline Access)#

How It Works#

Cache Location#

Managing the Schema Cache#

Use Cases#

Multi-Platform Support#

Specifying a Target Platform#

Runtime Auto-Fetch (During Solution Execution)#

Official Provider Auto-Resolution#

Official Providers#

How It Works#

run provider Auto-Resolution#

When to Declare bundle.plugins#

Strict Mode#

Disabling Auto-Resolution#

Example: End-to-End Workflow#

Troubleshooting#

Plugin not found in any catalog#

Version constraint not satisfied#

Cache corruption#

`run provider` Auto-Resolution#

When to Declare `bundle.plugins`#