Plugins#

Purpose#

Plugins are the extension mechanism for scafctl. They allow external binaries to contribute functionality to the system in a controlled, versioned, and discoverable way.

The primary purpose of plugins is to supply providers and auth handlers. “Plugin” is an internal implementation term - users interact with “providers” and “auth handlers” as catalog artifact kinds.

Terminology#

• Plugin: Internal term for a go-plugin binary. Not exposed to users.
• Provider Artifact: A plugin binary distributed via the catalog that exposes one or more providers. Users push/pull “providers” not “plugins”.
• Auth Handler Artifact: A plugin binary distributed via the catalog that exposes one or more auth handlers.

What a Plugin Is#

A plugin is an external process that implements one or more scafctl extension interfaces and communicates with scafctl over RPC.

Plugins are:

• Discovered and loaded at runtime
• Versioned independently from scafctl
• Isolated from the core process
• Capable of exposing multiple providers or auth handlers

scafctl uses hashicorp/go-plugin to manage plugin lifecycle, transport, and isolation.

What a Plugin Is Not#

A plugin is not:

• A provider itself
• A resolver
• An action
• A workflow engine
• A scripting environment

Plugins do not participate directly in execution graphs. They only expose capabilities that scafctl invokes.

Primary Use: Provider and Auth Handler Distribution#

Plugins exist primarily to distribute providers and auth handlers.

Under this model:

• Providers define behavior (data fetching, transformations, actions)
• Auth handlers define authentication flows (Entra, GitHub, custom identity providers)
• Plugin binaries package these capabilities
• scafctl orchestrates execution

A plugin may expose one or more providers OR one or more auth handlers (not both).

Catalog Artifact Kinds#

When distributed via the catalog, plugins are categorized by their purpose:

Users interact with these as distinct artifact kinds:

# Push a provider artifact
scafctl catalog push aws-provider@1.0.0 --catalog ghcr.io/myorg

# Pull a provider artifact  
scafctl catalog pull ghcr.io/myorg/providers/aws-provider@1.0.0

# Push an auth handler artifact
scafctl catalog push okta-handler@1.0.0 --catalog ghcr.io/myorg

# Pull an auth handler artifact
scafctl catalog pull ghcr.io/myorg/auth-handlers/okta-handler@1.0.0

Why Plugins Exist (Instead of Built-ins Only)#

Plugins exist to:

• Avoid baking all providers into scafctl
• Enable third-party and internal extensions
• Allow providers and auth handlers to evolve independently
• Isolate failures and crashes
• Support multiple languages via gRPC boundaries
• Keep the core binary small and stable

This mirrors patterns used by Terraform, Vault, Nomad, and Packer.

Plugin Architecture#

Dependencies#

• External: hashicorp/go-plugin (gRPC-based plugin system)
• External: google.golang.org/grpc (gRPC for plugin communication)
• External: google.golang.org/protobuf (Protocol buffers)

scafctl uses go-plugin with gRPC-based handshake.

Protocol buffer definition for plugin communication.

syntax = "proto3";
package plugin;
option go_package = "github.com/oakwood-commons/scafctl/pkg/plugin/proto";

// PluginService is the main plugin service
service PluginService {
  // GetProviders returns all providers exposed by this plugin
  rpc GetProviders(GetProvidersRequest) returns (GetProvidersResponse);
  
  // GetProviderDescriptor returns metadata for a specific provider
  rpc GetProviderDescriptor(GetProviderDescriptorRequest) returns (GetProviderDescriptorResponse);
  
  // ExecuteProvider executes a provider
  rpc ExecuteProvider(ExecuteProviderRequest) returns (ExecuteProviderResponse);
}

message GetProvidersRequest {}

message GetProvidersResponse {
  repeated string provider_names = 1;
}

message GetProviderDescriptorRequest {
  string provider_name = 1;
}

message GetProviderDescriptorResponse {
  ProviderDescriptor descriptor = 1;
}

message ProviderDescriptor {
  string name = 1;
  string description = 2;
  Schema schema = 3;
}

message Schema {
  map<string, Parameter> parameters = 1;
}

message Parameter {
  string type = 1;
  bool required = 2;
  string description = 3;
  bytes default_value = 4; // JSON-encoded
}

message ExecuteProviderRequest {
  string provider_name = 1;
  bytes input = 2; // JSON-encoded input map
}

message ExecuteProviderResponse {
  bytes output = 1; // JSON-encoded output
  string error = 2;  // Empty if no error
}

Conceptually:

• scafctl discovers a plugin binary
• scafctl negotiates protocol version
• Plugin advertises capabilities
• scafctl registers providers exposed by the plugin
• Providers are invoked through gRPC

The plugin process lifecycle is managed entirely by scafctl.

Plugin Capabilities#

Today, plugins are intended to expose providers.

Future capability types may include:

• Provider sets
• Schemas
• Validation helpers

However, plugins should not become a generic execution environment. Any new capability must align with scafctl core concepts.

Provider Exposure Model#

A plugin declares the providers it implements.

Conceptual example:

plugin: scafctl-provider-api
provides:
  - provider: api
  - provider: http

Each provider exposed by a plugin:

• Has a stable name and version
• Declares capabilities (from, transform, validation, authentication, action)
• Declares an input schema (with typed parameters)
• Declares output schemas per capability for the Data property within ProviderOutput
• Provides catalog metadata (description, category, tags, examples, maintainers)
• Is invoked deterministically

scafctl treats built-in providers and plugin-provided providers identically. All providers expose a ProviderDescriptor that includes identity, versioning, schemas, capabilities, and catalog information.

Invocation Flow#

When a provider is used:

1. scafctl resolves all inputs
2. scafctl validates inputs against the provider schema
3. scafctl invokes the provider via gRPC
4. The plugin executes provider logic
5. Provider returns ProviderOutput containing data, warnings, and metadata
6. scafctl validates output against the provider’s output schema for the current capability
7. scafctl continues orchestration

Providers never see unresolved CEL, templates, or resolver references. All provider responses use the standardized ProviderOutput structure.

Plugin Discovery#

Plugins are discovered via multiple mechanisms:

• The local catalog (built or pulled plugins)
• Configured plugin directories on disk
• Explicit configuration
• Environment-based paths
• Solution dependencies (automatically fetched from remote catalogs)

When a solution declares plugin dependencies (under bundle.plugins):

bundle:
  plugins:
    - name: aws-provider
      kind: provider
      version: "^1.5.0"
      defaults:
        region: us-east-1

scafctl will:

1. Check if the plugin exists in the local catalog
2. Pull missing plugins from configured remote catalogs
3. Validate version constraints are met
4. Load the plugin binary
5. Apply plugin defaults (shallow-merged beneath inline inputs)

See catalog-build-bundling.md for the full design of bundle.plugins, including the kind field, ValueRef-aware defaults, and lock file integration.

Discovery does not execute plugins. Execution occurs only when a provider is invoked.

Versioning and Compatibility#

Plugins declare:

• Supported protocol version
• Provider versions
• Optional feature flags

scafctl enforces compatibility at load time.

Incompatible plugins are rejected early.

Security Model#

Plugins are isolated processes.

Security properties:

• No direct memory access to scafctl
• Explicit gRPC boundaries
• No implicit filesystem or network access beyond what the plugin implements
• Providers are the only exposed surface
• All inputs validated before sending to plugin
• Schema validation at scafctl boundary
• Type checking for all parameters

Plugin execution is explicit and auditable.

Why Plugins Are Not a Separate Concept from Providers#

Conceptually:

• Providers define behavior
• Plugins deliver providers

Introducing plugins as a separate user-facing concept would add unnecessary indirection.

Users reason about:

• Providers
• Actions
• Resolvers

Plugins are an implementation detail that enables extensibility.

Design Constraints#

• Plugins must not orchestrate execution
• Plugins must not resolve data
• Plugins must not mutate scafctl state
• Plugins may only expose declared capabilities
• Providers remain the sole execution primitive

Notes#

• Plugins use hashicorp/go-plugin (same as Terraform, Vault, Packer)
• gRPC communication provides language flexibility (Go, Python, Rust, etc.)
• Plugin providers and built-in providers are indistinguishable to users
• Plugins are the primary extensibility mechanism
• Plugin crashes are handled gracefully
• Plugin directory is configurable
• Plugins may expose multiple providers
• Provider names must be unique across built-ins and plugins

Auto-Fetch & Runtime Loading#

When a solution declares plugin dependencies under bundle.plugins, scafctl automatically fetches missing binaries from configured catalogs at runtime.

Architecture#

Flow#

1. Solution is loaded with bundle.plugins entries
2. Fetcher.FetchPlugins() iterates declared plugins
3. For each plugin, the cache is checked first (by name + version + platform digest)
4. On cache miss, ChainCatalog queries configured catalogs in order
5. PluginFetcher extracts the platform-specific binary from the OCI artifact
6. Binary is stored in the content-addressed cache (atomic write, 0o755)
7. RegisterFetchedPlugins() adds cached paths to the plugin registry

CLI Commands#

• scafctl plugins install — Pre-fetch plugin binaries from catalogs before a build or run
• scafctl plugins list — List cached plugin binaries with digest, size, and platform info

Both commands live in pkg/cmd/scafctl/plugins/.

Cache Layout#

$XDG_CACHE_HOME/scafctl/plugins/
└── <sha256-digest>/           # Content-addressed binary

See the Plugin Auto-Fetching Tutorial for a complete walkthrough.

Multi-Platform Support via OCI Image Index#

Plugin binaries are platform-specific — a Linux x86-64 binary cannot run on macOS ARM64. To distribute a single plugin that works across OS/architecture combinations, scafctl supports OCI image indexes (fat manifests).

Architecture#

OCI Image Index Structure#

A multi-platform plugin is stored as an OCI image index referencing per-platform image manifests:

image index (application/vnd.oci.image.index.v1+json)
├── platform: linux/amd64
│   └── manifest → config + binary layer
├── platform: darwin/arm64
│   └── manifest → config + binary layer
└── platform: windows/amd64
    └── manifest → config + binary layer

Platform Resolution Strategy#

When PluginFetcher.FetchPlugin() is called:

1. OCI image index — If the catalog implements PlatformAwareCatalog, try FetchByPlatform() which resolves the platform from the image index. If the artifact IS an image index but the platform is missing, return PlatformNotFoundError (no fallback — the artifact is explicitly multi-platform).

2. Annotation matching (legacy) — Fall back to listing artifacts and matching the dev.scafctl.plugin.platform annotation on individual manifests.

3. Direct fetch — Fall back to fetching the artifact directly (single-platform artifacts without platform metadata).

Supported Platforms#

• linux/amd64
• linux/arm64
• darwin/amd64
• darwin/arm64
• windows/amd64

Building Multi-Platform Artifacts#

scafctl build plugin \
  --name my-provider \
  --kind provider \
  --version 1.0.0 \
  --platform linux/amd64=./dist/linux-amd64/my-provider \
  --platform darwin/arm64=./dist/darwin-arm64/my-provider

See the Multi-Platform Plugin Build Tutorial for a complete walkthrough.

Summary#

Plugins are the extensibility layer of scafctl. They exist to supply providers in an isolated, versioned, and scalable way using go-plugin. Plugins are not a new execution model or abstraction. They are the mechanism by which providers are distributed and invoked, keeping the core system small, stable, and extensible.

Plugins are distributed through the catalog system as OCI artifacts, enabling:

• Versioned plugin releases with semantic versioning
• Multi-platform support (linux/amd64, darwin/arm64, etc.)
• Offline distribution via scafctl save/load
• Automatic dependency resolution when solutions declare required plugins