Fingerprint Input Hashing Implementation Plan#

Related: fingerprinting.md (original design), #241 (state persistence)


1. Problem Statement#

The current fingerprint system only hashes file contents matched by sources and generates glob patterns. Resolver outputs – which are the dynamic data flowing into actions (template variables, CEL expressions, provider configs) – are invisible to the fingerprint check.

When a resolver value changes but the source template files are untouched, the staleness check sees “same files, same outputs” and incorrectly skips the action.

Reproduction Scenario#

resolvers:
  environment:
    value: "staging"

workflow:
  actions:
    generate-config:
      provider: go-template
      sources:
        - "templates/config.yaml.tpl"
      generates:
        - "output/config.yaml"
      inputs:
        data:
          expr: "_.environment"
  1. First run: action executes, fingerprints stored. Output contains staging.
  2. User changes resolver environment to "production".
  3. Second run: templates/config.yaml.tpl unchanged (sources hash matches), output/config.yaml unchanged (generates hash matches).
  4. Result: Action incorrectly skipped. Output still contains staging.

Root Cause#

The fingerprint Check() runs before resolveInputs() in the executor (line 554 of executor.go). Resolved input values are not available at check time, and even if they were, the Check() method only receives file glob patterns – not resolver data.


2. Solution: Two-Phase Fingerprint Check#

Split the fingerprint check into two phases:

  1. Phase 1 (pre-resolve): Check sources + generates file hashes. If files changed, skip input hashing and re-run immediately (cheap fast path).
  2. Phase 2 (post-resolve): If files match, hash all resolvedInputs and compare against stored inputs hash. If inputs changed, re-run.

This follows the approach used by Bazel, Nx, and Nix – every input that influences the output is part of the cache key.

Design Decisions#

DecisionChoiceRationale
Check strategyTwo-phase (files first, then inputs)Avoids unnecessary input resolution cost when files alone prove staleness
Input scopeAll resolved inputsSimple, correct. Static input changes (YAML edits) also trigger re-run correctly
Non-deterministic inputsNo special handlingDocumented limitation. If inputs contain timestamps/UUIDs, action always re-runs
State migrationForce re-run on missing inputs hashSafe one-time cost. Missing __fingerprint:*:inputs key treated as first run
Serializationencoding/json.Marshal (sorted keys)Deterministic by default for Go maps. No custom sorting needed

3. Updated Staleness Matrix#

SourcesGeneratesInputsResultReason
No prior stateStalefirst run
ChangedStalesources changed
MatchMissingStalegenerates missing
MatchChangedStalegenerates modified
MatchMatchNo prior stateStalefirst run
MatchMatchChangedStaleinputs changed
MatchMatchMatchSkipup-to-date

4. Updated Evaluation Order in Executor#

1. Check when: condition         --> skip with SkipReasonCondition if false
2. Fingerprint Phase 1 (files)   --> if stale, proceed to step 3
                                     if fresh, proceed to step 3 then check Phase 2
3. Resolve inputs
4. Fingerprint Phase 2 (inputs)  --> skip with SkipReasonUpToDate if up-to-date
5. Execute action
6. On success: record fingerprints (sources + generates + inputs)

5. Task Breakdown#

#TaskFile(s)SizeDepends On
1Add ReasonInputsChanged constantpkg/fingerprint/fingerprint.goS
2Add InputsHash / PreviousInputsHash to Resultpkg/fingerprint/fingerprint.goS
3Add HashInputs(inputs map[string]any) (string, error)pkg/fingerprint/hash.goS
4Add inputsKey() / LoadInputsHash() / update SaveHashes()pkg/fingerprint/state.goS
5Split Check() into CheckFiles() + CheckInputs()pkg/fingerprint/fingerprint.goM1–4
6Restructure executor: two-phase fingerprint integrationpkg/action/executor.goM5
7Update Record() to also store inputs hashpkg/fingerprint/fingerprint.goS3, 4
8Unit tests for HashInputspkg/fingerprint/hash_test.goM3
9Unit tests for CheckInputs + two-phase flowpkg/fingerprint/fingerprint_test.goM5
10Unit tests for inputs state helperspkg/fingerprint/state_test.goS4
11Update executor tests for two-phase skippkg/action/executor_test.goM6
12Integration test: resolver change triggers re-runtests/integration/M6
13Update design docdocs/design/state/fingerprinting.mdSAll

6. Interface Changes#

6.1 New Reason Constant#

// pkg/fingerprint/fingerprint.go

const (
    // ReasonInputsChanged indicates resolved action inputs have changed since last run.
    ReasonInputsChanged Reason = "inputs changed"
)

6.2 Updated Result Struct#

// pkg/fingerprint/fingerprint.go -- add to existing Result struct

// InputsHash is the SHA-256 hash of current resolved inputs.
InputsHash string

// PreviousInputsHash is the stored hash of inputs from last successful run.
PreviousInputsHash string

6.3 Input Hashing#

// pkg/fingerprint/hash.go

// HashInputs computes a deterministic SHA-256 hash over the resolved action
// inputs map. Uses json.Marshal which sorts map keys lexicographically.
// Returns empty string and nil error if inputs is nil or empty.
func HashInputs(inputs map[string]any) (string, error)

6.4 State Helpers#

// pkg/fingerprint/state.go

// inputsKey returns the state entry key for an action's inputs fingerprint.
// Format: "__fingerprint:<actionName>:inputs"
func inputsKey(actionName string) string

// LoadInputsHash retrieves the previously stored inputs hash from state.
// Returns empty string if not found.
func LoadInputsHash(data *state.Data, actionName string) string

// SaveHashes stores the current fingerprint hashes into state data.
// Updated signature: now accepts inputsHash in addition to sources and generates.
func SaveHashes(data *state.Data, actionName, sourcesHash, generatesHash, inputsHash string)

6.5 Split Check Method#

// pkg/fingerprint/fingerprint.go

// CheckFiles performs Phase 1: compares source/generates file hashes against
// stored state. If Stale==true, the caller should execute the action without
// proceeding to Phase 2. If Stale==false, the caller should resolve inputs
// and call CheckInputs.
func (c *Checker) CheckFiles(ctx context.Context, actionName string,
    sources, generates []string, baseDir string) (*Result, error)

// CheckInputs performs Phase 2: compares resolved input hashes against stored
// state. Only call this when CheckFiles returned Stale==false (files match).
// resolvedInputs is the full map returned by resolveInputs().
func (c *Checker) CheckInputs(ctx context.Context, actionName string,
    resolvedInputs map[string]any) (*Result, error)

6.6 Updated Record#

// pkg/fingerprint/fingerprint.go

// Record stores the current fingerprints (sources + generates + inputs) after
// a successful action. Call this after action execution to persist the new
// baseline.
func (c *Checker) Record(ctx context.Context, actionName string,
    sources, generates []string, baseDir string,
    resolvedInputs map[string]any) error

7. Executor Integration Detail#

Current flow (pkg/action/executor.go lines 554–690):

when check --> fingerprint Check() --> resolveInputs() --> execute --> Record()

New flow:

// Phase 1: File-based fingerprint check (cheap, before input resolution)
var filesFresh bool
if e.fingerprintChecker != nil && !e.noCache && len(action.Sources) > 0 {
    fpResult, fpErr := e.fingerprintChecker.CheckFiles(ctx, actionName,
        action.Sources, action.Generates, e.cwd)
    if fpErr != nil {
        logger.FromContext(ctx).V(0).Info("fingerprint check failed, executing action",
            "action", actionName, "error", fpErr.Error())
    } else if !fpResult.Stale {
        filesFresh = true
    }
}

// Resolve inputs (always needed -- for execution and Phase 2 check)
resolvedInputs, err := e.resolveInputs(ctx, action, graph.AliasMap)
if err != nil { /* existing error handling */ }

// Phase 2: Input-based fingerprint check (only if files were fresh)
if filesFresh {
    fpResult, fpErr := e.fingerprintChecker.CheckInputs(ctx, actionName, resolvedInputs)
    if fpErr != nil {
        logger.FromContext(ctx).V(0).Info("fingerprint inputs check failed, executing action",
            "action", actionName, "error", fpErr.Error())
    } else if !fpResult.Stale {
        e.actionContext.MarkSkipped(actionName, SkipReasonUpToDate)
        if e.progressCallback != nil {
            e.progressCallback.OnActionSkipped(actionName, string(SkipReasonUpToDate))
        }
        return nil
    }
}

// ... continue to execute action ...

// After success: record with inputs
if e.fingerprintChecker != nil && len(action.Sources) > 0 {
    if recordErr := e.fingerprintChecker.Record(ctx, actionName,
        action.Sources, action.Generates, e.cwd, resolvedInputs); recordErr != nil {
        logger.FromContext(ctx).V(0).Info("fingerprint record failed",
            "action", actionName, "error", recordErr.Error())
    }
}

8. Testing Strategy#

HashInputs Tests (pkg/fingerprint/hash_test.go)#

Test CaseInputExpected
DeterminismSame map called twiceIdentical hash
Key ordering{"b":1, "a":2} vs {"a":2, "b":1}Identical hash
Nested structuresMaps with nested maps/slicesConsistent hash
Nil/empty inputsnil / map[string]any{}Empty string
Value change{"env":"staging"} vs {"env":"prod"}Different hash
Type sensitivity{"port": 8080} vs {"port": "8080"}Different hash

CheckInputs Tests (pkg/fingerprint/fingerprint_test.go)#

Test CaseSetupExpected
First run (no prior inputs hash)Files match, no __fingerprint:*:inputs in stateStale, ReasonFirstRun
Inputs changedFiles match, inputs hash differs from storedStale, ReasonInputsChanged
Inputs unchangedFiles match, inputs hash matches storedUp-to-date
Two-phase: files staleSources changedStale from Phase 1, Phase 2 never called
Two-phase: files fresh, inputs changedSources match, resolver value changedStale from Phase 2
Two-phase: everything freshAll hashes matchSkip

Executor Tests (pkg/action/executor_test.go)#

Test CaseSetupExpected
Resolver change triggers re-runSame sources, different resolved inputsAction executes
Full cache hit skips actionSame sources + generates + inputsAction skipped
File change short-circuitsSources changed, inputs not checkedAction executes

Integration Test (tests/integration/)#

Solution with a resolver feeding into a template action. Run once, change resolver value, run again. Assert the action re-executed and output reflects the new value.


9. Performance Characteristics#

  • No regression for the common case: When files change (most runs), Phase 1 catches it immediately – no input hashing overhead.
  • Input hashing cost: One json.Marshal + SHA-256 per action, only when files haven’t changed. Negligible for typical input sizes.
  • resolveInputs always runs: This is a behavior change – inputs are now resolved even when the action will be skipped by Phase 1. However, the resolved inputs are needed for Phase 2 and the cost is minimal (CEL/template evaluation is fast).

10. State Migration#

Existing state files with __fingerprint:*:sources and __fingerprint:*:generates keys but no __fingerprint:*:inputs key will trigger a one-time re-execution (ReasonFirstRun on missing inputs hash).

  • No manual migration step required.
  • No YAML schema change – sources and generates fields are unchanged.
  • The Check() / Record() method signatures change – direct callers (MCP tools, tests) need updating.

11. Risks and Edge Cases#

RiskMitigation
Non-deterministic inputs (timestamps, UUIDs) cause perpetual re-runsDocument limitation. Users should avoid sources/generates on actions with non-deterministic inputs, or use --no-cache
Large resolved inputs slow down hashingjson.Marshal + SHA-256 is fast even for large maps. Benchmark to confirm
resolveInputs side effectsresolveInputs is pure evaluation (CEL/template). No side effects. Safe to call before skip decision
Float precision in JSONencoding/json uses consistent float formatting. Not a practical concern for config values
Breaking API changeCheck() and Record() signatures change. Keep deprecated Check() wrapper during transition if needed

12. Non-Goals#

  • Exclude list for inputs: Not implementing per-input fingerprint.exclude. Use fingerprint.scope: files to skip all input hashing instead.
  • Partial input hashing: Not differentiating static vs dynamic inputs. All resolved inputs are hashed uniformly (when scope is all).
  • Input-only fingerprinting: sources is still required to opt into fingerprinting. Actions without sources run every time, as before.