# Sources and Provenance

## Why This Exists

IR consumers need stable source identities for diagnostics, package resolution, cache freshness, and generated POSIX scripts. Source maps are not a debugging afterthought; they are part of the runtime diagnostic contract.

Generated scripts may run in CI or standalone release contexts where the original `.ops`, `.op.sh`, stdlib, or package files are absent. The IR therefore carries either embedded source content or enough package provenance to identify the exact source artifact.

## Producers

- **Parser lowering:** emits source IDs, spans, and embedded user source content when allowed.
- **Build:** resolves imports to logical source IDs and package provenance.
- **Registry loading:** supplies `.op.sh` package/path provenance for operation definitions consumed by resolver diagnostics.

## Consumers

- **Resolver:** reports graph, type, env, import, and safety diagnostics against original source spans.
- **Emitter:** embeds diagnostic metadata in POSIX scripts and standalone bundles.
- **Machine output/drilling:** exposes stable locations and provenance in JSON diagnostics and help text.
- **`opx_error` formatter:** receives enough source ID, span, snippet/provenance, and context data to reproduce the same formatted diagnostics from generated POSIX scripts.

## Shape

Source entries and sourcemap nodes use array payloads:

```json
[
  {
    "role": "source",
    "name": "file:ci.ops",
    "payload": [
      { "role": "origin", "payload": "user" },
      { "role": "path", "payload": "ci.ops" },
      { "role": "content", "payload": "main @release {\n  test()\n}\n" }
    ]
  },
  {
    "role": "source",
    "name": "pkg:shellops-std@1.0.0/log.ops",
    "payload": [
      { "role": "origin", "payload": "package" },
      { "role": "package", "payload": "shellops-std" },
      { "role": "version", "payload": "1.0.0" },
      { "role": "path", "payload": "log.ops" },
      { "role": "digest", "payload": "sha256:..." }
    ]
  },
  {
    "role": "source",
    "name": "registry:git.bump",
    "payload": [
      { "role": "origin", "payload": "operation" },
      { "role": "package", "payload": "git" },
      { "role": "path", "payload": "operations/git.op.sh" },
      { "role": "digest", "payload": "sha256:..." }
    ]
  }
]
```

In OPX:

```
Sr01_ 0.source
nSr01 1.name:string."file:ci.ops"
oSr01 1.origin:string."user"
pSr01 1.path:string."ci.ops"
cSr01 1.content:string."main @release {\n  test()\n}\n"
Sr02_ 0.source
nSr02 1.name:string."pkg:shellops-std@1.0.0/log.ops"
oSr02 1.origin:string."package"
pkS02 1.package:string."shellops-std"
vrS02 1.version:string."1.0.0"
ptS02 1.path:string."log.ops"
dgS02 1.digest:string."sha256:..."
Sr03_ 0.source
nSr03 1.name:string."registry:git.bump"
oSr03 1.origin:string."operation"
pkS03 1.package:string."git"
ptS03 1.path:string."operations/git.op.sh"
dgS03 1.digest:string."sha256:..."
```

Sourcemap nodes reference source IDs and carry span data:

```json
{
  "role": "sourcemap",
  "payload": [
    {
      "role": "node",
      "name": "pipelines.release.0",
      "payload": [
        { "role": "source", "payload": "file:ci.ops" },
        {
          "role": "span",
          "payload": [
            { "role": "start", "payload": [2, 3, 17] },
            { "role": "end", "payload": [2, 9, 23] }
          ]
        }
      ]
    }
  ]
}
```

In OPX:

```
Sm01_ 0.sourcemap
Nd01_ 1.node
nNd01 2.name:string."pipelines.release.0"
sNd01 2.source:string."file:ci.ops"
spN01 2.span
stSp1 3.start
s1St1 4.item:int.2
s2St1 4.item:int.3
s3St1 4.item:int.17
enSp1 3.end
e1En1 4.item:int.2
e2En1 4.item:int.9
e3En1 4.item:int.23
```

Span coordinates use `[line, column, byte]` tuples as leaf payloads.

## Invariants

- Source IDs are logical, stable within one IR unit, and never depend on absolute host paths.
- User-authored `.ops` entries should include `content` unless an explicit privacy policy disables embedding.
- Package/stdlib/registry entries may omit content only when `package`, `version`, `path`, and `digest` identify the exact artifact.
- Every executable statement, declaration, env entry, import, and generated diagnostic site must have a source-map entry.
- Source maps carry spans and metadata only. They must not embed raw `@secret` values.

## Additional Uses

Source maps are also useful outside error display:

- Generated-code debugging: trace POSIX lines or functions back to `.ops` declarations.
- Cache invalidation: detect whether an emitted artifact still corresponds to the same source/package digest.
- CI annotations: emit machine-readable file/line metadata for test and workflow UIs.
- Standalone audit trails: explain what source package/version produced a bundled script.
- Drilling/help provenance: show why a flag or dependency appears in generated help.

## Design References

- [architecture](../architecture.md) describes the compiler flow from parser to resolver to POSIX emitter.
- [strategies/error-catalog](../strategies/error-catalog.md) owns diagnostic identity and rendering strategy.
- [drilling](../drilling.md) requires generated artifacts to provide help and diagnostics without reparsing source.
- [posix-emission](./posix-emission.md) defines how these maps are embedded in generated scripts.
