# Pipelines and Steps

## Why This Exists

Pipelines are the executable center of the IR. The resolver needs graph-friendly structure, while the emitter needs exact invocation boundaries, capture names, decorator intent, and source locations for generated diagnostics.

## Producers

- **Parser lowering:** emits pipeline declarations, body statements, call nodes, decorators, bindings, and block-op bodies.
- **Resolver:** enriches the graph with registry metadata, hooks, transitive dependencies, and validation results.

## Consumers

- **Resolver:** builds the dependency graph, validates operation references, resolves same-file pipeline calls, and checks dependency decorators.
- **Emitter:** generates group scripts, standalone scripts, drill files, exports, captures, and POSIX-safe invocation code.
- **Drilling:** uses names, captures, and graph prefixes for `./ops pipeline step` and `@VAR` lookup.

## Pipeline Shape

Pipeline metadata (decorators, safety, dependencies) and body steps share a single payload array. The generator distinguishes them by role: structural roles (`entrypoint`, `safety`, `need`, `want`) are metadata, execution roles (`call`, `branch`, `loop`) are body steps.

```json
{
  "role": "pipeline",
  "name": "release",
  "payload": [
    { "role": "entrypoint" },
    { "role": "safety", "payload": "remote" },
    { "role": "need", "payload": [
      { "role": "pipeline", "payload": "configure" },
      { "role": "var", "payload": "VERSION" }
    ]},
    { "role": "want", "payload": [
      { "role": "pipeline", "payload": "lint" }
    ]},
    {
      "role": "call",
      "payload": [
        { "role": "target", "payload": "git.bump" },
        { "role": "export", "payload": "VERSION" },
        { "role": "capture", "payload": "bump" },
        { "role": "source", "payload": "#/sourcemap/pipelines.release.0" }
      ]
    }
  ]
}
```

In OPX:

```
Pl01_ 0.pipeline
nPl1_ 1.name:string."release"
ePl1_ 1.entrypoint
sPl1_ 1.safety:string."remote"
nD01_ 1.need
pN01_ 2.pipeline:string."configure"
vN01_ 2.var:string."VERSION"
wD01_ 1.want
pW01_ 2.pipeline:string."lint"
cP01_ 1.call
tC01_ 2.target:string."git.bump"
xC01_ 2.export:string."VERSION"
cpC1_ 2.capture:string."bump"
sC01_ 2.source:string."#/sourcemap/pipelines.release.0"
```

Pipeline names are canonical same-file identifiers. Imported pipeline references live in `import` nodes and step/reference values, not in flattened top-level keys.

## Step Forms

String shorthand is valid only for a bare invocation:

```json
"bash.shellcheck"
```

In OPX:

```
bSh01 0.call:string."bash.shellcheck"
```

Call nodes are required when any metadata exists:

```json
{
  "role": "call",
  "payload": [
    { "role": "target", "payload": "bash.shellcheck" },
    { "role": "args", "payload": [
      { "role": "positional", "payload": "tests" },
      { "role": "named", "name": "format", "payload": "gcc" }
    ]},
    { "role": "need", "payload": [
      { "role": "pipeline", "payload": "git.config" }
    ]},
    { "role": "capture", "payload": "shellcheck" },
    { "role": "source", "payload": "#/sourcemap/pipelines.lint.0" }
  ]
}
```

In OPX:

```
cL01_ 0.call
tL01_ 1.target:string."bash.shellcheck"
aL01_ 1.args
pA01_ 2.positional:string."tests"
nA01_ 2.named
nnA1_ 3.name:string."format"
vnA1_ 3.value:string."gcc"
nL01_ 1.need
pN01_ 2.pipeline:string."git.config"
cpL1_ 1.capture:string."shellcheck"
sL01_ 1.source:string."#/sourcemap/pipelines.lint.0"
```

Block ops use the same call shape with a `block` marker. Body steps share the payload alongside the call's metadata:

```json
{
  "role": "call",
  "payload": [
    { "role": "target", "payload": "std.retry" },
    { "role": "args", "payload": [
      { "role": "positional", "payload": 3 }
    ]},
    { "role": "block" },
    "infra.deploy",
    { "role": "source", "payload": "#/sourcemap/pipelines.release.1" }
  ]
}
```

In OPX:

```
cB01_ 0.call
tB01_ 1.target:string."std.retry"
aB01_ 1.args
pAB1_ 2.positional:int.3
bB01_ 1.block
dB01_ 1.call:string."infra.deploy"
sB01_ 1.source:string."#/sourcemap/pipelines.release.1"
```

Nested block ops compose naturally — the body of one block op contains another call node with its own `block` marker:

```json
{
  "role": "call",
  "payload": [
    { "role": "target", "payload": "std.optional" },
    { "role": "block" },
    {
      "role": "call",
      "payload": [
        { "role": "target", "payload": "std.timeout" },
        { "role": "args", "payload": [{ "role": "positional", "payload": 600 }] },
        { "role": "block" },
        {
          "role": "call",
          "payload": [
            { "role": "target", "payload": "std.retry" },
            { "role": "args", "payload": [{ "role": "positional", "payload": 3 }] },
            { "role": "block" },
            "infra.deploy"
          ]
        }
      ]
    }
  ]
}
```

In OPX:

```
cN01_ 0.call
tN01_ 1.target:string."std.optional"
bN01_ 1.block
cN02_ 1.call
tN02_ 2.target:string."std.timeout"
aN02_ 2.args
pAN2_ 3.positional:int.600
bN02_ 2.block
cN03_ 2.call
tN03_ 3.target:string."std.retry"
aN03_ 3.args
pAN3_ 4.positional:int.3
bN03_ 3.block
dN03_ 3.call:string."infra.deploy"
```

## Decorators and Bindings

In the previous schema, decorators and bindings were separate arrays alongside `payload`. Under the payload-shape rule, they merge into the payload as individual role-tagged items. The generator queries by role:

**Dependency decorators** use `need` and `want` roles directly:

```json
{ "role": "need", "payload": [
  { "role": "pipeline", "payload": "install" }
]}
```

In OPX:

```
nD02_ 0.need
pND2_ 1.pipeline:string."install"
```

**Safety decorators** use `safety` with a scalar payload:

```json
{ "role": "safety", "payload": "dangerous" }
```

In OPX:

```
sD01_ 0.safety:string."dangerous"
```

**Bindings** use `export` and `capture` roles with payload as the binding name:

```json
{ "role": "export", "payload": "VERSION" }
{ "role": "capture", "payload": "bump" }
```

In OPX:

```
xB01_ 0.export:string."VERSION"
cpB1_ 0.capture:string."bump"
```

**Timing and future decorators** follow the same pattern:

```json
{ "role": "timing", "payload": "record" }
```

In OPX:

```
tmD1_ 0.timing:string."record"
```

This keeps downstream generation open to new decorator roles without inventing new top-level fields for each feature.

## Naming and References

- Same-file pipeline calls are represented as normal call targets and resolved against pipelines before registry operations.
- Imported pipeline paths reference an import namespace plus a pipeline name.
- Registry operations remain qualified operation names.
- Variable references use `{ "role": "var", "payload": "VERSION" }`.
- Capture references use `{ "role": "capture-ref", "payload": "bump" }` or dotted access for nested fields.

## Invariants

- Exactly one `entrypoint` marker is allowed per pipeline.
- Pipeline names cannot collide with reserved top-level IR fields because pipelines are role-tagged items in the file's payload.
- Step argument boundaries are structured; emitters must never re-split an already-lowered shell string.
- Block ops are registry-driven by the `--block` contract, carry a `block` marker, and always carry nested body steps in their payload.
- Resolver diagnostics should preserve the original source location for both the pipeline declaration and each step.

## Design References

- [imperative-blocks](../features/imperative-blocks.md) defines pipeline declarations, `main`, block ops, and group invocation semantics.
- [steps](../features/steps.md) defines modifiers such as `needs`, `export`, and `as`.
- [invocation-arguments](../features/invocation-arguments.md) defines positional, named, object, and spread arguments.
- [pipeline-markers](../features/pipeline-markers.md) defines `remote`, `dangerous`, and `deprecated` pipeline metadata.
- [drilling](../drilling.md) defines drill files and `@VAR` lookup behavior.
