Plan And State

Plan, state, and live reconciliation current; unreleased and unverified against live Azure

The normalized plan is the immutable contract between user intent, extensions, policy, deployment engines, and evidence. Run state records what happened to an approved plan; it does not replace the plan.

YAML, JSON, Markdown, and HTML are not alternatives

These formats appear at opposite ends of the pipeline and are easy to confuse.

your YAML  ──►  resolve + plan  ──►  Export-AzureDataLabPlan
 19 lines        21-key plan         Json │ Markdown │ Html
 6 keys                              (renderings of the plan)

YAML is the authoring format — input only. You write it; the toolkit never produces it. examples/sqlvm-minimal.yaml is 19 lines with six top-level keys, roughly “a SQL VM, in this subscription, in this region”.

JSON, Markdown and HTML are renderings of the resolved plan — output only. They are three views of one artifact, not three ways to say the same thing as the YAML. The plan has 21 top-level keys, and almost none of them appear in your YAML:

In the plan	Not in your YAML
resources, actions	the exact Azure resources and the ordered action graph
provenance	per-leaf value, type, source and derivation — which layer set what
decisions	resolved secret-store, access, compute-security and sensitivity decisions
policyFindings, approval	blocking findings and what you must acknowledge
planHash, intentHash, contracts	determinism and contract-version bindings
unverifiedFacts	what could not be proven without Azure access
supportEvaluation	which support-matrix axes were checked
context	derived deterministic resource names

Put plainly: your YAML states intent; the plan states consequences. Markdown and HTML are curated human-review documents — they deliberately omit the subscription and tenant identifiers. JSON is the complete machine-readable plan and does carry them.

Why the plan is not exported as YAML

YAML admits many valid serializations of the same data, which is exactly what the canonical form exists to prevent — planHash is only meaningful because two serializations of one plan are byte-identical. The toolkit already treats YAML as a format to parse defensively rather than to produce: anchors and aliases are rejected outright, along with duplicate keys, explicit tags and multiple documents. Emitting the plan as YAML would either weaken the determinism guarantee or require a second canonicalization story. JSON is the machine format.

So if you want a richer authoring experience, that work belongs on the input surface — the configuration schema, templates, selectable options, and the browser wizard that writes YAML. The export formats are a separate concern.

Normalized plan envelope

The plan uses a versioned JSON-compatible schema. YAML is the authoring format, but canonical JSON serialization is used for hashing and interchange.

Field	Contract
planSchemaVersion and kind	Version and document type for the normalized plan
toolkit	Toolkit name and prerelease version that resolved the plan
contracts	Input, plan, catalog, target-provider, capability, solution-pack, and selected-engine contracts
canonicalization and intentHash	RFC 8785 profile and hash of complete resolved intent
target and context	Target support plus explicit Azure tenant, subscription, cloud, and location context
configuration and provenance	Fully resolved typed input and source or derivation for every leaf
resources	Stable IDs, Azure type and API version, desired properties, external ID, ownership, and teardown intent
actions	Stable IDs, execution scope, dependencies, mutation, idempotency, preconditions, postconditions, and reconciliation
catalogSelections and supportEvaluation	Exact metadata choices and structured support-matrix results
decisions	Authentication, secret, access, identity, disk, licensing, cost, data, backup, and lifecycle decisions
policyFindings and approval	Structured findings, blockers, acknowledgements, and plan-hash binding requirements
unverifiedFacts	External facts unavailable during offline planning
planHash	Hash of the canonical envelope excluding the hash field itself

Resource and action IDs are stable across repeated planning when semantic input is unchanged. They do not depend on array position, timestamp, machine path, or random generation. Their format is dot-delimited semantic IDs such as azure.compute.virtual-machine.primary and action.compute.virtual-machine.primary. The logical key comes from configuration identity rather than an eventual Azure resource ID. Each current action declares:

• directed acyclic graph dependencies;
• an offline, azure, or guest execution scope;
• preconditions and postconditions;
• expected mutation and retry class;
• ownership effect and teardown intent;
• scope-specific state sources and stop-on-unknown reconciliation.

Offline rendering actions are non-mutating and reconcile against local artifact state. Azure actions require live control-plane observation. Guest actions require target readiness, explicit guest-execution authorization, and a guest probe. A local renderer therefore cannot inherit a misleading Azure precondition merely because its artifact will eventually be used by a VM.

The plan is rejected if dependencies contain a cycle, IDs collide, a secret value is embedded, ownership is ambiguous, or the selected engine lacks a required capability.

Canonical hash

Canonical serialization uses RFC 8785 JSON Canonicalization Scheme over UTF-8. Before canonicalization, set-like collections are sorted by stable ID; ordered collections preserve declared order. The digest is SHA-256 and is written as sha256:<lowercase-hex>. Changing the canonicalization or digest algorithm requires a new normalized-plan contract version.

Approval, engine exports, state events, and reports bind to planHash. Any semantic change creates a different hash and invalidates approval.

Action idempotency keys bind to intentHash. The current intent hash includes the toolkit version, active provider, capability, and solution-pack contract and implementation versions, the selected engine contract, content hashes for the bundled catalogs, resolved configuration, and exact catalog selections. Changing any of those inputs changes every action key; a newer contributor cannot accidentally reuse an older implementation’s execution identity.

desiredProperties is an engine-facing normalized contract, not permission to send arbitrary fields to Azure. Resource-specific schemas tighten the shape as each provider reaches implementation. The current SQL IaaS resource uses the Microsoft 2023-10-01 property names, records the underlying VM’s full ARM resource ID, and rejects unknown or incorrectly typed properties.

Generated Azure Storage account and Key Vault names reserve the final 12 characters for a 48-bit hash of the complete naming intent. Human-readable names are shortened before that suffix, so two names with the same visible prefix do not silently collapse to the same planned global name. Live collision checks still fail closed because a hash reduces collision risk but cannot prove that a globally unique Azure name is available.

Plan and WhatIf

-Plan:

• requires no Azure authentication;
• performs no network call to Azure and no mutation;
• resolves deterministic names, defaults, dependencies, policies, and actions;
• marks quotas, resource existence, live prices, permissions, and other external facts as unverified when they cannot be proven locally;
• produces the same canonical plan for the same versioned inputs and extensions.

-WhatIf:

• requires Azure authentication;
• accepts an immutable normalized plan;
• queries live state but never mutates it;
• resolves unverified facts where possible;
• reports every node as create, reuse, update, replace, conflict, drift, or no-change;
• records query time, scope, identity, provider responses, and uncertainty.

reuse means desired and observed state can be consumed without ownership transfer. drift means a previously managed resource differs from recorded state. conflict means safe intent cannot be established. Engine-native previews are attachments, not replacements for this vocabulary.

Ownership and teardown

Every resource has exactly one ownership class:

Class	Lifecycle meaning
owned	Created by the run; eligible for approved teardown
adopted	Existing and explicitly brought under a recorded lifecycle scope
reused	Existing and consumed without ownership transfer
external	Referenced or observed only

Deletion requires owned or eligible adopted status, matching live identity, explicit teardown intent, successful teardown preview, and approval bound to the teardown-plan hash. reused, external, ambiguous, and unknown resources are never deleted.

Evidence envelope

Every operation emits a versioned envelope:

• evidence contract version, event ID, run ID, and plan hash;
• operation, status, start and end times;
• actor and execution identity references;
• provider, engine, schema, and catalog versions;
• affected stable IDs and correlation IDs;
• findings, risks, approvals, diagnostics, and redacted outputs;
• integrity hash and references to attachments.

The current plan can be exported as canonical JSON, Markdown, or standalone script-free HTML. Markdown and HTML include desired properties, external resource IDs, teardown intent, action dependencies and postconditions, RBAC details, blockers, and acknowledgement IDs for human review. WhatIf, cost, deployment, probe, teardown, cleanup-proof, and their evidence envelopes remain committed design or backlog.

Local run state

The default backend is a protected per-user local directory keyed by immutable run ID:

Windows:       %LOCALAPPDATA%\AzureDataLabToolkit\runs\<runId>
macOS/Linux:   ${XDG_STATE_HOME:-$HOME/.local/state}/AzureDataLabToolkit/runs/<runId>

Each run directory contains:

• plan.json, the immutable normalized plan;
• events.ndjson, the append-only event log;
• snapshot.json, the latest replaceable projection and source event sequence;
• approvals/, approval records bound to a plan or teardown hash;
• evidence/, operation envelopes and content-addressed attachments;
• engine/, references and receipts from the selected engine;
• run.lock, the lease record and fencing token.

It never stores secret values, access tokens, signed URLs, or private keys. The backend requires owner-only access where the operating system supports it and refuses mutation when it cannot protect state from other local users.

Terminal runs are retained for 90 days by default. Incomplete runs, runs with unknown cleanup state, and runs that still own resources are never automatically pruned. A user or organization policy may retain evidence longer. Expiration removes local evidence only after required audit export; it does not delete Azure resources.

Future external state backends must implement the same append, snapshot, lock, integrity, retention, and access-control contract. Backend choice cannot change lifecycle meaning.

Remote run state

CI, webhooks, schedules, and other unattended mutation cannot use the local backend. They require an encrypted remote backend with:

• OIDC or managed-identity authentication and least-privilege data-plane access;
• atomic append or equivalent event ordering;
• renewable leases and monotonically increasing fencing tokens;
• idempotency-key and duplicate-trigger detection;
• resumable snapshots and content-addressed evidence;
• retention, legal-hold, recovery, and deletion policy;
• no secret values, access tokens, private keys, or reusable signed URLs.

Azure Blob Storage is the first reference design. Another backend can be added only by passing the same contract and failure tests. Loss of the lease, fencing authority, identity scope, or event integrity stops mutation.

Lock, resume, and reconciliation

Only one mutating operation may hold a run lock. The lock has an owner, 60-second lease, heartbeat at least every 20 seconds, operation, and monotonically increasing fencing token. A contender may take an expired lease only after live reconciliation; it cannot rely on elapsed time alone. Read-only operations may run concurrently against an immutable snapshot.

Resume:

1. authenticates the requester and acquires the lock;
2. verifies plan hash, approval scope, versions, and retention state;
3. compares events, latest snapshot, engine state, and live resources;
4. classifies each action and resource;
5. resumes only approved, idempotent work;
6. stops on unknown state, ownership ambiguity, conflict, replacement, or wider spend.

Snapshots accelerate reads but are reproducible from the event log. Corrupt or missing evidence stops mutation. A future schema migration preserves the original events and records the migration tool, source version, target version, and resulting integrity hash.