Skip to content
Azure Data Lab Toolkit

Overview

Azure Data Lab Toolkit is a planned PowerShell module for creating repeatable data labs in Azure. Its purpose is to bring infrastructure, databases, sample data, assessment, community tools, testing, cost controls, and teardown into one understandable workflow.

The first implementation target is SQL Server on Azure VM. That target gives the project a demanding proving ground for identity, networking, storage, guest configuration, software installation, testing, and cleanup before the same core is extended to managed data services.

Intended experience

Section titled “Intended experience”

The planned workflow starts with a schema-validated YAML file:

  1. Select a template or describe a lab.
  2. Validate prerequisites, permissions, regions, quotas, and policy.
  3. Review a deterministic plan, an Azure-aware comparison, and an explicit cost estimate.
  4. Review the target’s secret-store decision and, for a VM, its administrative-access decision.
  5. Run applicable assessment without treating assessment as migration or a prerequisite for every earlier review step.
  6. Confirm any sensitive-data or destructive operation.
  7. Deploy through PowerShell initially; Bicep and Terraform follow before Fabric delivery.
  8. Load approved sample, community, generated, private, local, or URL-based data.
  9. Install approved software and solution packs.
  10. Probe the environment and export an HTML evidence report.
  11. Shut down, resume, or remove the lab with lifecycle guardrails.

Templates, command-line flags, and a future terminal UI are intended to serve different experience levels without creating different underlying behaviors.

Plan and WhatIf

Section titled “Plan and WhatIf”

The committed command design gives -Plan and -WhatIf different jobs:

  • -Plan works without Azure access. It validates the YAML and explains defaults, derived values, resources, actions, and guardrails.
  • -WhatIf runs after Azure sign-in. It compares the resolved plan with live resources and reports create, reuse, update, replace, conflict, drift, or no-change without making changes.

Neither command exists yet. The first implementation proof point must demonstrate both behaviors for SQL Server on Azure VM before deployment, probes, reporting, and confirmed teardown.

Architecture at a glance

Section titled “Architecture at a glance”

Azure Data Lab Toolkit architecture showing YAML input flowing through Core and extension contracts to data-lab targets and evidence outputs

The design separates reusable decisions from target-specific behavior:

  • Core owns configuration, validation, planning, policy, run state, reporting, and lifecycle controls.
  • Target providers understand deployable platforms such as SQL Server on Azure VM, managed or self-hosted PostgreSQL, or Microsoft Fabric.
  • Capability modules connect supporting services such as Key Vault, Bastion, storage, Git, identity, assessment, and cost data.
  • Catalog sources describe governed data, software, and templates; solution packs compose them into scenarios.
  • Deployment engines translate approved actions; probes verify outcomes and create evidence.

Read Architecture Overview for the full model.

Delivery direction

Section titled “Delivery direction”

SQL VM is followed by Azure SQL Database and SQL Managed Instance. The project then delivers managed PostgreSQL, PostgreSQL on an Azure VM, and containerized PostgreSQL before adding Bicep, Terraform, and the broader Git and CI/CD adapters. Fabric follows those segments, then Databricks and later platforms.

Every target and solution pack must state whether it will deploy or reuse Key Vault, use an approved external secret store, or explain why no secret store applies. Every Azure VM defaults to Azure Bastion with no public IP unless a reviewed plan explicitly approves another administrative-access decision.

Read the Roadmap for the full S0-S22 delivery plan and quality gates.

Project relationship

Section titled “Project relationship”

Azure SQLVM Toolkit is an unfinished predecessor and a source of lessons for the new architecture. Its published documentation remains online as a record of that work.

Azure Data Lab Toolkit is a new repository and design. Features from the older toolkit are not automatically available here.

Reading status correctly

Section titled “Reading status correctly”

Published releases and release notes will be authoritative for Available behavior. The repository shows current artifacts, the architecture records committed design, and the roadmap records backlog, exploratory, and deferred work. Read Project Status for the full vocabulary.

Where to go next

Section titled “Where to go next”