How Azure Automation Cut Our Microsoft Fabric Cost by 78%

The problem was not the pipelines

Cost optimization conversations often start inside the workload.

Can the notebook run faster? Can transformations be simplified? Can refresh frequency be reduced? Can the pipeline use fewer activities?

Those are all valid questions. But they were not the biggest opportunity here.

Our pipelines already had reasonably short execution windows. The much larger inefficiency existed outside them.

Imagine a restaurant that serves customers for two hours every evening but keeps the entire kitchen operating 24 hours a day. Making the chef work 10% faster might help.

Closing the kitchen when there are no customers helps much more.

Our Fabric capacity had essentially the same problem.

Fabric knew how to execute our pipelines. It did not know when our organization actually needed the compute.

We had to provide that intelligence ourselves.

First, I mapped the real compute windows

Before automating anything, I documented when compute was actually required.

The weekday workload begins at approximately 6:00 PM Mountain Time.

Because it orchestrates the complete bronze → silver → gold chain and normally finishes around 6:30 PM, I did not want the capacity to resume exactly when the pipeline was scheduled to begin.

The platform should already be available before the first workload starts.

So I added a 15-minute startup buffer.

Saturday required a larger window.

The first processing stage starts around 4:00 PM. Another stage begins around 6:00 PM, with the complete workload normally finishing around 6:30 PM.

That gave us four automation events.

Sunday needs no automation event because the desired state is simply paused.

Automation without stored credentials

Once the compute windows were clear, the next question was authentication.

Something has to call Azure and tell the Fabric capacity to pause or resume.

The easy implementation would be to put credentials into a script or run the process under an account with broad Azure permissions.

I didn't want either.

Instead, I created an Azure Automation Account and enabled its system-assigned managed identity.

That gave the Automation Account its own identity in Microsoft Entra ID.

The automation authenticates to Azure as itself.

But authentication was only half of the problem.

The identity still needed authorization to operate the capacity.

I didn't give the automation Contributor access

The runbook had one responsibility: manage the lifecycle of one Fabric capacity.

Giving it Contributor access across an entire subscription would have worked technically, but it would also have granted the automation far more authority than it needed.

Instead, I created a custom Azure RBAC role containing only the Fabric capacity operations required by the process:

Microsoft.Fabric/capacities/read
Microsoft.Fabric/capacities/write
Microsoft.Fabric/capacities/suspend/action
Microsoft.Fabric/capacities/resume/action

I assigned that role to the Automation Account's managed identity at the individual Fabric capacity resource scope.

The automation did not receive broad control over every Azure resource.

It received permission to operate the resource it was responsible for.

Cost automation should not create a new security problem.

One runbook controls both actions

I could have created separate runbooks for pause and resume.

I deliberately didn't.

Both operations act on the same resource and differ only by the requested action, so I created one parameterized PowerShell runbook.

param (
    [Parameter(Mandatory = $true)]
    [ValidateSet("Pause", "Resume")]
    [string]$Action
)

The four schedules call the same runbook with either:

Action = Resume or Action = Pause.

At a high level, the runbook performs six steps.

1. Authenticate Use the Automation Account's managed identity.
2. Read state Check the current state of the Fabric capacity.
3. Compare Determine whether a state change is necessary.
4. Act Call either the suspend or resume operation.
5. Verify Poll the capacity until the requested state is reached.
6. Fail clearly Raise an error if the transition does not complete.

The state check is worth having.

Suppose somebody manually resumes the capacity earlier in the afternoon.

At 5:45 PM the automation still runs, but instead of blindly submitting another resume operation, the runbook checks whether the capacity is already active.

If it is, there is nothing to do.

The same applies to pause.

In other words, the automation is designed to be idempotent.

The core control logic

# Configuration values intentionally anonymized

if ($Action -eq "Pause") {
    $apiAction    = "suspend"
    $desiredState = "Paused"
}
else {
    $apiAction    = "resume"
    $desiredState = "Active"
}

# Avoid unnecessary operations
if ($currentState -eq $desiredState) {

    Write-Output "Capacity is already in the desired state."

    exit 0
}

# Submit the requested capacity operation
Invoke-WebRequest `
    -Uri $actionUri `
    -Method POST `
    -Headers $armHeaders `
    -ContentType "application/json"

The production runbook goes one step further.

After the API accepts the request, it continues polling the Fabric resource until the expected state is actually reached.

An accepted API request is not the same thing as a completed state transition.

I tested both directions before scheduling anything

I did not attach the runbook to production schedules immediately.

First, I manually tested the pause operation using the Azure Automation test pane.

Authenticating using Automation Account managed identity...

Fabric capacity: [redacted]
Current state: Active
Requested action: Pause

Submitting Pause request...

Request accepted. HTTP status: 202

State check 1 : Pausing
State check 2 : Paused

SUCCESS: Fabric capacity is now Paused.

That single test validated several components of the design at once.

• Managed identity authentication worked.
• The least-privilege RBAC assignment was sufficient.
• The Fabric capacity management endpoint was accessible.
• The pause request was accepted.
• The runbook observed Active → Pausing → Paused.

I then ran the same test with:

Action = Resume

and verified that the capacity returned to Active.

Only after both directions worked manually did I publish the runbook and connect the schedules.

A scheduled automation that has never been tested manually is just a scheduled failure waiting for the right time.

Then I attached the four schedules

Once the runbook was proven, the remaining implementation was surprisingly small.

Four schedules call the same runbook with different times and parameters.

The schedules use Mountain Time rather than manually calculated UTC timestamps.

That keeps the Fabric capacity window aligned with the workload schedule through daylight-saving time changes without maintaining separate summer and winter schedules.

The normal weekday flow

Saturday follows the same principle, just with a longer capacity window around the scheduled workloads.

168 hours became 9.5

This was the number that made the decision particularly interesting.

A week contains 168 hours.

Our permanent scheduled capacity windows total approximately:

Scheduled availability therefore changed from:

That represents approximately a 94% reduction in scheduled capacity runtime.

I intentionally do not describe that as a 94% reduction in the Azure bill.

Cloud billing is rarely that simple.

Storage remains. Actual workload consumption matters. Platform accounting matters. Other services can also contribute to the overall cost.

So rather than assuming that a 94% reduction in scheduled runtime would translate directly into a 94% reduction in cost, I used a deliberately conservative financial model.

The model allows for the reality that not every Fabric-related cost disappears when capacity is paused, and that actual workload consumption can vary from week to week.

Even after allowing for that additional consumption and variability, the expected reduction in Fabric capacity cost is approximately 78%.

The biggest saving required no pipeline optimization

What I like most about this change is what we did not have to do.

• We did not redesign the medallion architecture.
• We did not rewrite bronze ingestion.
• We did not change the silver transformations.
• We did not simplify the gold layer.
• We did not reduce the frequency of business workloads.
• We did not move processing to another platform.

The pipelines continue to operate exactly as they did before.

The optimization happened around them.

That is an important distinction in cloud architecture.

Sometimes the biggest cost improvement is not hidden inside SQL, Spark, Python, a notebook, or a data pipeline.

Sometimes the resource simply should not be running.

The risk: what if a workload runs long?

Scheduled shutdown introduces one obvious operational risk.

What happens if a workload normally finishes at 6:30 PM but one day is still processing at 7:00 PM?

At that point, a hard capacity pause stops being a cost optimization.

It becomes an availability problem.

This is why the schedules include completion buffers instead of pausing the capacity immediately after the expected processing time.

Today, workload execution is predictable enough that a fixed window is a reasonable trade-off.

But I would not consider this design permanently finished.

If processing durations become more variable, the next iteration would be workload-aware suspension.

For the current workload volume, that additional complexity is not yet justified.

I do not want to build a sophisticated control plane for a problem four schedules already solve reliably.

Seasonal workloads stay seasonal

Another data source becomes active only during a specific part of the year.

I intentionally did not expand the permanent weekly compute window to accommodate it.

When that processing season starts, we can temporarily add or modify the relevant capacity schedule.

When the season ends, the additional compute window can be removed.

This prevents the infrastructure schedule from gradually accumulating permanent runtime for workloads that are active only occasionally.

Capacity should follow demand.

What I will monitor from here

The automation is working, but the cost story should now be validated against real operational and billing data.

The automation that saves money has now become part of the operating platform.

It deserves monitoring like any other production dependency.