Normally, the first thing to try is to wrap everything in a try...catch block.

try {
    $ErrorActionPreference='Stop'
    az group deployment create -g ....
    # Error in az group
    # More az cli commands
}
catch {
    Write-Host "ERROR: $Error"
}

Aaaaand it doesn't work.

This is when you scratch your head and realize that we are dealing with Azure CLI commands and not Azure PowerShell. They are not native PowerShell commands which would honor $ErrorActionPreference, instead, (as bad as it sounds), we have to treat each Azure CLI command independently as if we were running individual programs (in the back end, the Azure CLI is basically aliases which run python commands. Ironically most of Azure PowerShell commands are just PowerShell wrappers around Azure CLI commands ;-)).

Knowing that the Azure CLI will not throw a terminating error, instead, we have to treat it like a program, and look at the return code (stored in the variable $LASTEXITCODE) to see if it was successful or not. Once we evaluate that, we can then throw an error:

az group deployment create -g ....
if($LASTEXITCODE){
    Write-Host "ERROR: in Az Group"
    Throw "ERROR: in Az Group"
}

This then can be implemented into a try...catch block to stop the subsequent commands from running:

try {
    az group deployment create -g ....
    if($LASTEXITCODE){
        Write-Host "ERROR: in Az Group"
        Throw "ERROR: in Az Group"
    }
    # Error in az group
    # More az cli commands
}
catch {
    Write-Host "ERROR: $Error"
}

Unfortunately this means you have to evaluate $LASTEXITCODE every single time you execute an Azure CLI command.

Update - PS Version 7 error trapping for Az CLI (or kubectl)

This behavior changed significantly in PowerShell v7:

• https://github.com/PowerShell/PowerShell/issues/4002
• https://github.com/PowerShell/PowerShell/pull/13361

I ended up using the following solution instead which is consistent with PowerShell v5/7. It has the advantage of accepting pipeline input for commands like kubectl where stdio can be used for applying configuration etc.

I used scriptblocks since this integrates nicely into existing scripts without breaking syntax:

# no error
# Invoke-Cli {cmd /c echo hi}

# throws terminating error
# (with psv7 you can now call Get-Error to see details)
# Invoke-Cli {az bad}

# outputs a json object
# Invoke-Cli {az account show} -AsJson

# applies input from pipeline to kubernetes cluster
# Read-Host "Enter some json" | Invoke-Cli {kubectl apply -f -} 

function Invoke-Cli([scriptblock]$script, [switch]$AsJson)
{
    $ErrorActionPreference = "Continue"

    $jsonOutputArg = if ($AsJson)
    {
        "--output json"
    }

    $scriptBlock = [scriptblock]::Create("$script $jsonOutputArg 2>&1")

    if ($MyInvocation.ExpectingInput)
    {
        Write-Verbose "Invoking with input: $script"
        $output = $input | Invoke-Command $scriptBlock 2>&1
    }
    else
    {
        Write-Verbose "Invoking: $script"
        $output = Invoke-Command $scriptBlock
    }
    
    if ($LASTEXITCODE)
    {
        Write-Error "$Output" -ErrorAction Stop
    }
    else
    {
        if ($AsJson)
        {
            return $output | ConvertFrom-Json
        }
        else
        {
            return $output
        }
    }
}

Handling command shell errors in PowerShell <= v5

Use $ErrorActionPreference = 'Stop' and append 2>&1 to the end of the statement.

# this displays regular output:
az account show

# this also works as normal:
az account show 2>&1

# this demonstrates that regular output is unaffected / still works:
az account show -o json 2>&1 | ConvertFrom-Json

# this displays an error as normal console output (but unfortunately ignores $ErrorActionPreference):
az gibberish

# this throws a terminating error like the OP is asking:
$ErrorActionPreference = 'Stop'
az gibberish 2>&1

Background

PowerShell native and non-native streams, while similar, do not function identically. PowerShell offers extended functionality with streams and concepts that are not present in the Windows command shell (such as Write-Warning or Write-Progress).

Due to the way PowerShell handles Windows command shell output, the error stream from a non-native PowerShell process is unable (by itself) to throw a terminating error in PowerShell. It will appear in the PowerShell runspace as regular output, even though in the context of Windows command shell, it is indeed writing to the error stream.

This can be demonstrated:

# error is displayed but appears as normal text
cmd /c "asdf"

# nothing is displayed since the stdio error stream is redirected to nul
cmd /c "asdf 2>nul"

# error is displayed on the PowerShell error stream as red text
(cmd /c asdf) 2>&1

# error is displayed as red text, and the script will terminate at this line
$ErrorActionPreference = 'Stop'
(cmd /c asdf) 2>&1

Explanation of 2>&1 workaround

Unless otherwise specified, PowerShell will redirect Windows command shell stdio errors to the console output stream by default. This happens outside the scope of PowerShell. The redirection applies before any errors reach the PowerShell error stream, making $ErrorActionPreference irrelevant.

The behavior changes when explicitly specified to redirect the Windows command shell error stream to any other location in PowerShell context. As a result, PowerShell is forced to remove the stdio error redirection, and the output becomes visible to the PowerShell error stream.

Once the output is on the PowerShell error stream, the $ErrorActionPreference setting will determine the outcome of how error messages are handled.

Further info on redirection and PowerShell streams

https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_redirection?view=powershell-7.2

You may use the automatic variable $?. This contains the result of the last execution, i.e. True if succeded or False if it failed: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_automatic_variables?view=powershell-5.1#section-1

Your code would look something like this:

az group deployment create -g .... 
if(!$?){ 
  Write-Error "Your error message"
  # Handle your error
} 

Unfortunately, and same to @HAL9256 answer, you will need to add this code every single time you execute azure-cli

The sample from HAL9256 didn't work for me either, but I did find a workaround:

az group deployment create -g .... 
if($error){ 
  Write-Host "ERROR: in Az Group" 
  #throw error or take corrective action
  $error.clear() # optional - clear the error 
} 
# More az cli commands

I used this to try to deploy a keyvault, but in my environment, soft delete is enabled, so the old key vault is kept for a number of days. If the deployment failed, I'd run a key vault purge and then try the deployment again.

Building from HAL9256's and sam's answers, add this one-liner after each az cli or Powershell az command in a Powershell script to ensure that the catch block is hit when an error occurs:

if($error){ throw $error }

Then for the catch block, clear the error, e.g.

catch {
    $ErrorMessage = $_.Exception.Message
    $error.Clear()
    Write-Warning "-- Operation Failed: $ErrorMessage"
}