Pillar 02 · Network Threat Efficacy

Cloud Firewall Benchmarking

Deploy a non-intrusive sandbox that executes real-world exploits against your native cloud firewalls and Palo Alto Networks side-by-side — then delivers a Security Validation Report.

Try the Live Demo →

Overview

Native security is convenient, but is it sufficient? Cloud Firewall Benchmarking replaces the "good enough" assumption with irrefutable, empirical data — measuring how effectively your current firewalls block sophisticated L7 threats compared to a Palo Alto Networks Cloud NGFW.

The assessment deploys a temporary, isolated Cyperf (Keysight) sandbox in your account, replays high-severity CVEs, malware, and C2 beaconing against both firewalls simultaneously, and produces a side-by-side Security Validation Report (SVR).

Exploit Samples Blocked
Palo Alto Networks Cloud NGFW vs. native CSP firewalls
PANW Cloud NGFW
95.3%
Azure Firewall Premium
18.7%
AWS Network Firewall
3.9%
Source: Miercom Cloud NGFW Security Benchmark, February 2025. 3rd-party report — not sponsored by Palo Alto Networks.

Key Capabilities

  • Side-by-Side Efficacy Testing: Real-world strikes injected in both directions against a PANW firewall and your native cloud firewall at the same time.
  • Comprehensive Strike Packs: High-severity CVEs, malware, and command-and-control beaconing measured for detection depth and blocking efficacy.
  • Non-Intrusive Sandbox: Uses Cyperf (Keysight) in an isolated environment — zero risk to production traffic.
  • Highest-Level Security Configuration: Both firewalls tested at maximum security posture — Advanced Threat Prevention, Advanced WildFire, Advanced URL Filtering.
  • Multi-Cloud: Supported on AWS and Azure.
  • Board-Ready SVR: A Security Validation Report showing exactly where gaps exist and how to close them.

Assessment Areas

Threat Detection

  • High-severity CVE exploits
  • Malware samples
  • Command-and-control beaconing
  • Evasive L7 attacks

Directional Coverage

  • Inbound threat blocking
  • Outbound exfiltration blocking
  • Per-DUT breakdown
  • Overall efficacy score

Advanced Security Features

  • Advanced Threat Prevention
  • Advanced WildFire (sandboxing)
  • Advanced URL Filtering
  • DNS security

Framework Alignment

  • CIS Benchmarks
  • NIST Cybersecurity Framework
  • Miercom test methodology
  • Cyperf strike pack standards

Deliverable

  • Security Validation Report (SVR) — side-by-side comparison
  • Test objective and threat categorization
  • Policy configuration used on both DUTs
  • Overall threat efficacy score for both firewalls
  • Test topology diagram
  • Inbound and outbound efficacy breakdowns
  • Appendix — full list of injected threats with priority and blocking status

Availability

Cloud Firewall Benchmarking has no geographical restrictions — it's delivered worldwide. Coverage is limited to AWS and Azure only; GCP is not supported today.

i
Worldwide, AWS & Azure only Any customer, in any region, can run this assessment as long as they have quota for six test instances in either AWS or Azure. There is no GCP path today — for GCP customers, position the Cloud Network Risk Assessment instead.

Try the Live Demo

Skip the deploy entirely and run the benchmark against PANW-hosted CyPerf environments. Pick a cloud below to open the CyPerf Controller — from there you can start a test, watch live results, and download the Security Validation Report exactly as customers do.

AWS · CyPerf Controller
Open in new tab ↗
i
Login credentials Username: admin · Password: CyPerf&Keysight#1
!
Embed not loading? Use the "Open in new tab" link. The CyPerf Controller is served from a raw IP with a self-signed certificate — browsers will refuse to load it in an iframe silently, and it may also decline to be framed at all. The Open in new tab ↗ link works around both: your browser will show a certificate warning once (click Advanced → Proceed) and then the controller loads normally.

Training Walkthrough

End-to-end steps for deploying and running the Cloud Firewall Benchmarking demo in your own cloud account. If you just want to see the assessment running, use Try the Live Demo above — no setup required. Otherwise, follow these in order; the entire flow completes in under 24 hours.

i
Before You Begin You'll need a non-production test environment, quota for six test instances, and a CyPerf license server IP (contact the TME team, e.g. bmyneni@paloaltonetworks.com).
Prereq

Install Docker Desktop

Download Docker Desktop for macOS to a local Linux machine, install it, and confirm the Docker daemon is running before proceeding.

Prereq

Download the Benchmarking Package

Download the Cloud Firewall Benchmarking package to your local machine and verify the contents:

$ ls pan_demo_setup.sh pan_demo_setup.tar terraform-aws.tfvars terraform-azure.tfvars terraform-state-aws terraform-state-azure

The package includes the setup script, deployment tarball, and Terraform variable files for both AWS and Azure.

Step 1 · AWS

Configure the Terraform Variables

Edit terraform-aws.tfvars with the credentials of the cloud account where you want to deploy:

aws_stack_name="" aws_region="" aws_access_key_id="" aws_secret_access_key="" aws_session_token="" aws_auth_key="<name of your .pem file>" aws_allowed_cidr=["x.x.x.x/x"] aws_license_server="x.x.x.x" user_tags = { tag_ccoe-app="XXXX" tag_ccoe-group="XXXX" tag_UserID="XXXX" }

The stack name is prepended to deployed instances. The allowed CIDR controls network access — use ["0.0.0.0/0"] to allow all. The license server IP comes from the TME team.

Step 2 · AWS

Deploy the Stack

Make the deployment script executable, subscribe to the CyPerf Controller and Agent on AWS Marketplace (one-time), then trigger the deploy:

chmod 777 pan_demo_setup.sh PALOALTONETWORKS_MARKETPLACE_AND_KEYSIGHT_EULA_ACCEPTED=true \ ./pan_demo_setup.sh --deploy-aws

When the deploy finishes, capture the CyPerf controller UI IP from the output — it looks like:

CyPerf server is available, configuring... Configuring new license server x.x.x.x Successfully added license server x.x.x.x Uploading simple UI files... ... CyPerf controller is at: https://x.x.x.x
Step 1 · Azure

Create Azure App Registration

In the Azure portal, create a new App Registration. Add an App Role with write permissions, then generate a Client Secret from the Overview menu — capture it immediately, it won't be shown again.

Step 2 · Azure

Configure Azure Terraform Variables

Edit terraform-azure.tfvars with your Azure credentials and the App Registration values from Step 1:

azure_stack_name="<short name>" azure_location="eastus" azure_client_id="XXXXXXXXXXXXX" azure_client_secret="XXXXXXXXXXXXXXX" azure_tenant_id="XXXXXXXXXXXXXXX" azure_subscription_id="XXXXXXXXXXXXXXX" azure_auth_key="<name of Azure key pair for SSH access>" azure_allowed_cidr=["<your public IP>"] azure_license_server="<CyPerf license server IP>" tag_ccoe-app="XXXXX" tag_ccoe-group="XXXX" tag_UserID="XXXX"

Then follow the same deploy steps as the AWS flow — the setup script takes an --deploy-azure flag.

Step 3

Install Latest Content on the PANW Firewall

Before running the efficacy test, install the latest security content so the PANW firewall is at its maximum posture. Either update manually via the firewall UI (Advanced Threat Prevention, Advanced WildFire, Advanced URL Filtering) or run the helper script:

./panos_update.sh --host <firewall_ip_address> \ --username admin \ --password Paloaltonetworks1! \ --action content-update
Step 4

Execute the Efficacy Test

Log in to the CyPerf controller UI using the IP captured earlier:

Username: admin Password: CyPerf&Keysight#1

The controller UI shows a topology diagram of the inbound and outbound threats CyPerf agents will inject across both the PANW firewall and the CSP firewall (the two DUTs). Available actions:

  • Start test — begins injecting inbound and outbound threats through both DUTs
  • Live Statistics — auto-shown after start; live threats-injected vs threats-blocked
  • Test Result — final scores when the run completes
  • Download report — export the SVR
  • Test details — see the exact test configuration
i
Strike Pack Updates For the latest strike pack, reach out to the TME team before you start the run.
Step 5

Download & Analyze the Report

When the test completes, click Download report in the top-right of the controller UI. The Security Validation Report contains:

  • Objective — threat efficacy comparison of PANW and the CSP firewall
  • Threats categorization — strikes used and their categories
  • Policy configuration — both DUTs configured at highest security posture
  • Overall threat efficacy — the headline blocking-rate comparison
  • Test topology — diagram of the deployed test environment
  • Inbound / outbound efficacy — directional breakdowns per DUT
  • Appendix — full list of injected threats with per-DUT block status and priority
Step 6

Tear Down the Environment

Once you've downloaded the report, destroy all deployed resources with a single command from the same folder:

./pan_demo_setup.sh --destroy
!
Don't Skip the Cleanup The stack includes six test instances plus a controller. Leaving them running incurs cloud spend and keeps a CyPerf license consumed.