Skip to content
MAF Learning Hub
Advanced Govern Observe Duration: 35 min

Automating tests with gateway policies and Platform APIs

Prerequisites: graph-auth-and-governance-testing

The taxonomy is only useful if it runs. This capstone lesson turns the concepts into an execution model: what to do by hand, what a human kicks off but tooling completes, and what runs unattended - all on GA Anypoint tooling.

What you will learn

  • The manual, semi-automated, and fully automated testing ladder.
  • How gateway policies act as automated, fast-fail test gates.
  • Which GA Platform/Monitoring APIs feed trace-to-test and drift detection.

The automation ladder

A. Manual (human-driven). Exploratory testing against the broker via an API client (Postman/Insomnia) for one-off debugging; reviewing individual execution traces in the Anypoint Monitoring UI; and authoring the initial golden dataset entries and LLM-as-judge rubric - always a human judgment call that automation only scales afterward.

B. Semi-automated (human curates, tooling executes). After a change, re-run a case and compare the new Monitoring trace against the previous baseline (timing, connections called, path taken) - done manually or scripted by pulling both traces via the Platform APIs and diffing them. Trace-to-test conversion: pull a bad production trace, then shape it into a permanent regression case (retrieval scriptable; deciding it is a genuine failure is a human step). Sampled human review of LLM-as-judge output catches rubric drift.

C. Fully automated (scheduled/CI-triggered). Gateway policies as gates, CI/CD promotion gating, and scheduled drift/regression runs - covered next.

Automate the scale, not the judgment

Humans author golden sets and rubrics and decide what counts as a real failure; tooling then runs those judgments at scale, every commit and every night.

Gateway policies as automated test gates

Apply Omni/Flex Gateway policies scoped to the specific API/connection instance in API Manager - not gateway-wide - so unrelated connections stay untouched.

  • DataWeave Body Transformation Policy - rewrite a request/response payload in flight, for example translating an A2A v0.3.0 JSON-RPC method name to a v1.0 equivalent, to test protocol-version handling. Pair with a Header Injection policy when a specific protocol header is needed (e.g. A2A-Version: 1.0).
  • A2A / MCP Schema Validation - send a deliberately malformed request/response and confirm the policy rejects it.
  • PII Detector (A2A) - feed synthetic PII through the pipeline and confirm it is redacted/blocked before assuming the guardrail is effective.
# DataWeave Body Transformation - scoped to ONE connection instance.
# Test protocol-version handling by rewriting a v0.3.0 method to v1.0.
policyRef:
  name: dataweave-body-transformation-flex
config:
  requestFlow: "onRequest"
  script: |
    %dw 2.0
    output application/json
    ---
    payload update {
      case m at .method ->
        if (m == "message/send") "SendMessage" else m
    }

Validate the DataWeave script locally against a sample payload before applying it live, and scope both the transformation and any paired header policy to the specific connection instance.

  • CI/CD gating - wire these tests into your CI tool (Jenkins/GitHub Actions/Azure DevOps) as a pipeline step, using MuleSoft’s Platform CLI/REST APIs to gate promotion (DEV to UAT to PROD) on green.
  • Scheduled drift/regression runs - run the full golden-dataset suite (all four layers) nightly, independent of code changes, and automate baseline-vs-new-run trace comparison via the Platform APIs.

Forward-looking

The DataWeave policy is illustrative. Confirm exact policy identifiers, config schema, and method-name mappings against the official Included Policies directory and A2A spec versions before applying to a live connection.

Platform APIs for logs and MCP telemetry

Validate broker behavior using your own logs and telemetry through GA, officially supported Anypoint options - pick based on what is already enabled:

  • Manual - review individual traces in the Anypoint Monitoring UI (included with any Anypoint Platform subscription). Lowest-effort starting point.
  • Anypoint Monitoring Archive API - bulk/batch pull of historical metrics and logs for programmatic analysis (retention up to 1 year on Titanium), feeding both trace-to-test and scheduled drift detection.
  • Anypoint Metrics/Observability API (AMQL) - query API-level metrics (latency percentiles, request/response size, route taken) to build automated performance-regression checks.
  • Application/runtime log export - stream or pull runtime, application, and API management logs into your own stack (Splunk, ELK).
  • Telemetry Exporter / OpenTelemetry export (if Advanced Monitoring/distributed tracing is enabled; Mule Runtime 4.11.0+) - export traces/logs to a standard OTel collector (Datadog, New Relic, Splunk HEC, Azure Monitor) in near real time.
  • MCP tool invocations - MCP calls made through the Omni Gateway flow through the same centralized logging/auditability pipeline as regular API traffic, so every option above also covers MCP tool-invocation logs.

Forward-looking

Confirm with MuleSoft whether OpenTelemetry export captures Agent Network 2.0 / Agent Script broker execution traces specifically, and verify that full real-time telemetry/audit export may require a specific subscription tier (e.g. Integration Advanced or Titanium) before designing automation that depends on it.

Testing taxonomy checklist

Regression

  • Deterministic/structural checks for routing, schema, tool invocation, guardrails.
  • Semantic similarity for generative content, not exact-match.
  • LLM-as-judge with a versioned rubric for nuanced/high-risk correctness.
  • Human review sampled for the highest-risk/adversarial cases.
  • Golden datasets per component, versioned with the config they validate.
  • Trace-to-test pipeline turning real failures into permanent coverage.
  • Scheduled drift detection, not just pre-release.
  • Mocked dependencies for isolated per-component testing.
  • Full pyramid: unit to integration to E2E (including the front end).
  • Explicit handling for task-based vs. context-managing components.
  • Suites pinned and retired in step with broker version lifecycle.

Beyond regression

  • Protocol conformance per A2A/MCP version in use.
  • Agent card / MCP spec validation on every publish or change.
  • Cross-version interoperability/mock testing.
  • Agent network / graph-level topology and Agent Script wiring tests.
  • Per-connection auth/authz tests, including explicit MCP authorization checks.
  • Governance/policy consistency across all network assets.

Where to go next

You have completed Track 6: Testing & Assurance and the full MAF learning path. Adapt this generic taxonomy to a real engagement by mapping each suite onto the actual architecture in scope, then revisit any track from the home page or explore lessons by capability on the capability view.

References