Skip to main content

Contract Tests When Integrating External Systems

What This Concept Is​

A contract test asserts that two independently deployed services agree on the shape of the messages they exchange. Instead of spinning up the full external system in a test, you assert against the contract -- the request and response formats each side commits to.

Two common styles:

  • Consumer-driven contract tests (the Pact model): the consumer writes a test describing the requests it will send and the responses it expects. A contract file is produced. The provider verifies that its real implementation matches every interaction in the contract.
  • Schema-based contract tests: both sides validate messages against a shared OpenAPI, JSON Schema, gRPC proto, or similar specification. Breaking the schema breaks the test.

For capstone work, Pact-style consumer-driven contracts are the most common choice because you typically control only one side (the consumer) and want to prove the external system's current behavior matches what you expect. Pactflow's framing is useful: CDC turns the consumer's expectations into executable artefacts the provider is obligated to satisfy.

Contract tests live in the gap between unit tests and integration tests: fast enough to run on every PR, real enough to detect when a remote provider's shape has drifted from what your code assumes.

Why It Matters Here (In the Capstone)​

Capstones almost always integrate with something external: GitHub, Stripe, Slack, OpenAI, AWS SDK. Testing those interactions has three options:

  • hit the real service in tests (slow, flaky, costs money, leaks secrets);
  • mock everything (fast but proves nothing about the real contract);
  • use a contract test (fast, deterministic, and proven against both sides).

Integration tests against a fake (Concept 5) get you deterministic runs. Contract tests add the second half: a mechanism that breaks when the real external service changes its behavior. Without contracts, you will discover provider drift by paging the on-call -- exactly the failure mode S10 Module 4 wants you to avoid.

Concrete Example(s) -- from a real capstone​

In the task-manager capstone, the code consumes GitHub's issues API. A Pact-style contract on the consumer side:

def test_list_issues_contract(pact):
    expected = {"number": 1, "title": "Fix readme", "state": "open"}
    (pact
     .given("repo with one open issue")
     .upon_receiving("a request for issues")
     .with_request("GET", "/repos/acme/task-svc/issues")
     .will_respond_with(200, body=[expected]))

    with pact:
        result = github_client.list_issues("acme/task-svc")

    assert result[0].title == "Fix readme"

This produces a task-svc-consumer-github-api.json pact file. The CI pipeline can then:

  • replay the pact file against a local stub (keeping consumer tests deterministic and offline);
  • periodically run the pact file against the real GitHub API in a nightly job to detect breakage in the real provider.

If GitHub changes its response shape, the nightly job fails loudly. If your code changes its expectations, the unit-level consumer test fails immediately. The two tests cost seconds to run and cover a class of failure that integration tests against a fake cannot.

Common Confusion / Misconceptions​

The biggest misconception is that contract testing replaces integration testing. It does not. Contract tests assert shape, not behavior. You still want at least one integration test that exercises the adapter against a real or fake server and confirms your parsing, retrying, and error translation actually work.

Another is that you need both sides to participate for a contract test to be useful. For a capstone that does not control the provider (GitHub, Stripe), you run the "provider side" against the real external API periodically and treat drift as a bug.

A third is thinking contract tests are only for microservices. They are equally useful for any API boundary -- external third-party, internal, or cross-team.

A fourth is writing a "contract" that is really just a recorded mock response. If nothing re-verifies the recording against the real provider, it has the same value as a plain stub -- which is to say, close to zero for drift detection.

How To Use It (In Your Capstone)​

For each external seam in the capstone, ask:

  1. Am I the consumer, the provider, or both?
  2. Does the external side publish a schema (OpenAPI, proto)? If yes, prefer schema-based contract testing.
  3. If not, write a consumer-driven contract that captures what your code actually needs.
  4. Run the contract against a local stub in every build; run it against the real provider on a schedule.
  5. Treat contract drift as a first-class defect, not a warning -- triage it through Concept 10.
  6. Version the pact files so you can bisect when drift appeared.
  7. Pair every contract test with an integration test (Concept 5) at the same seam.

A practical setup:

Schema-Based vs Pact: Which When​

SituationRecommended style
Provider publishes OpenAPI / proto / GraphQL schemaSchema-based -- let the tooling validate requests and responses against the spec.
Provider is a public third party (GitHub, Stripe, OpenAI)Pact-style consumer-driven; your provider verification job runs against the real API on a schedule.
Your capstone is the provider and you have internal consumersPact-style; each consumer produces a contract, your CI verifies all of them.
Cross-team microservices inside one repoPact-style with shared broker; lets teams evolve independently.
Internal library boundary (same repo)Usually unnecessary -- integration tests are enough.

Anti-Patterns to Recognize​

  • Provider-written consumer contracts. Contracts dictated by the provider miss consumer needs.
  • "Pact as integration test." Pact files assert message shape, not full behavior.
  • Static mocks labeled as contracts. If a test records "API returns { status: 200 }" and never re-verifies against the real API, it is a mock.
  • Ignored nightly drift. The provider verification job goes red and the team silences the alert.

The Full Picture at a Seam​

For any meaningful external seam, the capstone needs three layers of coverage, not one:

  1. Unit-level adapter tests. Parsing, retry logic, error translation -- tested with no external involvement.
  2. Integration tests against a fake. Deterministic exercise of the adapter against an in-process fake server.
  3. Contract tests against the real provider. Scheduled, non-blocking, alerts on drift.

Missing any layer leaves a blind spot. The adapter might handle parsing but not retries; the fake might be perfect while the real API has changed; contracts might pass while retries are silently broken.

See also (integrative)​

External references:

Check Yourself​

  1. Why is a contract test not a substitute for an integration test?
  2. What is the role of the consumer in consumer-driven contract testing?
  3. When is a schema-based contract test simpler than Pact-style, and when does it break down?
  4. What are the three coverage layers you need at any real external seam?
  5. If your nightly provider-verification job has been red for a week, what is the incident severity and who fixes it first?

Mini Drill or Application (Capstone-scoped)​

  1. Pick one external seam in your capstone. In 45 minutes, write one consumer-driven contract test for one request you make.
  2. Produce the pact file and commit it alongside the test. Replay it against a local stub in CI.
  3. Add a nightly pipeline job that replays the pact against the real provider and opens a triage entry on failure.
  4. Version the pact file (semantic tag or checksum) so you can bisect when drift appears.
  5. Pair the contract test with an integration test (Concept 5) and a unit-level adapter test; confirm the "three-layer" coverage at that seam.

Source Backbone​

Capstone implementation applies earlier code-quality, testing, and refactoring material. These books are the source backbone for that practice.