Villa Backend SDK — Testing Guide

This document explains **how we test the SDK**, **why tests are organized the way they are**, and **how to add new tests** following the same methodology.

Test-driven development (TDD) approach

Development follows a **test-first mindset**:

1. **Define the behavior** — what should PrePaymentValidator do when isDrafting=true?

2. **Write a failing test** — assert result.valid is False and error mentions "drafting"

3. **Implement the minimum code** — add the drafting check in the pipeline

4. **Refactor safely** — tests guard against regressions

We do not require every commit to be strictly red→green (real projects rarely do), but **every module has tests written alongside or before implementation**, and **CI blocks merges when tests fail**.

Test pyramid

                    ┌───────────────┐
                    │  Integration  │  4 tests — multi-service flows
                    │   (slow-ish)  │
                    └───────┬───────┘
                            │
              ┌─────────────▼─────────────┐
              │         Unit tests         │  ~45 tests — single class/function
              │  fast, isolated, mocked    │
              └─────────────┬─────────────┘
                            │
              ┌─────────────▼─────────────┐
              │      Schema contract       │  fixtures from villaMasterSchema
              │  deterministic, offline    │
              └───────────────────────────┘

We **do not** hit shop.villamarket.com in CI. All network I/O is mocked so tests are deterministic and do not require credentials.

Directory layout

tests/
├── conftest.py              # shared fixtures (orders, config, HTTP mocks)
├── fixtures/                # JSON payloads from villaMasterSchema testData
│   ├── goodSample.json      # valid order → must pass schema validation
│   ├── wrongType.json       # orderId is integer → must fail
│   └── extraCol.json        # extra field (reference fixture)
├── helpers/
│   └── fakes.py             # FakeCognitoClient shared across auth tests
├── unit/                    # one file per SDK module
│   ├── test_auth.py
│   ├── test_client.py
│   ├── test_cli.py
│   ├── test_config.py
│   ├── test_http.py
│   ├── test_order_service.py
│   ├── test_payments.py
│   ├── test_schema_contract.py
│   └── test_validation.py
└── integration/
    └── test_checkout_flow.py  # end-to-end flows with composed mocks

What each test file covers

test_auth.py — Cognito authentication

**Mocking strategy:** inject FakeCognitoClient instead of calling boto3.

test_config.py — configuration

test_http.py — HTTP transport

test_order_service.py — orders module

test_schema_contract.py — schema contract tests

Uses **parametrized tests** with fixtures from villaMasterSchema:

@pytest.mark.parametrize(
    ("fixture_name", "should_pass"),
    [("goodSample.json", True), ("wrongType.json", False)],
)
def test_schema_contract_fixtures(...):
    ...

This ensures the bundled order.yaml and our validator stay aligned with the official schema repo.

test_validation.py — pre-payment pipeline

test_payments.py — payment service

test_client.py — facade

test_cli.py — CLI smoke tests

Uses click.testing.CliRunner to invoke commands without a shell:

test_http_integration.py — wire-level HTTP (responses)

Uses the responses library to intercept **real requests calls** at the network layer. Unlike MagicMock, this verifies:

test_cognito_integration.py — AWS Cognito (moto)

Uses moto to emulate Cognito in-process:

test_checkout_flow.py — composed business flows

Higher-level scenarios combining multiple SDK modules (validate → pay, auth → order fetch).

test_checkout_flow.py — composed business flows

Shared fixtures (conftest.py)

Fixtures keep tests **short and readable** — each test focuses on one behavior, not JSON loading boilerplate.

Mocking conventions

HTTP — patch session.request

with patch.object(http_client.session, "request", return_value=mock_response):
    result = http_client.get_json(...)

We mock at the **session level** (not requests.post globally) so each HttpClient instance is isolated.

Cognito — inject fake client

service = CognitoAuthService(config, client=FakeCognitoClient())

Constructor injection avoids patching boto3 and keeps auth tests pure unit tests.

API validator in integration tests — replace on client

client.api_validator = MagicMock()
client.pre_payment = PrePaymentValidator(client.schema_validator, client.api_validator)

When testing composed flows, replace the **validator collaborator** rather than patching deep internals.

Markers

Tests use pytest markers defined in pyproject.toml:

@pytest.mark.unit
def test_something(): ...

@pytest.mark.integration
def test_checkout_flow(): ...

Run selectively:

pytest -m unit
pytest -m integration

Running tests locally

# Install with dev dependencies
pip install -e ".[dev]"

# Full suite
pytest

# Verbose with coverage
pytest -v --cov=villa_backend_sdk --cov-report=term-missing

# Single file
pytest tests/unit/test_validation.py -v

# Single test
pytest tests/unit/test_validation.py::test_pre_payment_validator_rejects_draft_order -v

Expected result: **all 67 tests pass** with no live backend access required.

CI pipeline

GitHub Actions (.github/workflows/ci.yml) runs on every push and pull request:

Python 3.10 ──┐
Python 3.11 ──┼── pip install -e ".[dev]"
Python 3.12 ──┘
        │
        ├── pytest tests/unit -v --cov
        ├── pytest tests/integration -v
        ├── pytest -v                  (full suite)
        └── python -m build            (package still builds)

CI uses the same commands as local development — **if it passes locally, it should pass in CI**.

Adding a new test (checklist)

When you add a feature, follow this checklist:

1. **Pick the layer**

- Single function/class change → tests/unit/test_<module>.py

- Multiple services interacting → tests/integration/

2. **Name the test after behavior**

- Good: test_pre_payment_validator_rejects_draft_order

- Avoid: test_validator_1

3. **Arrange → Act → Assert**

```python

# Arrange

validator = PrePaymentValidator(...)

draft_order = order.model_copy(update={"is_drafting": True})

# Act

result = validator.validate(draft_order, verify_cost=False, verify_grand_total=False)

# Assert

assert result.valid is False

assert "drafting" in result.errors[0]

```

4. **Mock external I/O** — never call live APIs in unit/integration tests

5. **Run the suite**

```bash

pytest -v

```

6. **If adding schema fields** — update fixtures or add new fixture JSON from villaMasterSchema testData/

Schema fixture policy

Fixtures in tests/fixtures/ come from villaMasterSchema/order/testData:

When villaMasterSchema changes, **update bundled schemas and fixtures together**, then rerun contract tests.

Coverage expectations

We aim for high coverage on **service logic**, not boilerplate:

Run coverage report:

pytest --cov=villa_backend_sdk --cov-report=term-missing

Uncovered lines in CLI argument parsing or __init__.py re-exports are acceptable. Uncovered lines in validation or payment paths are not.

CI/CD

See **CI_CD.md** for the full pipeline documentation.

Quick summary — CI runs on every push/PR:

Related documents