Test Data Management and Coverage

Question

Scenario: You're testing an online shopping cart API. Your team has written 200 test cases but the API still has bugs in production that weren't caught during testing. Analysis shows the test coverage is: endpoint_coverage=95%, scenario_coverage=65%, data_coverage=40%. Your manager asks: 'Why are bugs escaping to production despite high endpoint coverage?' Provide analysis and propose a solution to improve test quality.

Accepted Answer

ROOT CAUSE ANALYSIS - Why bugs escaped despite 95% endpoint coverage: ENDPOINT COVERAGE MISCONCEPTION: Endpoint coverage = Are all endpoints tested? (Yes, 95% of endpoints have at least 1 test). But this doesn't mean endpoints are THOROUGHLY tested. EXAMPLE: Cart API has 20 endpoints. All 20 have 1 test each = 95% endpoint coverage. But: - POST /api/cart/add has only happy path test (add valid item), - Never tested: adding duplicate items, adding quantity=0, adding negative quantity, exceeding inventory, exceeding user credit limit. SCENARIO COVERAGE = 65% is the real problem: Let's say /api/cart/add endpoint has these scenarios: 1. Add valid item to empty cart -> tested, 2. Add duplicate item (should increase quantity) -> NOT tested, 3. Add item when user credit limit exceeded -> NOT tested, 4. Add item when item out of stock -> NOT tested, 5. Add item with invalid product ID -> NOT tested, 6. Add more items than inventory available -> NOT tested. So only 1/6 scenarios tested = 17% scenario coverage, Not 95% as endpoint coverage suggests. THIS IS THE BUG SOURCE. DATA COVERAGE = 40% means test data is insufficient: Test data currently used: Valid product IDs: {1, 2, 3}, Valid quant

Type	Description	Target
Endpoint Coverage	All endpoints tested	100%
Scenario Coverage	Happy path + error cases + edge cases	80-90%
Data Coverage	Valid, invalid, boundary values tested	75%+
Code Coverage	Source code executed by tests	70-80%

Test Data Management and Coverage

📌 Definition:

📖 Detailed Explanation:

📎 Coverage Types:

📎 Test Data Guidelines:

🎯 Scenario-Based Interview Question: