Track A: Testing & Verification Overview

Purpose: Build verification and testing habits that ensure code correctness and design quality
Philosophy: Testing is not cleanup after coding - it's part of design thinking from the beginning

Track Integration

This track runs alongside your semester coursework, starting immediately in Pre-Semester and evolving through all phases. Apply these skills to every project and implementation throughout the degree.

---

Track Progression by Phase

Phase	Level	Focus	Skills Developed
Phase 00-0	Level 1	Foundation habits	Unit tests for every implementation, assertion-driven development
Phase 1-2	Level 1	Mathematical verification	Test mathematical implementations, algorithm correctness verification
Phase 3	Level 2	Testable design	Design for testability, characterization tests for refactoring
Phase 4-5	Level 2	Systems testing	Integration tests, concurrent system testing, systems verification
Phase 6-7	Level 3	Architecture verification	Contract testing, API verification, architecture fitness functions
Phase 8	Level 3	Distributed testing	Service testing, distributed system verification, reliability testing
Phase 9-10	Level 4	Production verification	Infrastructure testing, end-to-end production verification, operational testing

---

Level 1 Starter Guide: Your First Tests

Getting Started Today (30 minutes)

Set up your testing environment:

1. Install pytest for Python:

   pip install pytest pytest-cov

2. Create your first test file (test_example.py):

   def add_numbers(a, b):
       return a + b
   
   def test_add_numbers():
       assert add_numbers(2, 3) == 5
       assert add_numbers(-1, 1) == 0
       assert add_numbers(0, 0) == 0

3. Run the test:

   pytest test_example.py

4. See it pass, then make it fail (change expected result) to understand test feedback.

Core Testing Principles (Start Here)

Every implementation needs at least one test that verifies:

1. Happy path: Normal input produces expected output
2. Edge cases: Boundary conditions and special values
3. Error conditions: Invalid input handled appropriately

For mathematical implementations, also test:

• Mathematical properties: Symmetry, associativity, identity elements
• Invariants: Properties that should remain true regardless of input
• Equivalence: Multiple implementations of the same concept should produce identical results

Your First Testing Workflow

For every module implementation exercise:

1. Write the test first (or immediately after understanding the problem)
2. Implement to pass the test
3. Add edge case tests
4. Refactor implementation while keeping tests passing
5. Document any assumptions or limitations

Example: Binary Search Implementation

def test_binary_search():
    # Happy path
    assert binary_search([1, 3, 5, 7, 9], 5) == 2
    
    # Edge cases
    assert binary_search([1], 1) == 0
    assert binary_search([], 5) == -1
    
    # Not found
    assert binary_search([1, 3, 5, 7, 9], 4) == -1
    
    # First and last elements
    assert binary_search([1, 3, 5, 7, 9], 1) == 0
    assert binary_search([1, 3, 5, 7, 9], 9) == 4

---

Practical Testing Techniques by Content Area

Mathematical Implementations (Phases 00-1)

Property-based testing:

def test_gcd_properties():
    # Test mathematical properties
    assert gcd(12, 18) == gcd(18, 12)  # Commutative
    assert gcd(12, 0) == 12            # Identity
    assert gcd(12, 6) == 6             # Divisor relationship

Verification against known results:

def test_fibonacci_known_values():
    known_values = [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
    for i, expected in enumerate(known_values):
        assert fibonacci(i) == expected

Algorithm Implementations (Phase 2)

Correctness testing:

def test_sorting_algorithm():
    # Test various input sizes and patterns
    test_cases = [
        [],                    # Empty
        [1],                   # Single element  
        [1, 2, 3],            # Already sorted
        [3, 2, 1],            # Reverse sorted
        [3, 1, 4, 1, 5, 9],   # Random with duplicates
    ]
    
    for test_input in test_cases:
        result = my_sort(test_input.copy())
        assert result == sorted(test_input)

Performance verification:

import time

def test_algorithm_performance():
    # Verify O(n log n) behavior for sorting
    small_input = list(range(1000, 0, -1))
    large_input = list(range(10000, 0, -1))
    
    start = time.time()
    my_sort(small_input)
    small_time = time.time() - start
    
    start = time.time()  
    my_sort(large_input)
    large_time = time.time() - start
    
    # Should scale roughly as n log n
    expected_ratio = (10 * math.log(10)) / (1 * math.log(1))
    actual_ratio = large_time / small_time
    
    # Allow some variance for system effects
    assert actual_ratio < expected_ratio * 2

Systems Programming (Phases 4-5)

Integration testing for C programs:

// test_shell.c - testing a simple shell implementation
#include <assert.h>
#include <string.h>

void test_command_parsing() {
    char input[] = "ls -la /home";
    char** tokens = parse_command(input);
    
    assert(strcmp(tokens[0], "ls") == 0);
    assert(strcmp(tokens[1], "-la") == 0);  
    assert(strcmp(tokens[2], "/home") == 0);
    assert(tokens[3] == NULL);
    
    free_tokens(tokens);
}

void test_file_operations() {
    // Test file I/O functions
    write_file("test.txt", "hello world");
    char* content = read_file("test.txt");
    assert(strcmp(content, "hello world") == 0);
    
    cleanup_test_files();
}

---

Essential Testing Tools by Phase

Phase 00-2: Python Testing Foundation

• pytest: Primary testing framework with rich assertion introspection
• pytest-cov: Code coverage measurement to identify untested code paths
• hypothesis: Property-based testing for mathematical and algorithmic code
• unittest.mock: Testing components in isolation with controlled dependencies

Phase 3: Design and Refactoring Tests

• Characterization tests: Capture existing behavior before refactoring
• Test doubles: Mocks, stubs, and fakes for testing in isolation
• Refactoring patterns: Safe code transformation with test protection
• Design by contract: Preconditions, postconditions, and invariants

Phase 4-5: Systems and Integration Testing

• System testing: Testing C programs with file I/O, networking, and process management
• Memory testing: Valgrind for memory leak detection and corruption analysis
• Performance testing: Profiling and benchmarking for systems code
• Integration testing: Testing component interaction and system behavior

Phase 6-8: Service and Architecture Testing

• API testing: REST/HTTP service verification with different clients
• Database testing: Transaction behavior, data integrity, performance characteristics
• Contract testing: Service interface verification and backward compatibility
• Architecture fitness: Testing that system design constraints are maintained

Phase 9-10: Production and Infrastructure Testing

• Infrastructure testing: Verifying that infrastructure code produces expected resources
• End-to-end testing: Full system testing from user perspective
• Reliability testing: Fault injection, chaos engineering, disaster recovery
• Operational testing: Monitoring, alerting, and incident response verification

---

Common Testing Scenarios by Semester

Pre-Semester and Semester 0

• Study system verification: Test that your habit tracking and note-taking systems work as intended
• Development environment: Test that your toolchain setup works correctly across different projects
• Basic algorithm implementations: Unit tests for search, sort, and simple data structure operations

Semester 1 (Mathematical Foundations)

• Proof verification: Test mathematical implementations against known theoretical results
• Combinatorial algorithms: Test counting functions, permutation/combination generators
• Probability simulations: Verify that Monte Carlo simulations converge to theoretical expectations
• Linear algebra operations: Test matrix operations, eigenvalue computations, decompositions

Semester 2 (Algorithms)

• Algorithm correctness: Comprehensive test suites for complex algorithms (graph traversal, dynamic programming, greedy algorithms)
• Performance verification: Testing that implementations achieve expected time/space complexity
• Data structure invariants: Testing that custom data structures maintain their promised properties

Semester 3 (Software Design)

• Design pattern implementations: Test that patterns solve the intended problems without introducing new issues
• Refactoring safety: Characterization tests and regression test suites
• Code quality: Automated testing for code smells, design violations, and maintainability metrics

Semesters 4-5 (Systems Programming)

• Memory safety: Testing for memory leaks, buffer overflows, and resource management
• Concurrent programming: Testing thread safety, deadlock avoidance, and race condition prevention
• Network programming: Testing client-server communication, protocol implementation, and error handling

Semesters 6-8 (Services and Architecture)

• Service integration: Testing API contracts, data consistency, and service composition
• Database integration: Testing transaction behavior, query performance, and data integrity
• Architecture compliance: Testing that implemented systems satisfy architectural constraints and quality attributes

Semesters 9-10 (Cloud and Production)

• Infrastructure verification: Testing that IaC code produces expected cloud resources with correct configuration
• Production readiness: Testing deployment procedures, monitoring systems, and operational runbooks
• End-to-end system verification: Testing complete system behavior from user perspective

---

Building Your Testing Mindset

Questions to Ask Before Implementation

1. What behavior am I promising? (This becomes your test specification)
2. What could go wrong? (This identifies edge cases and error conditions)
3. How will I know if this works correctly? (This defines your verification strategy)
4. What assumptions am I making? (These become test preconditions and documentation)
5. How will I verify this still works after changes? (This drives regression test design)

Red Flags That Indicate Weak Testing

• "It works on my machine" without systematic verification across different conditions
• Testing only happy paths without considering error conditions and edge cases
• Manual testing only without automated verification that can catch regressions
• Testing implementation details rather than promised behavior and interfaces
• Skipping tests due to time pressure then spending more time debugging mysterious failures

Green Flags for Strong Testing Practice

• Test-driven development: Tests written before or immediately after implementation
• Comprehensive coverage: Happy paths, edge cases, and error conditions all tested systematically
• Clear test names: Test names explain what behavior is being verified
• Fast feedback loops: Tests run quickly and provide immediate feedback on changes
• Regression protection: Changes to code immediately reveal impact through test failures

---

Resources for Continued Learning

Books by Phase

• Phase 00-2: Test-Driven Development: By Example (Beck) - Fundamental TDD practices
• Phase 3: Working Effectively with Legacy Code (Feathers) - Testing and refactoring strategies
• Phase 4-5: The Art of Unit Testing - Advanced testing techniques for different languages
• Phase 6-8: Testing Microservices with Mocha and Chai - Service testing strategies
• Phase 9-10: Site Reliability Engineering - Production testing and reliability practices

Online Resources

• Testing frameworks documentation for each language you learn
• Martin Fowler's testing articles - Architectural perspective on testing strategies
• Google Testing Blog - Industry practices and advanced testing techniques
• Ministry of Testing - Community and resources for testing professionals

Practical Exercises

• Code katas with TDD: Practice test-first development on algorithmic problems
• Legacy code refactoring: Practice adding tests to existing code before modifying it
• Integration testing: Build multi-component systems with comprehensive test coverage
• Performance testing: Develop benchmarking and performance regression detection skills

Remember: Testing is a design activity, not a quality assurance afterthought. Good tests clarify what your code is supposed to do and provide confidence that it actually does it correctly.