Test Strategy and Execution Guide

Overview

gist-cache-rs’s test strategy is structured in a three-layer architecture: unit tests, integration tests, and E2E tests.

Current Coverage: 68.95% (533/773 lines) — not re-measured in this update, see note below Number of Automated Tests: 251 (Unit 190 + Integration 61) Manual E2E Tests: 26 cases

Test Pyramid Structure

Principles of the Test Pyramid:

• Most unit tests (78%) - Fast, no external dependencies
• Integration tests in the middle (22%) - Verifies actual process execution
• Minimal E2E tests (manual) - Comprehensive user-centric verification

Test Execution

Basic Test Execution

# Run all tests
cargo test

# With verbose output
cargo test -- --nocapture

# Run specific tests only
cargo test test_cache_content

Running Tests with ignore Attribute

# Run tests including those with the ignore attribute
cargo test -- --include-ignored

# Run only tests with the ignore attribute
cargo test -- --ignored

Coverage Measurement

# Display coverage to standard output
cargo tarpaulin --out Stdout

# Generate HTML report
cargo tarpaulin --out Html --output-dir coverage

# See docs/testing/COVERAGE.md for details

Test Configuration

1. Unit Tests (190)

Location: src/ within #[cfg(test)] module

Coverage Target:

• Data structures and serialization (cache/types.rs)
• Cache management logic (cache/content.rs, cache/update.rs)
• Search logic (search/query.rs) and interactive picker logic (search/interactive.rs)
• CLI argument processing and interpreter detection (cli.rs)
• Configuration management, incl. extension-based interpreter mapping (config.rs)
• Error handling (error.rs)
• Basic functionality of the execution runner (execution/runner.rs) and syntax highlighting (execution/highlight.rs)
• GitHub API mock (github/client.rs)

Features:

• Fast execution (no external dependencies)
• Excludes GitHub API dependencies with MockGitHubClient
• Automatable in CI/CD
• 5 tests in github/api.rs are #[ignore] (require a real, authenticated gh CLI)

2. Integration Tests (61)

Location: tests/ directory

2.1 CLI Tests (tests/cli_tests.rs) - 33 (32 on Windows)

• Verification of command-line argument processing
• Subcommand operation verification (update, run, cache, config, completions)
• Error case verification (authentication errors, no cache, invalid input, etc.)
• Flag combination verification (--preview, --force, --filename, --description, --id, etc.)
• Shell completion generation for all 4 shells, incl. a bash subcommand-completion regression test (#[cfg(unix)] only, so it does not compile on Windows)

2.2 Interpreter Integration Tests (tests/integration_test.rs) - 16

• Bash, Python, Node.js execution tests
• TypeScript (ts-node, deno, bun) execution tests
• Ruby, Perl, PHP execution tests
• PowerShell execution, argument passing, preview mode, and failure tests
• Argument passing, error handling
• Preview mode operation verification

2.3 Runner Tests (tests/runner_test.rs) - 12

• Detailed verification of script execution logic
• Cache creation operation verification
• Download mode operation verification
• Force file-based execution verification
• Multi-file Gist selection logic
• PowerShell-specific variants of the above (cache creation, download mode, force execution, multi-file selection, preview+download, empty arguments)

Features:

• Verifies actual process execution
• Each test is either Unix-only or Windows-only via #[cfg_attr(not(all(unix, not(target_os = "windows"))), ignore)] (bash-based tests) or the PowerShell equivalent — the non-matching half is #[ignore]d at runtime on a given OS, not removed from the count
• Automatically skipped if interpreter is not installed

3. E2E Tests (26 cases, manual)

Location: docs/tests/

Test Sets:

1. Caching functionality (test_set_01_caching.md) - 8 cases
2. Search functionality (test_set_02_search.md) - 6 cases
3. Interpreter (test_set_03_interpreter.md) - 7 cases
4. Preview functionality (test_set_04_preview.md) - 5 cases

Features:

• Comprehensive verification using actual Gists
• User-centric workflow verification
• Detailed reproducible steps included

Testing Policy

What to Cover with Unit Tests

✅ Targets:

• Business logic
• Data transformation/serialization
• Error handling
• Mockable external dependencies

❌ Not Targets:

• External process execution (bash, python, etc.) → Verified by integration tests
• GitHub CLI (gh command) → Replaced by MockGitHubClient, or #[ignore] tests
• User input processing → Verified by E2E tests

Test Quality Metrics

Target Coverage: 60-70% (Standard for CLI tools) Current Coverage: 68.95% ✅ Target achieved

Reasons for Achievement:

• Core logic has high coverage (types 100%, config 96%, content 83%, cli 78%)
• External process dependent code (runner.rs 20%, api.rs 8%) verified by integration tests
• Low coverage for thin wrappers is acceptable

Troubleshooting

Tests are filtered out

Cause: Execution in non-Unix environment, or interpreter not installed

Solution:

• Unix environment recommended for integration tests
• Install interpreters (bash, python, node, etc.)
• Or, automatic skipping is normal behavior

Coverage cannot be measured

Cause: tarpaulin is not installed

Solution:

cargo install cargo-tarpaulin

Integration tests fail

Cause: Interpreter (bash, python, node, etc.) not installed

Solution:

• Install necessary interpreters
• Or, skipping is normal behavior

Detailed Documentation

• Coverage Measurement: COVERAGE.md - Measurement methods and module-specific details
• Test Inventory: TEST_INVENTORY.md - Classification and overview of all tests
• GitHub CLI Related Test Evaluation: GH_TESTING_EVALUATION.md - Evaluation of gh command tests

References

• Rust Testing Guide
• cargo-tarpaulin
• mockall

Last Updated: 2026-08-20 (test counts only; coverage % below is carried over from 2025-11-06 and has not been re-measured — see Test Inventory for the same caveat) Current Coverage: 68.95% Number of Automated Tests: 251 Covered Lines: 533/773 lines