Home/Testing Strategies/Property-Based Testing

Property-Based Testing

Generative testing with proptest

advanced
proptestquickcheckgenerative
🎮 Interactive Playground

What is Property-Based Testing?

Property-based testing generates random inputs and verifies that properties (invariants) hold for all of them. Instead of testing specific examples, you define what should always be true about your code.

The Problem

Traditional example-based testing has limitations:

  • Missed edge cases: You can't think of everything
  • Bias: You test what you expect, not what breaks
  • Maintenance: Many example tests to maintain
  • Coverage: Hard to achieve comprehensive input coverage

Example Code

// Add to Cargo.toml:
// [dev-dependencies]
// proptest = "1.0"
// quickcheck = "1.0"
// quickcheck_macros = "1.0"

use std::collections::HashMap;

/// A simple stack implementation to test
#[derive(Debug, Default, Clone)]
pub struct Stack<T> {
    items: Vec<T>,
}

impl<T: Clone> Stack<T> {
    pub fn new() -> Self {
        Stack { items: Vec::new() }
    }

    pub fn push(&mut self, item: T) {
        self.items.push(item);
    }

    pub fn pop(&mut self) -> Option<T> {
        self.items.pop()
    }

    pub fn peek(&self) -> Option<&T> {
        self.items.last()
    }

    pub fn len(&self) -> usize {
        self.items.len()
    }

    pub fn is_empty(&self) -> bool {
        self.items.is_empty()
    }
}

/// String utilities to test
pub fn reverse_string(s: &str) -> String {
    s.chars().rev().collect()
}

pub fn sort_string(s: &str) -> String {
    let mut chars: Vec<char> = s.chars().collect();
    chars.sort();
    chars.into_iter().collect()
}

/// Numeric functions
pub fn abs_diff(a: i32, b: i32) -> u32 {
    (a as i64 - b as i64).unsigned_abs() as u32
}

pub fn clamp(value: i32, min: i32, max: i32) -> i32 {
    if value < min {
        min
    } else if value > max {
        max
    } else {
        value
    }
}

/// Serialization round-trip
#[derive(Debug, Clone, PartialEq)]
pub struct Person {
    pub name: String,
    pub age: u32,
}

impl Person {
    pub fn serialize(&self) -> String {
        format!("{}:{}", self.name.replace(':', "\\:"), self.age)
    }

    pub fn deserialize(s: &str) -> Option<Person> {
        let parts: Vec<&str> = s.rsplitn(2, ':').collect();
        if parts.len() != 2 {
            return None;
        }
        let age = parts[0].parse().ok()?;
        let name = parts[1].replace("\\:", ":");
        Some(Person { name, age })
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    // Manual property test (without external crate)
    fn test_property<T, F>(generator: impl Fn(u64) -> T, property: F, iterations: u64)
    where
        F: Fn(T) -> bool,
    {
        use std::collections::hash_map::DefaultHasher;
        use std::hash::{Hash, Hasher};

        for seed in 0..iterations {
            let input = generator(seed);
            assert!(property(input), "Property failed for seed {}", seed);
        }
    }

    // Property: Reversing twice gives original
    #[test]
    fn prop_reverse_twice_is_identity() {
        let strings = ["", "a", "ab", "hello", "rust is great", "🦀🔥"];

        for s in strings {
            let reversed_twice = reverse_string(&reverse_string(s));
            assert_eq!(s, reversed_twice, "Double reverse should be identity");
        }
    }

    // Property: Reversed string has same length
    #[test]
    fn prop_reverse_preserves_length() {
        let strings = ["", "test", "longer string", "emoji 🎉"];

        for s in strings {
            assert_eq!(
                s.len(),
                reverse_string(s).len(),
                "Reverse should preserve byte length"
            );
        }
    }

    // Property: Sorted string has same characters
    #[test]
    fn prop_sort_preserves_characters() {
        let strings = ["dcba", "hello", "aabbcc"];

        for s in strings {
            let sorted = sort_string(s);
            let mut original_chars: Vec<char> = s.chars().collect();
            let mut sorted_chars: Vec<char> = sorted.chars().collect();
            original_chars.sort();
            sorted_chars.sort();
            assert_eq!(original_chars, sorted_chars);
        }
    }

    // Property: abs_diff is symmetric
    #[test]
    fn prop_abs_diff_symmetric() {
        let pairs = [(0, 0), (1, 2), (-5, 5), (i32::MIN, i32::MAX), (100, -100)];

        for (a, b) in pairs {
            assert_eq!(
                abs_diff(a, b),
                abs_diff(b, a),
                "abs_diff should be symmetric for ({}, {})",
                a, b
            );
        }
    }

    // Property: abs_diff is always non-negative (guaranteed by return type)
    #[test]
    fn prop_abs_diff_non_negative() {
        let pairs = [(0, 0), (-1, 1), (i32::MIN, 0)];

        for (a, b) in pairs {
            let diff = abs_diff(a, b);
            // u32 is always >= 0, but we verify the computation doesn't panic
            assert!(diff >= 0);
        }
    }

    // Property: clamp returns value in range
    #[test]
    fn prop_clamp_in_range() {
        let test_cases = [
            (5, 0, 10),   // Value in range
            (-5, 0, 10),  // Below min
            (15, 0, 10),  // Above max
            (0, 0, 0),    // All same
        ];

        for (value, min, max) in test_cases {
            let result = clamp(value, min, max);
            assert!(
                result >= min && result <= max,
                "clamp({}, {}, {}) = {} should be in [{}, {}]",
                value, min, max, result, min, max
            );
        }
    }

    // Property: clamp is idempotent
    #[test]
    fn prop_clamp_idempotent() {
        let test_cases = [(5, 0, 10), (-5, 0, 10), (15, 0, 10)];

        for (value, min, max) in test_cases {
            let once = clamp(value, min, max);
            let twice = clamp(once, min, max);
            assert_eq!(once, twice, "Clamping twice should give same result");
        }
    }

    // Property: Stack push then pop returns the item
    #[test]
    fn prop_stack_push_pop() {
        let items = [1, 2, 3, 100, -50];

        for item in items {
            let mut stack = Stack::new();
            stack.push(item);
            assert_eq!(stack.pop(), Some(item));
        }
    }

    // Property: Stack length increases by 1 after push
    #[test]
    fn prop_stack_push_increases_length() {
        let mut stack = Stack::new();

        for i in 0..10 {
            let before = stack.len();
            stack.push(i);
            assert_eq!(stack.len(), before + 1);
        }
    }

    // Property: Serialization round-trip preserves data
    #[test]
    fn prop_person_roundtrip() {
        let people = [
            Person { name: "Alice".to_string(), age: 30 },
            Person { name: "Bob:Smith".to_string(), age: 25 }, // Name with colon
            Person { name: "".to_string(), age: 0 },
        ];

        for person in people {
            let serialized = person.serialize();
            let deserialized = Person::deserialize(&serialized)
                .expect("Deserialization should succeed");
            assert_eq!(person, deserialized);
        }
    }
}

// Using proptest crate (when available)
#[cfg(test)]
#[cfg(feature = "proptest")]
mod proptest_tests {
    use super::*;
    use proptest::prelude::*;

    proptest! {
        // Property: reverse(reverse(s)) == s
        #[test]
        fn prop_reverse_involution(s in ".*") {
            let result = reverse_string(&reverse_string(&s));
            prop_assert_eq!(s, result);
        }

        // Property: abs_diff is symmetric
        #[test]
        fn prop_abs_diff_symmetric(a in any::<i32>(), b in any::<i32>()) {
            prop_assert_eq!(abs_diff(a, b), abs_diff(b, a));
        }

        // Property: clamp result is in bounds
        #[test]
        fn prop_clamp_bounds(
            value in any::<i32>(),
            min in any::<i32>(),
            max in any::<i32>(),
        ) {
            // Ensure min <= max
            let (min, max) = if min <= max { (min, max) } else { (max, min) };

            let result = clamp(value, min, max);
            prop_assert!(result >= min);
            prop_assert!(result <= max);
        }

        // Property: Stack operations
        #[test]
        fn prop_stack_push_pop(items in prop::collection::vec(any::<i32>(), 0..100)) {
            let mut stack = Stack::new();

            // Push all items
            for item in &items {
                stack.push(*item);
            }

            // Pop all items (should be in reverse order)
            for item in items.iter().rev() {
                prop_assert_eq!(stack.pop(), Some(*item));
            }

            prop_assert!(stack.is_empty());
        }

        // Property: Person roundtrip
        #[test]
        fn prop_person_serialize_roundtrip(
            name in "[a-zA-Z0-9 ]*",
            age in 0u32..150,
        ) {
            let person = Person { name, age };
            let serialized = person.serialize();
            let deserialized = Person::deserialize(&serialized);

            prop_assert_eq!(Some(person), deserialized);
        }
    }
}

// Using quickcheck crate (when available)
#[cfg(test)]
#[cfg(feature = "quickcheck")]
mod quickcheck_tests {
    use super::*;
    use quickcheck::{quickcheck, TestResult};

    #[test]
    fn qc_reverse_involution() {
        fn prop(s: String) -> bool {
            reverse_string(&reverse_string(&s)) == s
        }
        quickcheck(prop as fn(String) -> bool);
    }

    #[test]
    fn qc_abs_diff_symmetric() {
        fn prop(a: i32, b: i32) -> bool {
            abs_diff(a, b) == abs_diff(b, a)
        }
        quickcheck(prop as fn(i32, i32) -> bool);
    }

    #[test]
    fn qc_clamp_bounds() {
        fn prop(value: i32, min: i32, max: i32) -> TestResult {
            if min > max {
                return TestResult::discard();
            }
            let result = clamp(value, min, max);
            TestResult::from_bool(result >= min && result <= max)
        }
        quickcheck(prop as fn(i32, i32, i32) -> TestResult);
    }
}

fn main() {
    println!("Run property tests with: cargo test");
    println!("With proptest: cargo test --features proptest");
    println!("With quickcheck: cargo test --features quickcheck");
}

Why This Works

  1. Random generation: Finds edge cases you wouldn't think of
  2. Shrinking: When a failure is found, reduces to minimal failing case
  3. Property focus: Tests invariants rather than specific values
  4. High coverage: Thousands of test cases in seconds

⚠️ Common Properties

| Property | Description | Example |

|----------|-------------|---------|

| Inverse | f(f_inv(x)) == x | encode/decode |

| Idempotent | f(f(x)) == f(x) | sort, clamp |

| Commutative | f(a, b) == f(b, a) | add, max |

| Associative | f(a, f(b, c)) == f(f(a, b), c) | concat |

| Identity | f(x, id) == x | add 0, multiply 1 |

proptest vs quickcheck

| Feature | proptest | quickcheck |

|---------|----------|------------|

| Syntax | Macro-based | Function-based |

| Shrinking | Integrated | Requires Arbitrary impl |

| Strategies | Powerful combinators | Simpler |

| Use case | Complex input generation | Simple properties |

⚠️ Anti-patterns

// DON'T: Property that's always true
fn prop_always_true(_x: i32) -> bool {
    true // This tests nothing!
}

// DON'T: Property that tests implementation
fn prop_sort_uses_quicksort(items: Vec<i32>) -> bool {
    // Testing algorithm choice, not behavior
}

// DO: Property that tests observable behavior
fn prop_sort_produces_ordered(items: Vec<i32>) -> bool {
    let sorted = sort(items);
    sorted.windows(2).all(|w| w[0] <= w[1])
}

Exercises

  1. Write properties for a HashMap wrapper (insert/get roundtrip)
  2. Test a JSON serializer with roundtrip properties
  3. Implement properties for a custom number type
  4. Create custom generators for domain-specific types

🎮 Try it Yourself

🎮

Property-Based Testing - Playground

Run this code in the official Rust Playground