Specialty · TestingLesson 2 of 5Free to read

Edge cases · Software Testing

Which inputs should you test?

Why you need it

You can't test every possible input, and most bugs don't live in the middle of the range. They hide at the edges: the empty list, exactly the limit, one past it, zero. Choosing a handful of inputs well catches more bugs than a hundred random ones.

The intuition

Split inputs into groups that should behave the same (valid / too short / too long; positive / zero / negative). One test per group is usually enough.
Test the boundaries between groups: if the rule is "8 to 64 characters", test 7, 8, 64 and 65. Off-by-one mistakes (< instead of <=) only show up there.
Test the special values: empty string or list, one item, zero, negative, very large, None.
Test invalid input too. If the function should raise an error, check that it does:

def test_rejects_negative():
    try:
        sqrt(-1)
    except ValueError:
        return                          # the error we wanted
    assert False, "expected ValueError"

(pytest writes this as with pytest.raises(ValueError): sqrt(-1).)

To check that your tests are strong, imagine small mistakes in the code (< for <=, a missing rule). Each one should make at least one test fail. Tools that do this automatically are called mutation testing.

Definition

Boundary value

An input exactly at, just below or just above a limit, where off-by-one bugs appear.

Equivalence class

A group of inputs the code should treat the same way; testing one of them stands for the group.

Mutation testing

Checking tests by making small deliberate bugs (mutants) and seeing whether some test fails.

Examples

Find the boundary bug

Try it yourself

The middle of each range works; exactly 90 doesn't.

def grade(score):
    if score > 90:
        return "A"
    if score >= 75:
        return "B"
    return "C"

for score in [95, 80, 50, 90, 75, 74]:
    print(score, grade(score))

Output

95 A
80 B
50 C
90 B
75 B
74 C

Step by step

L2The rule says 90 and above is an A, but this says above 90.
L895, 80 and 50 sit in the middle of their groups and look fine. 90 is the boundary: B is wrong.

Testing that an error is raised

Try it yourself

A test passes when the right error appears, and fails when none does.

def withdraw(balance, amount):
    if amount <= 0:
        raise ValueError("amount must be positive")
    return balance - amount

def raises(fn, error):
    try:
        fn()
    except error:
        return True
    return False

print(raises(lambda: withdraw(100, -5), ValueError))
print(raises(lambda: withdraw(100, 0), ValueError))
print(raises(lambda: withdraw(100, 5), ValueError))

Output

True
True
False

Step by step

L6A helper: run fn and say whether it raised that kind of error.
L13lambda delays the call, so raises() can run it inside try.
L140 is the boundary: it should be rejected too.
L15A valid amount: no error.False

Common mistakes

Testing only typical values

50 and 95 pass even when 90 is broken. Put a test at each limit and just past it.

Forgetting the empty case

Empty strings and lists cause crashes (index 0 of nothing) and division by zero. Test them on purpose.

A test for an error that never checks

try: f() except ValueError: pass passes even if f() raises nothing. End with assert False (or use pytest.raises).

Quick quiz

1. A form accepts ages 13 to 120. Which set of test values is best?

A12, 13, 120, 121
B20, 40, 60, 80
C13, 50, 100

Show the answer

Answer: A

Each limit, and one step past each limit.

2. What does this print?

def is_teen(age):
    return 13 < age < 20

print([age for age in [12, 13, 19, 20] if is_teen(age)])

A
```
[19]
```
B
```
[13, 19]
```
C
```
[13, 19, 20]
```

Show the answer

Answer: A

13 < 13 is False, so 13 is wrongly excluded: exactly the kind of bug a boundary test finds.

Practice

3 exercises in three steps: predict what a program prints, change a program to meet a new requirement, then write one from scratch. Your code runs in the browser and is checked the moment you submit.

PredictWhich test catches it?
This function should say anyone 18 or older is an adult. Which single test would catch the bug?
ModifyThe last chunk
chunk(items, size) splits a list into pieces of size. It loses the last, shorter piece: chunk([1, 2, 3, 4, 5], 2) should be [[1, 2], [3, 4], [5]]. Fix it, and make sure the edge cases in the tests work too.
WriteTest the password rules
password_ok(pw) accepts passwords of 8 to 64 characters with at least one digit and one letter. Write test_ functions that pass on it and catch all four broken versions (plus a hidden one). Think boundaries: which lengths matter?

Practise this lesson

Open the interactive lesson to edit and run these examples and solve the exercises with instant feedback. Free to start, no sign-up.

Open the interactive lesson Course map

Interactive lessons open in order. If you have not done the earlier ones yet, you start at the course map.

Terms in this lesson

Term	Chinese	Definition
Boundary value	边界值	An input at or just past a limit.
Edge case	边界情况	An unusual input: empty, zero, the largest value…
Mutation testing	变异测试	Testing your tests by planting small bugs.