Debug a Defect with Comparable Before and After Evidence
Reproduce a local defect, preserve a failing baseline and compare the same check after a minimal fix. Use command results and watched hashes to explain what was actually verified.
Fastest way
Ask your AI to do it
Paste this into the AI agent you already use:
Use Debug Evidence Kit for my explicitly scoped local defect. Ask for expected behavior, actual behavior, a finite reproduction command and the relevant source and tests. Capture the failing baseline, make a minimal fix and rerun the same command with unchanged watched tests. Inspect the captured exit code, complete logs and hashes before building a case in a fresh folder. Mark missing evidence unknown and review private logs before sharing.
What makes a fix report credible?
Debug Evidence Kit captures a finite local command before and after a change, then builds an evidence report. Comparable commands, complete logs, stable inputs and preserved tests are essential. A passing check supports that reproduction; it does not prove every feature or the production service is healthy.
Define the reproduction before changing code
- State expected behavior, actual behavior, the smallest input and the relevant source and tests. Read references/diagnosis.md; use the target project's real runtime.
- Use an explicit finite command, a consistent working directory and watched source/test paths. The capture tool runs argv without a shell and bounds time and output.
- Save the failing baseline before fixing. Keep watched test files unchanged and inspect captured exitCode rather than relying on the capture wrapper's exit status.
Distinguish saved evidence from a passing check
| Observation | Supported conclusion | Next check |
|---|---|---|
| Capture wrapper exits zero | The evidence file was written | Inspect captured status, exitCode and truncation |
| Same failing check now exits zero | The scoped reproduction now passes | Compare source and unchanged test hashes; run adjacent checks |
| Commands differ or logs are incomplete | Comparison needs more evidence | Repeat a comparable capture; do not report a verified fix |
Run an isolated before/fix/after exercise
From the package folder, Python 3.10+ runs this standard-library-only exercise described in references/first-run.md. It copies a fictional numeric-sort helper, runs the genuinely failing tests, fixes only the copied source and reruns unchanged tests. It leaves the installed package intact. Open practice-debug/report/index.html and its private evidence files.
Read this release’s setup before running the example
Confirm installation and invocation in your host
python3 scripts/run-example.py --output ./practice-debugPractice the investigation without an automatic fix
The one-command example is a guided demonstration. To practice the investigation yourself, work on a new copy from the installed package directory. These commands capture the failing test without changing the installed fixture. The capture command can exit zero because it saved evidence even though the captured test failed. Inspect practice-manual-evidence/before.json: exitCode must be nonzero.
cp -R examples/fixture ./practice-manual-project
python3 -B scripts/capture-check.py --cwd ./practice-manual-project --watch sort_prices.py --watch test_sort_prices.py --label baseline --output ./practice-manual-evidence/before.json -- python3 -B -m unittest test_sort_pricesChange one expression, then rerun the unchanged test
Read the assertions and explain the failure first: sorting the numbers as strings puts 10 before 2. In practice-manual-project/sort_prices.py only, replace sorted(values, key=str) with sorted(values). Keep practice-manual-project/test_sort_prices.py unchanged, then run the commands below. Open practice-manual-report/index.html. A credible result has a failing before exitCode, a zero after exitCode, changed source hashes, unchanged test hashes and eight passing comparisons.
As a negative control, copy after.json to a new evidence file, alter its argv and build a separate report with that file. It should say needs-evidence. This tests comparison integrity, not the correctness of an arbitrary real-world fix. Do not alter the original captures.
python3 -B scripts/capture-check.py --cwd ./practice-manual-project --watch sort_prices.py --watch test_sort_prices.py --label post-fix --output ./practice-manual-evidence/after.json -- python3 -B -m unittest test_sort_prices
python3 -B scripts/build-case.py --input examples/case.json --before ./practice-manual-evidence/before.json --after ./practice-manual-evidence/after.json --output ./practice-manual-reportCheck the comparison, not just the green label
- Inspect evidence/before.json and evidence/after.json: the same command should fail before and pass after, with complete logs and stable watched files during each run.
- The sample report should show reproduction-now-passing and eight passed comparisons. The source hash should change while the watched test hash stays the same. That is real execution on fictional sample code.
- For your project, add a meaningful regression test, then capture the agreed baseline and after-check with the same watched test. Run relevant adjacent tests and record their separate scope.
Inspect the current example output → — Compare the visible result with the exported files and checks described in this section.
Share a scoped explanation with reviewed logs
Explain the defect, change and observations without promoting a passing check to root-cause proof. Missing or incomparable evidence stays unknown. Private captures may contain paths, logs and sensitive values; redaction is best effort, so review before sharing. Production deployment and live verification remain separate work.
Inspect page evidence after an SEO-related code fix
Prepare an evidence-based support handoff
Check whether the failure belongs to installation or execution — A reproducible application defect and a missing skill need different fixes.