Test order dependency: why a passing test fails when shuffled
When a test that always passed turns red for no visible reason, the team often reaches for one word: flaky. That word usually means the test passes and fails for…
You have read this
Worked testing lessons built on real cases. Page 2 of 6.
When a test that always passed turns red for no visible reason, the team often reaches for one word: flaky. That word usually means the test passes and fails for…
You have read this
Take a tester with 12 years on a payroll and HR product. Her team adds an assistant to the employee portal. It is built on a large language model (LLM)…
You have read this
Take a tester on a checkout team who finds a broken discount code late on a Thursday. They log a defect report titled "Checkout totally broken again!!" in the bug…
You have read this
Take a surgical robot whose console software gets an update. The update fixes a stutter in the camera display. It does this by changing how video frames are buffered. The…
You have read this
A visual test compares a screen with an image the team has already accepted, called the visual baseline. The test can show a change, but a reviewer still needs to…
You have read this
A search test can return the right products while taking too long to serve the customer. Checking the items on screen does not measure that delay.
You have read this
What does a passing test tell you about an AI fix? In SWE-Gate, 644 fixes passed functional tests, but 221 failed separate checks for review rules. The agents had received…
You have read this
Monday's AI agent passes its test.
You have read this
A test that went red overnight passes this morning, and nothing in the build changed. How do you tell your team whether that run found a defect? Non-deterministic testing starts…
You have read this