"The Ongoing Revolution in Software Testing" by Cem Kaner (2004) - разбор общепринятых утверждений о тестировании/тестировщиках

Благодаря давнишнему твиту Alan Page узнал про чудесную статью Cem Kaner, который еще в 2004 году разобрал популярные утверждения (мифы?) про тестирование и роль тестировщиков.

Что именно так разбирается:

The Role of Testers

• The primary reason to test is to find bugs?

• The primary reason to test is to prove the program works correctly?

• Testers are THE advocates of quality on a project.

• Test groups should evolve into quality assurance groups.

• The test group should have the power to block release if product quality is too low.

• Testers and programmers have conflicting interests.

• The test group should work independently of the programmers.

• Testers should push their project teams to follow "appropriately professional

development models," like the Waterfall, that require people to THINK before they act.

• Testers should base test cases on documented characteristics of the program. If the software documentation is inadequate for this, testers should assert a quality control function and demand better specification and requirements documentation. 

• The primary reason to test is to find bugs. 

• The primary reason to test is to prove the program works correctly.

Test Planning and Documentation

• Testers should specify the expected result of every test, in advance.

• Exploratory testing, if done at all, should be done only by experts.

• There should be at least one thoroughly documented test for every requirement item or

specification item.

• Testers should design most or all tests early in development.

• Testers should design all tests for reuse as regression tests.

• Testers should document manual tests in great procedural detail so that they can be

handed down to less experienced or less skilled testers.

• Testers should document each test case individually, ideally storing them in a test case

management system that describes the preconditions, procedural details, postconditions,

and basis (such as trace to requirements) of each individual test.

The Practice of Testing

• Testers should assume that the programmers did a light job of testing and so should

extensively cover the basics (such as boundary cases for every field).

• Testers should develop automated tests at the user interface level.

• Testers should design tests without knowledge of the underlying code.

• Testers should design the build verification tests, even the ones to be run by

programmers.

• Most testers don't need to be programmers, because they only have to specify how the

external program will behave in their tests. Testing tools will allow business experts to

develop automated tests without having to program.

• The pool of tests should cover every line and branch in the program, or perhaps every

basis path.

• All failures must be reported into a bug tracking system and carefully counted.

• We can measure the individual effectiveness of software testers by counting how many

bugs they find, perhaps with an adjustment for bug severity.

• We can tell how close we are to release of the product by examining the bug curve that

shows bug finds per week.

Все тут обсуждать/переводить не буду, оставлю только то, что очень зацепило. Но вы обязательно почитайте все остальное тоже, если интересуетесь организацией процесса тестирования или вашем местом в этом процессе.

QA - точно про assurance?

Попробуйте ответить на эти вопросы и еще раз подумать действительно ли вы сейчас QA-инженер?

• Do testers have the authority and cash to provide training for programmers who need it?

• Do testers have the authority to settle customer complaints? Or to drive the handling of

customer complaints?

• Do testers have the ability and authority to fix bugs?

• Do testers have the ability and authority to either write or rewrite the user manuals?

• Do testers have the ability to study customer needs and design the product accordingly?

"Каждый тест должен иметь ожидаемый результат"

One fundamental problem with this idea is that it is misguided. Every test has many results. No one specifies them all. An "expected result" points the tester at one or a few of these results, but away from the others.