Technical Skills to Becoming a Software Tester

Close-up of a hand typing on a laptop that shows green code text on a black screen, while the other hand holds a phone also displaying code.
In this post
  1. Validation vs. Verification
  2. Correctness versus Testing
  3. Static vs. Dynamic Testing
  4. Faults vs. Failures
  5. Testing vs. Debugging
  6. Test Case
  7. Test Harness and Scaffolding
  8. Seven Principles of Software Testing

Do you find software testing interesting? Read this article to understand key concepts and software tester skills you will have to develop to determine if it is a good career choice for you.

Software testers play an important role in quality assurance for various companies. If you are pursuing a software testing career, it may be beneficial to acquire excellent computer system analysis skills and knowledge in order for your software under test to run efficiently, effectively, and meet customer expectations. This article provides an introduction to the key concepts a software tester needs to know.

Before we begin, test your knowledge of some key concepts we will introduce in this article. Can you correctly match the following 17 items...

  1. Answers the question “Are we building the right software?”
  2. Answers the question “Are we building the software right?”
  3. Testing based on the requirements.
  4. Testing based on the program structure.
  5. Test with a scope that is a complete application and hardware units, with the goal of testing within the environment.
  6. Test with the goal of testing the interactions between modules/units of the system.
  7. Test of a particular unit with the goal of early detection of problems as the code is developed.
  8. Testing strategy that divides the input space into groups, which are hoped to have the property that any member of the group will cause a failure, if a bug exists in the code related to that group.
  9. Missing or incorrect code.
  10. Probability that failure will occur after a certain period of usage.
  11. Testing approach that tries to discover software faults by providing randomized inputs to find test cases that cause a crash.
  12. A specification of what the implementation under test should produce from the test inputs.
  13. A specification of the pretest state of the implementation under test and its environment, the test inputs or conditions, and the expected result.
  14. A set of values that input or state variables of the implementation under test can take.
  15. A testing approach that changes a program to check if a test set is able to find the changed code.
  16. The ability of the code to hide faults from a test suite.
  17. The percentage of branches executed at least once by a test suite.

...using the following words and phrases?

Verification, Validation, Fault sensitivity, Branch coverage, System testing, Integration testing, Partition testing, Unit testing, Failure, Functional testing, Fault, Structural testing, Failure rate, Test case, Domain, Mutation testing, Expected results, Fuzzing

Validation vs. Verification

Verification: The process of determining whether the products of a given phase of the software development process fulfill the requirements established during the previous phase. This answers the question "Are we building the product right”. The software should conform to its requirements and specifications.

Validation: The process of evaluating software at the end of software development to ensure compliance with intended usage. This answers the question "Are we building the right product”? The software should do what the user wants.

Verification and Validation

Validation & Verification should be applied at each phase in the software engineering development lifecycle. There are two goals for any verification or validation activity:discover defects and assess whether or not the system is useful and useable in its intended environment.

Correctness versus Testing

Correctness attempts to establish that software is error/vulnerability free while testing attempts to find if there are any errors in it. However thorough testing does not demonstrate that a program is error free. Testing, debugging, and the fault removal contribute to increasing confidence in the correct functioning of the software.

Validation testing aims to ensure that the software meets the user needs and requirements. A successful test shows that a use case has been met. Defect testing aims to discover defects/vulnerabilities in the software where its expected behavior differs from the observed (actual) behavior. A successful test exposes a defect/vulnerability in the system.

Static vs. Dynamic Testing

Static testing is testing without running the program, including software inspections/ audits and some types of static analyses. Static tests are good at finding possible defects / vulnerabilities (i.e., problems that could lead to defects/vulnerabilities). Examples are SPLINT, SSVChecker, and many others.

Dynamic testing is testing by executing the program with real inputs, including traditional software testing approaches and tools. Examples are manual test execution, PITSS, Junit, and many others.

Faults vs. Failures

Software Fault is a static defect in the software; a “bug”. The software Failure is the external, observable, behavior that differs from the expected behavior with respect to the requirements or user expectations. If a defect in the code is executed, this may cause a failure, but not necessarily in all circumstances. For example, some defects require very specific inputs or preconditions to trigger a failure, which may occur rarely or never. In addition to failures caused due to defects in the code, failures can also be caused by environmental conditions. For example, radiation, electromagnetic fields, and pollution can cause defects in firmware or influence the execution of software by changing hardware conditions.

Testing vs. Debugging

Testing is the process of determining if a program has any errors. While debugging is the process to determine the cause, and removal, of an error discovered via testing. Subsequent confirmation testing checks whether the fixes resolved the defects.

Test Case

A test case is defined as a set of input values, execution preconditions, expected results and execution postconditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement.

A test case is therefore made up of a combination of:

  1. Test values - The input values that directly satisfy a test condition / requirement
  2. Expected results: The behavior results expected from the test after executing the test

Test Harness and Scaffolding

Scaffolding is additional program code that is necessary to support software testing (e.g., Junit). A stub simulates the results of a call to a method, database, library, etc. that has yet to be implemented. A test driver simulates a method that calls to a method that is being tested.

Test Harness and Scaffolding

Seven Principles of Software Testing

  1. Testing shows the presence of defects, not their absence. Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, testing is not a proof of correctness.
  2. Exhaustive testing is impossible. Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Rather than attempting to test exhaustively, risk analysis, test techniques, and priorities should be used to focus test efforts.
  3. Early testing saves time and money. To find defects early, both static and dynamic test activities should be started as early as possible in the software development lifecycle. Early testing is sometimes referred to as shift left. Testing early in the software development lifecycle helps reduce or eliminate costly changes.
  4. Defects cluster together. A small number of modules usually contains most of the defects discovered during pre-release testing, or is responsible for most of the operational failures. Predicted defect clusters, and the actual observed defect clusters in test or operation, are an important input into a risk analysis used to focus the test effort.
  5. Beware of the pesticide paradox. If the same tests are repeated over and over again, eventually these tests no longer find any new defects. To detect new defects, existing tests and test data may need changing, and new tests may need to be written. (Tests are no longer effective at finding defects, just as pesticides are no longer effective at killing insects after a while.) In some cases, such as automated regression testing, the pesticide paradox has a beneficial outcome, which is the relatively low number of regression defects.6. Testing is context dependent
  6. Testing is done differently in different contexts. For example, safety-critical industrial control software is tested differently from an e-commerce mobile app. As another example, testing in an Agile project is done differently than testing in a sequential lifecycle project.
  7. Absence-of-errors is a fallacy. Some organizations expect that testers can run all possible tests and find all possible defects, but principles 2 and 1, respectively, tell us that this is impossible. Further, it is a fallacy (i.e., a mistaken belief) to expect that just finding and fixing a large number of defects will ensure the success of a system. For example, thoroughly testing all specified requirements and fixing all defects found could still produce a system that is difficult to use, that does not fulfill the users’ needs and expectations, or that is inferior compared to other competing systems.
ISTQB Foundation Level certification is the most important software certification you can attain to launch your career. Learning new things and improving your software tester skills is inevitable in order to keep up with trends and stay up to date. Each tester needs to be able to do a lot of things: Investigate requirements, ask insightful questions, understand product information, communicate with stakeholders, and interrogate and critically evaluate the software to stay competitive and relevant.

"Software Testing Techniques and Tools" by Josh Dehlinger, NSA Funded Curriculum is licensed under CC BY-NC-SA 4.0.

Keep reading