Hedge Fund Software

Test managers often hold brief phone interviews to screen out candidates for QA/testing positions. With resume in hand, which the test manager has probably not even reviewed, before the interview commences the test manager is expected to determine within an hour or less whether the interviewed candidate would be a good fit for the project. Trying to evaluate in such a short period time a candidate’s ability to perform the project’s testing and QA tasks can be an inexact science. It can also be a subjective undertaking if the test manager fails to ask the appropriate questions from the interviewing candidate. Even relying on a resume to determine the candidate’s aptitude for a position is unreliable since many candidates embellish their resumes.

A recommended approach for interviewing candidates is to draft a specific list of questions before the interview begins that are pertinent to the position that needs to be filled. These questions should be based on the candidate’s QA accomplishments and skill sets as documented in the resume. Below are some sample questions that can help demonstrate a candidate’s experience in the areas of quality assurance, and also their creativity and ability to comprehend basic testing concepts. The questions below can serve as criteria to screen out inexperienced candidates during the interviewing process.

Describing Testing/QA Terms

An experienced testing candidate with several years of experience will exhibit knowledge and understanding of well-established testing principles and testing terminologies. It behooves the test manager to ask the testing candidate to describe these concepts. Some suitable examples would be:

What is the objective of a peer review?
What is the Unified Modeling Language?
What are the components of a test plan?
What are the benefits of automated testing?
What are the benefits of Configuration Management?
What are the characteristics of a good test requirement?
What is a requirements traceability matrix is and why it is necessary?
What is the criterion for composing a test readiness review list (TRR)?
Provide descriptions for testing approaches (i.e. white box versus black box, etc).

The test manager needs to ascertain if the testing candidate is familiar with industry accepted terminologies that are commonly used within the project where the tester is being considered.

Thinking On Their Feet

In addition to understanding the testing requirements, a good testing candidate should have creativity and ingenuity when testing a software application. A tester should always be alert to potential scenarios that could cause an application to fail or yield defects and/or errors; even if such scenarios are not documented or presented in requirements. A thorough tester executes a particular test scenario with different sets of data, and conducts boundary testing to ensure that an application would not be deployed into production with overlooked problems.
A suggested question to discern a candidate’s testing meticulousness is to have a candidate provide use cases and test cases for a commonly used machine, or a self service application. An example would be: what test cases and use cases can the candidate think of for operating a beverage dispensing machine, or for purchasing books via a website. The candidate should generate an extensive list of test cases and use cases for the two aforementioned examples.

Development of Test Scenarios and Test Scripts A well written test scenario has information about: pre-conditions, post-conditions, traceability to a requirement, description, identification of authorship, a peer review and sign off section, roles to be tested, etc. A test script or test procedure on the other hand has detailed test steps with valid data values and expected results for each test step. In addition, a test script provides information about the test execution results for “passes/failures” and mapped requirements for the test step.
The test manager can have the tester provide information as to how test cases and test scripts were documented at the previous project. What level of detail was presented for the test cases and test scripts? What exactly was documented for the test cases and test scripts? Another suggestion is to have the tester send a sample test case and test script that he/she documented at a previous project for review. A well-documented test case and test script will demonstrate the tester’s attention to detail.

Life Cycle Methodologies

An experienced tester should have experience working with one or more IT methodologies such as waterfall spiral, evolutionary, incremental, rapid prototyping, etc. Some methodologies are more appropriate when requirements are well known/defined, or when requirements are not well known, or when the project has high risks, etc. The tester vying for the position should understand what the differences are between the main software development lifecycle methodologies. The test manager can present the tester with different hypothetical scenarios of IT projects and ask the tester what methodology would be most fitting based on the presented scenario.

Test Procedures and Test Standards

What testing standards and procedures the candidate has been exposed to is of paramount importance in determining whether the candidate would either be a good fit or adapt to a new testing environment. Did the candidate come from a regimented and disciplined type work environment like a CMM work environment with repeatable and defined processes? Or, did the candidate work in a chaotic test team that did not have any standards, procedures or defined processes?

Testers that come from a loose testing environment sometimes have difficulty adjusting to regimented testing environments that have defined processes and standards for things such as: lessons learned, test plans, naming standards, version control, test case templates, test execution matrix, test logs, test folders, reporting of test results, etc. Conversely, a tester that follows strict testing standards and procedures may struggle in a work environment that does not have defined processes, well documented scenarios, no structure for test cases, or templates for test scripts, etc. Questions that identify what the candidate’s experience is with testing standards and procedures are critical for assessing how well the candidate would adjust to the current QA environment.

Defects

It’s advisable to learn what sorts of defects a tester has identified and reported on in previous projects. Based on the candidate’s answers, the test manager can learn whether the tester focuses on cosmetic/minor defects or defects that are show stoppers and would have caused havoc for a released or production deployed application.

The test manager can have the candidate expound on a significant defect that was discovered and, what the impact of the defect would have been had the candidate not caught it. In particular, the test manager can focus on the tester’s approach for identifying the defect and how the tester re-tested the defect subsequently leading to the closing of the defect.

Test Script Automation Many candidates list in their resumes that they have experience with various automated testing tools. However, when confronted with technical questions about their experience with automated testing tools many candidates respond that their experience with automated test tools is limited to capturing and playback. Candidates that manifest to have only recorded and played back test scripts are in fact devoid of significant experience with automation test tools. Even candidates who claim to hold certifications with test tools should have their background probed and examined with technical questions.

The test manager should find out whether the candidates understand what a data driven test script is, how to create a parameterized test script, and why it would be necessary to construct a data driven script. The test manager should also ask the candidates to explain what data correlation is, how to create a driver script, how to synchronize scripts, why it is necessary to synchronize scripts, and how to report the execution results from a test script. Other key questions would be techniques for identifying and verifying recorded objects including providing examples of object’s attributes that can be verified, how to create checkpoints, how to debug an automated test script, and in what modes a recorded script can be re-played. The candidate should also know how to map and learn custom objects that the recording test tool does not identify or recognize.

The test manager can present more technical questions based on the nuances of the recording test tool that is present at the project site. The main objective of technical questions is to ensure that a candidate has experience that goes beyond merely recording and playing back scripts.