When do we say testing is complete?!

1 06 2010

This is one of the favourite questions asked during an interview for Test Professionals. The answers that I have heard in various interviews have been as varied. The answer to this question could be a simple – “When a tester has exhausted all the test cases he built for the application under test”. But, recounting on my experience, the answer is not this simple. The requirement for ending a test cycle and saying that the product is “Ok Tested”, takes in a lot more than this. It depends mostly on what the team perceives as the “testing” being completed. It may be as basic as running a single command to as complex as creating a whole new suite of test frameworks to check the product/application.

A good answer from my end would be more like, all the basic user interfaces (user can be internal or external, api or ui, etc.) have been verified as working and all the user scenario’s possible (with regard to the business requirements documented) have been validated. This exercise should result in zero P1/S1 bugs and x% of other bugs (as defined differently across projects and/or Organizations), which have been carefully filed and are reproducible with the given steps. Comments to the contrary are welcome 🙂


Cardinal Rule for Automation

28 05 2010

So let me first state what is the first rule we learn about Automation of the testing for any project/product: Do not automate anything before the application comes to a stable state. But seems like this basic rule of Automation is forgotten by people when they become managers. Or they have never done any automation and feel that anything and everything should and must be automated to make any impact.

Well! People need to understand that Automation is not just a buzzword which needs to be done to create an impact on your own ratings and seniors. It is a serious study and work, which needs to be carefully thought out and then worked on through the design, development and minimal of maintenance. Not something which just on the whim and fancy of saying that Automation is on the agenda and has been completed.

Update: I fully agree with what @meDilbert has to say. Automation has become a word in process trainings and the CV’s of people wanting to portray that they have worked on it. It brings accolades to the person who implements it and brickbats for those who even mention that it might be nonviable. 🙂

Language Barrier!!

13 05 2007

There is one field where I have not as yet encountered a language barrier and that is testing. An interesting concept came out the other day while I was attending a conference – “Whatever comes out of the Developer’s desk, for testing the same, we require a simple understanding of the application and what is desired from it. Innovation is thing which is best left to Developers, but an analytical understanding is to be acquired for testing out anything. This is and should be the language of the Tester. And for this, the Developer can write in any human readable language, but the thinking mind will always be able to decipher it and work out the wrong it in… “.

Although, the above is true; we should not forget that the Developers also have the common logical thinking language similar to the analytical thinking language… 🙂

Automation Tools

7 01 2007

How do we use tools which claim that they can help in the process of automating your testing procedures? This is one of the questions which has come up to me on many occasions. Involved the the process of testing software created by developers for the last 5 1/2 years [of which 2 1/2 years I was the one developing and testing it also], I have realised one thing that is common to all software development practices, software developed by someone can never be completely tested.

Do we make our own tools and use them to test the software or do we take the commercially and/or open source tools which are available to us. I think it mostly depends on what we are trying to test. If we have a propriety application which is mostly console based, then i don’t think there is a appropriate application which we can use which is available in the proper form. We either have to tweak them [and this can be a major pain in the ass] to work for the product to be tested or we have the option of creating our own. I would prefer to go with the second option.

Automating the process makes the software even more susceptible to bugs on the whole. We tend to let the automated framework which is the creation of a usually a single person [till he leaves the organization] to run on without much intervention and think that the process of testing is going on. But the code which this test framework is working on is changing and there might be changes which require some changes in the framework as such. This scenario is particularly seen in startups, where the software is in a continuous flux of change and so is the design of the same changing. The framework creator has to curb his desire of not wanting to change the software which is HIS baby and think on the broader terms of the software which is being tested. As testers we need to be detached from the test code and the tools that we create. We have heard of a developers nature regarding his creation [and I have gone through the same], but as testers we need to try and work in a manner that we break the code written by the developer and stick to our point when the developer tries to defend the same.

Documenting the development process!

17 04 2002

Got a good scolding from my Sr. Manager today on the way I am using to code stuff. Had written an earlier post on how to code or rather begin code, and I seem to have done the exact opposite. Specially I grew excited on creating this first PHP script which was to be the beginning of the Defect Tracking Page and in my enthusiasm to display my created product forgot to follow the guidelines and got it ‘smack’ on… 😀 (knew it was coming)
Now I need to really remember the mantra – document, document, test, document, code…