Regression Testing

28 05 2010

Regression with an automated test suite is a frequently used/abused phrase. People need to first understand what “regression” actually means.

Regression testing in a nutshell relates to the activity, by which we check a new application/product, for all the previous functionality; while keeping the newly added features unused. Basically, we validate that the new code has no effect on the older functionality. Regression is not used/should not be used to check for a functionality which has changed. As a change in functionality means that it is a new feature and needs to be verified first before it is validated.


Cardinal Rule for Automation

28 05 2010

So let me first state what is the first rule we learn about Automation of the testing for any project/product: Do not automate anything before the application comes to a stable state. But seems like this basic rule of Automation is forgotten by people when they become managers. Or they have never done any automation and feel that anything and everything should and must be automated to make any impact.

Well! People need to understand that Automation is not just a buzzword which needs to be done to create an impact on your own ratings and seniors. It is a serious study and work, which needs to be carefully thought out and then worked on through the design, development and minimal of maintenance. Not something which just on the whim and fancy of saying that Automation is on the agenda and has been completed.

Update: I fully agree with what @meDilbert has to say. Automation has become a word in process trainings and the CV’s of people wanting to portray that they have worked on it. It brings accolades to the person who implements it and brickbats for those who even mention that it might be nonviable. 🙂

I cannot Code!!

22 02 2009

The realization struck me, when I tried my hand at some automation that I was planning on doing for my recent project. It was supposed to be a simple Perl framework and it didn’t take me much time to complete the initial design and architecture of what would unfold into a testing suite for some of the new functionality added…

Then came the coding portion of it.. and here is where I got stuck!! How do I do a file open, what to implement – use or require (why?), what is the syntax for a for and a foreach and which variable goes where!!!

It really was an eye opener for me 😦

UI Automation Tools on the OSS Platform

17 07 2008

People seem to be getting back to the UI testing tools these days. UI is gaining ground and the commercial products available for the same are again coming into focus. There is a lot of hype being created around tools like Selenium and these are actually living upto that hype. Thoughtworks is one company which does consultation for the Open Source tools in use today and they have an intensive Selenium knowledge base. People should be encouraged to use these tools, which are although OSS, they compensate in man hours spent if we take the training consultation from companies such as Thought Works.
Another one which has come up from the same stable of is CubicTest. This uses the Selenium and Eclipse as a base and builds on top of that. A very useful tool, once the engineer knows how to use it. The learning curve for these tools may be high now due to the lesser penetration, but it is growing and soon might surpass the commercial ones soon enough.
The new Selenium and CubicTest versions are said to be major enhancements and robust stable ones.
So get onto this if you have a Java or Perl background or even with PHP now

Automation Tools

17 05 2007

There is a proliferation of automation tools in the markets these days. From the Open Source variety which may be paid or free, to the Commercial tools which claim to be the best in the market place. The main issue with all these is that they are meat for a particular kind of generic testing. The Commercial tools makers say that their tools are meant for a particular solution or are generic enough that they can be used in any kind of situation; but the issue here is that they are not configurable enough to be used in all kinds of situations. Similar are the issues related to the Open Source tools flooding the market these days. (Just run a search on Automation Tools, it gives about 25,600,000 results and 8,250,000 results for Testing Tools)

Of all the above results, each is advertising that their tool is the best. Although I have gone through plenty of these in my career as a automation engineer, I almost always had to tweak each one of these to suit my needs. This is required also, and for this purpose the makers of these tools have given each a tool command language (don’t confuse with TCL/Tk). These languages are usually related or belong to the family of one of the scripting languages (mostly it seems to be VBScript in the some of the popular Commercial tools).
Nowadays, I am using and trying out some Open Source and a few Commercial ones. Of these, Selenium and Waitr, I have found interesting for the work I am doing. TestComplete is the Commercial one being used, along with Test Link; hence am using these. For Selenium, Flash integration looks a good prospect and I am planning on using FlexUnit along with Selenium tools for such. Lets see what happens and comes out of this going forward… 🙂

Automating the UI process

17 03 2007

An interesting them is emerging from the markets these days. Cross-platform and cross-browser automation of the UI. The Web has gone from being just the Internet as a platform for sharing information to a Internet as a platform to share almost anything which can be along with entertainment for the users. Now this sort of a shift requires a lot of innovation on the part of the developers to capture the users imagination or as it is popularly known as the eye-balls of the users. This then again makes the issue of a cross-browser compatible feature more interesting, as the varied users of the Internet world have that many varied browsers on which they wish to portray their Internet experience on. The browsers in themselves display the static HTML displays very proficiently, but when it comes to dynamic display (excluding Flash and its brothers/sisters) the browsers have a very nasty habit have having quirks which would get any developer totally frustrated on the way of making these work.

Now comes the work of the person who has to automate this process. The developer would have used many a tricks up his sleeve to get the cross-browser functionality working, and when it comes to testing this, the person doing so, also has to use his/her ingenuity. Most of the time we can get away with checking the same on a few generic functionalities, but it would hurt the business when RIA’s come into the picture and people try to use these and encounter a place where they are about to make the highest score, but due to a browser incompatibility they are unable to proceed. This is where tools like Selenium jump in.

Selenium is cross-browser and can be used for cross-platform testing also. It can be based on a variety of scripting and proper languages, with which the person automating the process is aware of or has worked on. In my previous organization, they had used Perl and here we plan to use either Java or Perl. Along with this plans are also underfoot to use Flash and try to automate testing the Flash applications so that it makes work easier and more fruitful, as these days we seem to be getting a lot more of Flash based applications which are proliferating the Web world.

More on this later…

Work with UI

3 03 2007

This is an interesting aspect of the software industry work life which has come out into the fore front of all the other technologies. The User Interface or GUI as it is more popularly known as has become an object of desire for almost the complete industry and most of the Organizations have started giving it a coveted position. These days selling software is more of how and where you catch the users eye and then how you tend to utilize that capability to the best of your interests [basically these translate to where the Organization can earn to make your salary a reality].

So has also been the shift in testing of the software created. We are also becoming more UI centric nowadays. Due to this technologies like JavaScript and MFC have gained popularity in the QA world. These days the QA part is more intensive and require the knowledge of all kinds of tools and languages, which for a normal ‘testing’ person were not mandatory to learn and work on just a few years ago.

Need to get more updated on this, and will bring in more topics on what is to be done for this. For now, do go over the book: Effective Software Test Automation – Kanglin Li and Mengqi Wu [Developing an Automated Software Testing Tool]. In this they have described the creation of a automation test tool using C++ and MFC. It is both GUI and command console based. I think we need to get something on similar lines working for GNU/Linux also.