Lessons from GUI Testing

11 01 2012

I recently started working on the GUI testing space again. This is an interesting space, with loads of commercial and open source tools being available. Although all the tools might have their own unique features which they bring to the fore; I realized that there are some basic fundamental steps, which need to be brought up to get things moving in the right direction. I have tried to put these steps as succinctly as possible in this post.

The initial step is to realize that although all GUI web based applications may vastly differ from each other, they have one common control which needs to be looked into – the ‘objects’ which create the page. Each web page (or for that matter GUI based application) would have these. Each tool has its own unique way of looking at and identifying these objects on the web page. The basic assumption being that the Dev’s have done a spot of good coding and provided meaningful and unique names to all the visible objects on a Web based application 🙂

Map these objects to the web applications page and half the work of automating the web based app is complete. The crucial part is that the automation engineer should realize that he has to use the names provided by him during this initial setup and mapping stage. We cannot rely on names provided by the Dev Team, as these may be generic and/or not properly worded; to provide the correct identification of the object on the web page.

So from my viewpoint, you need to start any GUI Automation by first mapping all the objects and providing proper names to these. With this work done, now arrange them into a proper flow, so that you create the required test scenario as has been provided by either the Business or the Customer. Having the initial mapping of the objects, is the biggest help that can be obtained. Will further post on the different tools and how to build this great library of objects with each tool.

What does testing require?

26 09 2010

Testing is not an easy job. In India, software began in a big manner due to the test capabilities that were advertised for gaining a ground in the software field. That does not mean that we were not making good software, but test capabilities were the ones which catapulted us. It is not as simple as just writing a few scripts in shell or for the GUI. As James Whittaker has written, in his article on testing, it takes skill and a good knowledge of the domain that you are testing for. It is tougher than development of the same. Developers need to know the technology and they know the domain. Testers need to know much more. They need to know the workings of the application and the domain, along with how the user will use it.

From the viewpoint of the tester, it is never just a small portion of the feature that is being done work on. He has to know what all inputs can come into the product/feature and what kinds of output are expected by the downstream/next to make it to work. I learnt it that way, and that is why I love the profession of test. I know the product from the user viewpoint and also from the viewpoint of the Dev (the inner workings). Along the way, I made a lot of learnings. Although I would say that I lost the ability to program in any specific language; I learned a lot about logic and analysis of a problem.

Along with the above, testing also requires a lot of understanding of the tools that you need to use to implement the tests. This may be in the form of commercially available tools (Mercury QTP, SilkTest, Rational, WinRunner, etc.) or open source tools (Selenium, Watir, Fitnesse, etc). You can create your own using scripting languages or regular languages. After testing for a few years on different technologies and platforms, you should be able to shift from one to the other, which is not as easy for the developers who are working on a particular technology, but they find it easier to shift domains. What do you think…?! 🙂

Functional Testing, is it truly functional?!

2 06 2010

Most people who have been into the game of testing, drop words like functional, regression, end-to-end, system, integration testing like they are the ones who invented them. You ask the person on what exactly he wishes to attribute that kind of testing to, and you get a blank stare, which states – “You Stupid! Don’t you even know this much? And you wish to test my knowledge of it??!!” 🙂

Well! To me the concept of Function Testing is rather vague. It usually starts when the developers complete a component and hand it over to the test team. What needs to be checked and verified is the functionality of the component. How does one go about doing this? In the true sense, most of the time, functional is not about functionality, but more about the interface between multiple components in an application.

(more coming up…)

Regression Testing

28 05 2010

Regression with an automated test suite is a frequently used/abused phrase. People need to first understand what “regression” actually means.

Regression testing in a nutshell relates to the activity, by which we check a new application/product, for all the previous functionality; while keeping the newly added features unused. Basically, we validate that the new code has no effect on the older functionality. Regression is not used/should not be used to check for a functionality which has changed. As a change in functionality means that it is a new feature and needs to be verified first before it is validated.

Thought Processes

11 09 2006

They say that we are the thinking brains and hence we have risen to wherever we are today. Now, if someone wishes to stop that thinking process and make us all into just workers, what will happen. This is the thought that has been crossing my mind for a long time now. Specially since there have been many new developments at my workplace which are leading me along that path. I have often wondered on the aspects of Software Development, is it only just coding and testing that code that goes into it, or is there something else also involved. I always had the thought that it should have a lot more than just the coding and testing aspects. Then came the day when I entered the world of Software Development. Luckily for me the experience started with a lot of thought process going into it and also along with that a lot of logical reasoning and documentation.

The process in initialization into this field was not an easy one and that is what I had thought the way it should be. Just when I had started to feel that I had started to learn what goes into making and creating good software, I was introduced to the life of a mundane worker, who just is a glorified typist. This person has to follow whatever is being ordered of him and follow those instructions to the T. Forgotten were the previously taught things about logic and applying your own thinking power to create something. It all boiled down to deadlines and the best possible ways that you could go ahead to the challenge of meeting these and appeasing the top bosses. Although in this regime, you never got the credit for the long typing hours you spent trying to get tunnel carpal and eye fatigue. All the credit went to the guy who showed the top bosses the snazzy presentation slides made out of mindless mambos of jargon and technical quotes which neither the guy presenting it nor the people listening to him croon, could make any sense out of. But, all just nodded their heads in the fashion that they really were doing something great and had finally got near to achieving this too. [never understood till now that it was just another bonus cheque in their pockets]. This is when I revolted and ran to the next place which came across as a place where I would be allowed to think. And so it turned out to be until the time came when that same type of people again flocked back over me…

This is real frustrating when the person above you informs you to please shut down all thinking senses and start working according to set down rules and regulations!! What can one do in this case except change and expect that the new place will be a lot better and let you think??!!


8 04 2006

Back to the crux of all things – documentation. I suddenly realized why we need the documentation when suddenly a lot of people started to leave and the things that they had started here went for a toss as they have as yet not created any documentation in a Word Doc but only in their minds…

My PM came over today and the thing on his mind was to assimilate and get all the work that had been done by the people, into a single entry point on the version control system and then work on integrating the same for all the products and applications. Currently, the task I am doing is majorly centered on the Windows platform, but now I will have to check on all the products and all the platforms and see where all optimizations can be made so that re-work is not required. It takes a lot of effort and thought to create something but only a little bit of negligence to completely destroy the same. A prime example of the same is the test framework I am currently working on. Now we had a perfect version checker written in Perl, which used to check the version of the product installed by us and the version of the product as given by the executable. But the build and installer person decided that [or rather forgot] that he has to update the version number in the installed package. This made the situation that the version obtained from the package was different from the version given in the setup directory name and again different from what the user was entering it as. So came the problem that when we installed the product, the versions would not match and the product was again uninstalled, but as we were not checking the product version when installing, just the executable file was being checked, we used to install the product again, reboot and then get the version from the command line tool. This was matched with the version given by the user and again a loop as no match means uninstall the product… 😀

The main thing I next need to learn from this was never to trust anything where a human can make an error, as the famous words go, “if anything can go wrong it will” [along with “to err is human”] and the axiom added by my Project Manager from the first organization I worked in – “the thing that goes wrong will happen in front of the customer”, which is this case was the QA team who were supposed to execute the automated framework. They took the build and the process went into a loop… 😦

So coming back to the documentation part now, I was as is by habit creating all these documents, but could not comprehend this eventuality of the loop as I have given above. So exactly what does the documentation do? Well! for the first part, it got us immediately to the solution of this problem. From the given documentation we already knew that something was going wrong in the comparable algorithm as every time we got the message that ‘Version not Match’ and then uninstallation of the product and after that again an installation of the same. On going through the comments for the installation, I realized that I had not catered for human error and thus the solution to the problem was to cater for this, a major thing which I realized for Automation of anything was FAULT TOLERANCE...!! This has to be built into the system, else all our automation efforts fail, if we are just spending time in monitoring the application which has been automated.

As for the documentation part of all the other tasks being done for the automation, I realised that none of the others had any sort of documentation to go with them, except the typical programmers prospective of code and go, the logic and document is within the brain of the programmer who created the wonder of an application in the first place and as I have said earlier an unmaintainable application which if not understood and needs change might be discarded later. Thus, I am now to get down and create a document or ask the people who created these applications in the first place to create the same... [ the second option definitely looks appealing... 😛 ]

More coming' up...!!

Batch Scripting

2 04 2006

Did you know what all you could do with DOS Batch files??!! Did you think all you could do was start another batch file, call a few commands to run in sequence!! Well! With DOS and batch commands you can do a lot more, right – you can even read and write to a file, and also it has a somewhat restricted sense of sub-routines in it. So, finally I was introduced to the world of DOS Batch Programming. I learned that Batch processing is a lot more than I would have ever thought it to be. Can you image my surprise when I learned that you can do file manipulations using DOS??? And to add to that, even binary files can be manipulated using the simplest batch commands of DOS?!! Ever heard of the command FOR in DOS! Well! If you add the /F switch with the same, you get a wonderful file handling capabilities for DOS. Read up more on the posts by Mic, an avid and resourceful DOS Batch person, who I think would be able to answer all DOS Batch related questions you post to Batch World group on Yahoo!: Batch World

Well! I have had a fine time trying to work out what all I could to with DOS to help in my Automation work; as currently what I have created is a PXE server which can boot a machine into DOS, from here I have to carry on and cal the Acronis Command Line tool to accomplish my work, but all has to be done using DOS commands only as the environment into which the system is booted into is PURE MS-DOS with a few network enhancements. I now had to work out a way by which I can get my system to work by calling the Acronis Command Line tool and install an image for the system which will be Windows platform and configuration as selected by the user, as this is a dynamic thing, I have had to do a lot of work to create dynamic fiels which would do my work and heer I learned about DOS and Batch File commands. Currently, I have managed to get a working script which will help me in installing images stored by me on a server and a user configuration in which user selects his choice of OS file system to install and execute his test framework on:
FOR /F %%y IN (c:\OSCount) DO CALL :READ %%y
SET /A countWr=%my_return% 1
ECHO %my_return%
> c:\OSCount echo %countWr%
ECHO %countWr%

SET /A v_sum=%1
SET /A count=1
FOR /F %%x IN (c:\MyFileA.txt) DO CALL :ABC %%x
ENDLOCAL & SET /A my_return=%v_sum%

IF %count%==%v_sum% ECHO This File %1
ENDLOCAL & SET /A count =1


The above script is reading a file (MyFileA.txt) and getting the line number to be read from another (OSCount) file, which is then incremented for the next execution (the file is written to, we can also append to the file by using double greater than symbol – >>). Also, the variables are being passed as parameters from one sub-routine to another and then returned. So, finally, when we reach the first sub-routine again, the OSCount file is contains an incremented new value and when we run it again, we can get the next line in the file MyFileA.txt.

Although with the above script you can find out many things you can do with DOS batch commands – read a particular line in a file, write or append a line to a file, and even sub-routines in DOS!!! I never knew these were there. The SETLOCAL – ENDLOCAL actually defines a sub-routine in DOS. For debugging DOS batch files, you just do an @ECHO ON at the beginning of the file and execute the script through the DOS command line and viola, you can actually pin-point where you are going wrong – which line or variable is returning what value and where and when.

So, folks all I can say is that get your Windows machines pumped up and start using the powerful features of DOS which you might have never know existed… (PS: I have not added the code sent by Mic here as the link should take you to that, which is more nicely structured and concise!! My code is just a bunch of statements more meant to give an idea of what all DOS is capable of doing…)