Automation Tool across Web, Mobile and Web Services!

26 03 2012

Earlier in the week, I was sent across a request from one of our Senior Management on what could be a best tool, used for the automation of tests across the spectrum of Web (HTML & Flash), Mobile (iPhone, Android, Windows, etc.) and Web Services. What I could come up on this is the following. People may disagree with these options and may have different opinions and views on it… please feel free to comment and put them through, to improve on the content 🙂

Looking into the problem from the requirements viewpoint, I believe Selenium would be the tool best suited for the above automation work. The issue which might go against it, is that their Mobile product is still in Beta, and they are not the best for Web Services Testing, Watir being the frontrunner in the Open Source (i.e., Free) tools in that category. There are other Commercial Tools also which are available with good support and good interface, making it easier for the Automation to be maintained; which is somewhat of a problem with the Open Source tools, if not properly designed initially. Commercial products also have a big following and hence are cost-effective in the long run, although they might be expensive to procure, but getting a resource who is great in an Open Source product can sometimes be a big recruitment headache.

That said, Flash/Flex is a group, which almost with all tools requires a debug/special build to be provided for testing. Each tool has their own quirks and libraries with which the Flash/Flex application needs to be compiled with. So, you might wish to go more into each tools individual ability and reviews of their Flash library functionality; especially for Web Based applications.

Coming to mobile applications, the market for these exists as a very fragmented field for testing successfully. With Android Browser, iPhone Safari, IE Mobile and Firefox being the major browser contenders for the Automation tools available, along with testing of the Apps within the iOS, Android, Windows Phone and the various other vendors out there. I have seen many people refer to the Experitest SeeTestMobile tool, which might be becoming a tool of choice for many, these days.

I plan to go over some of the tools which might help out in each group, and some which might have multiple categories covered below. These opinions are my own through what I have experienced with them, and all are free to criticize and cajole me into making changes as is reflected “great” for them…

Selenium

Advantages: Good for Web GUI Testing. Great tools available for Firefox browser and the new WebDriver combined with PageObjects concept make it a great cross-browser test tool for the HTML/JavaScript Web. It even has a Flex/Flash plug-in for compatibility with the [debug/developer] flash applications. Can be coded in multiple languages (Java [most popular], Perl, PHP, Python, C#, etc.). This is a Free Open Source Tool.

Disadvantages: Not very intuitive, depends on coding skills and good design. New WebDriver is good, but there are not many in the market who can create some really good frameworks and know how to use it properly. Requires knowledge of XPath and JUnit type of coding to do anything great with the tool. Mobile product is still in Beta. Not many people available and consultation fees with consultants and resources can be high.

HP Quick Test Pro

Advantages: Well supported and lots of resources available who have certifications, but mostly used in Financial Institutions. Integrated add-ons for Flex, Web Services, Silverlight, and Web HTML. Framework issues can be easily taken care of with Odin AXE framework, which uses XML and simple interface.

Disadvantages: Ability to recognize complex UI and dynamic content hinders the tool. Mostly used in Data-driven web testing, which makes use of Excel sheets; easy for the user to use, but may cause issues in maintainability. Windows System only focused. Not suitable for Unix-Clones and Mac OS. High deployment costs.

MicroFocus / Borland SilkTest

Advantages: Good tool for Web and Flash. (MicroFocus has recently bought it after Borland failed, not sure of its development path going into the future). Has support for other platforms and operating systems.

Disadvantages: Learning curve, due to its test coding language. Not many people available with the tool knowledge.

Watir

Advantages: Good Open Source Tool for Web Services and Web Testing. Used with Fitnesse, produces easy to create and support web tests and web services tests. Not too good with Flash and Mobile.

Disadvantages: Uses Ruby as the language of choice, which is a skill getting hard to find for Testing.

 

SAHI

Advantages: Great tool for Web testing. Has good variety of plug-ins for the various other technologies. Available as Free version and supported paid version. Support for the same is great, the Developer of the tool is quite helpful in working out the issues with the Test Team. Good for complex websites, where other tools may sometimes fail. Unlike Selenium, it does not make use of XPath to identify objects; and can be used across browsers for recording tests.

Disadvantages: Only used for Web Testing for now. [not sure if it has been updated with plug-ins for others]. Limited use, thus not many people know about it.

 

SmartBear SoapUI

Advantages: Great tool for Web Services Testing from Smart Bear.

Disadvantages: Only useful for Web Services Testing. (but this might be an advantage, as they plan to make this a separate activity)

TestComplete

Advantages: Good tool, very similar to HP QTP, with a good interface and price. Overall good for Flash/Flex, with the included Libraries. SmartBear has a full stable of tools, which if bought together may be helpful in pricing and overall deployment and support. Uses VBScript/VBA for coding. People with QTP Experience may find it easy.

Disadvantages: Flash/Flex testing is still not very stable, sometimes fails to recognize the separate objects.

Microsoft Visual Studio Test Professional

Advantages: Is natively attached to the Visual Studio product line. Great for Cloud and .NET application testing. Good is you have Windows Phone applications. “CodedUI” is an excellent tool for testing cross-browser and web HTML testing. MS does deals to get the testing community to start using their tools 🙂

Disadvantages: Only for MS Technologies mostly. Not good for Firefox and Android. Only uses C# or Python.

Odin AXE Framework

Advantages: Great tool for building a wrapper over the existing tools scripts; actually it converts the tools identified objects into a XML recognizable format and has a great and easily understandable format for Automation testers.

Disadvantages: None that I can think of for now, except the use of a tool is somewhat a compulsory need for the framework created in AXE to work. Odin has done a good job of making the tool robust for Web Testing tools and it is compatible with almost all other commercial tools available.

Tricentis TOSCA

Advantages: Combines the best of Requirements, Test Case Design and Test Case execution, all in one single application. Good when there are business testers who know what the application is doing and there is good documentation available for doing it.

Disadvantages: Not very flexible when it comes to handling of unexpected behaviour within the application. Likes to have a clean interface to run through test cases and offer a “happy” path.

I can provide some more research into the new tools (and some less known but good ones), but the above are some of the common ones in use.

I am not advocating the use of any one tool above and to each depends on what he has worked with and would be comfortable in using.

Advertisements




Testing is Complete!

23 09 2010

A lot of time during the test process, we get to hear the phrase ” When will testing be complete?!”. The common refrain we can think of is – “Never”. In a way this might be the correct phrase; but that does not alienate us from not being responsible if the product gets a bug. As test people, we do not have the liberty to say – “Look I told, testing was not completed!” 🙂

There are many different aspects of judging when a particular test cycle is complete. One criteria depends on the test plan. The Test Plan, which is created using the design and engineering specifications document, should cover the common scenario’s. Along with the common scenario’s, we should also have what we in test parlance call the border cases and negative cases. The border cases are those which are not so common scenario’s, while the negative ones are if we use the application in a way it was not meant to be used, what is the behaviour.

Another very very imporatant aspect of development and subsequent testing, I was taught early on in my software education, was documentation. Whatever happens, the documentation should be kept on top. May it be a specification discussion or related to any new test that you add during development or testing. Mostly, one should not feel it as an attack on your ego, if someone else informs you on it, and then to mention why it was added and not have pangs of being crucified for missing it in the initial draft. That is the reason a draft is created. But mostly it is not so, and we miss some critical testing in this manner.

Coming to the topic of being “test complete”. The best bet is to write a comprehensive test plan based on the specifications and get it reviewed by all – dev and test teams, as well as the product managers. Once this exercise is done, rest assured that you have covered a good percentage of scenario’s on how the user can use your application. As they say in test – “Think like the user, not the owner of the product/component” 🙂





Appraisal System

14 11 2009

Was talking with a colleague on the current appraisal system that is adopted across most of the Organizations. They usually go with an yearly system, in which the person to be appraised is told to write out his goals for the year and then he gets to finish whatever he has stated in the goals over a period of a year. Well! most of the time, when the final appraisal is done, it is for the work done in the past quarter… and the person who has been consistent throughout the year loses out.





Web World & Advertising

5 04 2007

There seem to have been a lot of enhancements going on in the Web sphere. Initially, the web was just a place to share information on academic topics around the world. But recently, the Web seems to have acquired a characteristics of its own, where the individuals as a collective drive what they think would succeed and what would not. This has seen the emergence of the various communities on-line where each is trying to grab a bigger pie of the on-line users and thus establish themselves as the front-runner is this race which has started off to attract users.

Where does all this lead to?
The basic answer to the above would be more Advertising revenue and hence more sales and thus more jobs to cater for these factors. Alone, the Web would not have propagated to where it is currently. Organizations realized during the initial days of the emergence of the Internet that this media was going to be one of the major revenue earning streams and along with that also a great tool for advertising their wares and again generate revenue in the process. These Organizations went about promoting the Internet for their own needs and hence the transmogrification of the Web from an academia interest to the entertainment interest of now.

Advertising has become the way of life of many these days. We have professionals and complete Web/Internet Companies/Organizations dedicated to this way of life and users who are happy on clicking through and checking the various fares thus advertised. This is beneficial for all, as the Companies get a good exposure of their products and employ more as sales increase and the people get more jobs and better pays which they can again spend in what they like from these advertisements. Money is circulated and this means good for the Governments, which can also grow their own Countries economies and make sure that the citizens get what is due to them. Taxes grow and so do incomes.

Who says Advertising is bad??





Work @ Big Corporates

20 12 2006

So, I have started work at another one of those so-called Big Corporates which they say have the best policies for their employees. But does the Work Culture and the kind of work being offered also effect the employee’s or are they to be silent spectators in the grand wide scheme of things of churning out one badly conceptualized product after another?
Even the best of organizations are unable to offer the type of employee satisfaction that is required when working. I think it mostly depends on the seniors to ensure the type of quality work and environment which compels the employee to stick around and do the work given sincerely and to his own satisfaction level rather than that of the Organization. We do have a responsibility towards all those who are working with or for us to ensure that they are constantly energized to do the kind of work that is required from them. work and Money do not go hand-in-hand everywhere, but with the rising expectations [on the Money front] of the new breed of IT people who are coming into the field has fueled a sort of a mad rush where everyone just seems to be running for the money and not the work that is offered.
We are ready to take up a downright insulting work to our intellect rather than compromise on the Money part. Then the cribbing starts…
Why do we do this? Have we been ignoring these issues in the past so that the monster has become uncontrollable? I think so, we try to think of everything in terms of Money and in this we tend to spend a lot of it even when we might have not earned it, this way we start looking for avenues to make more and end up in the classic trap of making more Money without looking at the Work we may be doing to earn the same. I think it is high time we understood our priorities and started doing what we really wish for instead of chasing that elusive Money which might never give us the satisfaction we need.

Then again, it is also the prerogative of the seniors who need to start making the Work more interesting so as to reduce the problem of Attrition being faced by all. If we start making work interesting along with the money then we might just manage to curb this fascination of people to splurge and then look for avenues to retrieve back that lost Money. This way we might even manage to save the ever impeding problem of salaries in India crossing the profitability mark for the Corporates and save ourselves from a Big Depression…