Comparing Commercial Test Tools

1 03 2011

This post is more about the comparison of 2 tools, which finally made the cut for an application/product, I have to Automate for my current Organization. They are a renowned name in the Smart Grid domain and have their own Smart Meters manufacturing. The application is the software API on top of these meters and their firmware, which allows the readings from the multiple meters (mostly in the thousands), to be collected and provides a Business Intelligence abstraction layer for the actual Hardware and Firmware. There were certain criteria which were needed to be implemented and taken into consideration before the final tool choice was to be made.

The team went through many .NET enabled Software Test tools – both commercial and open source, before finalizing on the below two, due to long-term stability and robustness. Also, we had to cater for emulators being used to test things which were critical to the business. All these points might not be mentioned below in the actual comparison of the tools, which has been made more generic for the purposes of posting on the blog.

Criteria on which the tool has been analysed HP QuickTest Professional v11 Visual Studio Test Professional 2010
Actual end user simulation: Is the test conducted using this tool equivalent to an end user action? QTP claims to perform end user simulation, in other words executing QTP scripts are equivalent to a person performing those steps manually on the application. Using the Coded UI tests, we can create UI test cases as they have been done using actual user interaction. You can execute tests with the browser minimized also, like Selenium, as it can use XPATH and DOM.
Support for UI Components QTP requires extra add-ins (plug-ins, not free) to work with .NET and other components, like Java, JavaScript, etc. Visual Studio natively supports .NET components. Also, JavaScript and other web scripting languages support is present, without additional plug-ins.
Object Management & Storage QTP comes built-in with Object Repository. Object Repository management is quite easy in QTP. Objects are recorded and added automatically to the Object Repository. Visual Studio Coded UI interface provides a limited set of Object Repository. It creates the user interaction internally in XML format and can be used in conjunction with screen position or the object name and ID.
Support for Dialog Boxes QTP supports all kinds of IE dialog boxes. These are helpful, when parsing error messages in the application under test. Especially when we expect a popup dialog to appear. Good support for embedded and IFrame dialog boxes. This has better support for IE browsers, being a Microsoft product.
Support for web browsers Cross-browser support is lacking in QTP. Scripts created for one browser may not run on another. It has cross browser support for IE, Safari and Firefox. These have been built by the specific vendors themselves.
Object Oriented Language Support & Scalability (as in Integration with External tools utilities and libraries). VBScript has limited OO support and QTP has limitations with using any other language for framework development. Supports C# as the major language. It is very similar to Java and has full OO support. Also, there is a large base of resources who are working with C# and .NET
Integration with Test Management tool With HP Quality Center and Test Director Integrated with Visual Studio Test Manager and Team Foundation Server.
Types of application supported Web, Windows (.NET, VB,  Power Builder, TCL/TK), Terminal Emulation, Command Prompt, Windows Desktop Native .NET, Command Prompt, Windows Desktop Native, Web Applications.
Support for different Operating Systems / Platforms QTP only supports Windows Current implementation of Coded UI, can support test cases on Windows and Linux boxes, as the application creates XML based code.
Technical Support QTP offers technical support by phone and mail, HP also has a web-forum. QTP user community is vast and questions posted on online forums get answered quickly. Although the technical support is available through phone and mail, the forums are not that intuitive now. But Microsoft has made efforts to have multiple Evangelists create blogs and forums to discuss user issues.
Cost Costly. $9,000 per seat license. Separate costs for Quality Center and other development and SCM related tools from HP. Costs $11,000, when bought with the Visual Studio Ultimate edition. But has other products bundled with it  – TFS, Test Manager, Visual Studio, etc.
Test Development Environment Reasonable but not the best. QTP tests can only be developed using QTP or Notepad like application. Best in the world. (my opinion, after Eclipse, it is the best and those who use it love the interface) 🙂
Integration with development process No real integration possible. Has plug-ins now which can integrate with other tools and development processes. Tests developed using VS TP can be easily part of the development project. Using tools like TFS and VS Build, Continuous Integration is easier.
Data Driven Testing Support for Data-driven and Keyword-driven testing, implemented using MS Excel. Good support for both Data-driven and Keyword-driven testing. With XML as the base for Keyword-driven.
Database Testing SQL (Structured Query Language) is integrated with QTP. Can make use of SQL statements from within QTP. Native SQL Server DB API’s are present. Allows command-line driven testing for validation and verification of the DB Integrity also. Support for Oracle is also present. (Will need to investigate this further)

Update: Integration of HP QTP scripts with Microsoft TFS is now also possible, as has been given in this excellent Lecture Series: StickyMinds.com Lecture Series

http://testingcircus.com

Advertisements

Actions

Information

One response

21 01 2014
Iamvichu

Useful… thanks….

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




%d bloggers like this: