Automating with Agile

31 12 2013

Agile is not a new word to the world of Information Technology. Automation has been said to be one of the key practices to making an Agile project possible. This in many ways may be considered true. I have been going through some good established practices of Agile, where most have been based on some basic level of automation, which helped in making a success of the project. I have penned down some thoughts on what it means to have test automation along with Agile practices.

There are many schools of thought which have gone into the agile way of doing a project and managing the various components that finally lead to the delivery of the project. When we consider Agile and its various derivatives, we come to realize that each Organization has their own way of dealing with the complexities that come with it. There is mention of starting with a session to discuss and elaborate on what the scope of the project is and how it can be broken into smaller pieces which then become the initial requirements. These can then be distributed to the team and made into story cards, attached with t-shirt sizes and put up on the scrum board to be picked up in batches by the team and worked on. In this fashion, a project progresses with the minimal of friction  and gets completed within the estimates provided by the t-shirt sizes. Most of the time this might not be entirely true, but this is how some Organizations perceive Agile and practice it; in the process giving negligent time for the automation, as manual tests take up the majority of time to complete.

When we talk of automation in Agile, it not only consists of the testing component, but an overall ‘continuous integration’ component. From the check-ins, build, unit tests, defect handling, integration & system tests to the final deployment on the ‘test’ server for doing the User Acceptance Testing (UAT).  Agile shops majorly miss out on this flow which should be the first thing to be completed for a Project to perform smoothly through its life-cycle. There are a multitude of tools available for making these tasks simpler and more robust, to name just a few which I have used – Jenkins/AntHill, Maven/MSBuild/make/Ant, SVN/Git, JIRA/FishEye, Crucible, TOSCA/QTP/pCloudy.com/Selenium.

In the path to Agile, we forget that we need to do the planning for automating our complete build-deploy process also and that includes the crucial part for integration & system tests. A thoughtful planning would be to make the initial framework using stubs for the interfaces and when these get built, replace them with the real thing. Often what I see is the perception that automation should be started when a clear and stable build is provided; yes, in a way this might be true, but not for Agile, where you really need to be agile and think on your feet. Start by implementing a strategy, wherein you have stubs ready and a CI platform available to make sure that testing can be done without code. This was the first lesson I was taught, we were to create test cases based on the ‘pseudo-algorithm’ and  the interfaces that we have written. The tests need to be developed in a way that all fail initially and as and when the code is delivered they start to pass according to the requirements provided.

If you have done this then you have taken that crucial step towards Agile automation, that will take you a long way in making the project a success for you and your Team.

 

Advertisements




What to Automate?

19 09 2011

I had this interesting conversation on Automation the other day with my colleagues at my new job. It started off fairly innocently on how the automation should be thought out and what needs to be done to automate. We already have an existing framework and test scripts in place, which very efficiently work and report issues. The problem comes when we need to provide data on what automation actually does. How do you prove the effort spent on automation? You could easily say that it saves time and resources by checking for faults early in the development cycle, but how can we be sure that it actually covers scenarios that check the application? This is where a business requirement document becomes a necessity.

The issue which we increasingly face today is how do we relate the business requirements to what we test. There are a few things which are given in textbooks and across such certifications organizations like ISTQB, which provide information on such. The thing is ultimately it depends on the person who is sitting and working on the application (which most of the Program/Product Managers miss out on) and the person who is writing the scripts to automate the application testing. The best way to figure out for a restricted zone (propriety) application is to go and sit with the users and find out what they use the most (or run a key stroke capture software and see where all it goes, limitations of this later). With a public/global reach software, it is best to give out Beta versions of the same, like most of the Big Organizations do and see what is reported back.

The other end of the spectrum is propriety software, which might not have a user interface (system tools like Compilers is one example). For this the technique is basically to read through the Software Requirement Document/Specification and have confidence in your abilities to decipher the jargon written in those and convert them to simple English. I got my initial training on writing code on these only, and the golden rule for us was [in a summary]:

  • Read through the document and write what you have understood for each function
  • Create the Algorithm for each in plain English pseudo code
  • Convert these into test cases, and run those test cases
I think the above rules of going through the steps of Why, What and How has helped us out a lot to become what we are today 🙂




Comparing Commercial Test Tools

1 03 2011

This post is more about the comparison of 2 tools, which finally made the cut for an application/product, I have to Automate for my current Organization. They are a renowned name in the Smart Grid domain and have their own Smart Meters manufacturing. The application is the software API on top of these meters and their firmware, which allows the readings from the multiple meters (mostly in the thousands), to be collected and provides a Business Intelligence abstraction layer for the actual Hardware and Firmware. There were certain criteria which were needed to be implemented and taken into consideration before the final tool choice was to be made.

The team went through many .NET enabled Software Test tools – both commercial and open source, before finalizing on the below two, due to long-term stability and robustness. Also, we had to cater for emulators being used to test things which were critical to the business. All these points might not be mentioned below in the actual comparison of the tools, which has been made more generic for the purposes of posting on the blog.

Criteria on which the tool has been analysed HP QuickTest Professional v11 Visual Studio Test Professional 2010
Actual end user simulation: Is the test conducted using this tool equivalent to an end user action? QTP claims to perform end user simulation, in other words executing QTP scripts are equivalent to a person performing those steps manually on the application. Using the Coded UI tests, we can create UI test cases as they have been done using actual user interaction. You can execute tests with the browser minimized also, like Selenium, as it can use XPATH and DOM.
Support for UI Components QTP requires extra add-ins (plug-ins, not free) to work with .NET and other components, like Java, JavaScript, etc. Visual Studio natively supports .NET components. Also, JavaScript and other web scripting languages support is present, without additional plug-ins.
Object Management & Storage QTP comes built-in with Object Repository. Object Repository management is quite easy in QTP. Objects are recorded and added automatically to the Object Repository. Visual Studio Coded UI interface provides a limited set of Object Repository. It creates the user interaction internally in XML format and can be used in conjunction with screen position or the object name and ID.
Support for Dialog Boxes QTP supports all kinds of IE dialog boxes. These are helpful, when parsing error messages in the application under test. Especially when we expect a popup dialog to appear. Good support for embedded and IFrame dialog boxes. This has better support for IE browsers, being a Microsoft product.
Support for web browsers Cross-browser support is lacking in QTP. Scripts created for one browser may not run on another. It has cross browser support for IE, Safari and Firefox. These have been built by the specific vendors themselves.
Object Oriented Language Support & Scalability (as in Integration with External tools utilities and libraries). VBScript has limited OO support and QTP has limitations with using any other language for framework development. Supports C# as the major language. It is very similar to Java and has full OO support. Also, there is a large base of resources who are working with C# and .NET
Integration with Test Management tool With HP Quality Center and Test Director Integrated with Visual Studio Test Manager and Team Foundation Server.
Types of application supported Web, Windows (.NET, VB,  Power Builder, TCL/TK), Terminal Emulation, Command Prompt, Windows Desktop Native .NET, Command Prompt, Windows Desktop Native, Web Applications.
Support for different Operating Systems / Platforms QTP only supports Windows Current implementation of Coded UI, can support test cases on Windows and Linux boxes, as the application creates XML based code.
Technical Support QTP offers technical support by phone and mail, HP also has a web-forum. QTP user community is vast and questions posted on online forums get answered quickly. Although the technical support is available through phone and mail, the forums are not that intuitive now. But Microsoft has made efforts to have multiple Evangelists create blogs and forums to discuss user issues.
Cost Costly. $9,000 per seat license. Separate costs for Quality Center and other development and SCM related tools from HP. Costs $11,000, when bought with the Visual Studio Ultimate edition. But has other products bundled with it  – TFS, Test Manager, Visual Studio, etc.
Test Development Environment Reasonable but not the best. QTP tests can only be developed using QTP or Notepad like application. Best in the world. (my opinion, after Eclipse, it is the best and those who use it love the interface) 🙂
Integration with development process No real integration possible. Has plug-ins now which can integrate with other tools and development processes. Tests developed using VS TP can be easily part of the development project. Using tools like TFS and VS Build, Continuous Integration is easier.
Data Driven Testing Support for Data-driven and Keyword-driven testing, implemented using MS Excel. Good support for both Data-driven and Keyword-driven testing. With XML as the base for Keyword-driven.
Database Testing SQL (Structured Query Language) is integrated with QTP. Can make use of SQL statements from within QTP. Native SQL Server DB API’s are present. Allows command-line driven testing for validation and verification of the DB Integrity also. Support for Oracle is also present. (Will need to investigate this further)

Update: Integration of HP QTP scripts with Microsoft TFS is now also possible, as has been given in this excellent Lecture Series: StickyMinds.com Lecture Series

http://testingcircus.com





What does testing require?

26 09 2010

Testing is not an easy job. In India, software began in a big manner due to the test capabilities that were advertised for gaining a ground in the software field. That does not mean that we were not making good software, but test capabilities were the ones which catapulted us. It is not as simple as just writing a few scripts in shell or for the GUI. As James Whittaker has written, in his article on testing, it takes skill and a good knowledge of the domain that you are testing for. It is tougher than development of the same. Developers need to know the technology and they know the domain. Testers need to know much more. They need to know the workings of the application and the domain, along with how the user will use it.

From the viewpoint of the tester, it is never just a small portion of the feature that is being done work on. He has to know what all inputs can come into the product/feature and what kinds of output are expected by the downstream/next to make it to work. I learnt it that way, and that is why I love the profession of test. I know the product from the user viewpoint and also from the viewpoint of the Dev (the inner workings). Along the way, I made a lot of learnings. Although I would say that I lost the ability to program in any specific language; I learned a lot about logic and analysis of a problem.

Along with the above, testing also requires a lot of understanding of the tools that you need to use to implement the tests. This may be in the form of commercially available tools (Mercury QTP, SilkTest, Rational, WinRunner, etc.) or open source tools (Selenium, Watir, Fitnesse, etc). You can create your own using scripting languages or regular languages. After testing for a few years on different technologies and platforms, you should be able to shift from one to the other, which is not as easy for the developers who are working on a particular technology, but they find it easier to shift domains. What do you think…?! 🙂





Bidding Adieu to Yahoo!

16 06 2010

It has been an awesome journey for me in Yahoo!. I have made some great friendships, which would last a lifetime and also managed to get to know some good people with whom I would like to be associated with now and in the future. This was a long journey (one of my longest tenure in a single organization), which was filled with a lot of learning and strives, some I faced with confidence and others where I did falter. A good journey altogether, which matured me and made me look at life from the eyes of others who mattered. During this journey, I met the person with whom I plan to spend the rest of my lifetime and have been managing to steer myself through mood swings and great loving times.

Yahoo! as a company, is one Great place to work for. You get all the freedom to do what you want, get to travel (and expense it to company accounts – when you go for Campus or Regular Interviews). You imbibe a lot of learning, most of which is available to you through the labyrinth, which in Yahoo! yodel, we call the TWiki (the Technical Wiki), although most of the material available freely on it is no where near ‘technical’ 🙂

The Yahoo! culture is one great learning force, which enhances the knowledge of many a fresh faces and minds. Many have tried to introduce some sort of process into this madness, in the form of ‘Yet Another Technical Place’, but the culture of adhoc’ism prevails and drives the energy and work force to yet newer and innovative levels. (I know I am going to get a reprimand from my oldest mentor, for such a long sentence, but couldn’t help it 🙂 )

Yahoo! is a brand name, which lives in the life’s of online people around the world, it is synonymous with the Internet, and might have lost its sheen in search, but is still a force to contend with when it comes content. It supplies the best on the web, without doubt. It was “Jerry and David’s Guide to the Web” and has remained “THE Guide to the Web”. Critics have said that Jerry should have taken the decision in favour of Microsoft, but they are not the people who have nurtured the Organization to the stage at which it is today. Yes, Jerry does have sentimental values for a company he created, but he took the right decision. Yahoo! is a BRAND, which stands out on the big bad Web, and Jerry knew he could turn it into a profitable venture. He has brought in the right person to bring about this change. She may be a bit outspoken, but she is making the right decisions to keep all happy.

Ok, enough of this rant. To conclude, I would say that Yahoo! has great internal talent pool and a culture to moot for. Now that is being cultivated to bring about the required changes to make Yahoo! happen.

My colleagues and friends made it a GREAT day for me and I was for once really feeling hollow and devoid of words on what to thank them with. It really does not feel that I am leaving, and saying good-bye to all that I have enjoyed and nourished. I do not wish to do so, specially after 3 1/2 years of being here. But, as they say – “All Good Things Have To Come To An End!!”, so does my really great innings at Yahoo!.

Thanks All for being there and all the help and love rendered over this long fruitful journey!

Adieu, Afscheid and Sustantivo…





Personal Issues @ Work

28 02 2010

Should we begin work with thoughts other than work on our minds? Organizations which have matured have started dealing with the employees personal issues also. As they have understood that to have a great working force, they need to be fresh in mind when they enter Office. This makes them concentrate on work and leave their worries behind atleast for the time span that they are in the Office premises.

Most of the time it is the managers job to act as the personal shoulder weep in these situations, but things are changing. We gradually are seeing more professional help coming in with the introduction of counselors, whose job is to make the employees feel at ease. My point here being, why should the managers always be the ones to take care of the personal weeps of an employee. Managers themselves have enough to take care of in the professional front, with all the cost cutting and such activities, having to hold back employees is taking a toll on them. Over that, the extra burden of trying to resolve personal matters is another thing.

I have been going through a lot of this lately and have began to feel that the responsibility of personal matters should depend on the employee and professional help needs to be sort out. Taking recourse on the senior management is not an option and should not be used for matters where they are also helpless than to just give advice and hope the employee can get through with his work.





My coding exercise!

3 03 2009

So I have embarked on a journey to find my coding skills and try to find where and how I lost them. The journey does not seem to be too long, as I can remember the last time I had coded something was around the time I left my last Organization. Coming into the new atmosphere, I think I somewhere along the line lost the charm to code and that took its toll on my coding skill and brought me to this space.

The exercise began with me trying to understand what was going on under the hood of something which we call the collection of logs from users and their analysis and loading into the databases. The data is then given to the strategic data services, who do the data mining and get to work on analysis of how to make things better for our customers so that their ROI goes up and becomes better.

My main duty was to take care of this pipeline (or various pipelines for the different ways in which we collect the logs) and make sure it keeps flowing and garning more data about the various activities going on, in this somehow I lost the interest to code and innovate on things… [i think the right word was inovation]