DevOps: Skillset, but with a new Mindset

1 01 2020

Joe McKendrick wrote in an article at ZDNet about Devops — ‘requires multiple teams to work closely with each other, side by side, on a day-to-day basis, to meet the significantly shrunken delivery timelines.’

At my current organization, we have been working towards achieving the goal of Agile sprints and a DevOps/CloudOps culture. The attempts have been sincere, but the mindset change requires a lot of effort from both the management and the people working across the projects.

The management needs to understand that the workflow of a Dev+Ops cycle needs a lot of hand-holding and a certain degree of automation across the development, build, deploy and test phases of the application. On the other hand, people who work on the projects need to ensure that they work towards a goal of making tasks automated and easy to build and deploy through the use of scripts. All this entails that the test/QA team are involved with the design/development process from the requirement analysis phase. This is currently missing. And that works out against the concept of an Agile DevOps view.

DevOps can be a powerful antidote to the issues of Agile not working, when it is done properly, with a view to achieving an outcome beneficial for both the Organization and the Individual.

“Automating the testing and the QA aspects can deliver an ROI up to 250% to 300% month over month, according to Chris DeGonia, director of QA at International SOS. In a recent podcast with Kalyan Rao Konda, president and head of the North America East business unit at Cigniti, he credits the ability to automate the flow, across repeatable processes, checks, and balances in the system.”

The skillsets required for a DevOps project/practice to get success is already present in most of the team members, developers know how to use scripts and have worked with Puppet, Chef and Ansible and with CI/CD tools; the test team similarily has a good grip of C#, Jenkins, shell scripts, automation, performance and CI/CD tools. Most of the team have worked with Cloud and related Docker and Kubernetes systems too. But the mindset is something that needs to make a change:

  • Start small, so debugging becomes easy
  • Break stuff, so that you know where and what is going wrong
  • Embrace your mistakes and rectify them fast
  • Educate each other on tools and fixes
  • Project management needs disruption, don’t be caught up on costs and timelines
  • Promote collaboration with the team members and business stakeholders

All these would result in a DevOps (Agile, Collaborative) culture, where the following would hold true:

  • Collaboration between the development teams and the business
  • Faster and on-time delivery of products/projects
  • Employee engagement and happiness (they get to learn and implement the learnings)
  • Innovation in the form of the smaller increments, where direction can be changed with nimbleness and finesse.

Thus, teams need to embrace change and provide more guidance to each other to ensure that DevOps with CI/CD can be implemented successfully. The DevOps practise helps in the fast and improved delivery of the product/application, using tools and scripts to automate the build, deploy and testing of the software code.

In conclusion, there are six basic principles that define a DevOps mindset (as mentioned in the DevOps article on ZDNet):

  • Be about serving the customer: “DevOps organizations require the guts to act as lean startups that innovate continuously, pivot when an individual strategy is not (or no longer) working, and constantly invests in products and services that will receive a maximum level of customer delight.”
  • Create with the end in mind: IT organizations “need to act like product companies that explicitly focus on building working products sold to real customers, and all employees need to share the engineering mindset that is required actually to envision and realize those products.”
  • Encourage end-to-end responsibility: “Where traditional organizations develop IT solutions and then hand them over to operations to deploy and maintain these solutions, in a DevOps environment teams are vertically organized such that they are fully accountable from concept to grave.”
  • Promote cross-functional autonomous teams: DevOps teams “need to be entirely independent throughout the whole lifecycle,” and even “become a hotbed of personal development and growth.”
  • Continuously improve: “Minimize waste, optimize for speed, costs, and ease of delivery, and to continuously improve the products/services offered.”
  • Automate everything you can: “Think of automation of not only the software development process (continuous delivery, including continuous integration and continuous deployment) but also of the whole infrastructure landscape by building next-gen container-based cloud platforms that allow infrastructure to be versioned and treated as code as well.”

To close it all, Calvin & Hobbes is required! 🙂

Automating for the Future

1 01 2018

When we go about discussing on automation, we talk about frameworks and tools for automating the application or the user’s product. People, who want their applications to be automated, usually start off with taking up an open source tool (or commercially bought tool) and using it for a simple record and play script creation. This then starts the cycle of making those set of scripts more robust and make them work over the application. Finally, the scripts are joined together and the developers of those scripts start calling them frameworks. This is the beginning of the confusion and chaos for test automation.

It is the belief of testing teams that once a “framework” like this is created and it can then complete a regression cycle for a certain release or development, the same is the best piece of work they have created and it would work out for any and all releases they do from that time onwards. What they forget is the basic rule of any software that it evolves. And with it has to evolve the test software also. They create an application specific and tool specific “framework”, which might be just a combination of scripts, which execute the test cases for their application or product and nothing else. Sending out some rudimentary reports, which someone may one day see and realize that everything has been failing for the past 2 weeks 🙂


There is a plethora of test tools which are roaming the open source and commercial world of testing these days. They all are good for what they advertise themselves for. But there is an inherent problem with them all. They are generic (catering to out-of-box standards) in nature and require a framework to be developed over them, which will take care of specific needs for the user’s application and/or product.

Automation Tool across Web, Mobile and Web Services!

26 03 2012

Earlier in the week, I was sent across a request from one of our Senior Management on what could be a best tool, used for the automation of tests across the spectrum of Web (HTML & Flash), Mobile (iPhone, Android, Windows, etc.) and Web Services. What I could come up on this is the following. People may disagree with these options and may have different opinions and views on it… please feel free to comment and put them through, to improve on the content 🙂

Looking into the problem from the requirements viewpoint, I believe Selenium would be the tool best suited for the above automation work. The issue which might go against it, is that their Mobile product is still in Beta, and they are not the best for Web Services Testing, Watir being the frontrunner in the Open Source (i.e., Free) tools in that category. There are other Commercial Tools also which are available with good support and good interface, making it easier for the Automation to be maintained; which is somewhat of a problem with the Open Source tools, if not properly designed initially. Commercial products also have a big following and hence are cost-effective in the long run, although they might be expensive to procure, but getting a resource who is great in an Open Source product can sometimes be a big recruitment headache.

That said, Flash/Flex is a group, which almost with all tools requires a debug/special build to be provided for testing. Each tool has their own quirks and libraries with which the Flash/Flex application needs to be compiled with. So, you might wish to go more into each tools individual ability and reviews of their Flash library functionality; especially for Web Based applications.

Coming to mobile applications, the market for these exists as a very fragmented field for testing successfully. With Android Browser, iPhone Safari, IE Mobile and Firefox being the major browser contenders for the Automation tools available, along with testing of the Apps within the iOS, Android, Windows Phone and the various other vendors out there. I have seen many people refer to the Experitest SeeTestMobile tool, which might be becoming a tool of choice for many, these days.

I plan to go over some of the tools which might help out in each group, and some which might have multiple categories covered below. These opinions are my own through what I have experienced with them, and all are free to criticize and cajole me into making changes as is reflected “great” for them…


Advantages: Good for Web GUI Testing. Great tools available for Firefox browser and the new WebDriver combined with PageObjects concept make it a great cross-browser test tool for the HTML/JavaScript Web. It even has a Flex/Flash plug-in for compatibility with the [debug/developer] flash applications. Can be coded in multiple languages (Java [most popular], Perl, PHP, Python, C#, etc.). This is a Free Open Source Tool.

Disadvantages: Not very intuitive, depends on coding skills and good design. New WebDriver is good, but there are not many in the market who can create some really good frameworks and know how to use it properly. Requires knowledge of XPath and JUnit type of coding to do anything great with the tool. Mobile product is still in Beta. Not many people available and consultation fees with consultants and resources can be high.

HP Quick Test Pro

Advantages: Well supported and lots of resources available who have certifications, but mostly used in Financial Institutions. Integrated add-ons for Flex, Web Services, Silverlight, and Web HTML. Framework issues can be easily taken care of with Odin AXE framework, which uses XML and simple interface.

Disadvantages: Ability to recognize complex UI and dynamic content hinders the tool. Mostly used in Data-driven web testing, which makes use of Excel sheets; easy for the user to use, but may cause issues in maintainability. Windows System only focused. Not suitable for Unix-Clones and Mac OS. High deployment costs.

MicroFocus / Borland SilkTest

Advantages: Good tool for Web and Flash. (MicroFocus has recently bought it after Borland failed, not sure of its development path going into the future). Has support for other platforms and operating systems.

Disadvantages: Learning curve, due to its test coding language. Not many people available with the tool knowledge.


Advantages: Good Open Source Tool for Web Services and Web Testing. Used with Fitnesse, produces easy to create and support web tests and web services tests. Not too good with Flash and Mobile.

Disadvantages: Uses Ruby as the language of choice, which is a skill getting hard to find for Testing.



Advantages: Great tool for Web testing. Has good variety of plug-ins for the various other technologies. Available as Free version and supported paid version. Support for the same is great, the Developer of the tool is quite helpful in working out the issues with the Test Team. Good for complex websites, where other tools may sometimes fail. Unlike Selenium, it does not make use of XPath to identify objects; and can be used across browsers for recording tests.

Disadvantages: Only used for Web Testing for now. [not sure if it has been updated with plug-ins for others]. Limited use, thus not many people know about it.


SmartBear SoapUI

Advantages: Great tool for Web Services Testing from Smart Bear.

Disadvantages: Only useful for Web Services Testing. (but this might be an advantage, as they plan to make this a separate activity)


Advantages: Good tool, very similar to HP QTP, with a good interface and price. Overall good for Flash/Flex, with the included Libraries. SmartBear has a full stable of tools, which if bought together may be helpful in pricing and overall deployment and support. Uses VBScript/VBA for coding. People with QTP Experience may find it easy.

Disadvantages: Flash/Flex testing is still not very stable, sometimes fails to recognize the separate objects.

Microsoft Visual Studio Test Professional

Advantages: Is natively attached to the Visual Studio product line. Great for Cloud and .NET application testing. Good is you have Windows Phone applications. “CodedUI” is an excellent tool for testing cross-browser and web HTML testing. MS does deals to get the testing community to start using their tools 🙂

Disadvantages: Only for MS Technologies mostly. Not good for Firefox and Android. Only uses C# or Python.

Odin AXE Framework

Advantages: Great tool for building a wrapper over the existing tools scripts; actually it converts the tools identified objects into a XML recognizable format and has a great and easily understandable format for Automation testers.

Disadvantages: None that I can think of for now, except the use of a tool is somewhat a compulsory need for the framework created in AXE to work. Odin has done a good job of making the tool robust for Web Testing tools and it is compatible with almost all other commercial tools available.

Tricentis TOSCA

Advantages: Combines the best of Requirements, Test Case Design and Test Case execution, all in one single application. Good when there are business testers who know what the application is doing and there is good documentation available for doing it.

Disadvantages: Not very flexible when it comes to handling of unexpected behaviour within the application. Likes to have a clean interface to run through test cases and offer a “happy” path.

I can provide some more research into the new tools (and some less known but good ones), but the above are some of the common ones in use.

I am not advocating the use of any one tool above and to each depends on what he has worked with and would be comfortable in using.

I cannot Code!!

22 02 2009

The realization struck me, when I tried my hand at some automation that I was planning on doing for my recent project. It was supposed to be a simple Perl framework and it didn’t take me much time to complete the initial design and architecture of what would unfold into a testing suite for some of the new functionality added…

Then came the coding portion of it.. and here is where I got stuck!! How do I do a file open, what to implement – use or require (why?), what is the syntax for a for and a foreach and which variable goes where!!!

It really was an eye opener for me 😦

Journey’s and Life

15 12 2008

It is amazing how things turn out when you wish to go out an explore. Well! I am planning on doing a journey from Delhi to Bangalore along with a friend and get back my car. This is going to take about 3-4 days of travel and around 2100 kms will be covered in the same. The plan is currently very ambitious and I am not sure if we will ever conceptualize it in reality… but rhe thought is present.

We have made preparations by making sure of the route and also of the GPS system that we intend to use. It seems to be that Nokia phones have an excellent feature of OVI Maps, which can very well act like a GPS for Indian terrain! It now remains to be seen if we ever get this thing out of the drawing board and onto actual implementation 😀

Automating the Performance

7 12 2008

These days I have been quite occupied with a new development for checking the performance of applications and products. In this field, I came across a lot many applications (SLAMD, HTTPLoad, LoadRunner, etc.) and many new ways to check the performance based on API’s and white-box. It is interesting to know what a simple (single) line change and make to the performance of an application. Also, came to know about Mutex and Semaphores. These are also in some ways instrumental in making an impact on the performance of the tool based on how and where locking is made use of.

In the course of my investigations, I came across a great site for Q&A: This has been started by Joel (of joelonsoftware fame) and Jeff Atwood, along with a few other of their colleagues.

Back to the performance tools, I think SLAMD does a good job of checking web based applications. While for command line based ones or API checkers, I think the best way is to go with customized applications, which you build on your own, or start asking for help in the form of questions or professional consultancy 🙂

UI Automation Tools on the OSS Platform

17 07 2008

People seem to be getting back to the UI testing tools these days. UI is gaining ground and the commercial products available for the same are again coming into focus. There is a lot of hype being created around tools like Selenium and these are actually living upto that hype. Thoughtworks is one company which does consultation for the Open Source tools in use today and they have an intensive Selenium knowledge base. People should be encouraged to use these tools, which are although OSS, they compensate in man hours spent if we take the training consultation from companies such as Thought Works.
Another one which has come up from the same stable of is CubicTest. This uses the Selenium and Eclipse as a base and builds on top of that. A very useful tool, once the engineer knows how to use it. The learning curve for these tools may be high now due to the lesser penetration, but it is growing and soon might surpass the commercial ones soon enough.
The new Selenium and CubicTest versions are said to be major enhancements and robust stable ones.
So get onto this if you have a Java or Perl background or even with PHP now