What to Automate?

19 09 2011

I had this interesting conversation on Automation the other day with my colleagues at my new job. It started off fairly innocently on how the automation should be thought out and what needs to be done to automate. We already have an existing framework and test scripts in place, which very efficiently work and report issues. The problem comes when we need to provide data on what automation actually does. How do you prove the effort spent on automation? You could easily say that it saves time and resources by checking for faults early in the development cycle, but how can we be sure that it actually covers scenarios that check the application? This is where a business requirement document becomes a necessity.

The issue which we increasingly face today is how do we relate the business requirements to what we test. There are a few things which are given in textbooks and across such certifications organizations like ISTQB, which provide information on such. The thing is ultimately it depends on the person who is sitting and working on the application (which most of the Program/Product Managers miss out on) and the person who is writing the scripts to automate the application testing. The best way to figure out for a restricted zone (propriety) application is to go and sit with the users and find out what they use the most (or run a key stroke capture software and see where all it goes, limitations of this later). With a public/global reach software, it is best to give out Beta versions of the same, like most of the Big Organizations do and see what is reported back.

The other end of the spectrum is propriety software, which might not have a user interface (system tools like Compilers is one example). For this the technique is basically to read through the Software Requirement Document/Specification and have confidence in your abilities to decipher the jargon written in those and convert them to simple English. I got my initial training on writing code on these only, and the golden rule for us was [in a summary]:

  • Read through the document and write what you have understood for each function
  • Create the Algorithm for each in plain English pseudo code
  • Convert these into test cases, and run those test cases
I think the above rules of going through the steps of Why, What and How has helped us out a lot to become what we are today 🙂

My coding exercise!

3 03 2009

So I have embarked on a journey to find my coding skills and try to find where and how I lost them. The journey does not seem to be too long, as I can remember the last time I had coded something was around the time I left my last Organization. Coming into the new atmosphere, I think I somewhere along the line lost the charm to code and that took its toll on my coding skill and brought me to this space.

The exercise began with me trying to understand what was going on under the hood of something which we call the collection of logs from users and their analysis and loading into the databases. The data is then given to the strategic data services, who do the data mining and get to work on analysis of how to make things better for our customers so that their ROI goes up and becomes better.

My main duty was to take care of this pipeline (or various pipelines for the different ways in which we collect the logs) and make sure it keeps flowing and garning more data about the various activities going on, in this somehow I lost the interest to code and innovate on things… [i think the right word was inovation]


8 04 2006

Back to the crux of all things – documentation. I suddenly realized why we need the documentation when suddenly a lot of people started to leave and the things that they had started here went for a toss as they have as yet not created any documentation in a Word Doc but only in their minds…

My PM came over today and the thing on his mind was to assimilate and get all the work that had been done by the people, into a single entry point on the version control system and then work on integrating the same for all the products and applications. Currently, the task I am doing is majorly centered on the Windows platform, but now I will have to check on all the products and all the platforms and see where all optimizations can be made so that re-work is not required. It takes a lot of effort and thought to create something but only a little bit of negligence to completely destroy the same. A prime example of the same is the test framework I am currently working on. Now we had a perfect version checker written in Perl, which used to check the version of the product installed by us and the version of the product as given by the executable. But the build and installer person decided that [or rather forgot] that he has to update the version number in the installed package. This made the situation that the version obtained from the package was different from the version given in the setup directory name and again different from what the user was entering it as. So came the problem that when we installed the product, the versions would not match and the product was again uninstalled, but as we were not checking the product version when installing, just the executable file was being checked, we used to install the product again, reboot and then get the version from the command line tool. This was matched with the version given by the user and again a loop as no match means uninstall the product… 😀

The main thing I next need to learn from this was never to trust anything where a human can make an error, as the famous words go, “if anything can go wrong it will” [along with “to err is human”] and the axiom added by my Project Manager from the first organization I worked in – “the thing that goes wrong will happen in front of the customer”, which is this case was the QA team who were supposed to execute the automated framework. They took the build and the process went into a loop… 😦

So coming back to the documentation part now, I was as is by habit creating all these documents, but could not comprehend this eventuality of the loop as I have given above. So exactly what does the documentation do? Well! for the first part, it got us immediately to the solution of this problem. From the given documentation we already knew that something was going wrong in the comparable algorithm as every time we got the message that ‘Version not Match’ and then uninstallation of the product and after that again an installation of the same. On going through the comments for the installation, I realized that I had not catered for human error and thus the solution to the problem was to cater for this, a major thing which I realized for Automation of anything was FAULT TOLERANCE...!! This has to be built into the system, else all our automation efforts fail, if we are just spending time in monitoring the application which has been automated.

As for the documentation part of all the other tasks being done for the automation, I realised that none of the others had any sort of documentation to go with them, except the typical programmers prospective of code and go, the logic and document is within the brain of the programmer who created the wonder of an application in the first place and as I have said earlier an unmaintainable application which if not understood and needs change might be discarded later. Thus, I am now to get down and create a document or ask the people who created these applications in the first place to create the same... [ the second option definitely looks appealing... 😛 ]

More coming' up...!!

Object Oriented Work!!

14 08 2005

Well! High time it has been that I visited my page and continued on my Ramblings…
The past year has been a very stressful time with me on the tenterhooks all the time as I am VERY weak on OO concepts and that is what is the work that I am currently working on during the time that I have been employed in my present company… 🙂

Actually the work began with me being recruited during the interview for a compiler post and then it being transformed to a OO QA role later sometime. I never knew when it hit me that I will also have to go the OO route for me to be competitive enough in the field. But Managers being what they are I was told to complete the work given and start with the current concepts of learning about OO. The concept stage was easy, but implementation took the cake. To add to the injury, we had team mates who thought they knew all the concepts and ideas and wanted us to not learn mor as then we would advance leaving them in the background… 😦

I suppose the work culture that I wanted and had developed is slowing getting eroded away and soon it will be time when I shall again have to resort to the last available weapon with me – CHANGE!!! 😦

Work @ Acmet

15 06 2004


Seems like a long time ago I planned to start with this… Anyways, now that I have settled down to this new job, I think I can start planning and doing atleast (if not all) the things I had planned for…

What I actually wanted to start off with is what am I actually doing the whole day… But, now I think I will relate to the incidents and my learning a new computer language (new for me, Oldie for all those who are there or have used it… )

I start with my first experience with the computer language Java. It was first thrusted on me with the advent of a JVM for an Embedded Platform, which was to be built for a normal training project. This project was basically to start the new MCA (Master’s in Computer Application) trainees who would come to our Company for a basic 6 months of Course training. My Project Manager thought this would be a great way to teach them about the processes that we are following and also get them into the mold which would be followed, if they were to join the Company. Thus, started the great experience… I thought that I would have to start studying Java language and started off with just that… It was a great time as I started with the Java Language reference and then got on to the Java Code… that is where i drew a blank as till this time I was reading only the Preface and the Introduction… 🙂

Well! I think that is enough of my ramblings for today will start again tomorrow…