Best Practices for New Test Organisations

 

10 best practices for new test organisations

Best practice document on the Test Assurance priorities in new organisations such as technology start-ups, small organisations and even in new programme initiatives to the deliver a predictable customer experience that will create market differentiation and result in continued business viability.

1.    Assess the test maturity & Develop of a test maturity roadmap

To cope up with your growing IT needs you need a good test organisation which will expand and mature with those needs. Therefore the first principle of best practice is same as start point of any other business improvements – as assessment of your test organisations maturity level. It is advised to use industry accepted standards such as TMMi or CTP. Essentially you are trying to explain a realistic view of where you are [however bad it is]. Then a time scale of getting to various milestones in the industry growth model. Nothing works in isolation so it goes without saying that you need a mapping to your business plans and investment availability. This is your signed off plan with your senior management, so you have the executive sponsorship and support in this journey.

“If you're relentlessly focused on lowering cost, you'll quickly become oblivious to opportunities to increase value.” Michael Bolton on Twitter.

2.    Develop a set of improvement metric

Once you have a plan in place you need to devise a few measures which you can use to articulate month by month [sprint by sprint in Agile world] progress of your test organisations’ capabilities. The simpler the metrics the better it is. We are not inventing mason particle here. We don’t need trunks of data. Simple measures which is appropriately contextualized for its accuracy, that is what we should be looking for. We don’t want to be slaves of our own makings. All we need is a medium to communicate.

“Storytelling strikes me as a more powerful tool than quantification or measurement for what we do“, Mr Alan Cooper, on Twitter

3.    Communicate the strategic nature of initiatives AND responsible/accountable

You need that ‘here we come’ email at some point in time to communicate the initiatives and how important they are to business plans. It doesn’t matter its stating the obvious and perhaps you tell people what they already know. This is the essential marketing campaign you need, to get the new test organisation going. The new team, the sponsors and what they are going to do in the next few months should be clear. It is important that it comes from one of your C-levels. Get the initiatives into the news letters at organizational level. Sprinkle the bits and pieces in all possible channels. The communication on who is responsible / accountable is more from a motivational and recognition standpoint rather than pin pointing someone’s responsibilities. It is the recognitions and accolades which makes the difference for your employees. We want to see our test owners inspired by the goals we set, and that inspiration comes from recognition of their efforts. 

“It’s more important to have the right people involved than it is to follow the process exactly right – Rex Black

4.    Develop an integrated test approach [Organisation Test Policy]

Test Policy! Ask mature test organisations for this document and they will scramble around to get one and are more likely to say its somewhere in the attic and will get you when Santa is in the town next time. But that doesn’t mean that those organisations didn’t have one such document when they started. Its just that the processes and basic remits have formed the part of organisational collective memory and common understanding. In the new organisations we still need to create and cross reference it with various other policy documents such Release Processes and System Development Approaches. We have used the term integrated test approach to allude to the fact that you would need to articulate your approach to testing the service and customer development life cycle [acquisition to retirement; base to toppings; inclination for use to billing; validation to readiness]. This will cut across various test types and levels. The methodologies and standards to be followed put in place via this document. This document does is serving as a simple 10 pager for people to get a quick view of test assurance blueprint.

“The amount of formatting and verbosity applied to a test plan is inversely proportional to the amount of actual good testing represented by such plan.”, David Gilbert on his blog.

5.    Obliterate QA managers and Test managers; reincarnate them to Business Test Assurance Managers

Test organisations do not exist for themselves; they are part of the wider business case. If business can’t offer a good service or bring the services to market in time to differentiate from competition; there is no continuity for the business. In that wider context, the attitudes at the test organisations need to be aligned to that of Business Assurance Managers. This means that the test management needs to understand and be incorporated into the communications and business plan updates which is ongoing at the management level. Test managers can’t [Oops! Business Test Assurance managers] be the sensitive and support like people when they do not know the relevance and detail of the initiatives.

“If you're gonna manage testers or any highly cognitive work, then you need to participate in the work.” Interview with James Bach

6.    Envision a Test Tools Infrastructure and roadmap

This is quiet an obvious one. Test tools are required and none makes explicit case for them. The emphasis here is on the getting a holistic approach in place rather than evolving the tools landscape over a period of time by trial and error. A single, holistic, centralized test tools approach helps the tools with best functional coverage being selected [and avoiding investing on multiple tools with similar capabilities]. Think also in terms of the need for projecting the costs of building and maintaining a tools landscape which can be substantial for new organisations. Rather than piecemeal investment proposals to be submitted every quarter; a start proposal would be beneficial for management.

 

Test Tools seldom comes as direct pluggable components. They will require process changes to the way teams function. It will be particularly useful to have the tools with test process impact to be zeroed in first and implemented so that the later flux on changing the processes is reduced.

“(pure) ~Scripted and ~exploratory tests are two ARTIFICIAL ends of testing continuum. In real life I am doing neither. I just do testing.“, by Vipul Kocher on Twitter.

7.    Invest in a solid test automation approach 

Automation is not limited to testing alone, it forms integral part of the development approach in the form of unit test automation, code build, code deployment and configuration management. In the context of testing it refers to the element of testing you can automate so the manual test effort is reduced and the organisation has the flexibility to run them as and when needed. There are a whole suite of areas which can be automated for example; data comparators, functional testing, performance testing, device testing, security testing and so on. Automation is equivalent to a systems development in itself. It needs an investment in tools, frame work, specialists and automation effort itself. It needs to stay relevant to the build being tested therefore needs maintenance as well. We need a business case to be built to ensure ongoing executive sponsorship. It would be apparent that as the test needs grow the test organisation will not scale up without effective automation

“When it comes to automation - Under Commit and Over Deliver” – Anonymous.

8.    Develop a training Plan for your Test Engineers

New organisations need to ensure that they develop and support the training needs of employees in general, not just Test Organisations. An adequate and well laid out training plan will keep your test engineers focused on their jobs and enhance their speed of professional growth. In general a growing organisation is also one which learns or vice versa. Not just learn; but as well share the learning. Just to keep this simple you need an emphasis on the certifications and industry benchmarking for your work force. It should be rewarded; it should be recognized; to increase the overall employee loyalty and happiness.

“... we have as many testers as we have developers. And testers spend all their time testing, and developers spend half their time testing. We're more of a testing, a quality software organization than we're a software organization.”

 Bill Gates on trustworthy computing (Information Week, May 2002)

9.    Research & Develop a set of Test Libraries and Risk Weights associated

Surprisingly to our understanding the functionalities various businesses develop are lot similar. The way a mobile application or game will work has a lot of common functional elements regardless of the end use or business application. Similarly, there is lot in common for wireless devices regardless of the type of bandwidth or communication technology used. This is the starting point for the test libraries. Its beneficial to pro-actively look for sources of such libraries and start weighing the relevance to your application landscape. This is not to put the ownership of gathering business requirements onto test designers and test engineers, but to engage in understanding a variety of common usage situations for your applications or devices. The benefits are multifold; primary one is speeding up the maturity of your product in the market. We need to avoid or reduce the constant field reporting of functionality failings and ensuing panic. It goes without saying that the library built needs a good understanding of the impact of the failure should such scenarios occur, so Business Test Assurance Managers know that the potential fall out of not executing them and thus leads into the point on Risk Weights.

“In risk based testing, it's more important to have the right people involved than it is to follow the process exactly right.“, Rex Black

10. Develop an accounting token for every test activity to understand the cost of quality

Finally, testing needs is a business as much as an IT function. To be reckoned you need to ingrain the cost of the testing into your plans and projections. The token could be something as sophisticated as test point or man hours or man days. Test point token is preferred because it abstracts the relative value of each activity in test function and helps translating the financial cost of the testing at an appropriate management level. The token may not be optimal to start with; it will need calibration over few releases to get to a reliable measure of the cost. Once established it starts telling you where the test investments should go based on the complexity and business criticality of the function. More importantly every time management approves a testing budget its apparent what will be delivered at the cost. Certainly the method of adding up the test engineers and few other overheads is not an appropriate solution. At some point organisations think of what needs to be done in house and what needs to be outsourced and at such decision points a measure of test cost relating to the type and volume will be invaluable as well.

“If you don’t care about quality, you can meet any other requirement” – Gerald M. Weinberg