10 best practices for new test organisations
Best practice document
on the Test Assurance priorities in new organisations such as technology start-ups,
small organisations and even in new programme initiatives to the deliver a
predictable customer experience that will create market differentiation and result
in continued business viability.
1.
Assess the test maturity &
Develop of a test maturity roadmap
To cope up with your growing IT needs you
need a good test organisation which will expand and mature with those needs.
Therefore the first principle of best practice is same as start point of any
other business improvements – as assessment of your test organisations maturity
level. It is advised to use industry accepted standards such as TMMi or CTP. Essentially
you are trying to explain a realistic view of where you are [however bad it
is]. Then a time scale of getting to various milestones in the industry growth
model. Nothing works in isolation so it goes without saying that you need a
mapping to your business plans and investment availability. This is your signed
off plan with your senior management, so you have the executive sponsorship and
support in this journey.
“If you're relentlessly focused on
lowering cost, you'll quickly become oblivious to opportunities to increase
value.” Michael Bolton on Twitter.
2.
Develop a set of improvement
metric
Once you have a plan in place you need to
devise a few measures which you can use to articulate month by month [sprint by
sprint in Agile world] progress of your test organisations’ capabilities. The
simpler the metrics the better it is. We are not inventing mason particle here.
We don’t need trunks of data. Simple measures which is appropriately
contextualized for its accuracy, that is what we should be looking for. We
don’t want to be slaves of our own makings. All we need is a medium to
communicate.
“Storytelling strikes me as a more
powerful tool than quantification or measurement for what we do“, Mr Alan
Cooper, on Twitter
3.
Communicate the strategic
nature of initiatives AND responsible/accountable
You need that ‘here we come’ email at some
point in time to communicate the initiatives and how important they are to business
plans. It doesn’t matter its stating the obvious and perhaps you tell people
what they already know. This is the essential marketing campaign you need, to
get the new test organisation going. The new team, the sponsors and what they
are going to do in the next few months should be clear. It is important that it
comes from one of your C-levels. Get the initiatives into the news letters at
organizational level. Sprinkle the bits and pieces in all possible channels.
The communication on who is responsible / accountable is more from a
motivational and recognition standpoint rather than pin pointing someone’s
responsibilities. It is the recognitions and accolades which makes the
difference for your employees. We want to see our test owners inspired by the
goals we set, and that inspiration comes from recognition of their
efforts.
“It’s more important to
have the right people involved than it is to follow the process exactly right –
Rex Black
4.
Develop an integrated test
approach [Organisation Test Policy]
Test Policy! Ask mature test organisations for
this document and they will scramble around to get one and are more likely to
say its somewhere in the attic and will get you when Santa is in the town next
time. But that doesn’t mean that those organisations didn’t have one such
document when they started. Its just that the processes and basic remits have
formed the part of organisational collective memory and common understanding. In
the new organisations we still need to create and cross reference it with
various other policy documents such Release Processes and System Development
Approaches. We have used the term integrated test approach to allude to the
fact that you would need to articulate your approach to testing the service and
customer development life cycle [acquisition to retirement; base to
toppings; inclination for use to
billing; validation to readiness].
This will cut across various test types and levels. The methodologies and
standards to be followed put in place via this document. This document does is
serving as a simple 10 pager for people to get a quick view of test assurance
blueprint.
“The amount of formatting and verbosity applied to a test plan is
inversely proportional to the amount of actual good testing represented by such
plan.”, David Gilbert on his blog.
5.
Obliterate QA managers and Test
managers; reincarnate them to Business Test Assurance Managers
Test organisations do not exist for
themselves; they are part of the wider business case. If business can’t offer a
good service or bring the services to market in time to differentiate from
competition; there is no continuity for the business. In that wider context, the
attitudes at the test organisations need to be aligned to that of Business
Assurance Managers. This means that the test management needs to understand and
be incorporated into the communications and business plan updates which is ongoing
at the management level. Test managers can’t [Oops! Business Test Assurance
managers] be the sensitive and support like people when they do not know the
relevance and detail of the initiatives.
“If you're gonna manage testers or any highly cognitive work, then you
need to participate in the work.” Interview with James Bach
6.
Envision a Test Tools
Infrastructure and roadmap
This is quiet an obvious one. Test tools are
required and none makes explicit case for them. The emphasis here is on the
getting a holistic approach in place rather than evolving the tools landscape
over a period of time by trial and error. A single, holistic, centralized test
tools approach helps the tools with best functional coverage being selected [and
avoiding investing on multiple tools with similar capabilities]. Think also in
terms of the need for projecting the costs of building and maintaining a tools
landscape which can be substantial for new organisations. Rather than piecemeal
investment proposals to be submitted every quarter; a start proposal would be
beneficial for management.
Test Tools seldom comes as direct pluggable
components. They will require process changes to the way teams function. It
will be particularly useful to have the tools with test process impact to be
zeroed in first and implemented so that the later flux on changing the
processes is reduced.
“(pure) ~Scripted and ~exploratory tests are two ARTIFICIAL ends of
testing continuum. In real life I am doing neither. I just do testing.“, by
Vipul Kocher on Twitter.
7.
Invest in a solid test automation
approach
Automation is not limited to testing alone,
it forms integral part of the development approach in the form of unit test
automation, code build, code deployment and configuration management. In the
context of testing it refers to the element of testing you can automate so the
manual test effort is reduced and the organisation has the flexibility to run
them as and when needed. There are a whole suite of areas which can be
automated for example; data comparators, functional testing, performance
testing, device testing, security testing and so on. Automation is equivalent
to a systems development in itself. It needs an investment in tools, frame
work, specialists and automation effort itself. It needs to stay relevant to
the build being tested therefore needs maintenance as well. We need a business
case to be built to ensure ongoing executive sponsorship. It would be apparent
that as the test needs grow the test organisation will not scale up without
effective automation
“When it comes to automation - Under Commit and Over Deliver” –
Anonymous.
8.
Develop a training Plan for your
Test Engineers
New organisations need to ensure that they
develop and support the training needs of employees in general, not just Test
Organisations. An adequate and well laid out training plan will keep your test
engineers focused on their jobs and enhance their speed of professional growth.
In general a growing organisation is also one which learns or vice versa. Not
just learn; but as well share the learning. Just to keep this simple you need
an emphasis on the certifications and industry benchmarking for your work
force. It should be rewarded; it should be recognized; to increase the overall
employee loyalty and happiness.
“... we have as many testers as we have developers. And testers spend all
their time testing, and developers spend half their time testing. We're more of
a testing, a quality software organization than we're a software organization.”
Bill Gates on trustworthy
computing (Information Week, May 2002)
9.
Research & Develop a set of
Test Libraries and Risk Weights associated
Surprisingly to our understanding the functionalities
various businesses develop are lot similar. The way a mobile application or
game will work has a lot of common functional elements regardless of the end
use or business application. Similarly, there is lot in common for wireless
devices regardless of the type of bandwidth or communication technology used.
This is the starting point for the test libraries. Its beneficial to
pro-actively look for sources of such libraries and start weighing the
relevance to your application landscape. This is not to put the ownership of
gathering business requirements onto test designers and test engineers, but to
engage in understanding a variety of common usage situations for your
applications or devices. The benefits are multifold; primary one is speeding up
the maturity of your product in the market. We need to avoid or reduce the
constant field reporting of functionality failings and ensuing panic. It goes
without saying that the library built needs a good understanding of the impact
of the failure should such scenarios occur, so Business Test Assurance Managers
know that the potential fall out of not executing them and thus leads into the
point on Risk Weights.
“In risk based testing, it's more important to have the right people
involved than it is to follow the process exactly right.“, Rex Black
10. Develop an accounting token for every test activity to understand
the cost of quality
Finally, testing needs is a business as
much as an IT function. To be reckoned you need to ingrain the cost of the
testing into your plans and projections. The token could be something as
sophisticated as test point or man hours or man days. Test point token is
preferred because it abstracts the relative value of each activity in test
function and helps translating the financial cost of the testing at an
appropriate management level. The token may not be optimal to start with; it will
need calibration over few releases to get to a reliable measure of the cost.
Once established it starts telling you where the test investments should go
based on the complexity and business criticality of the function. More
importantly every time management approves a testing budget its apparent what
will be delivered at the cost. Certainly the method of adding up the test
engineers and few other overheads is not an appropriate solution. At some point
organisations think of what needs to be done in house and what needs to be
outsourced and at such decision points a measure of test cost relating to the
type and volume will be invaluable as well.
“If you don’t care about quality, you can meet any other requirement” –
Gerald M. Weinberg
