The Improvement Continuum
Abstract: The Improvement Continuum is a dual-pronged concept, containing both product and personal components, like two heads on the same animal. One head pertains to improvement within a solution, product or service, while the other concerns itself with the human mind, particularly our capacity for learning. This idea states that a viable candidate can never reach a point at which it legitimately plateaus in quality. Therefore, by extension, any perceived quality plateau, intentional aside, must be a product of human misunderstanding or mis-measurement of the current state, rather than a lacking of the candidate itself. For the purpose of this article, the term “candidate” is being used to refer to either a solution, product or service that is currently being used in the market, or the human’s individual capacity to increase mental operational quality through learning as long as that human is not inhibited by a medical condition otherwise. In other words, both products and humans have the capacity for improvement.
Introduction:
It is widely accepted that a product or service, can never reach a point at which it should intentionally arrive at a plateau in quality, unless that particular solution has been sunset. There’s an unseen and undiscussed black hole where sunset products go to die, but to avoid that, improvements must continually be invented, prioritized and implemented. This may seem like an obvious statement, but what’s not so obvious is how to move the entity forward.
In software development, an increase in quality does not simply mean finding and fixing bugs, as that is only one facet of how the product can be improved.
When evaluating the product as a whole, we should decide as a team which areas need the most focus. What areas of improvement is your division good at? Which areas may have been neglected? Do your efforts to improve as a tester align with the current product risk areas? First, we must establish that candidacy for improvement actually exists. In other words, is improvement warranted for a given feature, perhaps at least a quarterly cadence, or more rarely, for the product as a whole, perhaps on at least an annual evaluation schedule?
Determining Improvement Candidacy:
This section only applies to solutions (product and service offerings), not humans, since the latter is a candidate, inherently and indefinitely.
As long as a product or service is a viable offering, then it’s candidacy for improvement remains active. Even software solutions in maintenance-only mode, with a sunset timeline, are still candidates for being improved upon until EOL (the End-of-Life date). Keep in mind, just because something is a candidate for improvement, that does not mean improvement on that solution is mandatory. To understand this, we must look at the four types of improvement candidates that are abandoned.
Abandoned Candidates (Four Types):
Improvement candidates can be abandoned intentionally or unintentionally. How we handle these situations, says a lot abut our maturity as testers. This can be positive or negative, depending on how the abandonment was implemented. If you find yourself in this situation as a tester, take a moment of pause before getting upset. Try to realize that there’s a bigger picture outside of you, and the idea that we are the sole ‘gatekeepers’ of the product is archaic. Michael Bolton better frames this point in his blog post here Testers: Get Out of the Quality Assurance Business. Understanding why a given product is or is not getting attention can greatly help you do your job as a tester. It is easy to become disenfranchised with your product offering if you do not understand the business reasons behind the work being given to your team. Use the criteria below to be more informed about which camp your product lives.
- Warranted Intentional: The term ‘intentionally abandoned improvement candidate’ sounds like it’d always be a negative, but this can in fact be a sound business decision, thus warranted. In this case, product and upper management within the business has compared the risks of abandoning improvement from both a financial and reputation perspective. There are many sub-level considerations that play into each of these. Perhaps the revenue generated on the given product is negligible and efforts would be better focused on another solution or direction. Perhaps the known backlog of issues has been evaluated from a product risk perspective and deemed as a low-impact, and shoring up this backlog to maintain industry reputation would reside within the realm of diminishing returns.
- Unwarranted Intentional: Like the first type, management did at least make an intentional effort to get together and discuss the business needs, but due to a variety of reasons, of which only a small oligarchy may be aware, the product offering has abandoned without justified cause. Unfortunately, an unwarranted abandonment of a given product offering usually does not come with much transparency down to the teams. It’s may not even be a top-down decision, so tech leads and architects may have been involved, but it is possible that an unwarranted abandonment still took place in favor of a different alternative. Many times, unwarranted abandonment cannot be proven by those opposing the decision until it is too late. For example, the new offering craters after a few years in the market, but the previous solution would still be thriving. In this case, it may not be a matter of an oligarchy making the decisions, or lack of transparency, but rather a major miss on industry expertise and demand forecasting. This can sometimes plague startups that hire amazing talent with incredible knowledge and good intentions, but lacking in industry wisdom through experience.
- Warranted Unintentional: This is similar to the first type, except that external factors forced the abandonment. In the case of an innovative idea that does have a market, this usually only happens when a major mistake is made that threatens our humanity. For example, public exposure of a security hole in a financial product that shows it is storing PII (e.g. Home address, SSN, DOB, etc.) in clear text in an unencrypted database. This can cause irreparable reputation damage that can take a product out of the industry almost overnight, no matter what damage control may happen after the fact. You could argue that this is unwarranted, but that position is based on the human notion that everyone gets a ‘second chance’, but that often does not exist in many industries. Take a defibrillator, a medical device for example. What if the initial model from a new startup killed patients in some edge cases? As a startup, recovering from the resulting legal situation would be close to impossible. Now, this also happens when the product never had a market to begin with, thus the initial investment was done based on good intentions rather than research of data points within the target industry. However, this usually manifest in one way or another before a mass go-live since scarce target clients would be one of the obvious indicators of a product that is headed in this direction.
- Unwarranted Unintentional: This is the most rare type of abandonment, since smart intuitive ideas usually thrive in one vein or another. If in fact the product is sound and innovative with a need in the market, then this somewhat requires the planets to align, in that three factors must be present: External forces, sometimes even from those who wish to see a competing product fail. Bad salesmanship, marketing, and demographic targeting based on the product’s features, thus no traction is gained in the market. And finally, this requires an internal framework of individuals who lack ownership in the product.
Quality Plateaus:
Due to the nature of how improvement works, both within a product or a human, quality plateaus can be both intentional and unintentional. We’ve discussed various types of improvement abandonment, and now similarly need to discuss the different forms of quality plateaus that can take place. A product quality plateau can only be justified in the case of the Warranted Intentional abandonment case described earlier. A quality plateau within a human can only be justified in the case of rare medical conditions, but ethically there’d never be an intentional case of such condition, thus is it an outlier, external to this discussion.
Plateaus Within Products And Services:
Other than these edge cases, it has been established that an unaffected entity (see Abstract) cannot legitimately arrive at a quality plateau. By extension, any perceived quality plateau within a product must be a symptom of misunderstanding or mis-measurement of the current state in order to make that determination, rather than a failing of the product itself. This also means that such plateaus always require the need to be remedied.
Plateaus Within The Human:
In the case of the human mind, a tester’s skill is a qualitative property, and cannot be mathematically or objectively measured; therefore, a quality plateau would be subjective at best. If this plateau does exists within a tester, then it must be dynamically linked to an ethereal ‘learning to date’ + ‘ongoing learning’ measurement to equivocate to some qualitative scale or understanding. Most of the time this can be due to many easy to identify (and easy to remedy too) factors, such as: work ethic, laziness, lack of resources, poor management, misdirection and misinformation, etc. These problems are age old though, and can be fixed. However, for those who have reached a state of Unconscious Competence, this can be a very legitimate concern.
With that said though, these people are few and far between within the testing community and none of them of course have mastered all areas of being a good tester either.
The Perimeter Assumption:
Something else to be aware of when it comes to learning in various areas of testing, is The Perimeter Assumption. This is something that many struggle with when it comes to testing and learning. This is the idea that as long as I know the most important items (the extreme edges/test max capacity), and I understand the general framework, then I don’t need to worry about the little things (other considerations within those boundaries). This is something that is troubling but still influences us as testers. It can make us comfortable and complacent in our learning, if we are only worried about the most extreme scenarios when testing. For example, when testing a credit application, we focus on making sure the form submits but might miss that some negative testing exposes a major flaw in the website, exposing PII stored in the database. Remember, not all showstopper product risks are found from testing in high-risk areas.
Time Management:
Learning is hard. If it weren’t hard it wouldn’t be beneficial, nor have the allure like it does for so many. I used to say that time management was my biggest roadblock, but then realized that I created that problem for myself, so I needed to stop complaining about it. Learning is liquid. By saying this, I mean that learning is this huge phase space that has no boundaries, so it is easy to create insurmountable learning obstacles that we never address. Brian Kurtz, one of my mentors, often says, ‘If you give me one hour to test something, I’ll use that entire hour. If you give me one week to test that same item, then I will use the entire week.’ The same is true with learning; we will fill the time given with testing. However, there’s so much to learn and not all of it is valuable to us as testers. We have to prioritize what should be learned since business priorities force us to time box our learning to an extent.
So, I like to use this ‘gate’ visual to describe how we self-sabotage our own learning, in hopes that this might help others become more aware of their own potentially self-imposed learning roadblocks:
It may sound ironic, but unlike other roadblocks in my life, learning roadblocks have little to do with learning itself being the problem. Product feature roadblocks for example are usually based on knowledge about that feature being needed to continue development and testing. With learning, the roadblocks tend to come from all other angles, and this is simply because I have traditionally prioritized learning after all my other duties.
As humans, we will naturally seek to fill our time, and typically we will fill it with items that are familiar to us. We might not even enjoy some of these items. For example, excessive meetings can sometimes creep up in the scrum process; however, we get into a cycle of what we believe is expected of us and rarely challenge it. Sure, we challenge acceptance criteria, developers, bug reports and product management, but at some point through the sprint, testers have satisfied some of these priorities, yet continue to use the remaining team priorities as reasons why individual learning cannot be achieved.
Sometimes, we must work with our team to evaluate schedules and product risks, in order to open these gates consciously so that we can target the learning we want done. We must reprioritize, and minimize the time it takes to address some of these expectations. If we time-box our activities, then we’ll have time to address learning on a continual basis, and set aside time within within each sprint for this. So, stop self-imposing these barriers on yourself. We invent this structure, then complain and beat ourselves up when we don’t get to do the learning we want. When asked why we haven’t focused on using a certain test model or pursued reading that book about testing we said we would months back, we pass it off as not being good at time management. Make learning as a tester, one of your main priorities, and if you have to, work with your Scrum Master, Product Owner and Developers on your team to build time into your sprint to prioritize learning just like you would a user story.
Constantly improving as a tester, also involves a great amount of humility to realize we do not have all the answers. Imagine a contractor being called by a customer and they walk into the house with only a nail and a hammer, under the assumption they can handle anything the homeowner might throw at them. Well, when stated like that it’s a completely ludicrous assumption on the part of the contractor. So, why do we do this as testers? We try to take on all testing scenarios with our current knowledge set, but that set is just like having an inadequate toolbox for the jobs put before us. We need to ‘Fill our mental toolbox’, as my collegue Brian Kurtz says, for better ways to address this common pitfall.
Action Items For Testers:
Now that you have this knowledge, how do you use it? How does being aware of ideas like improvement abandonment and quality plateaus actually help you in your day to day to better understand how to continually improve? Ideally this article serves as jumping off platform. Awareness of a problem is the first beneficial step to moving your mental state out of Unconscious Incompetence, which is the “I don’t know what I don’t know” state. What’s the barometer to know if you are in this state? Simply ask yourself if this article came across a little heavy. Does this information seem confusing or foreign to you? If so, then perhaps it’s worth investigating if you genuinely desire to become a better tester, and combat any potential learning plateau of which you may or may not be aware.
So, where do you go from here? Read Quality Concepts and Testers Tell A Compelling Story. It is our job as testers to “cast light on the status of the product and its context, in the service of our stakeholders.” – James Bach. We can only do this effectively if we continually educate ourselves on how the product works, and continually improve our own mentality when it comes to how to test. If you are testing for the sake of testing, and simply present in your job to collect a paycheck, then I encourage you to take a introspective look at the reason why. Our responsibility as testers is to the product and its stakeholders, so ultimately if you are occupying a test position within a company, but know you are lacking that passion to be a constant learning for the betterment of our stakeholders, then it may be time to evaluate if testing is your true calling.
Conclusion:
The improvement continuum exists because there is no true zero. Conversely this also means there is no true 100%. In short, quit trying to quantify things that are only qualifiable; rather, concern yourself with identifying the actions needed to reach the next step on the infinite staircase of learning as a tester.
