My Testing Journey

Abstract: This is a personal experience story about where I started as a tester and how I have grown through my experiences and interaction with various mentors along the way. I’ve moved from a gatekeeper to an informer, mainly due to the influence of some smart minds along the way that took their time to invest in me. I’ve shifted paradigms along the way, pretty drastically since my formal entry intro into testing in 2005, and believe I am in a much more mentally sound and intentionally aware state today.

The following timeline is an account of my own personal journey as I moved through my testing career. It is my hope that by sharing this, I give you some context for not only where I am coming from, a barometer of sorts by which to judge all my other content, but also some perspective on how I might engage with you on various topics should we have discussions from time to time. I believe that learning about each other’s personal history and experiences can allow us to have more meaningful conversations and hopefully minimize the chance that we will talk past each other. I believe putting this information out there publicly is congruent with my guiding light of service to others which is my defining heuristic for determining my actions.

Pre/2005:

Early on in my testing career I simply tested what I wanted to test, and thought that was enough. I had no intention of reporting out on my testing other than writing bug reports, unless someone specifically asked, and even then it was only a shallow, non-compelling verbal response. It’s safe to say that I had my head in the sand when it came to learning new things and while I had good intentions, I did not take the responsible steps I needed to in order to be intentional about my learning. I used to champion my somewhat OCD-nature as a powerful swords. Finding everything was my goal and I had strict tunnel vision that required everything to be pixel perfect; this was obviously at a time when I thought perfection was something actually attainable. To make it worse, this endeavor did not come with any logical reason or continual evaluation of if the work I was doing was actually worthwhile. “Was I doing good testing?” was not a question I asked myself on a daily basis, as I thought good intentions were enough. I had little, if any awareness of the term “product risks” as I only wanted the user interface/backend to work in the way “I” thought it should. After all, “I am the tester, the gatekeeper, the last line of defense; therefore, I know best right?” Well, I now believe I was living under a detrimental paradigm; detrimental to the product, my team and frankly one that stifled my own advancement in the skill-craft of testing. Today, if I were to evaluate my 2005-self as a tester, I would tell him, “Under you current mindset, it will not be possible for you to do good testing,” and since using the word “good” is a value judgement coming from my future self, I believe there may have been a meaningful impact on the 2005-me. Anyone got a time machine?

2006:

At this point I moved from a test-everything mode, into a justification mindset, which happened to correspond to moving to a new company and working on a very different product platform. Was this coincidence? I think not. I felt I had to forever prove I was doing my job, but not for the sake of making the product better or informing my stakeholders. Instead, I was doing this for completely selfish reasons that probably ended up misleading my stakeholders and fooling myself more than it was helping. Again, I may have had good intentions all along, as I stated earlier, but you know what they say about those. This selfishness and tendency to mislead was not intentional in nature, but was akin to the blind leading the blind. I was still not up to the level at which I would claim I was doing good testing, because I naviaged uncharted waters with a very limited tool-belt. I made no effort to find the tools needed to alleviate those gaps in my thinking, unless they were already available. During this time I only tested to confirm what I already believed or suspected (see: Confirmation Bias) rather than practice intellectual humility to increase my knowledge and effectiveness as a tester. My reasons for testing were still flawed: I tested for the sake of statistics, higher bug count numbers, etc. So, I would generate reports on what I tested and the outcomes, but these were extremely biased, and made me out to be a hero tester, as if I were solely responsible for the product and code quality when I was never even involved in the creation. At this point, I was still a long way from becoming a good tester, and had no concept of the business and developer creating the initial quality, and testers simply being the informers of that build process. I would say things like “It’s my job to break things.” No it isn’t. “I create a quality product.” No I don’t.

2008 (early):

I then moved into the paradigm of ‘It is my job to find defects and report on the status of those defects but I still know better on what should be fixed since I am the tester’ and while I explored, testing outside of the acceptance criteria, I was doing ad-hoc/random testing rather than truly structured exploratory testing. I was still missing the boat on the true nature of what it meant to do exploratory testing and I wasn’t even thinking yet on reporting out on what I did not test, only what I did. I was getting to a point where I realized there were more important product risks, but still somewhat ignoring them because I had not fully broken out of the paradigm of the tester being the final “gatekeeper” of the product/release, which we are not. (Read more on that in Michael Bolton’s blog “Testers: Get Out of the Quality Assurance Business“)

2008 (mid/later):

By the end of 2008, I still lingered in the shallow end of the unconsciously incompetent exploratory tester pool (yes, incompetent), and to a fault, in that I reported anything and everything I found, even when 80% might be edge cases or of low value to the customer. I was not weighing product risks and priorities as part of my testing, and many times would insist we fix something before I would close a given story, yet it was outside of the scope of that story. I would try to convince developers they were being short sighted, which may have been true in many cases, since there was a love/hate relationship between Dev and Test at the company I worked for at the time. However, many times I was contributing to feature creep (AKA modifying a feature’s acceptance criteria while it is in progress, thus invalidating estimates and increasing TTL without warranted justification from Product Management). Fortunately (and through some prodding from my main mentor at the time), I quickly learned to log separate bugs for these issues I was finding, and had to come down from my high-and-mighty testing tower to do so. It took some humbling to realize that perhaps I did not know the full picture about the product and business priorities at play, so these bugs I kept reporting were in fact not as critical as I initially advocated.

2009/2010:

Events in my personal life, outside of work, made me reevaluate how I interacted with people. I realized I was rather selfish, and not putting the “other” first. Through some strong mentorship from some wiser people in my life, I turned a corner in how I operated as a human. This personal shift directly affected my work. I no longer saw myself as the ‘one and only’ expert on a given feature. I no longer saw being a ‘tester’ as more important when it comes to product releases. I began to shy away from telling others that testers were gatekeepers, and instead pushed the idea that we have to work with others to determine what is or is not important to the stakeholders.

2011:

I continued to champion product management’s goals over my own, but I still did not do it with any structure. Every new feature I approached was done with good intent, but no consistency. I could not look back on what I had tested from project to project and come up with any kind of internal metrics for myself to rate my efficiency as a tester. Eventually, I created A Personal Metric For Self-Improvement years later, but at this point, I was still ‘in the dark’ on how to be a good tester, and what really constitutes a good tester, but I desired to learn that even though I was not conscious of how to go about it, at least not on the surface.

2012:

It was not really until 2012, that I embraced the idea that testers truly are the informers of risks, not the final decisions makers. Once we inform the stakeholder of a certain bug or risk, and product management says ‘OK, we can push a release even with these bugs’, then we need to step back, as testers, and let the business run the business. This was also when I was first introduced to the context-driven testing community. This is a community that embraces the main principles listed here, on the Context-Driven Testing site, which I came to realize that I embraced these as well. Of all the testing environments in which I have been, I would say that Dealertrack has been my most intellectually beneficial experience. This is due to a number of influences and interactions, but mainly through mentors like Brian Kurtz. Brian also exposed me to the great minds in the community, like Michael Bolton, James BachElisabeth HendricksonGerald WeinbergCem Kaner and others. I refer to this as my “awakening” phase, as a tester, where I moved into thinking much more intentionally and critically about the skill-craft of testing. I used to test things ‘my way’ and that was enough justification for me, but I then realized that it was not enough justification for my stakeholders. Before, I had my own best interests at heart, and I was now realizing, through learning more about the depth of testing, how to have the customer’s best interests at heart, and chasing those superseded mine. I realized that I used to be part of the problem, not the solution, in that I was not actively trying to learn and improve my skill-craft in testing. I was still not using explicit models, heuristics, oracles, mnemonics and other tools that were available in the testing community. Sure, I inherently had my own as we all do, but those models and heuristics were limited. Once I learned about the availability of the external community, I quickly realized how small and obtuse I had been as a tester. I was humbled by the amount of new information and realized that I had a long way to go if I wanted to really consider my work good testing.

2013-Present:

Once the floodgates of possibilities in learning had opened for me mentally, the pathway was clear. I had, and still have, a lot of learning to do before I can consider myself consciously competent in certain areas. I started to get intentional about mapping out my strengths and weaknesses using personal metrics like this or this. I became more self-critical on what makes a good tester, and challenged myself in ways that I had not done before. I dove headlong into the context-driven testing community which cast even more light on my inadequate areas. Through discussions with other much wiser testers, I learned how to increase my skill-craft. I learned how to tell a compelling story to my stakeholders. I learned that learning takes more hard-work that I had previously thought, and like anything worth accomplishing, it does not just happen, you must create it for yourself. I became even more ever-aware of the needs of others around me, and how I could use my knowledge to aid in their endeavors rather than just stay in my own little bubble on my team, my project, my stories. I got used to saying, “Yeah, maybe you’re right and I need to re-evaluate why I believe what I believe,” instead of forging ahead with my own ideas based only on my limited experienced; limited in the sense that I had previously made decisions based only on what I had been through, rather than always seeking counsel and establishing relationships where others could break through my shield, expose me to my own biases, and do it in a way that genuinely cared for my advancement. I realized that I had to rely on gathering the perspectives of others to help shape my decision-making process and harden any actions I took before I carried them out. It also dawned on me that I needed to be much more intentional about my interaction with the online community so that I could reach those outside of my immediate walls. I rejoined Twitter, and created an account to engage with testers (@connorroberts). I began attending conferences such as CAST, Reinventing Testers, etc to engage with other minds in the community. I created this blog where I could share new ideas and tools with others. I started a local meet-up, DFW Testers, where those in and around Dallas/Ft. Worth, Texas could come together to explore the depths of testing and I continue to look for even more ways to engage with others about our skill-craft.

So, where I am now?

I am a work in progress, but I can safely say that I have honed the art of constantly becoming a more competent tester every week. I am involved in the larger community, I crave learning, I engage and collaborate with others, I know that practice won’t make me perfect, but it will make me more competent and as long as I am rapidly experimenting with new ideas, practices, tools and models, then I will avoid my greatest fear as a tester – becoming stagnant and ultimately irrelevant. In short, it is the skill of critical thinking and forever-learning that allows me to be at peace as a tester. I don’t need to fret over a user story, or worry about a feature deadline, because I am at peace with the knowledge that I have filled my tool-belt with things that allow me to do sufficient testing within any time frame. I also remind people that you are the average of the five people with which you spend the most time. Surround yourself with critical thinkers, and people who know more than you. As long as I continue to do that, I know I will have peers in my life who will hold me accountable and challenge my biases. I have no concerns for my own future.

3 thoughts on “My Testing Journey

  1. Thanh says:

    Hey Connor,

    Very good infographic and journey! I can see myself (well, a little bit) in your journey. It’s interesting that you and I have the same starting point, but it looks like you are ahead of me (in terms of knowledge you get and the transformation).

    I also agree with you that critical thinking still plays important role in our testing activities and I’m still learning it….Critical thinking is hard actually.

    Anyway, good post and wish you all success in your journey!

    Cheers,
    Thanh

Leave a Comment

Your email address will not be published. Required fields are marked *