Utilizing a Rubric to share expectations of the QA Engineer Role

Why Rubrics

When I was teaching English in South Korea I worked with a professor that taught me about using rubrics for grading our student’s performance. It is difficult to measure each others performance especially in the field of software development. Rubrics create transparency around performance and expectations of a role. I created a rubric to increase visibility and teach others about what Quality Engineers can bring to the table for their teams. I have worked in a few different roles and businesses and many times encountered a new role that I had no idea what they did and how they did it.

Creating Transparency

My intial attempt at creating a rubric came from thinking deeply about my own performance on a backend heavy team. I thought about the different ways that I contributed and where I could improve and extrapolated that out to a few different categories. I began to share that out with my peers and began to collectively shape this document to represent a role that reflected an average person that might work on any of our development teams. This became a jumping off point for Quality Engineers to either use it as it is or craft their own version to share with their teams.

Criteria	Excelling	Intermediate	Novice
Relationships with different roles. POs, Developers, TA	Able to give excellent and timely feedback. Understands and weighs needs of team members.	Able to give some level of feedback and is aware of other role’s needs.	Relationships are superficial. Difficult to give feedback and understand role’s point of view.
Influencing	Able to leverage mistakes into opportunities for others to learn from. Builds influence across business.	Able to effectively influence team members on an idea. Aware of various methods to build influence.	Finds it difficult to share a point of view or relate and uses only logical details to support an argument.
Code Ability	Able to step in to create complex test automation or assist with day to day tasks alongside developers. Read, writes, and debugs code. Points out risk.	Able to read, write, and debug code. Can point out where common issues increase risk for a project.	Able to read but difficult to understand limitations of code and where problems can occur for developers.
Bug Selling Ability/ Risk	Able to raise the right level of awareness on a bug or impactful change and gets it prioritized to fix.	Raises awareness on bugs or impactful changes.	Bugs are entered but not followed up on with product owner. Level of risk is not considered.
Design	Asks insightful questions in design phase about product and testing. Questions whether feature is adding value. Questions how to test and why to test. Identifies potential business/technical risks.	Questions how to test and why to test.	Questions in design only discuss how to test and not why.
Developer Test Coaching	Developers feel empowered to write different levels of testing and push the boundaries of how tests are written and for what reasons. Explains test pyramid and weigh the levels of risk, maintenance, and ROI for specific tests.	Able to explain testing pyramid to developers and weigh the levels of risk, maintenance, and ROI for specific tests.	Tests are written without regard for maintainability or level of testing required.
Business Domain Knowledge and Subject Matter Expertise	Excellent knowledge on end user needs and how the business works. Communicates potential improvements and risks from the end user and business perspective.	Adequate knowledge on end user needs and how the business works.	Limited knowledge on end user needs and how the business works.
Process Improvement	Guides and oversees implementation of new tools/processes to remove overly redundant and manual tasks. Finds and highlights areas of improvement.	Points out and suggests tools/processes to remove overly redundant and manual tasks. Takes part in some level of implementation.	Process improvement suggestions and implementation are limited.
Test Coverage and Risk	Test analysis is thorough and efficient. Engages team in risk analysis and provides recommendations. Ensures the team understands test coverage and risks.	Test analysis does not result in significant gaps. Adequate communication and visibility around test coverage and risks.	Test analysis may result in significant gaps. Limited communication and visibility around test coverage and risks.

Customizing the QA Quest Log to a team

Team 1

Criteria
Party Building	Communicates frequently and effectively to connect business and technical perspectives; Gives timely and well thought out feedback; Understands and weighs team member needs.
Bardic Skills	Sings the stories of heroic testing; Empowers Devs to push boundaries of testing; Explains feature risks as well as test reasoning, maintenance, and ROI.
Constitution	Ensures Tests and risk analyses are thorough, efficient, and understood by the team; Encourages and advises on logging and alerting;
Special Ops	Contributes to root cause analysis and troubleshooting; Doesn't just find bugs but also hunts down their source and eradicates them "with extreme prejudice";
Strategery	Asks insightful questions in design phase; Questions whether a feature is adding value and how to test that value; Identifies potential business and technical risks;

Team 2

Criteria
Constitution (Coverage & Risk)	Tests and analyses are thorough and efficient; Encourages and advises logging; Creates effective alerts;
Arcane Knowledge (Business)	Excellent knowledge of business and user needs; Communicates risks and improvements from >1 perspective.
Prioritization	Tests what is most impactful; Doesn't waste time on low-impact tests; Balances all responsibilities well;
Special Ops	Excellent root cause analysis and troubleshooting; hunts bugs to their source and helps to eradicate them;
Technical Ability	Can create complex automated tests; knows risks of code; uses knowledge to communicate and test backend;
Paladin's Zeal	Can create complex automated tests; knows risks of code; uses knowledge to communicate and test backend;

Giving a Voice to the Team and Testing

I am proud of my team members and taking this idea on a path of its own and customizing it. As much as we often want to develop a common standard for every group to use the fact of the matter is that there is some level of customization each team requires in maintaining their software. We should come to terms with constantly re-evaluating our skill sets within our teams.