Whenever I go to a QA conference, I’m struck by just how many managers relate how the training is well and good, but they can’t get their companies to implement it. The problem is usually a resource constraint, company culture, or lack of management buy-in. I wanted to understand this a little more, so using my role as founder of the DC Software QA and Testing Meetup, I reached out to my members and found two QA managers interested in discussing their teams.
About the Interviewees
Brian works for a federal government contractor on a development team consisting of 30 software and 5 QA engineers. The QA team has a manual testing background, with 2 of them having 3 years of UI automation experience. They all have a beginner to intermediate level of understanding of Agile processes, and largely worked in a Waterfall SDLC prior to their recent Agile adoption project. 3 of the members are aligned to Agile development teams while the other 2 are functioning in an extended fashion. Sue works on a smaller team composed of both in-house and 3rd party development and QA team members, giving a total of 8 developers and 4 QA (plus a business analyst acting as QA). The QA team is able to conduct both front- and back-end tests for a Web-based product that serves a mix of commercial and government customers. The team members are dedicated to projects and cross-trained to back each other up. Both managers are passionate, long term QA managers. They have worked with a variety of companies and projects and are both hands-on managers.
Describe your Development Cycle
Sue: Sprints are 2 weeks long, followed by a 1 week regression, and then a release. During the sprint, the team runs targeted tests around the scope of the changes and exploratory tests when the feature is integrated. Continuous Integration (CI) is only with the in-house team and it is only from the developer’s local machine to the deployment tool used. There is no CI to a test environment. Brian: Sprints are 2 weeks and the number is dependent on the size of the development project. The development team desires CI, but it has not been implemented.
Talk About Automation.
Brian: We are about 80% manual. Our automation tools consist of mostly UI-based automation through Rational Functional Tester (RFT). We have done a little work with Selenium IDE and are exploring a rollout of Selenium Webdriver as a possible replacement for RFT. We also just started using JMeter for performance testing of one application. Developers do not help with automation. Sue: Right now automation is about 25% and growing. We are using Test Studio by Telerik. Tests are developed using the recorder, and if things change, then the team updates the actual code. While developers haven’t used the tool yet, they do run the scripts for integration testing.
What are Your Biggest Pain Points?
Sue:
QA attends scrums, but they do not feel like they are part of the team. They bring up the issues that block them, but don’t seem to take them seriously. Quite often the QA manager must push the developers to react.
QA isn’t given enough time to do their work.
The requirements don’t have enough detail up front for scripts to be written. QA needs to be more involved upfront.
Manual testing is tedious and redundant. It’s hard to keep morale and interest up.
Interactions with the 3rd party team members can be tough. Poor communication plays a big part in this.
Brian: The biggest challenge is the security protocols used by our customers and how they limit access to our applications being tested. This is a major roadblock with regard to unattended automation. On top of this is the amount of busy-work and compliance with regulations, which can be limiting.
Tell Me About Your Successes
Brian: We do great in that we’ve initiated the first professional QA operation that the customer has experienced. We’ve introduced automation, standardized reporting of defects, capture of critical metrics, and improved the way work is handled across our program. Sue: Through strong training and mentorship, the QA team has become a ‘well oiled machine’. The team feels as one, and since they are cross-trained, they are able to support one another and have some variety in their tasks. Implementing “QA Guidelines”, and teaching the team how to be strong communicators has provided a strong foundation from which the team can build.
Is There Anything Else You’d Like to Bring Up?
Sue: It’s tough to get good QA analysts. While the focus in the industry seems to be on hiring for automation skills, having a QA mindset is the real key. Manual testing can be a bore. You need to find ways to be creative and make it fun — show your team new and interesting ways to test to keep them motivated and interested in testing. Teach them to think outside the box. Help them keep up-to-date with the latest tools, standards, and techniques in the industry. Brian: Get your developers to think of things from a quality perspective. The engineer should dialogue system requirements, specs, problems, etc. with their QA Engineer on a daily basis as part of an Agile team. As the developer becomes familiar with the work the QA team does, they gain a different perspective on quality. Unfortunately, in Waterfall shops that are highly siloed, engineers don’t get that perspective.
Summary
These interviews mimic what I have been hearing on the street. Companies are doing their best to be agile, and to implement automation. This is working to varying degrees of success. Joe Nolan is the Mobile QA team lead at Blackboard. He has over 10 years of experience leading multi-nationally located QA teams, and is the founder of the DC Software QA and Testing Meetup.
