Lisa Crispin - Agile Testing

Agile Values

What Do We Mean by “Agile Testing”?

A Little Context for Roles and Activities on an Agile Team

Customer Team

Developer Team

Interaction between Customer and Developer Teams

How Is Agile Testing Different?

Working on Traditional Teams

Working on Agile Teams

Traditional vs. Agile Testing

Whole-Team Approach

Summary

What’s an Agile Tester?

The Agile Testing Mind-Set

Applying Agile Principles and Values

Provide Continuous Feedback

Deliver Value to the Customer

Enable Face-to-Face Communication

Have Courage

Keep It Simple

Practice Continuous Improvement

Respond to Change

Self-Organize

Focus on People

Enjoy

Organizational Culture

Quality Philosophy

Sustainable Pace

Customer Relationships

Organization Size

Empower Your Team

Barriers to Successful Agile Adoption by Test/QA Teams

Loss of Identity

Additional Roles

Lack of Training

Not Understanding Agile Concepts

Past Experience/Attitude

Cultural Differences among Roles

Introducing Change

Talk about Fears

Give Team Ownership

Celebrate Success

Management Expectations

Cultural Changes for Managers

Speaking the Manager’s Language

Change Doesn’t Come Easy

Be Patient

Let Them Feel Pain

Build Your Credibility

Work On Your Own Professional Development

Beware the Quality Police Mentality

Vote with Your Feet

Summary

Team Structure

Independent QA Teams

Integration of Testers into an Agile Project

Agile Project Teams

Physical Logistics

Resources

Tester-Developer Ratio

Hiring an Agile Tester

Building a Team

Self-Organizing Team

Involving Other Teams

Every Team Member Has Equal Value

Performance and Rewards

What Can You Do?

Summary

Seeking Lightweight Processes

Metrics

Lean Measurements

Why We Need Metrics

What Not to Do with Metrics

Communicating Metrics

Metrics ROI

Defect Tracking

Why Should We Use a Defect Tracking System (DTS)?

Why Shouldn’t We Use a DTS?

Defect Tracking Tools

Keep Your Focus

Test Planning

Test Strategy vs. Test Planning

Traceability

Existing Processes and Models

Audits

Frameworks, Models, and Standards

Summary

The Agile Testing Quadrants

Tests that Support the Team

Tests that Critique the Product

Knowing When a Story Is Done

Shared Responsibility

Managing Technical Debt

Testing in Context

Summary

An Agile Testing Foundation

The Purpose of Quadrant 1 Tests

Supporting Infrastructure

Why Write and Execute These Tests?

Lets Us Go Faster and Do More

Making Testers’ Jobs Easier

Designing with Testing in Mind

Timely Feedback

Where Do Technology-Facing Tests Stop?

What If the Team Doesn’t Do These Tests?

What Can Testers Do?

What Can Managers Do?

It’s a Team Problem

Toolkit

Source Code Control

IDEs

Build Tools

Build Automation Tools

Unit Test Tools

Summary

Driving Development with Business-Facing Tests

The Requirements Quandary

Common Language

Eliciting Requirements

Advance Clarity

Conditions of Satisfaction

Ripple Effects

Thin Slices, Small Chunks

How Do We Know We’re Done?

Tests Mitigate Risk

Testability and Automation

Summary

Business-Facing Test Tool Strategy

Tools to Elicit Examples and Requirements

Checklists

Mind Maps

Spreadsheets

Mock-Ups

Flow Diagrams

Software-Based Tools

Tools for Automating Tests Based on Examples

Tools to Test below the GUI and API Level

Tools for Testing through the GUI

Strategies for Writing Tests

Build Tests Incrementally

Keep the Tests Passing

Use Appropriate Test Design Patterns

Keyword and Data-Driven Tests

Testability

Code Design and Test Design

Automated vs. Manual Quadrant 2 Tests

Test Management

Summary

Introduction to Quadrant 3

Demonstrations

Scenario Testing

Exploratory Testing

Session-Based Testing

Automation and Exploratory Testing

An Exploratory Tester

Usability Testing

User Needs and Persona Testing

Navigation

Check Out the Competition

Behind the GUI

API Testing

Web Services

Testing Documents and Documentation

User Documentation

Reports

Tools to Assist with Exploratory Testing

Test Setup

Test Data Generation

Monitoring Tools

Simulators

Emulators

Summary

Introduction to Quadrant 4

Who Does It?

When Do You Do It?

“ility” Testing

Security

Maintainability

Interoperability

Compatibility

Reliability

Installability

“ility” Summary

Performance, Load, Stress, and Scalability Testing

Scalability

Performance and Load Testing

Performance and Load-Testing Tools

Baseline

Test Environments

Memory Management

Summary

Review of the Testing Quadrants

A System Test Example

The Application

The Team and the Process

Tests Driving Development

Unit Tests

Acceptance Tests

Automation

The Automated Functional Test Structure

Web Services

Embedded Testing

Critiquing the Product with Business-Facing Tests

Exploratory Testing

Testing Data Feeds

The End-to-End Tests

User Acceptance Testing

Reliability

Documentation

Documenting the Test Code

Reporting the Test Results

Using the Agile Testing Quadrants

Summary

Why Automate?

Manual Testing Takes Too Long

Manual Processes Are Error Prone

Automation Frees People to Do Their Best Work

Automated Regression Tests Provide a Safety Net

Automated Tests Give Feedback, Early and Often

Tests and Examples that Drive Coding Can Do More

Tests Are Great Documentation

ROI and Payback

Barriers to Automation—Things that Get in the Way

Bret’s List

Our List

Programmers’ Attitude—“Why Automate?”

The “Hump of Pain” (The Learning Curve)

Initial Investment

Code that’s Always in Flux

Legacy Code

Fear

Old Habits

Can We Overcome These Barriers?

Summary

An Agile Approach to Test Automation

Automation Test Categories

Test Automation Pyramid

What Can We Automate?

Continuous Integration, Builds, and Deploys

Unit and Component Tests

API or Web Services Testing

Testing behind the GUI

Testing the GUI

Load Tests

Comparisons

Repetitive Tasks

Data Creation or Setup

What Shouldn’t We Automate?

Usability Testing

Exploratory Testing

Tests that Will Never Fail

One-Off Tests

What Might Be Hard to Automate?

Developing an Automation Strategy—Where Do We Start?

Where Does It Hurt the Most?

Multi-Layered Approach

Think about Test Design and Maintenance

Choosing the Right Tools

Applying Agile Principles to Test Automation

Keep It Simple

Iterative Feedback

Whole-Team Approach

Taking the Time to Do It Right

Learn by Doing

Apply Agile Coding Practices to Tests

Supplying Data for Tests

Data Generation Tools

Avoid Database Access

When Database Access Is Unavoidable or Even Desirable

Understand Your Needs

Evaluating Automation Tools

Identifying Requirements for Your Automation Tool

One Tool at a Time

Choosing Tools

Agile-Friendly Tools

Implementing Automation

Managing Automated Tests

Organizing Tests

Organizing Test Results

Go Get Started

Summary

The Purpose of Release Planning

Sizing

How to Size Stories

The Tester’s Role in Sizing Stories

An Example of Sizing Stories

Prioritizing

Why We Prioritize Stories

Testing Considerations While Prioritizing

What’s in Scope?

Deadlines and Timelines

Focus on Value

System-Wide Impact

Third-Party Involvement

Test Planning

Where to Start

Why Write a Test Plan?

Types of Testing

Infrastructure

Test Environments

Test Data

Test Results

Test Plan Alternatives

Lightweight Test Plans

Using a Test Matrix

Test Spreadsheet

A Whiteboard

Automated Test List

Preparing for Visibility

Tracking Test Tasks and Status

Communicating Test Results

Release Metrics

Summary

Be Proactive

Benefits

Do You Really Need This?

Potential Downsides to Advance Preparation

Advance Clarity

Customers Speak with One Voice

Story Size

Geographically Dispersed Teams

Examples

Test Strategies

Prioritize Defects

Resources

Summary

Iteration Planning

Learning the Details

Considering All Viewpoints

Writing Task Cards

Deciding on Workload

Testable Stories

Collaborate with Customers

High-Level Tests and Examples

Reviewing with Customers

Reviewing with Programmers

Test Cases as Documentation

Summary

Driving Development

Start Simple

Add Complexity

Assess Risk

Coding and Testing Progress Together

Identify Variations

Power of Three

Focus on One Story

Tests that Critique the Product

Collaborate with Programmers

Pair Testing

“Show Me”

Talk to Customers

Show Customers

Understand the Business

Completing Testing Tasks

Dealing with Bugs

Is It a Defect or Is It a Feature?

Technical Debt

Zero Bug Tolerance

It’s All about Choices

Decide Which Bugs to Log

Choose When to Fix Your Bugs

Choose the Media You Should Use to Log a Bug

Alternatives and Suggestions for Dealing with Bugs

Start Simple

Facilitate Communication

Testers Facilitate Communication

Distributed Teams

Regression Tests

Keep the Build “Green”

Keep the Build Quick

Building a Regression Suite

Checking the “Big Picture”

Resources

Iteration Metrics

Measuring Progress

Defect Metrics

Summary

Iteration Demo

Retrospectives

Start, Stop, and Continue

Ideas for Improvements

Celebrate Successes

Summary

What Makes a Product?

Planning Enough Time for Testing

The End Game

Testing the Release Candidate

Test on a Staging Environment

Final Nonfunctional Testing

Integration with External Applications

Data Conversion and Database Updates

Installation Testing

Communication

What If It’s Not Ready?

Customer Testing

UAT

Alpha/Beta Testing

Post-Development Testing Cycles

Deliverables

Releasing the Product

Release Acceptance Criteria

Release Management

Packaging

Customer Expectations

Production Support

Understand Impact to Business

Summary

Success Factor 1: Use the Whole-Team Approach

Success Factor 2: Adopt an Agile Testing Mind-Set

Success Factor 3: Automate Regression Testing

Success Factor 4: Provide and Obtain Feedback

Success Factor 5: Build a Foundation of Core Practices

Continuous Integration

Test Environments

Manage Technical Debt

Working Incrementally

Coding and Testing Are Part of One Process

Synergy between Practices

Success Factor 6: Collaborate with Customers

Success Factor 7: Look at the Big Picture

Summary