As the use of genetically targeted cancer therapies becomes mainstream, the demand for reliable NGS assays is skyrocketing. To help doctors personalize treatments to their patients’ genetic profiles – and to stand out from the competition – clinical laboratories like yours are constantly looking for ways to create more accurate, more comprehensive assays.
We call this the quest for the elusive “bulletproof” assay – one that exhibits the highest performance and reliability in delivering patient results. Because NGS assays are so complex, involving a dizzying number of genes, mutation types, and options for sequencing and bioinformatics, the bulletproof assay is no easy feat. But it can be done.
As we wrote in our last blog post, there are three key steps:
- Consulting with experts
- Outlining your validation and quality control (QC) strategies together
- Evaluate reference material options
In this article, we’ll discuss step two: strategically outlining the NGS assay validation study and QC plan together.
Doing the validation study and QC plan at the same time will give your lab the highest probability of achieving a functioning assay with high quality:
- Validation results set assay performance
- A well-designed QC management system will make sure the assay stays within specifications
- QC tracking and trending will also provide early indicators of assay drift or potential failure
How Validation and QC Are Better Together
Validation is expensive because there are prescribed numbers of samples to run to meet sensitivity, specificity, reproducibility, and performance specifications in a submission to regulatory bodies, such as the New York State Department of Health. However, just running samples during validation doesn’t provide cost effective guidance for QC.
Guidelines like CLSI’s require using positive and negative controls in every run. But there are multiple reference material compositions. What is the most effective material to run?
Ideally, you should evaluate control technologies during NGS assay validation and choose them based on a technical design that readily detects sources of error at each phase:
- Preanalytical sample prep
- Analytical library prep and sequencing
- Post-analytical data analysis
In many respects, choosing a control technology is just as critical as selecting an assay format and should be done in parallel with choosing the library prep kits and pipeline software.
In their recently published guidelines for NGS oncology assays, the Association for Molecular Pathology and the College of American Pathologists detail the importance of a QC management strategy. To meet design criteria, they endorse using any of three categories of controls for NGS assays:
- Reference cell lines
- Biosynthetic DNA fragments
- Engineered cell lines
The Wet Bench and the Dry Bench
As we wrote in step one, it is always best to learn from the experience of other labs designing QC management systems.
In a 2016 study, Francine B. de Abreu and her team described an effective quality management practice that divides NGS into two processes: the “wet bench” sample prep and sequencing and the “dry bench” bioinformatics process.
These authors highlight 12 QC check points across their workflow. They included:
- DNA extraction
- DNA concentration
- DNA quality
- Library prep quantification
- Sequencing run metrics such as Q scores
- Coverage
- Percent mapped reads
- Percent variant allelic frequency
Each checkpoint has a specification that must be met before proceeding to the next step.
An advanced laboratory will always run its control material through validation to gather enough data to establish performance metrics on a “truth set” before taking the assay live. Most statisticians recommend running 20 control samples as a training set to establish baseline values before going live with an assay; you can accomplish this during assay validation.
Up Next: How to Choose the Right Reference Materials
You’re two-thirds of the way to a bulletproof NGS assay. In this step, you learned why it’s important to plan your QC and validation strategy together. But to do QC and validation right (and control your lab’s costs) you need the right reference material.
What is the right reference material? In the next and final step, we’ll teach you.
