We call this the quest for the elusive “bulletproof” assay – one that exhibits the highest performance and reliability in delivering patient results. Because NGS assays are so complex, involving a dizzying number of genes, mutation types, and options for sequencing and bioinformatics, the bulletproof assay is no easy feat. But it can be done.
As we wrote in our last blog post, there are three key steps:
In this article, we’ll discuss step two: strategically outlining the NGS assay validation study and QC plan together.
Doing the validation study and QC plan at the same time will give your lab the highest probability of achieving a functioning assay with high quality:
Validation is expensive because there are prescribed numbers of samples to run to meet sensitivity, specificity, reproducibility, and performance specifications in a submission to regulatory bodies, such as the New York State Department of Health. However, just running samples during validation doesn’t provide cost effective guidance for QC.
Guidelines like CLSI’s require using positive and negative controls in every run. But there are multiple reference material compositions. What is the most effective material to run?
Ideally, you should evaluate control technologies during NGS assay validation and choose them based on a technical design that readily detects sources of error at each phase:
In many respects, choosing a control technology is just as critical as selecting an assay format and should be done in parallel with choosing the library prep kits and pipeline software.
In their recently published guidelines for NGS oncology assays, the Association for Molecular Pathology and the College of American Pathologists detail the importance of a QC management strategy. To meet design criteria, they endorse using any of three categories of controls for NGS assays:
As we wrote in step one, it is always best to learn from the experience of other labs designing QC management systems.
In a 2016 study, Francine B. de Abreu and her team described an effective quality management practice that divides NGS into two processes: the “wet bench” sample prep and sequencing and the “dry bench” bioinformatics process.
These authors highlight 12 QC check points across their workflow. They included:
Each checkpoint has a specification that must be met before proceeding to the next step.
An advanced laboratory will always run its control material through validation to gather enough data to establish performance metrics on a “truth set” before taking the assay live. Most statisticians recommend running 20 control samples as a training set to establish baseline values before going live with an assay; you can accomplish this during assay validation.
You’re two-thirds of the way to a bulletproof NGS assay. In this step, you learned why it’s important to plan your QC and validation strategy together. But to do QC and validation right (and control your lab’s costs) you need the right reference material.
What is the right reference material? In the next and final step, we’ll teach you.