The U.S. Food and Drug Administration on February 25, 2016 held an all-day public workshop entitled “Next Generation Sequencing-based Oncology Panels” at their White Oak campus in Silver Spring, Maryland. Companion diagnostics were a recurring theme of the initial FDA presentations at this Next Generation Sequencing-Based Oncology Panels Workshop and these important tests will be our focus here. The slides that were presented at the workshop are available (PDF) and are referenced below, and those presented by SeraCare’s CSO Russell Garlick start on page 196.
Companion Diagnostics (CDx)
Companion diagnostics differ from regular in vitro diagnostics in that they are qualitative assays (yes/no in relation to the clinical decision point) that were used to select the patients who were treated in (typically) a phase III clinical drug trial. They represent an important element of precision medicine because many of the new drugs target specific molecules that may only be relevant in a small percentage of patients and it is important to select such patients for trials (and later for treatment) in order to show that a given drug is both effective and safe – but the latter does not imply that there may not be severe side effects.
When there is a companion diagnostic for a drug, the safety and efficacy of that drug were established in a population that was selected using the output of the companion diagnostic. It does not mean that all patients who were selected for the trial obtained some benefit from the drug. It also does not mean that all patients who were not selected for the trial based on the companion diagnostic would not have obtained some benefit from the drug. However, the safety and efficacy of the drug in those patients who were not selected was not addressed in the trial.
The number of drugs that are available for precision medicine is increasing steadily (a 2013 McKinsey report indicates 3 mutations currently targeted with on-market drugs and an additional 53 drugs in the pipeline) and this has the potential to increase the number of different companion diagnostics. Since many of these tests are likely to rely on similar samples, this presents a problem for selecting the best treatments for patients when such samples are limiting. Additionally, having to run dozens of separate nucleic acid-based tests is not cost and time effective. As Dr. Elizabeth Mansfield indicated during her introductory statements, the FDA feels that NGS-based oncology panels are a good way to get all the information one needs out of a single test.
Follow-on Companion Diagnostics
The output of an approved companion diagnostic provides a rather limited answer: it tells patients whether they should have been treated with the drug during the trial or not. That presents a problem for the design and approval (or clearance) of follow-on companion diagnostics that would have a similar intended use to the original companion diagnostic but potentially a different output that could select a different population for treatment for which the safety and efficacy profile could be different.
On slide 10, (again the presentation slides are available here as a PDF) the FDA indicates that follow-on companion diagnostics should 1) “consistently and accurately select the same intended use patient population as the originally-approved companion diagnostic devices for the indicated therapeutic drug” and 2) “demonstrate the same or comparable level of analytical and clinical performance for specific mutations in the originally-approved companion diagnostic device.”
In a qualitative test, the limit of detection (LoD) is the amount of an analyte that is detected at least 95% of the time and is an important performance metric – especially, when the clinical decision point is the LoD. Presently, there are two companion diagnostics for Erbitux and Vectibix that are used to assess whether KRAS contains particular mutations that would cause treatment with these drugs to fail. The Roche cobas® KRAS Mutation Test (per FDA document PDF) has an LoD for KRAS mutations G12A, G12C, G12D, G12R, G12S, G12V and G13D of 2.93%, 2.61%, 1.64%, 5.78%, 2.55%, 2.48% and 1.67%, respectively. The QIAGEN therascreen® KRAS RGQ PCR Kit (per FDA document PDF) has LoDs of 0.8%, 1.5%, 6.4%, 2.6%, 5.6%, 1.6% and 6.4% for these same mutations. It is important to keep in mind that a sample with a KRAS mutation at a frequency below the LoD may still result in a positive result – just fewer than 95% of the time. For example, the LoD for G12D with the QIAGEN test is about 6.4%, but the test still picked up 1.56% G12D about a third of the time.
Tumor content is also important. The Roche test requires at least 10% tumor content while the QIAGEN test requires at least 20% tumor content. For a sample with 10% tumor content and an assay LoD around 5%, the entire tumor must be heterozygous for the mutation in order to be positive 95% of the time. At the same time, for a sample with 100% tumor content, only 10% of the tumor must be heterozygous for the mutation in order for it to be positive 95% of the time.
Which Companion Diagnostic to Use
The most recent (4/2015) labeling revision for Erbitux (cetuximab) (FDA document PDF) provides some interesting information. In the “Indications and Usage” section for colon cancer, Erbitux is indicated for the treatment of “K-Ras wild-type, EGFR-expressing, metastatic colorectal cancer as determined by FDA-approved tests.” Since both companion diagnostics differ in terms of tumor content and the LoD for a particular mutation, whether a patient would be treated or not can depend on the companion diagnostic being used. There is also the question of what “wild-type” KRAS actually means since the gene spans about 46,000 bases of which a bit fewer than 6,000 bases end up in the final mRNA and fewer than 600 bases code for protein. This is clarified in section 5.7, which indicates that “Erbitux is not indicated for the treatment of patients with colorectal cancer that harbor somatic mutations in exon 2 (codons 12 and 13), exon 3 (codons 59 and 61), and exon 4 (codons 117 and 146) of either K-Ras or N-Ras and hereafter is referred to as Ras” and that “use of cetuximab in patients with Ras mutations resulted in no clinical benefit with treatment related toxicity.”
Given that the revision to labeling happened about a year ago, this raises the question of which FDA-approved companion diagnostic allows a physician to assess whether any exon 3 and 4 mutations are present. It also raises the question of which companion diagnostic to use to assess mutations in N-Ras. Such a companion diagnostic – or follow-on companion diagnostic – may have been submitted to the FDA for review a year ago, but it has still not obtained approval.
LDTs to the Rescue
At the FDA workshop, the majority of panelists had experience with NGS-based LDTs that are presently used to assess tumors for mutations. Presently, LDTs are an essential component of precision medicine and provide the only option for physicians to assess tumors for mutations for which there is no FDA-approved test. It is anticipated that NGS-based companion diagnostics will be marketed after FDA clearance or approval, and several companies are working on such tests. However, there are still many questions related to how such tests should be designed, labeled, validated and compared; and those same questions exist for current LDTs. There are also questions related to how such tests could be updated in order to assess additional mutations without requiring lengthy supplemental reviews. An attempt was made to address these questions in panel discussions.
Pre-Analytical Challenges and Quality Control
The first panel discussion focused on how samples should be handled and analyzed and what types of samples are needed for validation. Slide 53 of the FDA slide deck, which was part of a presentation from Dr. John Pfeifer (Washington University, St. Louis MO), showed the amount of tumor material his lab typically receives for analysis. “In our experience, 6% of FFPE cases have <10ng of DNA, 13% <100ng, 25% <200ng, and 57% <750ng.” If one looks at the sample requirements for the Roche companion diagnostic, a minimum of 400 ng of nucleic acid is required and must be obtained from no more than 10 microns of material. According to the instructions for use where samples are eluted in 100 µL: “If the DNA Stock is still <4 ng/μL, then request another FFPET specimen.” A complaint from the panelists was that different companion diagnostics have different sample preparation requirements and that this presents a problem when sample amounts are limiting, which is common. This was also touched on at an earlier FDA workshop and also presents a problem for new cancer markers since a pathologist may only have several slides of material to work with and testing for a new marker could preclude testing for an established marker. Interestingly, it was mentioned that samples are still analyzed when minimum input amounts are not met since one does not want to deprive a patient of a chance for a diagnosis. That could mean that the performance of a companion diagnostic in a lab that does this is not the same as its performance during a clinical trial.
Certain types of cancer are more difficult to analyze than others because of substances found in and around those cancers. Thus, there was some discussion about whether it is necessary to validate the ability to detect particular mutations with the types of cancers within which those mutations may be relevant. Perhaps this could be addressed as an interfering substances study where it is only necessary to show that one can get sequence-able DNA out of different types of cancer.
Given that samples should be processed such that they give a patient the best chance at a diagnosis, given that this is often not the case, and given that essentially all nucleic acid-based companion diagnostics and many LDTs prescribe similar FFPE processing conditions (for example Foundation Medicine's recommendations here), I asked a question about whether pathologists should be trained in sample processing for precision medicine (e.g., as part of Continuing Medical Education). Having a set of best practices in sample preparation would help patients and well as those developing and validating assays.
Analytical Challenges (Analytical Validation and Bioinformatics)
The next panel focused on analytical challenges of NGS-based tests. Overall, there was not much new information but slide 93 of the FDA slide deck, which was part of Dr. Madhuri Hegde’s presentation (Emory University), mentions that “most assays can detect variants with an allele fraction as low as 10% with a sensitivity of 100%, the sensitivity for variants with allele fraction 5% and 1% is zero.”
Given that existing companion diagnostics have LoDs as low as 1% for some targets (i.e., they have 95% sensitivity at allele fractions of 1%), this suggests that many existing NGS-based LDTs in oncology could not substitute for existing companion diagnostics. If follow-on companion diagnostics are evaluated similar to 510(k) submissions where an existing companion diagnostic would serve as a predicate device, then there would be a potential requirement for LoDs of 1%. That could have implications on the NGS platforms that could be used for companion diagnostics – especially, if error rates are significant compared to the targeted LoDs. Finally, the minimum depth of coverage may also be substantial in order to have acceptable performance with variants at 1%. That could impact the cost of NGS-based tests in oncology and/or limit the number of different regions that can be sequenced by the tests. However, whether it is important to have acceptable performance with variants at 1% would also depend on the intended use and performance claims.
Clinical and Follow-on Companion Diagnostic Claims
Slide 159 of the FDA slide deck summarizes the FDA’s concern with the emergence of NGS-based tests that may be used as substitutes for existing companion diagnostics: “One concern with follow-on companion diagnostic claims is that the new assay may have different/better analytical sensitivity than the original assay… Imagine an assay with 100x greater sensitivity that would identify a significantly different patient population. How do you ensure that clinical benefit of the drug to this population would be assured?”
Provided that sensitivity and specificity exceed those of existing companion diagnostics, software could be used to report results as if they were lower (e.g., “the sample could have been reported as being negative for mutation X in gene Y by companion diagnostic Z, but the mutation is present”). For the assays that were discussed at the workshop, 100x greater sensitivity than existing companion diagnostics is unlikely for cost/benefit reasons and because of sample input amounts. For example, 10 ng of extracted DNA – which is the input amount for many amplicon assays – represents the genomes of a bit more than 1,000 cells. A variant at around 1% would be heterozygous in 20 cells and because of an underlying binomial distribution would be detected with a variant frequency between 0.6% and 1.5%, 95% of the time – assuming that 100% of the input DNA is usable. A variant at around 0.1% would not be detected at least 40% of the time because it will not be present in the input material 40% of the time. With KRAS, where 4 bases in codons 12 and 13 are important, and with sequencing at an average base quality of Q30, it would be more likely to observe a mutation as a result of sequencing error than from the 10 ng of input material. Thus, more input material would be required. Additionally, sequencing depth would need to be increased because a binomial distribution applies there too. But the issue of sequencing error remains.
Final Notes
The FDA is planning to have a workshop on liquid biopsies in July of 2016, announced at the start of the workshop. Such assays may need to have greater sensitivities in order to reliably detect variants at or below 0.1% and incorporate methods that mitigate the error rates of the sample preparation and sequencing steps. At the same time, the sample is not limiting, so liquid biopsies could be performed using multiple different tests and would not compete for the limited biopsy material that could be used for existing companion diagnostics.
More information about SeraCare’s oncology reference materials is available here.
