Sometimes a single remark can effectively summarize a very complex problem. While the November 12, 2015 FDA Public Workshop on “Standard Based Approach to Analytical Performance Evaluation of Next Generation Sequencing In Vitro Diagnostic Tests” focused primarily on tests that look for germline and somatic variants, a comment made by Jared Maguire – Counsyl’s ‎Director of Computational Biology – about 1 hour 14 minutes into the workshop explained a lot about the current state of non-invasive prenatal testing (NIPT) and what patients, physicians and insurers are experiencing; more on that later.

Backstory

About a week ago, my wife and I welcomed a perfectly healthy baby girl into the world. About 7 months prior to that, given our respective ages, her OB/GYN recommended getting an NIPT test. All of these tests currently fall into the category of Laboratory Developed Tests (LDTs) and are commonly believed to be nearly as accurate as invasive tests. However, it is difficult to compare their relative performance and that relative performance may change as the tests change in response to patient, market and regulatory pressures.

Developing reference materials for such tests at SeraCare and analyzing a lot of associated data has provided me with an in depth understanding of the different tests and their respective limitations. If we could choose – and if my wife’s insurance would pay for at least part of it – there is one test I would have preferred. However, my wife’s OB/GYN ordered a different one because that is what he always orders and has experience with. That test also happened to be out of network for my wife. Initially, the bill was quite substantial but after appealing the coverage denial her insurance eventually paid for most of it; in part, because there was no alternative.

Why no alternative?

Here is Jared’s comment from the workshop:

“For instance in the non-invasive prenatal tests, things are typically tuned to have a high false positive rate so that you don't miss anything because the follow-up is amniocentesis; it's not a final result; it's just a screen.”

Counsyl is one of Illumina’s many verifi^® Prenatal Test partners (PDF link to partner list). Unlike Illumina, they use the NIPS abbreviation (non-invasive prenatal screen) as opposed to NIPT (PDF link to Counsyl brochure with positive and negative predictive value charts). Since a screen is used to identify patients for further testing – in this case invasive testing – it is essential to have a low false negative rate. That comes at the expense of a higher false positive rate. The reported performance characteristics of the base Verinata/Illumina verifi Prenatal Test show high specificities (>99.9%) for the common trisomies and sensitivities above 98%. However, a 98.15% sensitivity for T13 Patau syndrome means that it – while rare – is missed about 2% of the time.

In order to improve sensitivity, all that is necessary is to adjust the positive/negative cutoff. Without performance standards for a particular type of test, laboratories are free to choose values they are comfortable with. For an otherwise identical test, one lab’s positive may be another lab’s negative if the cutoff values differ.

Counsyl provides the positive predictive values (PPVs) and negative predictive values (NPVs) for their Informed Pregnancy Screen (PDF link to Counsyl brochure). The PPV is important since it indicates how likely it is that a positive result from their screen would end up being confirmed as a true positive. When this test is typically performed (10-12 weeks gestational age), a 36 year old woman with a positive T13 result would only have about a 25 % chance of that being a true positive. Similarly, a positive T21 result would be a false positive about 30 % of the time. In their informed consent, Counsyl states the following:

“Any positive results should be followed by consulting with your healthcare provider regarding the option of pursuing diagnostic invasive testing (such as chorionic villus sampling and/or amniocentesis) and additional evaluation as well as genetic counseling. The decision of whether to pursue additional testing is entirely yours and should be made in consultation with your healthcare provider.”

For patients and their healthcare providers, it is important to take the PPVs into account because this screen is performed with an accuracy that is well below that of an invasive test. Whether or not a different NIPT/NIPS test has a similar PPV – even one offered by a different verifi Prenatal Test partner – is not clear. Thus, it is in the patient’s best interest for a healthcare provider to have experience with the test and the significance of its positive and negative results.

Healthcare providers, when they take insurance, are often part of several networks. That presents a problem for patients and insurers because the healthcare provider may prefer an NIPT/NIPS test that is not in network for a given patient. This could be alleviated partially by having accepted performance standards for such tests and uniform reporting.

Future Directions

Even with performance standards and uniform reporting, there are market pressures to reduce costs, which could lead to reduced performance in the future such that a given test becomes just good enough to meet the standards. For example, the verifi-like tests currently analyze somewhere around 20 million NGS sequence reads for each patient. For such tests, more reads typically correlates with improved sensitivity and specificity. More reads also correlates with a higher cost per sample. However, it is possible to use fewer reads and maintain either sensitivity or specificity by using an appropriately adjusted cutoff. Thus, for a screen where sensitivity is of primary importance, PPV may become lower over time as laboratories look to minimize costs while meeting certain performance standards for sensitivity.

At a minimum, that could lead to an increased amount of invasive testing – which non-invasive testing was supposed to reduce. Perhaps the initial follow-up to NIPS would be a reflex to a similar non-invasive test with a better PPV?

If you are looking for reference materials for non-invasive prenatal testing (or screening), you may be interested in SeraCare’s offerings here. Also if you are interested in additional background about non-invasive prenatal testing, we've written about 'What is NIPT?' here.