Too Good to be True?

Monday, May 04, 2009

The IVD community was all abuzz with the data Sequenom announced last June.  According to the company, the SEQureDx test for Down's Syndrome had 96.6% positive predictive value and 100% negative predictive value.  That's right, no false negatives at all.  Ever.

Perfection is a worthy goal, but it's not often we get to see it out in the field.  Were these results for real or maybe a little too good to be true? 

We won't know for sure which one it is until the product is either cleared or criminal indictments are handed out.  But yesterday's announcement that Sequenom's data was "mishandled" tends to suggest that the story was too good to be true.  Their investors appear to have jumped to this conclusion as well: their stock fell more than 75% on the announcement.  The class action suits have already gotten underway.

Just to be clear, i don't know any of the people involved and don't have any insider information on this one. 

Despite this lack of first-hand information, I think it's safe to offer some general observations:

  • If you show anything like 100% results, expect that your data will raise eyebrows.  Before you announce this kind of result, you should plan to subject your data to the kind of enhanced scrutiny that it will receive from FDA.  Hire outside statisticians and don't attempt to bias their findings.  It is far better to know what you actually have than to announce something you may need to take back later.

  • You can game the system, but you do so at great risk.  There is a certain amount of trust built into this system and short-term advantages can be had by cutting corners.  When we're talking about products that take at least 10 years to pay back development costs, the long-range view really ought to be a key element of your planning.  Don't make the mistake of interpreting short-term conditions like lax enforcement as solid long-term foundations for your business.  Over the course of your product's lifecycle, there will be at least one or two changes of administration.

  • When you're in a hole, stop digging.  If you have been caught with bad data, don't announce that you still expect clearance in a few weeks and attempt to make distinctions between your bad data and your wonderful product.  If ill-considered public statements have gotten you in trouble, make sure the next few things you say in public is well-vetted.  Remember, history eventually vindicated ImClone, but some people who made public statements about ImClone went to jail.

I'll be keeping on an eye on this situation and will add more as the situation develops.

Tags: Best Practices, Enforcement, FDA, FDA 101, Public Statements

Other Blog Authors

Dylan Reinhardt
Dave Kern
Steve Gutman
Jo-Ann Gonzales

Recent Blog Posts

The Staple That Changed an Industry LDT regulation and the perils of challenging Eminem to a rap battle
FDA Releases the Kraken Whichever way this breaks, it's long past time to have this conversation in a meaningful way
Illumina Acquires Myraqa Acquisition Strengthens Illumina’s Clinical Readiness
The Spirit of GLP A Best Practice for IVDs
Process Performance Qualification (PPQ) Lots So, how many lots are required?
The Case for Risk-based Monitoring Better the devil you know...
Form Follows Function OIR Reorganizes to Meet the Advancing Wave of Molecular Diagnostics
CDRH Unveils New PMA Guidance Documents Attention shoppers, it's two-for-one day!