As a biostatistician, it is amusing (in a painful sort of way) to hear the responses that I most often get when asking clients what the acceptance criteria are for the design validation studies.
Here then are the Top Ten Answers to "What are the acceptance criteria for this study?":
10. 10% just like always.
9. I don't know the answer yet, so how can I set the acceptance criteria?
8. We don't want to set the acceptance criteria and then fail!
7. I'll know what the acceptance criteria will need to be when I see it.
6. I just need to estimate the precision - isn't that the acceptance criteria?
5. I just want to see if it is in the ballpark.
4. I want to know if it works before I set the acceptance criteria.
3. Once we know what the product does, we'll figure out the acceptance criteria.
2. You know, it's a chicken and egg thing, so it doesn't matter where we start.
1. What's statistically significant?
None of these are appropriate as acceptance criteria. By the time development teams get to the design validation, they are focused on technical and timeline challenges. The time to set acceptance criteria in a less charged environment is long past. At this point, teams usually just want to know the answer to what sample size is required, as they were supposed to start the study yesterday. The thing that often isn't realized is that the team usually has the data and information from which to derive the answer.
It is important to focus the discussion so that appropriate acceptance criteria can be determined. What are the various sources of error, impact / risk of incorrect results and what total error in the system is acceptable across the assay range? How wil the product be used, what are the medical decision points, standard of care, and intended use? These are questions that are best addressed by a cross-functional team.
With an understanding of the acceptance criteria and the expected performance of the product, it is possible to look at the trade-offs in sample size and experimental error rates. The minimum sample size required by the FDA is not always the best choice for a study. In order to estimate the probability of meeting the acceptance criteria, you first must have acceptance criteria. Sometimes increasing the sample size of a study improves the probability of meeting the acceptance criteria and heance having a successful study.
Tags: Biostats, CDRH, Clinical Studies
