We are in the process of getting an instrument for testing. Is there anywhere to look with information about what all needs tested? How many samples should be used for each assay being validated? For antibody identification, should the reference method match the new instrument exactly? If the reference method shows Anti-E and new instrument shows Anti-E with 2 cells unexplained reactivity (but all antibodies ruled out).....is this considered a true positive (since antibody ID is correct) or false positive (since some cells of unexplained reactivity)? Also, if nothing is found via reference method, but antibody is found with new method....is this a true positive (since more sensitive)...or false positive (since not matching reference method). Any help in this matter would be appreciated.

There is no "rule" on how many samples are necessary to validate anything. The number 20 is bandied about, but you can determine how many you need to verify your instrument's functionality. With your examples, YOU will have to determine how to interpret conflicting results (reference vs instrument). If you are going to interface remember that you have to validate that also.

CLSI has a guideline for validating automated systems for immunohematological testing- you might want to check out their website for more info.

Edited October 14, 201014 yr by mhc
typo error

We pulled numbers magically out of the air. For the first instrument we did 100 samples of types and screens to "prove" methodology acceptance. For each subsequent instrument at the other hospitals we did decreasing amounts of testing. As for acceptable percentages, again we pulled numbers magically out of the air that seemed acceptable.

What David and Carrie said is true...no magic numbers....you have to decide how much testing you must do to feel comfortable/confident with the new instrumentation. You also have to decide whether the instrument is acceptable to you when its results do not correlate perfectly with your standard methodology.

You do need to spell out exactly what you are going to do in your Validation Plan, who is going to do what, then perform the testing, then evaluate the testing results, then document whether or not the results were acceptable. (If there's something about the results that you aren't comfortable, I imagine you could write an addendum to your plan and perform additional testing, then reevaluation.)

Donna

Echo rep visited w/us yesterday. She said validation for us wouldn't be that bad, as we do tubes w/PEG, and the LISS for capture had a similar potentiating strength/sensitivity as PEG. Woundn't be as hard as compareing tube LISS (less sensitive) to ie gel (more sensitive).

It has been a while since I have taken statistics but I do remember that there is a minimal number of samples, or data, needed to satisfy the "P" value, which confirms that the data being generated is do to specific reactivity, as opposed to the knoll hypothesis, which suggests that the aquired data are do to randomly occuring events and no specificity.

The current edition of the Technical Manual (16th ed) has an appendix 1-3 in Chapter 1 on Statistical Tables used to determine adequate sample size and level of confidence for validation pass/fail. I found this very useful in dealing with the new CAP checklist question regarding method to method correlation. Hopes this helps you out!

The validation guide for the Immucor ECHO is very thorough. Follow it and you should be fine. Correlate as many samples as you need to feel comfortable with the technology.

Thanks for all the information and input!!

Laura

The validation guide for the Immucor ECHO is very thorough. Follow it and you should be fine. Correlate as many samples as you need to feel comfortable with the technology.

And your Immucor installation specialist should be a good resource as you do your validation. Mine had been users of the instrument prior to their current jobs with Immucor and understood using the insturment in the real world. I suspect that would probably also be true for other instruments.

Installation Qualification, Operational Qualification and Performance Qualification all need to be done.

IQ & PQ the instrument supplier can help with (IQ must be done by the company and Certs issued ). The last you will need to set the parameters to be tested yourself. Correlation discrepancies will have to be explained by the known performance specifications of the two systems (e.g. manual V/S Echo). You would get some help by looking at External Quality Assurance Schemes and see what antibodies are missed frequently by what system (I know some but will not stick my neck out to be sued by supplier). Also liaise with other users. They will tell you what needs to be looked at. P values need at least 30 data points to be any way meaningful. Some statisticians will argue for more. My advice is to do as many as possible, given time and cost restraints. When all that is done, write an executive summary giving your reasons why it is safe to use the technology and "GO LIVE". Some prefer to do parallel runs for a while afterwards till confidence levels are higher amongst staff, but remember ther are a number of discrepancies due to differing specificities and sensitivities. You, with your medical director must decide if they are clinically significant. Get your director to sign off as well.

Good Luck with it all.

Cheers

Eoin

Edited October 28, 201014 yr by Eoin

Good point on the P value, Eoin.