PEPFAR Quality Control and Method Validation Activity 6: Presentation Slides

Determining our assigned mean and SD for the new control ACTIVITY: Parallel Testing ‐ Making History 1 When Do You Create a New LJ Chart? • Lot changes Controls (parallel testing) – Reagents – • Calibration Expected (part of instrument maintenance – schedule) –Unexpected (a problem during testing is corrected) • Major component changed on the instrument • Change in the testing process, usually manufacturer‐directed. 2 How SQC Works SD obs MEAN obs .. We did expect this point We did NOT expect this point(s) We compare today’s observed value with the past history of previous results to determine if the system is stable or undergoing a change. 1 HILS1754 Effective Date: 12/07/2016 Overview: Introducing a New Lot Number into Service +2 SD +1 SD MEAN obs SD obs Assigned MEAN ‐1 SD ‐2 SD In parallel with the Based on the past history lot number currently of previous results in‐use QC Materials • Produce a Gaussian distribution when analyzed repeatedly • Monitor the analytical phase only • Should mimic patient samples – behave in the same manner (i.e. if patients results undergo a shift, so should our QC results) • Should detect changes at medical decision levels 5 Commercially Prepared Controls • Assayed • Unassayed Tested by manufacturer using NOT tested by manufacturer – – multiple methods before sale before sale Sold with results of the Control values must be – – testing determined by the individual Common formats used with laboratory without having the – package inserts guideline information from • Mean and ±2 SD range the insert • Mean and 1 SD Usually less expensive than – Used to evaluate accuracy and assayed – precision Used to evaluate precision – Avoid laboratory errors when only – determining control values Avoid manufacturer errors in – determining control values 6 2 HILS1754 Effective Date: 12/07/2016 ISO 15189 5.6.2.2 Quality control materials The laboratory shall use quality control materials that react to the examining system in a manner as close as possible to patient samples. Quality control materials shall be periodically examined with a frequency that is based on the stability of the procedure and the risk of harm to the patient from an erroneous result. NOTE 1 The laboratory should choose concentrations of control materials, wherever possible, especially at or near clinical decision values, which ensure the validity of decisions made. NOTE 2 Use of independent third party control materials should be considered, either instead of, or in addition to, any control materials supplied by the reagent or instrument manufacturer. 7 Activity: Parallel Testing Role Play Purpose What will you do? You will work individually To list the steps necessary to  perform parallel testing in the order Observe the role play  presented during the role play List the steps as they unfold during the role play What will you need? Notepad Pen or pencil 15 minutes8 Activity: Parallel Testing SOP Purpose What will you do? To create a parallel testing SOP You will work in a group (table). for your site Work as a group to create a SOP . for parallel testing using the Handout as a guide  Select a spokesperson to What will you need? provide a review of the List of steps from the role play group’s SOP  2 ‐ Flipchart paper Spokesperson presents a 1 minute review to the class Flipchart markers  Participate in the classroom Handout: SOP Activity Template discussion 60 minutes9 3 HILS1754 Effective Date: 12/07/2016 WHERE WE ARE Accuracy X= n • Mean ( x ) the average of a set of values – primary indicator of accuracy – measure of systematic error (error in a given – direction) 10 WHERE WE ARE Precision >(x - x)2 • Standard deviation (SD) Used to measure dispersion/scattering of a group – of values around a mean Primary indicator of precision – Measure of random error ( error in any direction) – Measure of absolute variability – • Coefficient of Variation CV% = (SD / x ) * 100% ¯ The great statistical equalizer – Measure of relative variability – 11 Coefficient agreeing, collaborating, combining, concurring, coordinated, harmonious, joining, relating, uniting Variation changeability, fluctuation, inconsistency, unpredictability 12 4 HILS1754 Effective Date: 12/07/2016 WHERE WE ARE Precision CV% = (SD / x ) * 100% Coefficient of Variation ‐ The great statistical equalizer Measure of relative variability – Expresses the SD in percent – Comparison of overall precision, since standard – deviation typically increases as the concentration of the analyte increases Estimates the SD at the concentration level of interest – Provides an estimate of method precision – 13 The Great Equalizer Laboratory Applications Using %CV • Can compare methods that give different units • Provides a estimate of method precision over a range of concentrations • Can assist with the selection of new methods or instruments for your site • Compares the precision of several instruments used simultaneously by your site • Used as the basis for comparing with your laboratory’s peer group • Used at the cross‐over (parallel testing) between 2 different lot #s of control material when materials have a short out‐of date shelf life (i.e. whole blood materials) 14 On‐going Evaluation of Your Quality Control Systems Gaussian is the Key to Quality Control Alerts us to changes in Monitors and evaluates accuracy and precision method performance Sorry Umsings Ough NS EQA TEa MN Perform parallel testing to determine the Defines an acceptable change vs. observed mean and observed SD for the unacceptable change (TE < TEA) QC material Determines how far the mean can shift Determine if the system is stable or before erroneous results are reported undergoing a change (SEc and Sigma‐metric) Identify rule violations when a change occurs Determine the type of error (SE or RE) Select appropriate control rules based on the SEc or Sigma for that method present to aid with troubleshooting 5 HILS1754 Effective Date: 12/07/2016