ISO 13528 STATISTICAL METHODS USED IN PROFICIENCY TESTING THROUGH INTERLABORATORY COMPARISONS

ISO 13528:2022 is an international standard that covers the statistical methods used in proficiency testing through interlaboratory comparisons. This standard provides detailed information on the design of proficiency tests, the analysis of the data obtained, and the interpretation of the results.

Purpose of the Standard

• Increase the reliability of proficiency tests: The standard aims to enhance the reliability of results by ensuring that proficiency tests are conducted on a scientifically and statistically sound basis.
• Evaluate laboratory performance: It provides an objective framework for assessing how competent laboratories are in specific testing methods or measurement processes.
• Guide accreditation bodies: The standard guides accreditation bodies in the process of accrediting proficiency tests by providing evaluation criteria and acceptability limits.

Scope of the Standard

• Design of proficiency tests: It addresses topics such as statistical methods and sample size calculations for designing different types of proficiency tests (e.g., open, blind, double-blind).
• Data analysis: It provides guidance on suitable methods for statistical analysis of obtained data (e.g., analysis of variance, regression analysis) and interpretation of results.
• Evaluation of results: It provides information on statistical measures used to assess the performance of participant laboratories (e.g., z-score, Cohen's kappa) and acceptability limits.

Importance of the Standard

• Quality assurance: The reliability of measurement results affecting the quality of products and services can be verified through proficiency tests conducted in accordance with the ISO 13528 standard.
• International acceptance: This standard is a widely accepted reference point used in many countries.
• Continuous improvement: It helps laboratories continuously improve their performance.

Who Is It Important For?

• Laboratories: This standard is of great importance for laboratories, especially those that want to become accredited or maintain their current accreditation.
• Proficiency test providers: Organizations that conduct proficiency tests should design and evaluate their tests in accordance with this standard.
• Accreditation bodies: Accreditation bodies use the ISO 13528 standard as a basis for evaluation during the laboratory accreditation process.
• Customers: For customers who want to know the competence of the laboratories from which they receive products or services, this standard is an important indicator.

ISO 13528:2022 is an important document that covers statistical methods used for the reliable evaluation of laboratory performance. This standard provides numerous benefits for both laboratories and their customers.

Objectives of Proficiency Testing

Within the scope of the ISO 17043 standard, proficiency testing (PT) involves the use of interlaboratory comparisons to determine the performance of participants (which can include laboratories, accreditation bodies, or individuals) for specific tests or measurements and to monitor their ongoing performance. Most importantly, laboratory performance evaluation, identifying issues within laboratories, determining the effectiveness and comparability of test or measurement methods, providing additional assurance to laboratory clients, validating uncertainty claims, and training of participating laboratories are included. The applied statistical design and analytical techniques should be suitable for the specified objective(s).

Rationale for Scoring in Proficiency Test Schemes

Various scoring strategies are available and used for proficiency testing. While the detailed calculations may differ, most proficiency test schemes compare the participant's deviation from an assigned value to a numerical criterion used to determine whether the deviation is concerning. Therefore, strategies used to choose a criterion for the assignment of values and the evaluation of participant deviations are critical. Specifically, it is important to consider whether the assigned value and the evaluation criterion for deviations should be independent of the participant results or derived from the results provided. The ISO 13528 standard provides both strategies. However, attention is drawn to the discussion on the advantages and disadvantages of choosing assigned values or evaluation criteria that are not derived from participant results, as found in ISO 13528 Sections 7 and 8. Generally, it will be observed that choosing assigned values and evaluation criteria independent of participant results provides advantages.

This is particularly true for criteria used to evaluate deviations from the assigned value, such as standard deviation or measurement error for proficiency evaluation. In such criteria, a consistent selection based on the suitability of measurement results for a specific final use is particularly useful.

ISO 13528 and ISO/IEC 17043

This document provides support for the implementation of ISO/IEC 17043, particularly regarding the requirements for statistical design, validation of proficiency test items, review of results, and reporting of summary statistics. The statistical techniques used for a specific proficiency test scheme are expected to evolve as the scheme matures; scores, evaluation criteria, and graphical techniques will be developed to better meet the specific needs of the target participant group, accreditation bodies, and regulatory authorities.

ISO 13528 includes published guidance for proficiency testing in chemical analytical laboratories but also contains a broader range of procedures that can be used along with valid measurement methods and qualitative definitions.

ISO 13528 incorporates statistical techniques that are consistent with other international standards, particularly TC69/SC6, and specifically the ISO 5725 series on Accuracy: Precision and Accuracy. The techniques are also intended to reflect other International Standards where appropriate and aim to be consistent with ISO/IEC Guide 98-3 (GUM) and ISO/IEC Guide 99 (VIM).

Statistical Expertise

ISO/IEC 17043 requires that the proficiency test provider has access to statistical expertise and must authorize specific personnel to conduct statistical analysis. Neither ISO/IEC 17043 nor ISO 13528 can specify in detail what this required expertise entails. For some applications, an advanced degree in statistics is beneficial, but generally, the expertise requirements can be met by individuals with technical expertise in other areas who are familiar with basic statistical concepts and have experience or training in applying common techniques for analyzing data from proficiency test schemes. If a person is responsible for statistical design and/or analysis, it is crucial that this person has experience in interlaboratory comparisons, even if they have an advanced degree in statistics. Traditional advanced statistical training typically does not include exercises involving interlaboratory comparisons, and the unique causes of measurement error in proficiency testing may appear unclear. The guidance in this document cannot provide all the expertise needed to assess all applications and cannot replace the experience gained by working with interlaboratory comparisons.

Computer Software

The computer software required for the statistical analysis of proficiency test data can vary widely, from simple spreadsheet arithmetic using known reference values for small proficiency test schemes, to complex statistical software used for statistical methods based on iterative calculations or other advanced numerical techniques. Most of the techniques in this document can likely be carried out using conventional spreadsheet applications with routines customized for a particular proficiency test scheme or analysis; some techniques will require computer applications that are freely available. In all cases, users are expected to validate the correctness and accuracy of their calculations, especially when custom routines are entered by the user. However, even if the appropriate computer applications correctly and appropriately apply the techniques of the ISO 13528 standard, they cannot be applied without the attention of an individual with sufficient technical and statistical expertise to understand the nature of the applications and statistical assumptions, and to identify and investigate anomalies that may occur in any round of a proficiency test scheme.

Scope

ISO 13528 provides detailed explanations of the statistical methods that proficiency test providers can use to design proficiency test schemes and analyze data obtained from these schemes. ISO 13528 also provides recommendations for the interpretation of proficiency test data by participants in these schemes and accreditation bodies.

The procedures contained in ISO 13528 can be applied to show that measurement results obtained by laboratories, inspection bodies, and individuals meet the specified criteria for acceptable performance.

ISO 13528 can be applied to proficiency tests where the results are either quantitative measurements or qualitative observations regarding test items.

ISO 13528 Standard Content

4 General Principles
4.1 General requirements for statistical methods
4.2 Basic model
4.3 General approaches for performance evaluation

5 Guide for the statistical design of proficiency testing programs
5.1 Introduction to the statistical design of proficiency testing programs
5.2 The foundation of a statistical design
5.3 Considerations for the statistical distribution of results
5.4 Points to consider for a small number of participants
5.5 Guidelines for selecting a reporting format
5.5.1 General requirements for reporting format
5.5.2 Reporting repeated measurements
Reporting as "less than" or "greater than" a limit (censored data)
5.5.4 Significant figures

6 Guide for the initial review of proficiency test samples and results
6.1 Homogeneity and stability of proficiency test samples
6.2 Considerations for different measurement methods
6.3 Error removal
6.4 Visual review of data
6.5 Robust statistical methods
6.6 Outlier techniques for individual results

7 Determination of the assigned value and standard uncertainty
7.1 Selection of methods for determining the assigned value
7.2 Determining the uncertainty of the assigned value
7.3 Formulation
7.4 Certified reference material
7.5 Results obtained from a laboratory
7.6 Consensus value obtained from expert laboratories
7.7 Consensus value derived from participant results
7.8 Comparison of the assigned value with an independent reference value

8 Defining criteria for performance evaluation
8.1 Approaches for defining evaluation criteria
8.2 Based on expert perception
8.3 Based on previous rounds of a proficiency testing program
8.4 Using a general model
8.5 Using repeatability and reproducibility standard deviations from previous collaborative studies on a measurement method’s precision
8.6 From data obtained in the same round of a proficiency testing program
8.7 Monitoring interlaboratory agreement

9 Calculation of performance statistics
9.1 General considerations for determining performance
9.2 Limiting uncertainty of the assigned value
Deviation estimates (measurement error)
9.4 Z-scores
9.5 Z' scores
Zeta scores (ζ)
9.7 Scores
9.8 Evaluating participant uncertainties in the test
9.9 Combined performance scores

10 Graphical methods for defining performance scores
10.1 Application of graphical methods
10.2 Histograms of results or performance scores
10.3 Kernel density plots
10.4 Bar graphs of standardized performance scores
10.5 Youden plot
10.6 Plots of repeatability standard deviations
10.7 Split samples
10.8 Graphical methods for combining performance scores over several rounds of a proficiency testing program

11 Design and analysis of qualitative proficiency testing programs (including nominal order properties)
11.1 Types of qualitative data
11.2 Statistical design
11.3 Assigned values for qualitative proficiency testing programs
11.4 Performance evaluation and scoring for qualitative proficiency testing programs

Appendix A (Normative) Symbols
Appendix B (Informative) Homogeneity and stability of proficiency test samples
Appendix C (Informative) Robust analysis
Appendix D (Informative) Additional guidance on statistical procedures
Appendix E (Informative) Illustrative examples
Appendix F (Informative) Computer code example for drawing PT results and re-sampling analysis ("bootstrapping")

Comprehensive Information on ISO 13528:2022 Standard Training

The ISO 13528:2022 standard is a crucial international standard that comprehensively covers the statistical methods used in interlaboratory comparison tests. This standard aims to scientifically base the methods used to evaluate laboratories' competence and ensure the reliability of results.

Training Objective

The ISO 13528:2022 standard training aims to provide participants with knowledge and skills in the following areas:

• Purpose and importance of proficiency testing: Understanding why interlaboratory comparison tests are conducted, how these tests affect laboratory quality, and their role in accreditation processes.
• Fundamental concepts of statistical methods: Learning the basic principles of statistical methods such as data analysis, variance analysis, and regression analysis.
• Requirements of ISO 13528:2022: A detailed examination of the standard's content and learning which statistical methods should be used in interlaboratory comparison tests.
• Data analysis software: Gaining practical knowledge on the use of statistical analysis software such as SPSS, Minitab, etc.
• Reporting: Acquiring the necessary skills to report the results accurately and understandably.

Who Should Attend?

• Laboratory managers: Managers who want to evaluate and improve the competence of their laboratories.
• Quality assurance specialists: Experts aiming to improve the effectiveness of quality systems and ensure laboratories comply with international standards.
• Laboratory technicians: Technicians who aim to ensure the accuracy and reliability of test results.
• Accreditation body staff: Personnel involved in the evaluation of laboratories in accreditation processes.
• Statisticians: Statisticians working in this field and wishing to enhance their knowledge.

Training Content (Generally)

• Introduction: The importance of interlaboratory comparison tests and the purpose of the ISO 13528 standard.
• Basic concepts of statistical methods: Data distributions, measures of central tendency, measures of variability.
• Designing proficiency tests: Sample selection, test protocols, data collection.
• Data analysis: Data cleaning, visualization, statistical tests.
• Evaluating results: Performance evaluation criteria, reporting.
• Software applications: Practical applications using statistical analysis software.

What Will You Gain After the Training?

• Comprehensive knowledge of the ISO 13528:2022 standard.
• Ability to design and analyze interlaboratory comparison tests.
• Skills to effectively use data analysis software.
• Ability to increase the reliability of test results.
• Improved success in international accreditation processes.

Participant Profile: Organizers, laboratories wishing to be accredited according to ISO 17043 Standard, laboratory managers who want to apply, develop, and manage these systems, and HR professionals, laboratory staff who want to build a career in laboratory standards can participate in this training.

Exam: An exam may be conducted at the end of the training.

Instructor: The training is provided by our International Lead Auditors and Training Experts. Announced before the training.

Training Date: Announced before the training.

Training Duration: 3 days.

Training Location: Online/In-house.

Certificate: A certificate will be provided to participants at the end of the training.