How to Report Qualitative IVD Precision Near the Decision Boundary

What a near-boundary precision report should let you see

A qualitative in vitro diagnostic (IVD) assay gives alternating positive and negative calls when repeated aliquots are tested near its established decision boundary. The RA/QA reviewer needs to see which concentrations were tested, how often each result occurred, and under which conditions. Start with a concentration-by-site table that retains every initial result category and makes its denominators reproducible.

For the reporting approach used here, show both total initial attempts and valid binary outputs. Keep initial invalid and applicable equivocal results visible, then link any follow-up results to their originating attempts. State which variation factors were varied, held fixed or confounded. Assess the pattern against the assay's established result rules, justified study criteria and applicable guidance.

Nominal panel concentration describes the analyte level assigned to prepared material. Categorical device output describes the call returned on an individual replicate. Keep those fields separate so readers can follow one fixed concentration across repeated tests. Pooling all concentrations at the outset discards the level-specific detail needed to examine the boundary. The worked example below shows exactly how that information can disappear.

Identify the cutoff and the guidance that apply

First identify whether the assay uses an analytical detection cutoff or a cutoff selected from clinical data. Record any equivocal or retest zone. The dated sources below address different devices and submission settings. Their study recommendations require a documented applicability assessment for the assay under review.

This map identifies the supporting sections and the limited purpose for which each source is used here. The tables that follow provide an editorial reporting proposal.

A synthetic repeat-result table near the boundary

The synthetic panel has five levels at 0, 8, 10, 12 and 20 arbitrary units, with 10 identifying the boundary neighborhood in this teaching scenario. Each level has twenty initial attempts at each site, giving sixty attempts per level. In each site cell, P/N/I gives positive, negative and invalid initial counts in that order. The valid positive fraction uses P/(P + N). The initial invalid fraction uses I/(P + N + I). This example uses binary valid outputs.

F0 and F4 each have sixty valid results, all negative and all positive respectively. F1 has three positives and fifty-seven negatives. F3 has fifty-seven positives and three negatives. F2 has thirty positives, twenty-nine negatives and one invalid result. Every row therefore reconciles to sixty initial attempts, while the valid binary denominator changes at F2. The displayed percentages are calculated from those constructed counts and rounded to two decimal places.

Expanding F2 makes the denominator difference visible at the site where it arises. At Site C, eleven positives and eight negatives form the nineteen valid binary outputs. The invalid result is retained in the twenty-attempt total. The combined row adds the category counts across sites before calculating its percentage.

Site C's valid positive fraction is 11/19 = 57.89%, while its initial invalid fraction is 1/20 = 5.00%. Across sites, the valid positive fraction is 30/59 = 50.85% and the invalid fraction is 1/60 = 1.67%. These small fictional counts demonstrate accounting. A statistical comparison of site performance would require an appropriate design, analysis and uncertainty assessment using actual data.

Reconstruct each denominator from the replicate record

For this example, initial attempts means executed first attempts on assigned panel replicates. Planned replicates belong in a separate execution reconciliation, including any that were missed. Valid binary outputs are the initial attempts returning positive or negative under the specified result rules. For F2, 30 + 29 = 59 valid binary outputs and 59 + 1 = 60 initial attempts. The positive percentage is therefore 30/59, and the initial invalid percentage is 1/60. An assay with equivocal outputs needs that additional category and an explicit denominator definition.

Suppose the single invalid F2 attempt receives one protocol-defined follow-up test that returns positive. Keep the initial sixty-attempt table, and add the follow-up record linked to that invalid attempt. There are now sixty-one executed test records across the initial and follow-up populations. If the assay's predefined resolution rule uses that positive follow-up to resolve the original attempt, a separate final-outcome summary would contain thirty-one positives and twenty-nine negatives, giving 31/60 = 51.67%. Label that analysis population clearly so readers can reconcile it with the initial 30/59 result and initial 1/60 invalid fraction.

flowchart TD
    A["Executed initial attempt with unique ID"] --> B{"Initial output category"}
    B -->|"Positive or negative"| C["Count in initial-attempt and valid-binary totals"]
    B -->|"Equivocal when applicable"| D["Retain equivocal count and defined denominator"]
    B -->|"Invalid"| E["Retain invalid count and initial-attempt denominator"]
    D --> F{"Follow-up specified by assay protocol?"}
    E --> F
    F -->|"Yes"| G["Create linked follow-up record and record material source"]
    F -->|"No"| H["Retain original category in reporting"]
    G --> I["Report follow-up and final outcomes with defined populations"]

The following field set is an editorial proposal for reconstructing the worked tables. Use the assay protocol and data system to determine the implemented record structure. A reviewer should be able to start at a displayed cell, retrieve its contributing attempts, and follow any subsequent test through its parent-attempt link.

Set analysis populations and exclusion rules in the protocol. For an already completed study, report when those rules were established and document changes and deviations as they occurred. Present initial, follow-up and final-outcome populations with their own labels and denominators. Related workflow topics include consumer invalid-result handling and specimen-level comparator disagreements. Home-use invalid-result workflows. Reporting discordant IVD comparisons.

A useful arithmetic check starts at the smallest reporting cell. At every site and concentration, add positive, negative and invalid counts and compare the sum with executed initial attempts. Then add corresponding categories across sites and reproduce the combined denominator. For an assay with additional mutually exclusive categories, include every category in that reconciliation. Record missed planned replicates separately so a lower execution count remains visible.

Across all five synthetic levels, the table contains 300 initial attempts, 150 positives, 149 negatives and one invalid result. Pooling them gives 150/299 = 50.17% positive among valid initial attempts. That single percentage loses the distinction between F0, which is entirely negative, and F4, which is entirely positive. It also conceals the location of the invalid event. Start interpretation with the individual concentration rows and retain their counts through any subsequent summary.

Check combined percentages using summed numerators and summed denominators. For F2, the unweighted average of the three site percentages gives equal weight to nineteen valid outputs at Site C and twenty at each other site. The displayed combined fraction instead uses all thirty positives divided by all fifty-nine valid binary outputs. This distinction matters whenever the contributing denominators differ. Keep calculation precision until the final rounding step so readers can reproduce the displayed value.

State which sources of variation the study covered

Within-run replicates under fixed conditions address within-run repeatability. A design spanning sites, days, operators, lots or instruments examines the factors it actually varies. In this synthetic example, only concentration, site labels and outcome counts are specified. Operator, lot, instrument, extraction and day/run allocations would need separate records in a real study. Match every precision claim to those execution records and the analysis that the allocation supports.

In the Dual Submission setting, assigning one different reagent lot to each site prevents separation of site and lot effects. Section V.B describes evaluating multiple lots at each site, or a separate internal lot study while sharing the same lot across sites in the main study. Record the chosen allocation and its limits. FDA Dual Submission guidance, Section V.B.

Use a coverage matrix to make the execution design visible alongside the result tables. The entries below are proposed reporting fields. Replace the example descriptions with the actual allocation, including any factors held fixed or left unexamined.

For studies within the respiratory-panel guidance scope, Section 6.B.v also calls for per-sample numeric summaries and variance components, with factor-specific and overall assessment. Retain those analyses alongside the categorical table when applying that source. Select analyses for another assay through its own applicability assessment. FDA respiratory-panel guidance, Section 6.B.v.

Interpret alternating calls within the assay’s evidence

Read the completed table together with the panel definition and study design. F2 shows an almost even division of valid positive and negative calls at the fictional boundary level. That describes the constructed output pattern. An assessment of an actual assay also needs its expected response behavior, acceptance criteria, uncertainty and execution conditions.

The migration guidance discusses a C5–C95 interval that can be narrow relative to clinical specimen concentrations in ultrasensitive assays, making some panel members difficult to prepare. It separately addresses equivocal and retest zones. Match the assay situation to the appropriate part of that discussion before selecting or interpreting panel levels. FDA assay-migration guidance, Section VI.A.1.b(i).

F1 happens to produce 3/60 = 5.00% positive and F3 produces 57/60 = 95.00% positive in the fictional data. Those fractions provide an arithmetic illustration. Estimating concentrations associated with response probabilities requires actual study data, a justified method and its assumptions, and an uncertainty assessment. Keep an observed fraction attached to its tested concentration and conditions. Any clinical-performance conclusion also needs the evidence appropriate to that claim.

Prepare the reporting package for review

Prepare the reporting package so an independent reviewer can trace each summary to its supporting evidence. Broader validation, strain-and-matrix claim interpretation and submission strategy have separate coverage. IVD analytical performance validation. Reading respiratory-panel LoD claims. CLIA waiver submission strategy.

• Identify each panel member, its assigned concentration and units, matrix, preparation method and material traceability. State how the selected levels relate to the established assay boundary.

• Record the protocol and interpretation-rule versions, result categories, cutoff or zone definitions and follow-up algorithm used during the study. Explain any deviations and their effect on analysis.

• Present complete initial category counts at each concentration and site, combined totals and clearly labeled percentages. Include the underlying numeric-output summaries and analyses required by the applicable study approach.

• Reconcile planned replicates, executed initial attempts, valid binary outputs and other result categories. Retain linked follow-up records and define the population used for each final-outcome analysis.

• Provide the variation-factor matrix and execution allocation. Describe fixed and unexamined factors, confounding and the analyses supported by the actual design.

• State justified assay-specific acceptance criteria, sample-size rationale, statistical methods and uncertainty. Record the evidence and reasoning behind the resulting assessment.

• State the scope and limits of the conclusions, including the panel material, intended users and tested conditions. Record the guidance versions and the applicability assessment used for the report.

Sources

• Respiratory Viral Panel Multiplex Nucleic Acid Assay — Class II Special Controls Guidance for Industry and FDA Staff

• Assay Migration Studies for In Vitro Diagnostic Devices — Guidance for Industry and FDA Staff

• Recommendations for Dual 510(k) and CLIA Waiver by Application Studies — Guidance for Industry and Food and Drug Administration Staff

• CLSI Publishes the Third Edition of CLSI EP12—Evaluation of Qualitative, Binary Output Examination Performance