Development and validation of an automated algorithm for end point adjudication for a large U.S. national registry

Background: Clinical events committee (CEC) evaluation is the standard approach for end point adjudication in clinical trials. Due to resource constraints, large registries typically rely on site-reported end points without further confirmation, which may preclude use for regulatory oversight.

Methods: We developed a novel automated adjudication algorithm (AAA) for end point adjudication in the National Cardiovascular Data Registry Left Atrial Appendage Occlusion (LAAO) Registry using an iterative process using CEC adjudication as the "gold standard." A ≥80% agreement rate between automated algorithm adjudication and CEC adjudication was prespecified as clinically acceptable. Agreement rates were calculated.

Results: A total of 92 in-hospital and 127 post-discharge end points were evaluated between January 1, 2016 and June 30, 2019 using AAA and CEC. Agreement for neurologic events was >90%. Percent agreement for in-hospital and post-discharge events was as follows: ischemic stroke 95.7% and 94.5%, hemorrhagic stroke 97.8% and 96.1%, undetermined stroke 97.8% and 99.2%, transient ischemic attack 98.9% and 98.4% and intracranial hemorrhage 100.0% and 94.5%. Agreement was >80% for major bleeding (83.7% and 90.6%) and major vascular complication (89.1% and 97.6%). With this approach, <1% of site reported end points require CEC adjudication. Agreement remained very good during the period after algorithm derivation.

Conclusions: An AAA-guided approach for end point adjudication was successfully developed and validated for the LAAO Registry. With this approach, the need for formal CEC adjudication was substantially reduced, with accuracy maintained above an 80% agreement threshold. After application specific validation, these methods could be applied to large registries and clinical trials to reduce the cost of event adjudication while preserving scientific validity.