Measuring the diagnostic accuracy (DA) of an EEG device is unconventional and complicated by imperfect interrater reliability. We sought to compare the DA of a miniature, wireless, battery-powered EEG device (“microEEG”) to a reference EEG machine in emergency department (ED) patients with altered mental status (AMS). Two hundred twenty-five ED patients with AMS underwent 3 EEGs. Two EEGs, EEG1 (Nicolet Monitor, “reference”) and EEG2 (microEEG) were recorded simultaneously with EEG cup electrodes using a signal splitter. The remaining study, EEG3, was recorded with microEEG using an electrode cap immediately before or after EEG1/EEG2. The official EEG1 interpretation was considered the gold standard (EEG1-GS). EEG1, 2, and 3 were de-identified and blindly interpreted by two independent readers. A generalized mixed linear model was used to estimate the sensitivity and specificity of these interpretations relative to EEG1-GS and to compute a diagnostic odds ratio (DOR). Seventy-nine percent of EEG1-GS were abnormal. Neither the DOR nor the κf representing interrater reliabilities differed significantly between EEG1, EEG2, and EEG3. The mean setup time was 27min for EEG1/EEG2 and 12min for EEG3. The mean electrode impedance of EEG3 recordings was 12.6kΩ (SD: 31.9kΩ). The diagnostic accuracy of microEEG was comparable to that of the reference system and was not reduced when the EEG electrodes had high and unbalanced impedances. A common practice with many scientific instruments, measurement of EEG device DA provides an independent and quantitative assessment of device performance.