Objective An automated cognitive neurophysiological test is presented that characterizes how an individual was affected by a drug or treatment. The test calculates sub-scores for working memory task performance, cortical activation, and alertness, and combines the sub-scores into an overall score. Methods The test was applied in a double-blind, placebo-controlled study of alcohol, caffeine, diphenhydramine, and sleep deprivation in 16 healthy adults. Results The between- and within-day variability of the sub-scores and overall scores for placebo were all near zero, suggesting that the scores are stable. All treatments affected the overall score, while differential effects on sub-scores highlighted the added value of EEG measures. Conclusions The test is sensitive to relatively mild alterations in cognitive function. Its automation makes it suitable for use in large-scale clinical trials. Significance By combining task performance with EEG brain function measures, the test may prove to have better sensitivity and specificity in detecting changes due to drugs or other treatments than comparable neuropsychological test batteries that do not directly measure brain function signals.