1. Introduction {#s0005}
===============

Psychosis is a clinical syndrome associated with impairments in multiple domains including functioning, thought, behavior, and cognition. Most patients experience onset of psychosis in early adulthood, although most mental health systems fail to detect psychosis until a year or more has elapsed ([@bb0630]). However, early detection and intervention may prevent or slow the progress of the illness ([@bb0510]; [@bb0680]). There are a growing number of psychosis studies focused on predicting the onset of psychosis, examining features including speech ([@bb0125]) and cognitive deficits ([@bb0845]). Strategies to leverage technology and digital tools, including smartphones, to increase scalability and offer assessments and therapy directly to patients, hold high potential for the future ([@bb0185]; [@bb0520]; [@bb0765]; [@bb0760]; [@bb0785]).

In the early phases of psychotic spectrum illnesses like schizophrenia, patients can experience cognitive decline prior to onset of other symptoms such as delusions and hallucinations ([@bb0150]). Cognitive deficits are already present before and during the prodromal phase ([@bb0150]), but progress to more severe impairments in first-episode psychosis and may decline further afterwards ([@bb0155]; [@bb0675]). Cognitive deficits in first-episode psychosis can be experienced across all cognitive domains, with impairments of varying severity in general cognition, social cognition, processing speed, and verbal memory ([@bb0235]; [@bb0375]; [@bb0565]; [@bb0695]; [@bb0715]; [@bb0830]). Among these cognitive domains, verbal memory is often impacted and may be predictive for prognosis and functional recovery in first-episode psychosis ([@bb0010]; [@bb0020]; [@bb0295]; [@bb0565]; [@bb0705]; [@bb0750]; [@bb0830]; [@bb0850]). However, all cognitive domains decline before or in the first-episode with emerging evidence of continued decline after ([@bb0300]), including verbal memory ([@bb0160]; [@bb0845]). Research shows a possible underlying neurobiological mechanism for verbal memory deficits in first-episode psychosis, including gray matter decreases in left temporal lobe, hippocampal system, and frontotemporal circuits ([@bb0690]). Early detection is critical as cognitive remediation significantly improves verbal memory compared to other cognitive variables ([@bb0650]) and eventually mediates functional gains ([@bb0490]).

Verbal memory refers to the memory of verbally represented information ([@bb0755]) and has two main subdomains, which are declarative and non-declarative verbal memory. Among these two subdomains, declarative memory can be divided into semantic (fact memory) and episodic memory (event memory) ([@bb0010]; [@bb0735]). In order to measure verbal memory, a broad range of tasks are available including story recall, learning of word lists or sequences of paired words ([@bb0755]). Instruments used to measure verbal memory in psychiatric studies include Weschler Memory Scale (WMS), Rey-Auditory Verbal Learning Test (RAVLT), California Verbal Learning Test (CVLT), Hopkins Verbal Learning Test (HVLT), Buschke Selective Reminding Test (BSRT), Brief Assessment of Cognition in Schizophrenia (BACS), Cambridge Cognition Paired Associates Learning Test, and many others. While the literature shows the importance of verbal memory for early detection and intervention in first-episode psychosis, it is still not clear which tasks or tests are most widely used to measure verbal memory, what differential features they possess, and which may be most amenable to being translated to a digital format. There is a lack of information about the methods (computer, digital, pen-paper) used to administer these tests and frequency of follow-up measurements. In this systematic literature review, we aimed to find which verbal memory assessment tests are most widely used in first-episode psychosis (FEP) and could be applied via digital technologies (smartphone applications, chatbots etc.) with the goal of capturing dense and longitudinal data during this period of clinical and neurobiological transitions surrounding the first episode.

2. Methods {#s0010}
==========

In November 2019, we searched English language studies measuring verbal memory in first episode psychosis or first episode schizophrenia over past 10 years on PubMed (Medline) and PsycINFO databases. We chose 10 years as digital versions on mobile phones and portable consumer devices would not be available earlier. The following keywords were used in searching papers published from 2010 to date: "first episode psychosis and verbal memory" and "first episode schizophrenia and verbal memory". Inclusion criteria were first-episode psychosis and verbal memory measurement for patients. In this study, we identified a test as 'digital' if it was administered largely via a mobile device or tablet that was capable of also gathering dense and longitudinal data. While there may not be validated digital versions of some assessments, it is possible research on early adaptations has been conducted and we sought to identify these papers.

Two investigators independently screened the titles and abstracts. We excluded review studies, meta-analyses, commentaries, and editorials. We downloaded full-texts of included studies to identify the verbal memory measurement tests and tools used, follow-up frequencies, and sample sizes. The PRISMA diagram below shows the process ([Fig. 1](#f0005){ref-type="fig"}). A standardized, pre-piloted form was used to extract data from the included studies for assessment of study quality and evidence synthesis.Fig. 1PRISMA 2009 flow diagram.Fig. 1

3. Results {#s0015}
==========

We screened 233 papers and found that 120 original research studies measured verbal memory in first episode psychosis over past 10 years. Of these 120 studies, 31(%26) used Rey Auditory Verbal Learning Test (RAVLT), 29(24%) used California Verbal Learning Test (CVLT), 27(22%) used Weschler Memory Scale-Logical Memory Scale (WMS), 14(12%) used Hopkins Verbal Learning Test-Revised (HVLT), 4(3%) used both WMS and CVLT, 3(2%) used both WMS and RAVLT, and 11(10%) used other tests to measure verbal memory. Four (3%) of these studies specified that they used a computer, 24(20%) used paper-pen, 1(1%) study used both computer and pen-paper, and 91(76%) studies did not specify their verbal measurement application tools. 35(30%) studies had follow-up measurements of verbal memory, while 85(70%) studies had only a single time point verbal memory measurement during study period. Study sample sizes range from 6 to 498 ([Table 2](#t0010){ref-type="table"}).

Thirty-five (30%) studies had follow-up measurements of verbal memory, while 85(70%) studies had only a single time point verbal memory measurement during study period. Of these 35 studies, follow-up intervals range from 10 weeks to 10 years. Thirteen (39%) of these 35 studies have 1-year follow-up intervals, followed by 5(14%) studies with 6-month follow-up intervals.

3.1. Verbal memory tests ([Table 1](#t0005){ref-type="table"} and [Table 2](#t0010){ref-type="table"}) {#s0020}
------------------------------------------------------------------------------------------------------

### 3.1.1. California Verbal Learning Test (CVLT) {#s0025}

Developed in 1987 ([@bb0255]) and revised in 2000 ([@bb0260]), CVLT is one of the most widely used verbal learning and memory tests in neuropsychology ([@bb0645]). It can be used to assess both recall and recognition of the words from 4 different semantic categories ([@bb0735]). Administration time is approximately 20 min excluding 30 min delay intervals. Steps of administration are described in [Table 1](#t0005){ref-type="table"}.Table 1Verbal Memory Assessments.Table 1TestDevelopedUpdated versionAssessed ageAdministrationKey processMeasures obtainedAlternate versions availableCVLT1987CVLT-II (2000)16--8920 min and 30 min delay intervals: (1)5 trials recalling 16-word list A, (2) 1 trial Interference list B, (3) Short delay free and cued recall trials of list A, (4) 20 min delay interval of nonverbal tasks, (5) long-delay free-recall, long delay cued-recall of list A, (6) yes/no recognition trials of list AExplicit recall and recognition of semantically related words presented over multiple trialsImmediate and delayed recall, retrieval efficacy, acquisition rate, learning strategy, proactive and retroactive interference, motivationYesHVLT1991HVLT-R(2001)13--80+15 min and 20 min delay interval: (1) 3 learning trials of 12-word list, (2) a delay interval, (3) a delayed recall trial, (4) yes/no delayed recognition trialExplicit recall and recognition of semantically related words presented over multiple trialsLong Term memory, recognition, forgettingYesRAVLT1957RAVLT (1983 is the most commonly used)6--9715 min and 20 min delay interval:\
(1) 5 trials of free recall of 15-word list A, (2) Interference list of 15-word list B, (3) delayed recall of list A, (4) 20 min delay interval, (5) Recall of list A, (6) A story that uses all the words from list A and identifying of these words of list AExplicit recall and recognition of unrelated words presented overWorking and long-term memory, learning, recognition, forgetting, proactive and retroactive interference, temporal order, motivationYesWMS1945[@bb0820]16--8930 min and 30 min delay interval: (1) Immediate recall of two paragraphs read aloud, (2) 30 min delay interval, (3) Delayed recall of two paragraphs, (4) Yes/no recognition testExplicit recall and recognition of auditory informationImmediate and delayed recall and recognition of auditory material*No*[^1]Table 2Studies and Verbal Memory Tests.Table 2TestN and % of studies used including the studies used two tests (total *N* = 120)N studies used paper-pen (PP) toolsN studies used computerized (CMP) toolsN studies did not specify tools (NOS)Studies have follow-up assessments (excluding baseline assessment)List of studiesCVLT33(27%)9024*9 studies*: ([@bb0015])(*3 fup*), ([@bb0335])(*1 fup*), ([@bb0035])(*1 fup*), ([@bb0800])(*1 fup*), ([@bb3695]) (*1 fup*), ([@bb0090])(*4 fup*), ([@bb0095])(*3 fup*), ([@bb0075])(*2 fup*), ([@bb0120])(*1 fup*)PP: ([@bb0015]; [@bb0120]; [@bb0140]; Bioque et al., 2016; [@bb0285]; [@bb0335]; [@bb0440]; [@bb0585]; [@bb0835])\
CMP: --\
Both PP and CMP: --\
NOS: ([@bb0035], 2016; [@bb0070]; [@bb0075]; [@bb0080]; [@bb0085]; [@bb0090]; [@bb0095]; [@bb0175]; [@bb0240]; [@bb0250]; [@bb0245]; [@bb0280]; [@bb0290]; [@bb0330]; [@bb0345]; [@bb0385]; [@bb0465]; [@bb0570]; [@bb0600]; [@bb0605]; [@bb0635]; [@bb0740]; [@bb0800])HVLT14(12%)707*6 studies*: ([@bb0320])(*5 fup*), ([@bb0380])(*2 fup*), ([@bb0315]) (*1 fup*), ([@bb0615])(*2 fup*), ([@bb0770])(*1 fup*), ([@bb0795])(*2 fup*)PP: ([@bb0230]; [@bb0320]; [@bb0380]; [@bb0700]; [@bb0770]; [@bb0795]; [@bb0855])\
CMP: --\
Both PP and CMP:--\
NOS: ([@bb0195]; [@bb0315]; [@bb0355]; [@bb0410]; [@bb0420]; [@bb0555]; [@bb0615])RAVLT34(28%)4129*9 studies*: ([@bb0665])(*1 fup*), ([@bb0485])(*1 fup*), ([@bb0200])(*1 fup*), ([@bb0575]) (*1 fup*), ([@bb0550])(*2 fup*), ([@bb0055]) (*2 fup*), ([@bb0780])(*1 fup*), ([@bb0660])(*2 fup*), ([@bb0730]) (*2 fup*)PP: ([@bb0425]; [@bb0590]; [@bb0660]; [@bb0780])\
CMP: --\
Both PP and CMP: ([@bb0470])\
NOS: ([@bb0025]; [@bb0065], [@bb0060], [@bb0055]; [@bb0200]; [@bb0325]; [@bb0340]; [@bb0360]; [@bb0365]; [@bb0370]; [@bb0390]; [@bb0395]; [@bb0460]; [@bb0475]; [@bb0485], [@bb0480]; [@bb0500], [@bb0495]; [@bb0540]; [@bb0550]; [@bb0560]; [@bb0575]; [@bb0655]; [@bb0665]; [@bb0710]; [@bb0730]; [@bb0775]; [@bb0790]; [@bb0840])WMS34(28%)4129*10 studies*: ([@bb0665])(*1 fup*), ([@bb0015]) (*3 fup*), ([@bb0805])(*1 fup*), ([@bb0515])(*1 fup*), ([@bb0135])(*1 fup*), ([@bb0220])(*3 fup*), ([@bb0415]) (*1 fup*), ([@bb0215]) (*3 fup*), ([@bb0075])(*2 fup*), ([@bb0640])(*1 fup*),PP: ([@bb0015]; [@bb0140]; [@bb0190]; [@bb0535])\
CMP: --\
Both PP and CMP: ([@bb0470])\
NOS: ([@bb0005]; [@bb0045]; [@bb0070]; [@bb0100]; [@bb0110]; [@bb0135]; [@bb0145]; [@bb0170]; [@bb0205]; [@bb0210]; [@bb0215]; [@bb0220]; [@bb0415]; [@bb0430]; [@bb0435]; [@bb0455], [@bb0450]; [@bb0515]; [@bb0525]; [@bb0530]; [@bb0545]; [@bb0610]; [@bb0640]; [@bb0655]; [@bb0665]; [@bb0670]; [@bb0805]; [@bb0815])Other12(10%)246*3 studies*: ([@bb0305])(*1fup*), ([@bb0030])(*1 fup*), ([@bb0310])(*2 fup*)PP: ([@bb0040]; [@bb0400])\
CMP: ([@bb0180]; [@bb0265]; [@bb0305], [@bb0310])\
Both PP and CMP:\
NOS: ([@bb0030]; [@bb0445]; [@bb0580]; [@bb0620]; [@bb0685]; [@bb0745])

In our study, we found that 33 (27%) of the studies that measured verbal memory in patients with first-episode psychosis in the past ten years used CVLT. This result shows that CVLT is the second most widely used verbal memory measurement test in first-episode psychosis studies of past ten years. Of these 33 studies, 9 used paper-pen, none used computerized methods, and 24 study did not specify the tool. Nine of these studies had follow-up assessments of verbal memory while 24 of them had only a single assessment ([Table 2](#t0010){ref-type="table"}).

### 3.1.2. Hopkins Verbal Learning Test (HVLT) {#s0030}

Devised in 1991 ([@bb0165]) and updated in 1998 ([@bb0130]), HVLT is one of the most widely used verbal memory assessments, especially in dementia ([@bb0735]). HVLT is widely used by schizophrenia researchers as a part of Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Consensus Cognitive Battery ([@bb0595]). HVLT assesses verbal recall and recognition of the words from 3 different semantic categories. Administration time is approximately 15 min excluding a 20 min delay interval. Steps of administration are described in [Table 1](#t0005){ref-type="table"}.

Our results show that 14(12%) of the studies that measured verbal memory in patients with first-episode psychosis in the past ten years used HVLT. Seven of these studies used paper-pen, none used computerized methods, and 7 did not specify the tool. Six of these studies had follow-up assessments of verbal memory while 8 of them had only one ([Table 2](#t0010){ref-type="table"}).

### 3.1.3. Rey Auditory Verbal Learning Test (RAVLT) {#s0035}

Developed in early 1900s by Eduard Claparede ([@bb0225]) and published in 1958("[@bb0505] Neuropsychological Assessment. 2nd Edition, Oxford University Press, New York. - References - Scientific Research Publishing," n.d.), RAVLT is one of the first verbal memory measurement tests. Although it has been updated many times, the most widely used version is published in 1983("[@bb0505] Neuropsychological Assessment. 2nd Edition, Oxford University Press, New York. - References - Scientific Research Publishing," n.d.). RAVLT assesses working and long-term memory, learning, recognition, forgetting, proactive and retroactive interference, temporal order and motivation. Administration time is approximately 15 min excluding 20 min delay interval. Steps of administration are described in [Table 1](#t0005){ref-type="table"}.

We found 34 (28%) of the studies in the past ten years used RAVLT to assess verbal memory in first episode psychosis patient population. This shows that, excluding the studies using two different verbal memory measurement tests, RAVLT is the most widely used verbal memory tests to assess first-episode psychosis patients in psychiatric studies of the past decade. Four of these studies used paper-pen, one used computerized methods, and 29 did not specify the tool. Nine of these studies had follow-up assessments of verbal memory while 25 study had only one initial assessment ([Table 2](#t0010){ref-type="table"}).

### 3.1.4. Weschler Memory Scale-Logical Memory Scale (WMS) {#s0040}

WMS -LM was developed in 1945 ([@bb0810]) and updated to current form (WMS-IV) in 2009("WMS-IV Wechsler Memory Scale 4th Edition," n.d.). It is ranked the most widely used verbal memory measurement in neuropsychology field ([@bb0645]; [@bb0735]). Among the subtests of WMS, Logical Memory Scale assesses verbal memory. In particular, this subtest assesses immediate and delayed recall and recognition of auditory material. Administration time is approximately 30 min excluding 30 min delay interval. Steps of administration are as stated in the [Table 1](#t0005){ref-type="table"}.

We found that 34 (28%) of the studies in the past ten years used WMS to assess verbal memory in first episode psychosis patient population. This result shows that, besides of RAVLT, WMS is one of the two most widely used tests in first-episode psychosis research of past ten years including the studies used two different verbal memory measurement tests. Four of these studies used paper-pen, 1 study used computerized methods, and 29 study did not specify the tool. Ten of these studies had follow-up assessment of verbal memory while 24 study had only initial assessment ([Table 2](#t0010){ref-type="table"}).

### 3.1.5. Other verbal memory measurement tests {#s0045}

Out of 120 studies identified from our search, 12(10%) study used wide range of verbal measurement tools other than CVLT, WMS, RAVLT, and HVLT. Four studies ([@bb0400]; [@bb0580]; [@bb0685]; [@bb0745]) used Brief Assessment of Cognition in Schizophrenia (BACS), 3 studies ([@bb0040]; [@bb0445]; [@bb0620]) used Buschke Selective Reminding Test (BSRT), 2 studies ([@bb0305], [@bb0310]) used 12 Word Verbal Recall (CANTAB Software), 1 study ([@bb0265]) used 60 Word Test, 1 study ([@bb0180]) used Word Learning Test (WLT), and 1 study ([@bb0030]) used Paired Associate Learning Test (PAL) to assess verbal memory. Two of these studies used paper-pen, 4 studies used computerized methods, and 6 studies did not specify the tool. Three of these studies had follow-up assessment of verbal memory while 9 studies had only initial assessment ([Table 2](#t0010){ref-type="table"}).

4. Discussion {#s0050}
=============

We found no standardized verbal memory measurement across studies of first-episode psychosis. Although there has been an increase in use of technology in mental health services over the past decade, only a small number of the studies reviewed (5/120; 4%) used computerized methods to measure verbal memory. To date, continuous and periodic verbal memory measurement are not widely used to monitor cognitive function in first-episode psychosis. Additionally, we found that only 30% of the 120 studies assessed had a follow-up assessment of verbal memory, with the majority being cross-sectional studies. Additionally, we found that only 36 (30%) of the 120 studies had a follow-up assessment of verbal memory, with the majority being single time point studies. Most of the follow-up measurements in these studies show remarkable and continuous decline of verbal memory performance in psychosis patients. To that extent, follow-up measurements also prove that verbal memory decline in FEP can be improved or slowed with timely therapeutic interventions. However, most of these studies have 1 year or longer follow-up intervals, which decreases possibility of early detection and intervention in verbal memory decline. To date, continuous and periodic verbal memory measurement are not widely used to monitor cognitive function in first-episode psychosis.

Our results show that the most widely used verbal memory measurement tests are the RAVLT (28%) and WMS (28%), followed by CVLT (27%), and HVLT (12%). Each of the four presented here have notable differences. Three (CVLT, HVLT, RAVLT) consist of word lists while WMS consists of reading paragraphs. Additionally, WMS is the only test without alternative versions for follow-up assessments. Studies using HVLT had the greatest follow-up ratio (43%) followed by WMS (29%), CVLT (27%), and RAVLT (26%). While use of any tool does not determine whether the study will feature follow up, it is interesting to note that HVLT was featured in more studies with follow up compared to the three others. CVLT is the most comprehensive, reliable, and predictive verbal memory measurement test among the 4 main tests ([@bb0115]; [@bb0725]; [@bb0735]), and can be used for more detailed and multiple assessments of verbal memory measurement.

Few studies used technology to administer assessments. Only 5/120 studies used computerized methods and none used digital formats to assess verbal memory. Of these 5 studies, 2 ([@bb0305], [@bb0310]) used 12 Word Verbal Recall (CANTAB Software), 1 study ([@bb0265]) used 60 Word Test, 1 study ([@bb0180]) used Word Learning Test (WLT), and 1 study ([@bb0470]) used both RAVLT and WMS. In addition, of these 5 studies, only 1 study ([@bb0310]) a follow-up assessment of verbal memory. There is no detailed explanation of the computerized administration methods of WMS and RAVLT in the study of ([@bb0470]), and no visible examples of computerized or digitalized administration of HVLT or CVLT in first-episode psychosis research.

The successful digitization and validation of BACS ([@bb0050]) called the BAC App offers a practical model for digital tool conversation and adaption, as the tablet-based version offers results consistent with the paper BACS ([@bb0050]). Such results support the translation of other paper scales into other digital modalities such as smartphones. Given the increased scalability, accessibility, and portability of digital scales compared to their paper predecessors, the comparability of results supports further paper to digital translations. Other considerations to guide paper-to-digital scale translation, especially for scales to be used for longitudinal administration, include user experience (UX) testing to minimize use attrition in patients, and data security to ensure patients are comfortable engaging with digitized versions of these scales. When considering adopting digitized versions of these scales, comparisons against paper versions will need to be made in the context of measures such as convergent and discriminant validity (i.e. what are the degrees of relatedness and independence of different scale constructs). One recent study examined these validities using PCA (principal component analysis) results when comparing online (MyCQ) against paper-based (CANTAB) assessments of cognition in OCD, schizophrenia/schizoaffective disorder, and MDD patients ([@bb0270]). Another study has shown that results from digitally-administered tests can be used for separating first-episode psychosis patients from controls ([@bb0520]), which may be more useful in contributing to clinically actionable insights.

We propose that a brief verbal memory measurement via smartphone applications may facilitate longitudinal research and clinical care. Further benefits beyond longitudinal data include that timing of word reading would be uniform and spoken without an accent that might be consistent with a participant\'s expectation. Also, there would be no need for the rater to write words down while testing although often today it remains necessary for the rater to touch a box indicting the word was spoken. In the near future, natural language processing may enable fully remote testing and recognition of spoken words where a rater is not necessary. Software programs today can also automatically score results, removing any chance for human error. However, although one source of error will be removed, these tests are still scoring humans. As such, test performance validity will need to be assessed: is digital test performance impacted by patient engagement to the point that the test is no longer reflective of neurocognitive function ([@bb0350])? It will be important for these digital tests to consider their scored results in the context of phenomena such as patient effort, in order to concurrently assess performance and embedded validity indicators. The CVLT already contains such an embedded test that has been utilized for assessing psychosis inpatients ([@bb0105]), and creating and validating a digital version of this instrument will be key for fitting into a broader literature studying the interplay between effort and validity (e.g. [@bb0720]). Although efforts have been made to create a method that would lend itself to automated assessment of performance validity ([@bb0825]), attempts to generalize these methods have encountered varying measures of success ([@bb0275]). In addition, these assessments will need to be considered in the context of other digital systems that may affect effort and engagement, such as adherence-promoting reward systems ([@bb0625]) and other gamification-based elements of apps that integrate testing with multiple dimensions of care.

While validating any new assessment is not a simple process, the potential of longitudinal cognitive data combined with digital phenotyping data offering concomitant data on environments, social functioning, symptoms, and even physiology offers a new window into cognitive assessment the field should not ignore. Such a digitalized brief verbal memory assessment may be able to detect and monitor verbal memory changes in first-episode psychosis patients remotely and continuously, without burnout of the users and administrators. Considering smartphone use is common among patients with psychosis ([@bb0765]; [@bb0760]), we believe that a digital verbal memory assessment using the most-widely used assessments and digital mental health applications bears investigation and validation. Indeed, verbal memory tests are already being administered in a digital format and enabling automated algorithm-based scoring of story recall ([@bb0405]). Validation and developing norms around new digital scales via international deployment will be an especially salient opportunity given the previously discussed scalability of these technologies. While pen-and-paper neurocognitive assessments have been extensively assessed, global studies are now possible that take advantage of ubiquitous technologies such as smartphones that will allow for cross-cultural testing, rapid prototyping and retesting, and portable updates that can be applied to multiple platforms concurrently and in a monitorable, quantifiable manner. It will be informative study how these digital formats can best be taken advantage of different languages and cultures in standardizing assessments.

Our paper has several limitations. The search term used was broad and will have missed several studies that did not feature the keywords searched for in the title and abstract. For example, several studies utilize the verbal memory tests in the BACS and CANTAB battery but these did not show up in our title and abstract search. We also did not search [clinicalstudies.gov](http://clinicalstudies.gov){#ir0005} and thus our results reflect published research and not ongoing efforts which may be more likely to utilize newer technologies.

5. Conclusion {#s0055}
=============

Verbal memory measurement can predict clinical and functional disability in first-episode psychosis patients. Although the literature supports the importance of verbal memory measurement in first-episode psychosis, we found no universally used and recommended verbal memory measurement test in first-episode psychosis literature. Although many verbal memory tests are widely used, almost none of them have a computerized or digitalized administration example -- although some like the BACS do. Creating and validating further assessment tools that can gather longitudinal data offers a new window in cognition that when combined with other data streams from mobile devices could present a useful new tool for the field.

Declaration of competing interest
=================================

CMK, JB, DO, JT have no relevant COI. DO is supported by funding from the NIMH P50MH115846. JT is supported by funding from the NIMH K23 MH116130--02.

RSEK is an employee of Duke University and in the past 3 years has received research funding from the National Institute of Mental Health and Boehringer Ingelheim; in the past 3 years he has also received honoraria, served as a consultant, speaker, or advisory board member for AbbVie, Acadia, Aeglea, Akebia, Akili, Alkermes, Allergan, ArmaGen, Astellas, Avanir, AviNeuro/ChemRar, Axovant, Biogen, Boehringer-Ingelheim, Cerecor, CoMentis, Critical Path Institute, FORUM, Gammon Howard & Zeszotarski, Global Medical Education (GME), GW Pharmaceuticals, Intracellular Therapeutics, Janssen, Kempharm, Lundbeck, Lysogene, MedScape, Mentis Cura, Merck, Merrakris Therapetics, Minerva Neurosciences Inc., Mitsubishi, Montana State University, Monteris, Moscow Research Institute of Psychiatry, Neuralstem, Neuronix, Novartis, NY State Office of Mental Health, Orygen, Otsuka, Paradigm Testing, Percept Solutions, Pfizer, Pharm-Olam, Regenix Bio, Reviva, Roche, Sangamo, Sanofi, SOBI, Six Degrees Medical, Sunovion, Takeda, Targacept, Teague Rotenstreich Stanaland Fox & Holt, Thrombosis Research Institute, University of Moscow, University of Southern California, University of Texas Southwest Medical Center, WebMD, and Wilson Therapeutics. He receives royalties from versions of the BAC testing battery, the MATRICS Battery (BACS Symbol Coding), and the Virtual Reality Functional Capacity Assessment Tool (VRFCAT) and is a shareholder in VeraSci and Sengenix.

[^1]: This table is adapted from "A compendium of neuropsychological tests" ([@bb0735]). Copyright by the Oxford Publishing Limited. Reproduced with permission of the Licensor through PLSclear.
