Diagnostic startup Theranos struggles with testing technology
Theranos, a $9bn California-based company which offers over 240 types of diagnostic blood test, has struggled to use the lab instrument developed as the linchpin of its strategy. The device, named Edison, draws a tiny amount of blood from a patient’s finger. Former employees claim that Edison lacks accuracy and is used to handle just a small fraction of the tests then sold to consumers. Failure to disclose concerns about inaccuracy could be a violation of federal rules for laboratories.

Theranos has also failed to disclose its use of traditional machines purchased from companies such as Siemens. The company disputes the allegations but admits to not using the device for all tests that it offers. A formal complaint about testing at Theranos was received by the New York State Department of Health in April 2014 and was forwarded it to the Centers for Medicare and Medicaid Services.  Asked about the complaint, Theranos confirmed that the Edison system produced results for several tests last year that differed from results obtained from traditional equipment.

Theranos has previously received criticism from the scientific community over its secretive practices, but went some way to nullifying this criticism with its FDA approval for detection of the  herpes simplex virus, which required large amounts of data to be submitted to the FDA.

Recently, spectacular advances in diagnostic technologies, genomics, etc. offer unprecedented opportunities for widespread testing of asymptomatic individuals, in the hope that this testing will unravel early disease signs which could lead to preventative or more effective therapeutic measures. In particular, one commercial organization, Theranos, promises to revolutionize diagnostics by offering multi-analyte testing at low prices in commercial outlets, thus challenging the current paradigm of targeted and centralized diagnostic testing. In this paper, I analyze the Theranos technology and their promises, and contrast this information with the currently used technologies, to show that most of the company’s claims are exaggerated. While it remains to be seen if this technology will revolutionize diagnostics, in this Opinion Paper, I also draw attention of associated issues, such as self-testing and self-interpretation of results, over-testing, over-diagnosis and over-treatment, along with their associated harms. As the public is bombarded daily with new and revolutionary health-related advances, it is time to balance the enthusiasm of the seemingly obvious huge gains, by also explaining the associated possible harms.

Introduction My Vice-President of Operations at a major teaching Hospital in Toronto asked me to update him on a “new” company called Theranos that promises to revolutionize diagnostics by offering multi-analyte testing at bargain prices in commercial outlets such as pharmacies [1]. The interest of my Vice-President was triggered by an inquiry from the recently appointed Hospital President, who was wondering as to how Theranos could do diagnostic testing at 10% of the cost of centralized laboratories. I assumed from these discussions that my administrators are seriously thinking of adopting some kind of a “Theranos model” to drastically reduce current laboratory costs. Theranos seems to be a highly successful enterprise that managed to raise hundreds of millions of dollars in multiple rounds of fundraising and has a stellar governance body. The company was mentioned as an example of discovering disruptive technologies [2] and their technologies as being one of the top 10 medical and technological innovations in 2013 [3]. The young executive of Theranos was included in 2006 in the best “30 under 30” group [4] and labeled as “lifesaver” [4]. Are all these accolades enough to guarantee that the company will deliver the promised goods in healthcare? The answer is no. History teaches us that there are numerous examples of seemingly disruptive technologies in healthcare (and especially in diagnostics), developed by high-profile scientists, including Nobel Laureates, which later collapsed, due to their inability to deliver the promised goods. A few examples have been highlighted by this author elsewhere [5]. In this paper, I will examine the Theranos technology and model, and compare them to current technologies used in centralized laboratories and similar settings.

Theranos technology Details of the Theranos technology have not been disclosed to scientific journals and for this reason it is not possible to comment. In general, the technology involves a fingerstick, to draw a few microliters of blood into a disposable cartridge, which is then loaded into a “reader” for analysis. Results are sent wirelessly from the reader to a secure database, and then to the patient or patient’s physician. The reader can be at the same or different location from the site of blood collection. The perceived benefits include fast results and capability to analyze panels of tests (e.g., up to 30 blood tests can be performed on a single sample), thus reducing the cost per test. The following comments apply: The quality of the results are not known since the Theranos system has not been independently evaluated, nor do any published results exist to compare with conventional technologies. New diagnostic tests must be evaluated for their accuracy, precision, specificity and long-term robustness. Trueness and precision (accuracy) need to be maintained over months or years, and monitored by external quality assurance programs, so that patient’s data can be directly compared over long periods of time [6]. Without independent validation, Theranos technology’s quality and robustness will remain in question.

Speed of results Theranos claims that the usual delay of testing in centralized laboratories is approximately 3 days and that they will generate and deliver their data much faster (e.g., within 4 h). The 3-day delay claim is not accurate. The bulk of laboratory testing in centralized laboratories is completed within an hour or two (calculated from time of sample collection to time of results posting for physician review). For example, in our laboratory, more than 90% of creatinine and troponin requests from all wards are completed in <1 h and more than 97% in <2 h. It is thus questionable that Theranos’s technology will be able to deliver faster results than the ones mentioned. It is true that some tests, which are used for monitoring chronically ill patients (e.g., parathyroid hormone for end-stage renal disease patients) are submitted to centralized laboratories with turnaround times of days, instead of hours, but faster results in such cases are not critical for adjusting patient management. Consequently, faster analysis will not have a major impact on patient outcomes.

Costs It is claimed by Theranos that their cost per test is much lower (in the order of 10%) compared to the cost of centralized laboratories. In fact, the reagents/consumables costs of centralized laboratories are, in general, likely much lower than those of Theranos. For example, the cost of the reagent alone for running a commonly ordered test (i.e., glucose) is <1 cent. The majority of centralized laboratory costs are related to overhead and personnel costs, rather than the technology itself [7]. Also, it would not be appropriate to calculate costs per test based on multiparametric panels. For example, if a 30 analyte profile costs $30, the cost per test will be a $1 per test, but if the other 29 tests are not necessary, there is no benefit of running more tests. In fact, as I have explained in detail elsewhere [8], panel profiling, which was introduced in the 1970s as a way of identifying early biochemical changes of disease in asymptomatic individuals, had been abandoned in the 1980s, not so much for the cost. It has long been realized that with multiparametric testing, approximately 5% of results will be false positives, i.e., test results outside the reference intervals, in otherwise normal subjects. This is due to the definition of reference intervals, as being values between the 2.5 and 97.5 percentile of a reference (normal) population. The high cost of investigating seemingly abnormal results in normal people, and the added anxiety of patients, has led to the complete replacement of such biochemical profiling with what is now known as “discrete testing”. In the latter, tests are performed by the testing laboratory, only if requested specifically by the physician. It could be concluded that biochemical profiling of asymptomatic individuals with batteries of tests is not necessarily a good idea, and it could actually lead to harm instead of benefits, as outlined in detail elsewhere [9]. Some other seemingly advantageous aspects of the Theranos technology have been highlighted by its CEO in public media interviews [10]. For example, it is mentioned that for each test requested for a centralized laboratory, a separate tube of blood is necessary. This is not accurate. Currently, with a 7 mL tube of blood, 10–100 analytes can be routinely measured by conventional technologies. It has further been claimed that “with inexpensive and easy access of the information running through their veins, people will have an unprecedented window on their own health. A new generation of diagnostic tests could allow them to head off serious afflictions from cancer, to diabetes, to heart disease” [10]. This is not the first time that such claims have been made and, in fact, some prominent scientists [11, 12] and organizations, such as Google [13] are currently exploring this possibility. Proponents believe that extensive biochemical testing could identify early and asymptomatic disease, in hopes that early intervention can improve patient outcomes. In my previous analysis of this issue [8], I highlighted the fact that any multiparametric testing algorithm will identify at least one, and maybe more, false positive parameters (i.e., abnormal results in otherwise normal people). More importantly, with emerging high-throughput genomic testing, genetic changes will be identified, but of unknown clinical significance [14]. Genetic changes may not accurately predict disease predisposition as most diseases are also (more) influenced by environmental factors [15]. I highlighted earlier that false positive results may require invasive procedures to delineate, thus harming many patients instead of helping them [9]. I further suggested than even if we had effective screening diagnostic procedures for identifying serious diseases, such as cancer, problems of false positives and of finding abnormalities of unknown significance (incidental findings), may lead to more harm than good of the tested patients. This is why even the most high-profile screening programs for cancer, such as breast, prostate, lung and ovarian, are still highly controversial regarding overall patient benefit [16–19]. We can also consider that contrary to what was believed until recently, intensive and radical treatments do not necessarily lead to better clinical outcomes, in comparison to later and less-intensive treatments [20–22]. The problems of over-testing, over-diagnosis and over-treatment have been analyzed in more detail by this author in other forums [8, 9].

Afraid of needles The Chief Executive of Theranos admits that she is afraid of needles and that this was one of her motivations to develop her technology [10]. However, the fingerpricks require lancets and the success of the fingerprick is dependent of the depth of the wound and the ability of blood to flow freely from the prick. If the prick is not done properly (deep enough), the sample will consist mostly of interstitial fluid instead of blood, which will comprise the obtained results and may need a repeat of a fingerprick. However, there is an additional hidden possible benefit of people getting accustomed to needles. It is highly unlikely that in somebody’s lifetime there will be no need for blood drawing or transfusion, minor or major surgery, or other invasive medical intervention. One way to eliminate the fear of needles (and other phobias) is graduated exposure therapy, developed in the 1950s [23]. With this strategy, exposing yourself to a needle (e.g., during routine blood collection) will desensitize and reduce your fear. In reality, blood drawing is a simple procedure which takes less than a minute to complete and objectively, is not painful. Many believe that fingerpricks are more painful and the pain lasts longer, in comparison to venipuncture [24, 25].

Results to patients Having patients self-tested and then self-interpreting what the results mean is tricky and could lead to many problems. Despite the fact that in most cases if your glucose is 5 mmol/L and the reference range is 4–6 mmol/L you can assume that you are “ok”, there is a myriad of scenarios where result interpretation could be confusing. Three simple representative examples will illustrate the point. My sister has breast cancer, treated by surgery and monitored every 6 months for recurrence with the CA 15.3 tumor marker test (reference range <30 U/mL). For one, she is not aware that this test is weak in its predictions since many recurring patients do not have any elevations and some elevations could be due to other reasons, not cancer recurrence. If her CA 15.3 on one occasion is 25 U/mL and the next testing shows 24 U/mL she is having a party; if the next testing is 26 U/mL she loses sleep and panics. She has no knowledge of the simple concepts of biological and analytical variation and she will not understand that all three numbers mentioned above are practically the same. A lay person whose PSA is 20 μg/L will assume, based on statistics, that he would have a more than 50% chance of harbouring prostate cancer; and ask for a biopsy. However, if his PSA a few days earlier was 1 μg/L, his chances of having cancer are virtually zero, the likely cause of his PSA increase being acute prostatitis, a benign and treatable condition. A male with a positive “pregnancy test” will likely be totally confused but a trained physician would look for testicular cancer. One wonders if it is preferable for patients to be tested after a request from a physician, and leave the physician to interpret the result, or test and interpret results by themselves. As highlighted elsewhere, the current wisdom suggests that testing should be ordered only when there is a question to be addressed and the result of the test will aid in an intervention that will be useful to the patient [26]. Centralized laboratories and hospitals are now using point-of-care devices which incorporate many Theranos-like technologies, such as fingerprick-collected whole blood, microfluidics, wireless communication, etc. for testing in emergency, remote and/or small hospitals and for samples suspected of contamination with infectious agents (i.e., ebola). These devices are not used for widespread testing for two reasons: (1) they are not high-throughput; and (2) they cost more per reportable result.

Concluding remarks The spectacular successes of Theranos with fund-raising, for attracting high-profile individuals on their Board (although most with non-medical background, such as ex-politicians and military) and with widespread exposure to the public media are well-recognized. However, as mentioned earlier, their claims of superiority over current systems and practices are speculative, at best. There is an apparent lack of appreciation of the dangers of self-screening and self-interpretation of results by asymptomatic individuals who are trying to detect occult disease. An open discussion of the merits and shortcomings of the Theranos and similar approaches should take place in the scientific literature and other public forums, so that the benefits and harms are better understood by the public.