vancouver seattle study critiques open letter by dr paul h blackman research coordinator for nra ila nra official journal dear sir by now we are used to the new england journal of medicine s publication of small scale studies related to firearms from which conclusions are drawn which are quantum leaps from the data followed by announcements of momentus scientific findings these are regularly released to the press without the caveats which riddle the conclusory paragraphs and often accompanied by an editorial calling attention to the findings generally while they at least present a few interesting data however meaningless the studies misinterpret statistics and ignore or belittle serious studies by criminologists the latest effort handgun regulations crime assaults and homicide a tale of two cities by j h sloan et al with the accompanying editorial firearms injuries a call for science by two employees of the centers for disease control november however is an insult to the intelligence of any serious scholar in any field and have so few data and so many flaws that i feel compelled to write at some length to call attention to various major and minor failings in no particular order the authors misleadingly cite wright et al ref to support the statement that some have argued that restricting access to handguns could substantially reduce our annual rate of homicide wright et al in fact studied and rejected that contention the authors pretended that vancouver and seattle are very similar cities with similar economic circumstances histories demographic characteristics and the like in fact the cities are very different with very different demographic characteristics which appear to explain completely the higher homicide rate in seattle both cities are over three forths non hispanic white and the non hispanic white homicide rates are reported to be the same in seattle and vancouver it is the different back grounds problems circumstances and behaviors of the various ethnic minorities which explain the difference in homicide the authors pretend they are evaluating canada s gun law compared to washington state s but they do not examine at all the situation in vancouver prior to the gun law taking effect in as it happens in the three years prior to that vancouver averaged homicides per year one eighth involving handguns ref and in the seven years of the nejm article there were homicides per year one eigth involving handguns surely even the medical profession recognizes that one must look to see the prior situation was before concluding that a change made a difference would a physician conclude that a patient was benefiting from eating oat bran muffin each day for seven years because his cholesterol level was without at least seeing if it was before he started the regimen the authors pick two medium sized cities to evaluate a national gun law nothing can be learned from such a tiny and arbitrarily selected sample seattle appears to have been selected because it was convienient for the authors rather than for any scientific reason would physicians call something a scientific study which involved one experimental subject and one dissimilar control had different arbitrarily selected cities been chosen opposite scientific conclusions would follow vancouver s homicide rate exceeds that of such wild west cities in texas as el paso corpus christi austen and in colorado colorado springs ref the authors fail to clearly demonstrate that firearms or handguns are far more commonly owned in seattle than in vancouver they use two surrogate approaches in pretending to study the availability of firearms handguns the first is an apples and oranges effort to compare the number of carry permits in seattle to the number of registered handguns in vancouver but the number clearly understates the number of handguns in seattle and counts primarily protective handgun owners the second however tells nothing about the number of handguns in vancouver and counts non protective handguns for the most part where is it difficult to obtain handguns legally for protection registration figures are meaningless there are registered handguns in new york city new york daily news sept comparing the two that method suggests about handguns per population in new york city compared to in vancouver meaning vancouver has a greater prevalence of weapons than new york city the second method of measuring gun density is cook s gun prevalence index a previously validated measure of intercity differences but the validation was by cook of his own theory ref normally second opinions are sought from a different doctor more significantly the cook index is based on the average of the percentage of firearms involvement in suicide and homicide so the authors are basically taking a measure of misuse unsurprisingly gun misuse in homicide in seattle in vancouver is related to gun misuse in homicide plus suicide divided by two in seattle in vancouver the authors are not measuring the relative avail ability of firearms or of handguns in seattle and vancouver the authors misstate the laws of both washington and canada they neglect to mention the significant fact that washington has a waiting period and background check prior to the purchase of a handgun and that provisions exist in canadian law for owning and carrying handguns for personal protection the authors also make it appear that it is more difficult to get a handgun legally in canada than is actually the case the authors ignore all other factors which might explain the differences in crime rate beyond some vague mention of the penalities provided by law and the roughest of estimates of clearence for one particular offense homicide involving a firearm there is no measure of the differences in the number of law enforcement officers their aggressiveness in making arrests for gun law violations in the two jurisdictions arrest rates for other offenses conviction rates actual sentences imposed for gun related crimes violent crimes without guns or gun law violations or incarceration rates whereas social scientists would attempt to measure and hold for such differences the authors of the nejm tale of two cities fail even to mention most factors related to crime control the authors dismiss claims that handguns are an effective means for protection unless the criminal is killed such is not the case criminologists ref have found that almost americans annually use handguns for protection from criminals and that using a gun for protection reduces the liklihood that a crime rape robbery assault will be completed by the criminal and reduces the likelihood of injury to the victim it is interesting nonetheless that the authors reported the same number four of civilian justifiable homicides without firearms in each city but that less restrictive seattle accounted for of the reported civilian justifiable homicides involving firearms the centers for disease control which funded the study editorially praised the paper ref saying it applied scientific methods to examine a focus of contention betweeb advocates of stricter regulation of firearms particularly handguns there is nothing in the paper which could possibly be mistaken for scientific methods by a sociologists or criminologists the vancouver seattle study is the equivolent of testing an experimental drug to control hypertension by finding two ordinary looking middle class white males one aged and the other and without first taking their vital signs administering the experimental drug to the year old while giving the year old a placebo then taking their blood pressure and on finding the younger man had a lower blood pressure announcing in a special article a new medical breakthrough it would be nice to think that such a study would neither be funded by the cdc or printed by the nejm since the longstanding anti gun biases of the nejm and the cdc make them willing to present shoddy research as scientific breakthroughs in special articles and editorials relating to firearms we are obligated to correct the record by notifying the news media and those with congressional and executive oversight over the activities of the centers for disease control about the distortions contained in handgun regulations crime assaults and homicides a tale of two cities and firearm injuries a call for science clearly all scientific standards go by the wayside whenever the cdc and the new england journal of medicine seize an opportunity to attack firearms ownership in america references wright jd et al weapons crime and violence in america a literature review and research agenda washington d c department of justice scarff e evaluation of the canadian gun control legislation final report ottawa ministry of the solicitor general of canada p department of justice federal bureau of investigation crime in the united states uniform crime reports washington d c government printing office cook pj the role of firearms in violent crime in wolfgang m weiner na eds criminal violence beverly hills calif sage pp kleck g crime control through the private use of armed force social problems ziegenhagen ea brosnan d victim responses to robbery and crime control policy criminology lizotte aj determinants of completeing rape and assault journal of quantitative criminology sayles sl kleck g rape and resistance paper at the american society of criminology convention chicago mercy ja houk vn firearm injuries a call for science nejm guns and sputter by james d wright from july issue of reason free minds free markets someone once wrote statistics are like a bikini what they real is suggestive but what they conceal is vital the problem is demonstrated by the most recent entry in a long line of scientific research purporting to show a causal link between gun availability and homicide funded by the federal government and published last year in the new england journal of medicine the study compared homicide rates in seattle and vancouver and suggested that a handgun ban may reduce the rate of homicide in a community the nine medical doctors who published handgun regulations crime assaults and homicide essentially reasoned in three steps despite many historical social and demographic similarities vancouver has a markedly lower homicide rate because its stricter gun regulations make guns less available the second step in their reasoning seems indisputable the overall homicide rate in seattle for the period was per popuation compared with in vancouver homicide is definitely more common in seattle the question then becomes why the authors present a believable although not entirely accurate case to support the notion as claimed in the third step of their reasoning that vancouver s handgun regulations are much more stringent but their evidence on the difference in gun availabilty is indirect and unpersuasive indeed they acknowledge that direct evidence on the point does not exist they offer two fragments of inferential data in support of the claim that guns are more available in seattle but for all anybody knows as a matter of empirical fact the opposite could be true we are therefore being asked at the conclusion of the study to believe that a difference in gun availability explains the difference in homicide rates when a difference in gun availability has not itself been established indeed the situation is even more troublesome the first of the two indirect bits of evidence is a difference between the number of concealed weapons permits issued in seattle and the number of restricted weapons permits issued in vancouver differences between the two cities in the permit regulations render these two numbers strictly noncomparable the second bit of evidence is cook s gun prevalence index which stands at percent for seattle but only percent for vancouver cook s index however does not measure the relative prevalence of gun ownership in various cities it measures gun misuse it is an average of the percentage of homicides and suicides involving firearms in the present case the index shows only that in homicides and suicides firearms are more likely to be used in seatte than in vancouver to take cook s index as a measure of general firearms availability it must be assumed that the proportional involvement of guns in homicides and suicides is directly related to their relative availability in the general population but this is exactly what the authors are seeking to prove to assume what one is seeking to prove then to prove it on the basis of that assumption does ot constitute scientific evidence for anything even if we were to grant on the basis of no compelling evidence that guns are less common in vancouver we might still question what causes what the authors attribute seattle s higher crime rate to a higher rate of gun ownership but it might well be argued that low crime or homicide rates reduce the motivation for average citizens to obtain guns in other words that crime rates explain the variation in gun ownership not vice versa in fact it was once commonly argued that great britain s low rate of violent crime was a function of that nation s strict gun laws and the consequent low rate of gun ownership until british researcher colin greenwood found that great britain had enjoyed low rates of violent crime for many decades before strict firearms controls were enacted to invoke an ancient methodological saw correlation is not cause nor do the problems with this study end with its lack of direct data on gun ownership the authors say seattle and vancouver are similar in many ways implying that they differ mainly in gun availability gun law stringency and crime rates this is an evident attempt to establish the ceteris paribus condition of a sound scientific analysis that all else is equal among things being compared clearly the two cities are similar in some ways but a closer look reveals differences in ways that are relevant to their respective crime or homicide rates the cities are closely matched in what percentage of their population is white percent and percent but seattle is about percent black while vancouver is less than percent vancouver s minority population is overwhelmingly asian so although the authors show that th two cities are approximately comparable on a half dozen readily available demographic indicators they have not shown that all potentially relevant sources of variation have been ruled out in fact the differences in racial compositions of the two cities is particularly relevant in light of the study s breakdown of homicide rates according to the race of the victim for the white majority the homicide rates are nearly identical per in seattle in vancouver the differing overall homicide rates in the two cities are therefore due entirely to vastly different rates among racial minorities for blacks the observed difference in homicide rate is to and for hispanics to methodoligical complexities render the asian comparison problematic but it too is higher in seattle than in vancouver racial minorities are much more likely to be the victims of homicide in seattle than in vancouver the white majority is equally likely to be slain in either city since the nearly initial difference in homicide reates between the cities is due exclusively to or differences between minority groups it is fair to ask why postulated difference in gun availability or gun law strigency would matter so dramatically to minorities but not matter at all to whites can differential gun availability explain why blacks and hispanics but not whites are so much more likely to be killed in seattle than in vancouver studies in the united states incidentally do not show large or consistent racial differences in gun ownership or are other explanations more plausible could the disparity between canadian and american rates of poverty among racial minorities have anything to do with it what are the relative rates of drug or alcohol abuse of homelessness among each cty s minority population the city of seattle runs the largest shelter for homeless men west of the mississippi unemployment among young central city nonwhite men in the united states usually exceeds percent what is the comparable canadian percentage the crucial point is that canada and the united states differ in many ways as do cities and population subgroups with the two countries absent more detailed analysis nearly any of these many ways might explain part or all of the difference in homicide rates in gross comparisons such as those between seattle and vancouver all else is not equal the authors of this study acknowledge that racial patterns in homicide result in a complex picture they do not acknowledge that the ensuing complexities seriously undercut the main thrust of their argument they also acknowledge that socio economic status is probably an important confounding factor in our comparison remarking further that blacks in vancouver had a slightly higher mean income in than the rest of vancouver s population given the evidence presented in the article it is possible that all of the difference in homicide rates between seattle and vancouver results from greater proverty among seattle s racial minorities but the authors pay no further attention to this possibility since detailed information about household incomes according to race is not available for vancouver the largely insurmountable methodological difficulties confronted in gross comparative studies of this sort can be illustrated with as simple example if one were to take all u s couties and compare them in terms of pervalence of gun ownership and crime or homicide rates one would find an astonishing pattern counties with more guns have less crime would one conclude from this evidence alone that guns actually reduce crime or would one insist that other variables also be taken into account in this example the hidden variable is city size guns are more common in small towns and rural areas whereas crime is a big city problem if researchers failed to anticipate this variable or lacked the appropriate data to examine its possible consequences they coud be very seriously misled in the study at hand the authors matched two cities for size but not for minority poverty rates or other hidden variables and their results are impossible to interpret in the editorial firearm injuries a call for science accompanying the study two officials from the centers for disease control lauded the authors for applying scientific methods to a problem of grave public heath significance but in attempting to draw causal conclusions from nonexperimental research the essence of scientific method is to anticipate plausible alternative explanations for the results and try to rule them out absent such effort the results may well seem scientific but are little more than polemics masquerading as serious research that this study is but one of a number of recent efforts all employing practical identical research designs and published in leading scientific journals is cause for further concern james d wright is professor of sociology at tulane university he has researched extensively on the relationship of firearms and crime reason published monthly except combined august september issue by the reason foundation a nonprofit tax exempt organization subscription rate per year reason foundation ocean park blvd suite santa monica ca
