text:
Raw – Lessons From The Stand-Up Feature
Thoughts On: Raw (1987)
Eddie Murphy's second stand-up comedy feature film.
This is a weirdly significant film to me. Being way too young to have been watching this, Murphy's Raw was my first exposure to stand-up comedy. I can't even remember how old I was when pushing in the old video tape, but what I will never forget is the full-body reaction, the utter captivation, the drug-like sensation, of watching and listening to the most hilarious shit for an hour and a half straight. I can't stress, nor explain, how mind-blowing the experience of watching and re-watching this film was to me as a kid. It was basically a kind of magic to me, how someone could stand on stage and ignite a fiery, burning uproar in my gut. And it's that feeling of pain, of delirious joy as I laughed and laughed and laughed until I was blue in the face, that makes this film significant to me. Never have I howled as much, so wholeheartedly, so purely and thoroughly as I did when watching this film for the first few times.
Raw has then always stuck with me as it's one of those films that I watch at least a few times every year. And watching the film a few times a year means in totality. I'm constantly watching clips from both Delirious and Raw, showing them to friends and family – even if I've shown them a billion times already. What's more, I'll often wake up and watch Murphy talking about Michael Jackson, Um-Fufu, girlfriends going on holiday and taking half or Italian guys watching Rocky just to start the day off on some kind of high. So, as should be clear, Eddie Murphy has been a huge influence on me. And I suppose this manifests itself primarily in my sense of humour, but also desire to tell stories. Because Raw was such a revelatory experience to me I obviously had to find more – which meant other comedians. And that yearning or fascination with stand-up has lasted to this day. It was after Eddie Murphy that I went on to find Tim Minchin, Richard Pryor, Aries Spears, Sam Kinison, Chris Rock, Louis CK, Robbin Williams, Sebastian Maniscalco, Dave Chappelle, George Carlin, Chris D'Elia, Joe Rogan, Mitch Hedberg, Kevin Hart, Jim Jefferies, Joey Diaz, Patrice O'Neal, Bill Burr and so much more. Each and every one of these comics hit that special note and ignited that familiar feeling I first felt when watching Eddie Murphy scream, shout and swear in Raw. And with this came this strange philosophy, sensibility or way of thinking.
My favourite comics seem to have two key characteristics. The first is a harsh, unapologetic truth. From Pryor to Kinison to Diaz to Jefferies, I see 'dirty comics' that find funny in very human truths that we often don't like to talk about. The other attribute I'm drawn to is an intellectual commentary. You get this from CK, Carlin, Rogan and Burr. These comics not only tell stories that make you laugh, but expose and talk about thought provoking ideas. But, what I often find is that these two types, the intellectual commentator and un-PC genuine storyteller, are often one and the same. So, whilst Carlin was more of a ranter, he was also focused on a kind of shock, hard-truth kind of commentary. Simultaneously, whilst Joey Diaz has endless stories about his balls, coke, prostitutes and guns, you get to hear the lessons of his life in his comedy. So, what I love in comedy is the visceral storytelling and the personalities that can say shit no other entertainer can. But, plain shock comedy, just like alternative pseudo-intellectual comedy, grows bland real quick. That's why I like an element of truth or thought in comedy. When that thought meets visceral exposition, you find the purest form of truth in my opinion. This is because, with a non-PC approach, you can say the things we all (many of us) personally think about all the time. You may talk about sexual things, racial topics, subjects of gender, poverty, evil, corruption and selfishness. And by not just bringing up these hidden and pushed aside topics of truth, but having something to add to them with some kind of original thought, the truth and expression demonstrated becomes something so much more precious.
I think the "it's just comedy" side of comedy off-sets this idea of visceral, probing truth incredibly well as it gives a comic free-range to create stories with hyperboles, which leaves their ideas with an ambiguous cushion around them – one that engages the audience, entertains them and doesn't assume they're an utter nit-wit that has to be fed easy to swallow, plain and simple ideas. Louis CK is probably one of the best comedians when it comes to implying a subtle truth within a hyperbolic comedic premise. Whilst CK may talk about "If Murder Was Legal" and suggest that everyone would kill someone if they were allowed as a way of creating something funny, he also implies something about a side of human nature that isn't so loving, that does get annoyed, fed up and frustrated. Whilst I could go on to talk about this for thousands of words more, this is essentially why I'm so swept away by certain kinds of comedy. Not only is there a pure, unflinching truth, but its delivered and packaged in an intellectual way that engages as well as respects an audience in a manner that no other art form, or form of entertainment, can manage.
Having said this, I bring up Eddie Murphy's Raw to ask two questions. The first question is, is Raw a film and does it count as cinema? This is a subtly tricky question. The answer seems to simply be, no, not really. This answer would suggest that, whilst this is a 90 minute recording, one that was given a theatrical release, put on tape, DVD and Blu-ray, we go to see this film for an art form that is not cinematic. We watch this 'film' for stand-up comedy. And stand-up certainly doesn't need a camera to function – it just needs a crowd. What this certainly says is that stand-up features, or specials, fit into a weird realm of cinema. However, despite this, in my opinion, I certainly think that they count as films – just not in a strict way. They are loosely cinematic. After all, A Streetcar Named Desire and Dogville are films. These are basically stage plays. We don't need cinema to experience these stories, however, they are stories expressed on film and so they are movies. This is why I say that Raw counts as a movie, as cinema. Not only is this a 'story' captured by a camera. but it is one dictated, facilitated and contorted by cinematography, direction and editing – a uniquely cinematic combination.
Having asserted that Raw is a film – though a strange one – we can move onto the next, and more pressing, question: so what? What can we learn from Raw as a movie?
With the introduction, I've tried to convey what the stand-up comedy and comedians I like have taught me. Primarily, the likes of Pryor, Diaz, CK, Burr and O'Neal have taught me a key idea of truth. What the mentioned comics do like no other artist can is, within the comedic realm, expose hidden truths and provide commentary on them. They are allowed to do this as their form of storytelling is entirely focused on making a succinct point. I repeat, entirely focused. A joke will only work if the audience gets it. And to get a joke you must understand how it tries to manipulate and misguide you. A pretty flat example would be as follows:
What do you call a boy about to stand up to his bullies?
An ambulance.
What this very simple joke does is set up a narrative of a boy about to rise up against his conflicts – be a hero of his story. However, the laugh lies in a sharp turn. Reality barges its way in with the implimence that the boy is about to be beaten to a pulp as he tries to pull on the tights and spandex of a hero. We then laugh because we understand that bullied kids are bullied for a reason – they're usually weak. Our recognition of this is the laugh. It's kind of fucked up when you think of comedy in these terms, but that's simply how it is. And it's not a form of sadism or hatred to laugh at these kind of jokes as, as previously said, "it's just comedy". What this much repeated phrase suggests is the obvious: the boy, his bullies and the ambulance don't exist. This means that a joke is, to push towards slightly pretension depths, a form of philosophical questioning.
As a philosopher, you may, without flinching, ask: why don't we just rape the women we're attracted to? The reason why a philosopher asks this question has nothing to do with actual rape. The question means to expose a hidden truth. After all, if you were to ask an average passing person why we don't just rape women, you'd likely get a very unsatisfactory answer: it's not right. With "it's not right" being the only answer one could provide to this question, you can see a tremendous problem. To explain, imagine you stand with your best friend before a red button. You are in an empty room without windows, without a door, just the button. You have the urge to push the button. Your friend says not to. You ask why. They said "it's not right". Where do you go from here? You'll eventually push the button, right? However, you wouldn't if your friend explained that the button has a wire that's connected to him and that if you push that button, a bomb strapped to his chest would go off.
Now, I don't mean to compare rape with pushing a button out of curiosity – not so plainly anyhow. What I mean to suggest is that asking questions and providing answers provides clarity and so reason to a rhyme. So, whilst no one (blindly holding onto optimism) raised in modern society really needs to be told not to rape someone, we can imagine an alien civilisation landing on Earth who have no mating rituals, instead, just inseminate anyone who catches their eye. Assuming these aliens are intelligent and empathetic, we'd have to explain what rape is, why it's bad, such and so on. You couldn't do this by simply saying "it's not right". You're going to need the philosopher.
Comedy works in a similar way to philosophy in this respect. A great example would be found in the works of Bill Burr. A great bit of his is him confronting the assertion that "there is no reason to hit a woman". Burr accepts, as we all do (again, blindly holding onto optimism) that you just shouldn't hit women. However, we are all told this in a "it's not right" manner. Burr questions this, exposing the truth, that is so easily glossed over, that women are humans, that they can be infuriating, that they can certainly be deserving of a punch. He repeats that you shouldn't hit a woman, but, asserts that there is reason to nonetheless. This is funny in the same manner that explaining why rape is wrong to aliens is helpful. Just as explaining rape to the aliens would ignite a reaction of understanding in them, Burr exposing a caveat in a subject of domestic abuse is funny. There is thus, in both cases, a reflex response to validate these intellectual assertions, which demonstrates how there is both a philosophical questioning going on in comedy, but also the need for the joke/questioning.
Taking this concept of jokes as philosophy, we can extrapolate our first lesson from the stand-up feature film. A great way to approach a story is with this focus on making a point that all comedians have. What makes a joke great is the profundity of its point. What often makes films great is also their profundity, however, there is a reliance on image and action in film that allows weak points to be made. For example, just look to Michael Bay. His films have no profound point to them – not in my view. However, millions have seen and paid for them. Ask yourself this though: if Transformers had to be performed as a stand-up routine, would it get any laughs? I say no – not a chance. Look to a film like Dogtooth, however. There is a succinct and powerful point made in this film about the control parents have over their children…
Could this movie be adapted into a stand-up comedian's set and be funny? I certainly think so. This is not just because Dogtooth is a slight dark comedy, however, but because there is a conflicting and probing truth exposed by this story that a comedian could capitalise on. What this says about Dogtooth and the approach to cinema that Lanthimos takes is that poignant stories have a point, one that exposes a visceral, easily looked over truth. If you appreciate this kind of filmmaking, I think stand-up comedy is a great source of inspiration and lessons. It's watching Burr, Diaz, Rogan, Hedberg and Pryor that you'll then learn the importance of a premise, its twist and its resolution. If you can put this into a script, I believe you'll have the tools and means to create something special. And you don't have to be inspired to write a comedy. In simply recognising how comedians expose truth, you can construct powerful dramas, fantasies, crime-mysteries – anything. What comedy will teach you is how to make those stories profound and how to get a reaction out of an audience.
The next lesson that a film like Raw may teach you, which is connected to the previous, is the importance of words or dialogue. Whilst you could learn this from a play, say for instance the 2001 version of Waiting For Godot, stand-up is a much more expressive example of the power of dialogue. When we watch Raw, it is so easy to forget that all we're doing is watching Murphy talk….
… that's it. If a stand-up comedian can entertain people for up to 2 hours on stage and this may translate to film, does this not suggest a much greater scope of story that may work in a script? What Raw suggests is that not all films need to be Die Hard, instead, that they can be much more subdued and simple. By watching comedy, you come to understand just how to achieve this, how to fill up a large space of time with minimalist resources. The answer is an unsatisfying one – it's all about a great performance and some great writing. This is why stand-up comedy is so hard and a film that mimics its form would be so difficult to produce. But, as Murphy demonstrates, this is more than possible.
A lesson within this subject of dialogue that you may learn from stand-up is how people performing talk or sound. As any screenwriter could tell you, when you write a film, you are not trying to write real conversations. If you were to turn a transcript of a real conversation into a film, it'd likely be boring. As a result, when you write dialogue, you don't want to transcribe real speech. Instead, you want to write movie dialogue. You can learn to do this many ways, and I believe watching stand-up comedians perform is one of the best due to the pure focus on their words and voice. One of the best sources actually begins to move further away from cinema. Joe Rogan's 2016 special, Triggered…
… is shot almost entirely in a mid-shot or close-up with a few wides. There are no inserts of the crowd, we just see him talk. This emphasises the power of his voice and performance. And what makes this entirely minimalist approach almost a no-brainer when it comes to Rogan is, of course, his podcast…
These are 2, sometimes 3 or 4 hour long conversation-performances (I say conversation-performances as they are for a camera and mic to pick up). All you see is captured by the above shot; just a cut between Joe and his guest. Is this cinema? There is cinematography, a camera, editing and so it has to be in some way, right? Whilst you may call it just a podcast, I believe that the YouTube videos of The Joe Rogan Experience are tantamount to movies – movies I happily sit through dozens and dozens of times.
What this should further emphasis to any film lover or filmmaker is that minimalism and bare-bones storytelling is a viable option. Whilst it may seem commercially daunting to try this, what Joe Rogan and many other podcasters who stream on sites like YouTube show is that, with a good character, this is very possible and endlessly entertaining. And this is arguably what we see in a film like Locke…
However, coming back to the idea of learning how to make characters talk, look no further than podcasts and stand-up specials to learn what goes into performances as well as how conversations flow and structure themselves. Recognising that podcasts and comedy specials aren't just conversations or talks will then open up the world of cinema to explore and experiment with minimalist, character-driven forms we rarely come across.
These are three great lessons that take a lot of time to absorb. I won't lie and suggest that I have the universal answers on learning how to make a great movie from watching Raw or any other stand-up feature. What I think makes a lot of sense, however, is that looking at the stand-up feature as cinema is incredibly powerful. And so learning from Raw, Delirious, Triggered, Live At The Beacon Theatre or Bring The Pain in the same way you may a Bergman, Spielberg or Scorsese film is pivotal. If you like comedy, if you are interested in comedic storytelling or even verbal storytelling, I cannot recommend stand-up more.
Cries And Whispers – Graceful Mortality
Noah – Can It Make Sense?
Author danielslackdsuPosted on September 23, 2017 September 23, 2017 Categories Uncategorized
Previous Previous post: Cries And Whispers – Graceful Mortality
Next Next post: Noah – Can It Make Sense?


text:
101 FUN THINGS TO DO
Best In Corpus Christi
Christmas Guide
Fiesta de la Flor
Texas SandFest
Texas State Aquarium
USS Lexington
101 Network Host Locations
Browse Vacation News for more articles like this.
Find the Spirit of the Karankawa in Suter Park
Larry Running Turtle Salazar at Hans and Pat Suter Park in Corpus Christi during a blessing of the grounds Jan. 13. Salazar holds a hat with the logo for the Ishka Monument he hopes to someday see installed at the park. Photo by Chris deLong
By Suzanne Freeman
Known by archeologists as 41NU2, an area of this seaside city just outside the gates of Texas A&M University-Corpus Christi and Corpus Christi Naval Air Station was once a Native American burial ground. Locals know it as Hans and Pat Suter Park on Ennis-Joslin, a four-lane stretch of asphalt that ends at Ocean Drive. It is the second-largest Indian burial ground in the state of Texas.
The bones and artifacts of hundreds of Native Americans, some thousands of years old, were discovered there in 1933, unearthed by a hurricane. The first official dig was not conducted until the 1990s as the Texas Department of Transportation successfully kept the initial find quiet.
In 1994, archeologists hired to excavate during a road improvement project found the bones of a 13-year-old girl believed to be 500 years old. The word of this find was also kept under wraps until a reporter got wind of it and notified a local Native American activist.
"I went out there to pay my respects and to do my ceremonies for forgiveness for what they were doing," Larry Running Turtle Salazar said of the find. "(TxDOT workers) ran me out of there. I went back at midnight and was right there by the park singing my songs and praying, and the voices of the winds came in. I apologized for what was going on."
Salazar and members of the South Texas Alliance of Indigenous People hold a funeral procession in the area every March since that year. They use smudge sticks and herbs to bless the sacred ground.
"For as long as I am on this Earth Walk," Salazar said, "I will bless the grounds every single year on the date or close to the date I was notified that it was a burial ground."
An expert on area Native American tribes, Robert Ricklis was an architect at the site in the 1990s. He also wrote a book, "The Karankawa Indians of Texas," which tells the story of the Karankawa. According to his research, the tribe died out in the 1800s, meaning most of the bodies unearthed at 41NU2 are too old to be Karankawa.
"People have been living in (the Coastal Bend) for 14,000 years," Ricklis said. "As far as identifying Karankawa, we can only push that back in time about 1,000 years."
Most of the unearthed bones found at the site are now at the University of Texas at Austin, perhaps all that is left of a prehistoric people who once were the main residents of the Coastal Bend.
The Karankawa and other Native Americans were doomed, according to both Ricklis and Salazar, when the Spanish colonial system collapsed in the 1820s.
"The Spanish wanted to convert them to Christianity," Ricklis said. "The Anglo-Americans wanted to exterminate them and get them out of the way because they wanted their land."
Salazar, who is of Tsalagi and Apache descent, tells how the few Native Americans left cut their hair and adopted Hispanic names to pass as Mexican, which is where he gets his last name.
"My people took on Hispanic survival names," he said. "They learned the language and the culture. They became Mexicano to stay alive."
Both also agree on something else important about the Karankawas: They were not cannibals as often reported. The Karankawa economy was based on hunting, gathering, and fishing. The tribes were family-oriented and welcoming to outsiders.
A medicine wheel has been installed at Hans and Pat Suter Park along with temporary information markers Salazar hopes will take a more permanent form soon. He and the South Texas Alliance of Indigenous People are raising money for a monument to the Native Americans buried at the site. The March to Ishka monument was designed by the late artist Dave McGary, a world-renowned bronze sculptor. The group needs to raise $300,000 for McGary's widow to turn his model into the actual statue. The statue of a Native American in full head dress pointing to the east incorporates color and texture in the work.
"We get our blessings from the holy people right as the sun is rising," Salazar explained. "We say the word 'Ishka' as the sun comes up, which means, 'I will see you again. If not in this lifetime, the next.'"
Direct donations may be made by mailing checks to 1929 Yale St., Corpus Christi, TX 78416. Checks should be made out to "The Ishka Monument." Donations may also be made online at ishkamonument.webs.com.
Hans and Pat Suter Park is located at 909 Ennis Joslin Road in Corpus Christi. A wildlife refuge popular for bird-watching, it has biking and hiking trails and a picnic area. It is free and open 24 hours a day.
Find more articles like this in Vacation News
Leroy Pena — November 18, 2019 @ 07:13 pm
The video that i recorded was at the Burial Grounds at Hans and Pat Suter Park off Ennis Joslin. The war cries started after i offered tobacco and while i dropping cedar.
I have a video that i recorded last year where you can clearly hear Apache War Cries. The Lipan Apache were also in the area, i am an enrolled member of the Lipan Apache Tribe of Texas.
Featured Vacation News Articles
Texas State Aquarium Experience Not Hampered by COVID Protocols
2021 Beach Parking Permits on Sale in Corpus Christi
Blue Angels to Soar Over Buc Days 2021
Corpus Christi Fireworks New Year's Eve
More Vacation News
Subscribe to our E-mail newsletter
today to receive updates on the latest news.
We respect your privacy.Your information is safe and will never be shared.
Copyright ©2021 101CorpusChristi.com
- Attractions and Activities
- Beaches
- Beach Activities
- Eat and Drink
- Museums & Culture
- Parks
- Shopping
- Stay
- Best In Corpus Christi
- Christmas Guide
- Fiesta de la Flor
- RV Living
- Texas SandFest
- Texas State Aquarium
- USS Lexington
- Digital Products
- 101 Network Host Locations

redpajama_set_name: RedPajamaCommonCrawl


text:
LSE PH500 (May-June 2020)
Frequentists in Exile
Blog Bagel
Elbar Grease
(LSE) PH500
12-12-12 December 12 Seminar (10-12)
12-12-12 (background): Some Recipes for p-values, type 1 and 2 error probabilities, power, etc.
5 Dec. seminar reading (remember it is 10a.m.-12p.m.)
28 Nov. Seminar and Current U-Phil
AUTUMN SEMINARS: Contemporary Philosophy of Statistics
office hours week of Dec. 3-10 Dec:
SUMMER SEMINARS: Contemporary Philosophy of Statistics
W14Phil6334
Phil 6334 Assignments
6334 (Q-0)
6334-Excercise 3 Testing Recipes (Spring Break)
Short essay
Phil 6334 SLIDES
Phil 6334: Day 1 Slides
Phil 6334: Day 11 Slides
Phil 6334: Syllabus
Phil 6334 Syllabus: Fifth Installment
Feb 6 (participants)
SEV APP
PhilStat Spring 19
Syllabus: Second Installment
Notes Excursion 1
Mayo Slides Meeting #1 (Phil 6334/Econ 6614)
Mayo Slides: Meeting #2 (Phil 6334/Econ 6614) Part I (Bernoulli Trials)
Mayo Slides: Meeting #2 (Phil 6334/Econ 6614) Part II (Logic)
Mayo Slides #5 (Phil 6334/Econ 6614)
Mayo Slides Meeting #10 (Phil 6334/Econ 6614)
Spanos Lecture Notes 1 (Phil 6334/Econ 6614)
SIST Tour Summaries
Captain's Biblio with Links
Spanos ch 1, 2 & IID R.V. explained
Additional Stats help
Mayo Pubs
RMM Special Topic articles
Senn's posts+
Error Statistics Philosophy
S. Senn: Randomisation is not about balance, nor about homogeneity but about randomness (Guest Post)
Posted on April 20, 2020 by Mayo
Stephen Senn
Consultant Statistician
The intellectual illness of clinical drug evaluation that I have discussed here can be cured, and it will be cured when we restore intellectual primacy to the questions we ask, not the methods by which we answer them. Lewis Sheiner1
In their recent essay Causal Evidence and Dispositions in Medicine and Public Health2, Elena Rocca and Rani Lill Anjum challenge, 'the epistemic primacy of randomised controlled trials (RCTs) for establishing causality in medicine and public health'. That an otherwise stimulating essay by two philosophers, experts on causality, which makes many excellent points on the nature of evidence, repeats a common misunderstanding about randomised clinical trials, is grounds enough for me to address this topic again. Before, however, explaining why I disagree with Rocca and Anjum on RCTs, I want to make clear that I agree with much of what they say. I loathe these pyramids of evidence, beloved by some members of the evidence-based movement, which have RCTs at the apex or possibly occupying a second place just underneath meta-analyses of RCTs. In fact, although I am a great fan of RCTs and (usually) of intention to treat analysis, I am convinced that RCTs alone are not enough. My thinking on this was profoundly affected by Lewis Sheiner's essay of nearly thirty years ago (from which the quote at the beginning of this blog is taken). Lewis was interested in many aspects of investigating the effects of drugs and would, I am sure, have approved of Rocca and Anjum's insistence that there are many layers of understanding how and why things work, and that means of investigating them may have to range from basic laboratory experiments to patient narratives via RCTs. Rocca and Anjum's essay provides a good discussion of the various 'causal tasks' that need to be addressed and backs this up with some excellent examples.
It's not about balance and it's not about homogeneity
In discussing RCTs Rocca and Anjum write
'…any difference in outcome between the test group and the control group should be caused by the tested interventions, since all other differences should be homogenously distributed between the two groups,'
and later,
'The experimental design is intended to minimise complexity—for instance, through strict inclusion and exclusion criteria'.
However, it is not the case that randomisation will guarantee that any difference between the groups should be caused by the intervention. On the contrary, many things apart from the treatment will affect the observed difference. And it is not the case that the analysis of RCTs requires the minimisation of complexity. Randomisation and its associated analysis deals with complexity in the experimental material and although the treatment structure in RCTs is often simple this is not always so (I give an example below) and it was not so in the field (literally) of agriculture for which Fisher developed his theory of randomisation. This is what Fisher, himself had to say about complexity
No aphorism is more frequently repeated in connection with field trials, than that we must ask Nature few questions, or ideally one question, at a time. The writer is convinced that this view is wholly mistaken. Nature, he suggests, will best respond to a logical and carefully thought out questionnaire; indeed, if we ask her a single question, she will often refuse to answer until some other topic has been discussed.3 (p. 511)
This 1926 paper of Fisher's is an important and early statement of his views on randomisation and was cited recently by Simon Raper in his article in Significance4. Raper points out, that Fisher was abandoning as unworkable an earlier view of causality due to John Stuart Mill whereby controlling for everything imaginable was the way you made valid causal judgements. I consider Raper is right in thinking of Fisher's approach as an alternative to Mill's programme, rather than some realisation of it, so I disagree for example, with Mumford and Anjum in their book5 when they state
'Fisher's idea is the basis of the randomized controlled trial (RCT), which builds on J.S. Mill's earlier method of difference' (pp. 111-112).
I shall now explain exactly what it is that Fisher's approach does with the help of an example.
Breathing lesson
Before going into the example, which is a complex design, it is necessary to clear up one further potential point of confusion in Rocca and Anjum's essay. N-of-1 studies, are not alternatives to RCTs but a subset of them. RCTs include not just conventional parallel group trials but also cluster randomised trial and cross-over trials, including n-of-1 studies. The difference between these studies is at the level one randomises and this is reflected in my example, which has features of both a parallel group and a cross-over study. Thus, reading Rocca and Anjum's paper, which I can recommend, will make more sense if by their use of RCT is understood 'randomised parallel group trials'.
For the moment, all that it is necessary to know is that within the same design, I can compare the effect on forced expiratory volume in one second (FEV1), measured 12 hours after treatment, of two bronchodilators in asthma, which here I shall just label ISF24 and MTA6, in two different ways. First, I can use 71 patients who were given MTA6 and ISF24 on different occasions. Here I can compare the two treatments patient by patient. These data have the structure of a within-patient study. Second, within the same study there were 37 further patients who were given MTA6 but not 1SF24 and 37 further patients who were given ISF24 but not MTA6. Here I can compare the two groups of patients with each other. These data have the structure of a between-patient or parallel group study.
I now proceed to analyse the data from the 71 pairs of values from the patients who were given both using a matched pairs t-test. This will be referred to as the within-patient study. Note that this is an analysis of 2×71=142 values in total. I then proceed to compare the 37 patients given MTA6 only to the 37 given ISF24 only using a two-sample t-test. I shall refer to this as the between-patient study. Note that this is an analysis of 37+37=74 values in total. Finally, I combine the two using a meta-analysis.
The results are presented in the figure below which gives the point estimates for the difference between the two treatments and the 95% confidence intervals for both analyses and for a meta-analysis of both, which is labelled 'combined'. (The horizontal dashed line is the point estimate for a full analysis of all the data and is described in the appendix.) Note how much wider the confidence intervals are for the between-patient study than the within-patient study. This is because the within-patient study is much more precise.
Figure 1 Point estimates and confidence intervals for the between, within and combined estimates.
Why is the within-patient study so much more precise? Part of the story is that it is based on more data, in fact nearly twice as many data: 142 rather than 74. However, this is only part of the story. The ratio of variances is more than 30 to 1 and not just approximately 2 to 1, as the number of data might suggest. The main reason is that the within-patient study has balanced for a huge number of factors and the between-patient study has not. Thus, differences in 20,000 plus genes and all life-history until the beginning of the trial are balanced in the within-patient study, since each patient is his or her own control. For the between-patient study none of this is balanced by design. In fact, there are two crucial points regarding balance.
1. Randomisation does not produce balance
2. This does not affect the validity of the analysis
Why do I claim this does not matter? Suppose we accept the within-patient estimate as being nearly perfect because it balances for those huge numbers of factors. It seems that we can then claim that the between-patient estimate did a pretty bad job. The point estimate is 0.2L more than that from the within-patient design, a non-negligible difference. However, this is to misunderstand what the between-patient analysis claims. Its 'claim' is not the point estimate; its claim is the distribution associated with it, of which the 95% confidence interval is a sort of minimalist conventional summary and of which the point estimate is only one point. As I have explained elsewhere, such claims of uncertainty are a central feature of statistics. Thus, the true claim made by the between-patient study is not misleading. It is vague and, indeed, when we come to combine the results, the meta-analysis will give 30 times the weight to the within-patient estimate as to the between-patient estimate simply because of the vagueness of the associated claim. This is why the result from the meta-analysis is so similar to that of the within-patient estimate. Furthermore, although this can never be guaranteed, since probabilities are involved, the 95% CI for the between-patient study includes the estimate given by the within-patient study. (Note, that in general, confidence intervals are not a statement about a value in a future study, but about the 'true' average value6 but here, the within-patient study being very precise, they can be taken to be similar.)
This works because what Fisher's analysis does is use variation at an appropriate level to estimate variation in the treatment estimate. So, for the between-study it starts from the following observations
1) There are numerous factors apart from treatment that could affect the outcome in one arm of the between-patient study compared to the other.
2) However, it is the joint effect of these that matters.
3) This joint effect of such factors will also vary within each of the two treatment groups.
4) Provided I use a method of allocation that is random, there will be no tendency for this variation within the groups to be larger or smaller than that between the groups.
5) Under this condition I have a way of estimating how reliable the treatment estimate is.
Thus, his programme is not about eliminating all sources of variation. He knows that this is impossible and accepts that estimates will be imperfect. Instead, he answers the question: 'given that estimates are (inevitably) less than perfect, can we estimate how reliable they are?'. The answer he provides is 'yes' if we randomise.
If we now turn to the within-patient estimate, the same argument is repeated but in a first step differences are calculated by patient. These differences do not reflect differences in genes etc. since each patient acts as his or her own control. (They could reflect a treatment-by-patient interaction but this is another story I choose not to go into here7, 8. See my blog on n-of-1 trials for a discussion.) The argument then uses the variance in the single group of differences to estimate how reliable their average will be.
Note that a different design requires a different analysis and in particular because the estimate of the variability of the estimate will be inappropriate even if the estimate is not affected. This is illustrated in Figure 2 which shows what happens if you analyse the paired data from the 71 patients as if they were two independent sets of 71 each. Although the point estimate is unchanged, the confidence interval is now much wider than it was before. The value of having the patients as their own control is lost. The downstream effect of this is that the meta-analysis now weights the two estimates inappropriately.
Figure 2 Effect of analysing the within-patient information incorrectly as if it were between-patient. The point estimate is unchanged but the confidence limits are too wide and this in turn leads to the between-patient estimate being given too much weight in the meta-analysis.
Note also, that it is not a feature of Fisher's approach that claims made by larger or otherwise more precise trials are generally more reliable than smaller or otherwise less precise ones. The increase in precision is consumed by the calculation of the confidence interval9, 10. More precise designs produce narrower intervals. Nothing is left to make the claim that is made more valid. It is simply more precise. The allowance for chance effects will be less, and appropriately so. Balance is a matter of precision not validity.
The shocking truth
As I often put it, the shocking truth about RCTs is the opposite of what many believe. Far from requiring us to know that all possible causal factors affecting the outcome are balanced in order for the conventional analysis of RCTs to be valid, if we knew all such factors were balanced, the conventional analysis would be invalid. RCTs neither guarantee nor require balance. Imbalance is inevitable and Fisher's analysis allows for this. The allowance that is made for imbalance is appropriate provided that we have randomised. Thus, randomisation is a device for enabling us to make precise estimates of an inevitable imprecision.
I thank George Davey Smith, Elena Rocca and Rani Lill Anjum for helpful comments on an earlier version.
Sheiner LB. The intellectual health of clinical drug evaluation [see comments]. Clin Pharmacol Ther 1991; 50(1): 4-9.
Rocca E, Anjum RL. Causal Evidence and Dispositions in Medicine and Public Health. International Journal of Environmental Research and Public Health 2020; 17.
Fisher RA. The arrangement of field experiments. Journal of the Ministry of Agriculture of Great Britain 1926; 33: 503-13.
Raper S. Turning points: Fisher's random idea. Significance 2019; 16(1): 20-23.
Mumford S, Anjum RL. Causation: a very short introduction: OUP Oxford, 2013.
Senn SJ. A comment on replication, p-values and evidence S.N.Goodman, Statistics in Medicine 1992; 11: 875-879. Statistics in Medicine 2002; 21(16): 2437-44.
Senn SJ. Mastering variation: variance components and personalised medicine. Statistics in Medicine 2016; 35(7): 966-77.
Araujo A, Julious S, Senn S. Understanding Variation in Sets of N-of-1 Trials. PloS one 2016; 11(12): e0167167.
Senn SJ. Seven myths of randomisation in clinical trials. Statistics in Medicine 2013; 32(9): 1439-50.
Cumberland WG, Royall RM. Does Simple Random Sampling Provide Adequate Balance. J R Stat Soc Ser B-Methodol 1988; 50(1): 118-24.
Senn SJ, Lillienthal J, Patalano F, et al. An incomplete blocks cross-over in asthma: a case study in collaboration. In: Vollmar J, Hothorn LA, eds. Cross-over Clinical Trials. Stuttgart: Fischer, 1997: 3-26.
Appendix: The design of MTA/02
This was a so-called balanced incomplete blocks design necessitated because it was desired to study seven treatments (three doses of each of two formulations and a placebo)11 but it was not considered practical to treat patients in more than five treatments. Thus, patients were allocated a different one of the seven treatments in each of the five periods. That is to say, each patient received a subset of five of the seven treatments. Twenty-one sequences of five treatments were used. Each sequence permits (5×4)/2 = 10 pairwise comparisons but there are (7×6)/2= 21 pairwise comparisons overall and the sequences were chosen in such a way that any given one of the 21 pairwise comparisons within a sequence would appear equally often over the design. Looking at the members of such a given chosen pair one would find that in five further sequences the first would appear and not the second and vice versa. This leaves one sequence out of the 21 in which neither treatment would appear. The sort of scheme involved is illustrated in Table 1 below.
Table 1 Incomplete blocks design in five periods and seven treatments (A to G). Columns are periods, rows are sequences allocated at random to a patient. Yellow: patient received A & F. Blue: patient received A but not F. Grey: patient received F but not A. Purple: patient received neither A nor F.
The active treatments were MTA6, MTA12, MTA24, ISF6, ISF12, ISF24, where the number refers to a dose in μg and the letters to two different formulations (MTA and ISF) of a dry powder of formoterol delivered by inhaler. The seventh treatment was a placebo.
In fact, the plan was to recruit six times as many patients as there were sequences, randomising a given patient to a sequence in a way that would guarantee approximately equal numbers per sequence. This would have given 126 patients in total. In the end, this target was exceeded and 161 patients were randomised to one of the sequences.
Obviously, this is a rather complex design but I have used it because it enabled me to compare two treatments two different ways. First by using only the ten sequences in which they both appear. For this purpose, I could use each patient as his or her control. Second, by using the ten further sequences in which only one appears.
This thus permitted me to analyse data from the same trial using a within-patient analysis and a between-patient analysis. The analyses used above should not be taken too seriously. The analysis would not generally proceed and did not in fact proceed in this way. For example, I ignored the complication of period effects and ignored the fact that by including all the seven treatments in an analysis at once, I could recover more information. I simply chose two treatments to compare and ignored all other information in order to illustrate a point. The two treatments I compared, 'ISF24' and 'MTA6', were respectively, the highest (24μg) dose of the then (1997) existing standard dry powder formulation, ISF of the beta-agonist formoterol, and the lowest (6μg) of a newer formulation, MTA, it was hoped to introduce. The experiment is discussed in full in Senn, Lilienthal, Patalano and Till11.
The full model analysis that I showed as a dotted line in Figure 1 & Figure 2 fitted Patient as a random effect and Treatment and Period as fixed factors with 7 and 5 levels respectively.
Ed. A link to a selection of Senn's posts and papers is here. Please share comments and thoughts.
Categories: RCTs, S. Senn	| 29 Comments
29 thoughts on "S. Senn: Randomisation is not about balance, nor about homogeneity but about randomness (Guest Post)"
I am extremely thankful to Stephen Senn for a further installment on his discussions of randomisation on this blog. His response to the common criticism that RCTs don't achieve balance is extremely important and opened my eyes to where the critics (often philosophers) go wrong. I would like to draw Stephen out a bit more, though, on what he means in saying randomisation is about randomisation. Fisher said:
The purpose of randomisation . . . is to guarantee the validity of the test of
significance, this test being based on an estimate of error made possible by
replication. (Fisher [1935b]a1951, psection. 26) (SIST p. 286).
The meaning of' randomisation is about randomisation', I suggest, is that its function is to guarantee the error probability assessments (for estimators and tests). That is why its relevance comes into question by subjective Bayesians who question the relevance of error probabilities to inference (though some find a role for it in vouchsafing posterior probabilities). For the error statistician, it's a way to deliberately introduce the probabilistic considerations needed to appraise the reliability of the inference that a method outputs.
CauseHealth Risk and Safety
Thanks, Stephen, for engaging with our paper and for sharing this convincing argument about the non-necessity of the 'balance condition' in RCTs. As you correctly note, we don't touch upon this in our paper, despite the fact that some arguments for the same statement come also from the philosophy of science front (see for instance J. Fuller, 2018, 'The Confounding Question of Confounding Causes in Randomized Trials', Brit. J. Phil. Sci.).
For the point we want to make in our paper, there is no problem to adopt the view that RCT aims to have confounding factors randomly distributed and not homogeneously distributed among groups. In light of your post, we should have said that:
'…any correlation between the tested intervention and the difference in outcome between the test group and the control group can be estimated, since all other causally relevant factors have been randomly distributed between the two groups'
Rather than what we said, namely that:
'…any difference in outcome between the test group and the control group should be caused by the tested interventions, since all other differences should be homogenously distributed between the two groups'.
We believe that it would have been technically more correct given what you explain.
For the specific purpose of our analysis in the paper, however, we want to make clear why this is not a misunderstanding of the logic underlying this experimental design.
A quick background. The point in our paper is to argue for the necessity of plurality of methods to make a causal claim in medicine, and it is based on the philosophical idea that what medicine ultimately needs to know about are intrinsic properties (also called dispositions, capacities or causal powers). For instance, Ibuprofen causes gastrointestinal symptoms in a large number of patients. This observation is useful, because it points to an intrinsic property of ibuprofen. Now, whether one agrees with this idea or not, the problem remains that intrinsic properties are not directly observable in the same way that a correlation is, and different methods have some advantages and some blind spots to detect such properties. In the paper, we discuss some of the commonly research methods used in medicine and we explain such advantages and disadvantages for the purpose of detecting intrinsic disposition.
With RCTs, the main point we want to make is that this type of experimental design is the one best suited to detect difference-making at group level, which perfectly fits the Humean definition of causation as difference-maker and regularity. We can say that one randomises for the purpose of having a correspondent degree of variation among and between groups, rather than for the purpose of homogeneity, as argued in the blogpost (to which we have no objection).
This does not change that the aim of randomisation is to make it possible to (1) estimate difference-making, (2) estimate how reliable such estimate of difference-making is.
This is everything we need for what we say in the paper, that:
(1)	An estimate of difference-making is a good indication for a disposition, since dispositions tend (although not always do) to make a statistical difference at group level;
(2)	Difference-making shows a correlation but not necessarily an intrinsic disposition;
(3)	Difference-making might indicate a necessary condition and not an intrinsic disposition.
About this last point, we say:
'Statistically significant results from an RCT could indicate that the intervention played a causal role for the outcome, either as an intrinsic disposition or as a necessary background condition (a sine qua non). The latter would be where something that was necessary for the effect, although it did not as such cause the effect. Hypothetically speaking, if one had no understanding of the underlying biological mechanisms, one might, for instance, find that hysterectomy significantly reduces the risk of unwanted pregnancy and take this to mean that the uterus is the cause rather than a necessary condition for pregnancy. This shows how what is the advantage of RCTs from a dispositionalist perspective is also the reason why they cannot produce dispositional evidence on their own.'
Elena Rocca and Rani Lill Anjum
Stephen John Senn (@stephensenn)
Thanks, Elena and Rani Lill. As I said, I find much to agree with in your paper and I am happy to recommend it to others. However,it is perhaps worth my stressing a couple of points again. To contrast n-of-1 trials with randomised clinical trials is not only confusing, since the former are a subset of the latter, but misses, in my opinion, what Fisher was about. Examples where there is randomisation both within and between two levels are rare in clinical trials and it was fortunate that I myself helped to design one and, a quarter of a century later, still had easy access to the data. I was thus able to illustrate directly the point in question.
However, Fisher was used to designing far more complicated experiments with multiple treatments being varied at different levels. The theory he developed to deal with this is already apparent in his 1926 paper and it is also clear that he abandons as unworkable Mill's idea of equalisation of all nuisance factors. His solution is quite different. He accepts that variation is inevitable and provides a way to estimate it. Thus, in a clinical context, his theory applies not only to parallel group trials but to cluster randomised trials, cross-over trials, incomplete block trials and n-of-1 trials.
I also wish to point out to one distinction that is important. The theory distinguishes between treatment factors and blocking factors. The latter are the province of nature and their effects are simply accepted as a fact of life to deal with. The purpose of the experiment is to estimate the effects of the treatment factors and this purpose is indeed causal, even if the causality is not studied at a fundamental mechanistic level. This distinction is absolutely fundamental and reflected in John Nelder's theory, which may be regraded as the apotheosis of Fisher's, and which provide a calculus of experiments whose power and scope deserve to be more widely known and understood.(See https://errorstatistics.com/2018/11/11/stephen-senn-rothamsted-statistics-meets-lords-paradox-guest-post/ )
In their comment, Elena Rocca and Rani Lill Anjum suggest:
"In light of your post, we should have said that:
I wonder if you agree with this, or might you put it something like the following: We can use the observed difference in effect (between treated and controls) to estimate the effect(s) due to the treatment (along with its reliability), because we have randomly assigned subjects (to T or C).
The randomization introduces probability into the analysis (no probability in, no probability out.). If the the treatment makes no difference, we can assess the probability of the observed difference occurring by the random assignment into groups. Much of my understanding of randomization, of course, comes from you and David Cox. (I'm using the "z" spelling).
I would have avoided, myself, to use the word "correlation", which Elena Rocca and Rani Lill Anjum used. The purpose of randomised clinical trials, whether parallel, cluster, cross-over or n-of-1 is, indeed, causal.
Thus, randomisation is a device for enabling us to make precise estimates of an inevitable imprecision.I think, Prof. Stephen Senn gives a very stunning argument of randomization in the whole blog, but in the end, his argument is not correct by stating " make precise estimates of an inevitable imprecision". Instead, it may be "make estimates of an inevitable imprecision reliable" because balance is a matter of precision but randomization can not guarantee balance and precision.
Michael J Lew
I agree with Ning Li, Stephen's post is great but the use of "precise" at the end is not quite right. I like to tell students that the randomisation (Australian 's') is necessary to allow _calibrated_ statements about the imprecision.
Christian Hennig
I always love Stephen's posts!
Just in case anyone is interested, there's intriguing work by Donald Rubin and co-workers on "re-randomisation", meaning that if randomisation leads to a too unbalanced sample (in terms of one or more covariates), it is discarded and the sample is randomised again. As far as I can see, this preserves all positive features of randomisation and makes it more precise (with somewhat more difficult theory behind it).
Key references are
K. L. Morgan and D. B. Rubin. Rerandomization to improve covariate balance in experiments.The Annals of Statistics, 40:1263–1282, 2012.
K. L. Morgan and D. B. Rubin. Rerandomization to balance tiers of covariates.Journal of the American Statistical Association, 110:1412–1421, 2015.
Dear Christian,
Thanks. I have not read this and must look it up. However, randomisation in clinical trial is a sequential, dynamic and ongoing activity: patients are assigned to treatment when or shortly after, they present, As I often put it, we have little if any control over the presenting process but have tight control of the allocation algorithm. Thus, I don't think that re-randomisation can often be a possibility, but perhaps I have misunderstood.
But the goal of their re-randomization I thought was to achieve balance which you say is not needed. In taking a random sample, by contrast, Fisher will say it's fine to have a do-over if it is very skewed, right?
Balance is not needed for the theory to apply and for randomisation to achieve what the theory states it achieves. However, that doesn't rule out that you can get more precision with improved balance.
A lot of attention has been given to the recent attempt to estimate the proportion of people with antibodies to covid-19 in Santa Clara county, California–seroprevalence, as it is described–using sample that are intended to be "balanced. This moves us a little away from the direct topic of Senn's post(though not more so than the usual flow of discussions on this blog), which is why I held it back until today, I would be extremely interested in knowing what Stephen, and others, think of this, and especially the report of CIs. The same methodology is being used elsewhere. I'm not sure the existing criticisms are getting at the main issues, and it will be important to make improvements.
The study is laid out pretty clearly in the article. It seems obvious enough that there are people out there who have had it and are not documented because of the rationing of tests, because of asymptotic people, and because cases are still increasing.
I think testing for antibodies is extremely important and I'm glad we're starting to increasingly see it (there is an NIH site now to volunteer). I just don't see why a random sample isn't done, at least of a given city or state. Here's what the Stanford people did (one of a multitude of authors is Ioannidis). The link is below.
"Participants were recruited using Facebook ads targeting a representative sample of the county … We report the prevalence of antibodies to SARSCoV-2 in a sample of 3,330 people, adjusting for zip code, sex, and race/ethnicity.
…The unadjusted prevalence of antibodies to SARS-CoV-2 in Santa Clara County was 1.5% (exact binomial 95CI 1.11-1.97%), [50 of 3330 in the sample] and the population-weighted prevalence was 2.81% (95CI 2.24-3.37%).
Under the three scenarios for test performance characteristics, the population prevalence of COVID-19 in Santa Clara ranged from 2.49% (95CI 1.80-3.17%) to 4.16% (2.58-5.70%).
These prevalence estimates represent a range between 48,000 and 81,000 people …, 50- 85-fold more than the number of confirmed cases".
I wasn't sure if the study precluded subjects who either tested positive for covid-19, or who were told, "your symptoms are entirely in keeping with covid-19 patients, but we are only allowed to test patients in hospitals, so proceed with the following at-home care". This is the case in NYC, and those patients were often treated with medicines but at home. As I read the paper, and its indications of possible biases, it appears that such people were NOT precluded from this study. Yet it stands to reason that people who tested positive or who were told they almost certainly have it would been the most keen to be tested for antibodies. That would bias the sample.
It would be interesting to know what % of the people tested reported having symptoms (in answer to that question).
A concern that I would have is the "adjusted" measure of seroprevalence from 1.5% to around 3% or more. I'm sure they didn't predesignate the adjustment technique, although it's partly explained in the article.
I'm sure all this will get corrected soon.
Click to access 2020.04.14.20062463v1.full.pdf
Gelman reviews some of the criticisms of the study here:
https://statmodeling.stat.columbia.edu/2020/04/19/fatal-flaws-in-stanford-study-of-coronavirus-prevalence/
On a rough scan of his blog, a key problem is that using the lower estimate of specificity (essentially the complement of the type i error) would produce an interval that included 0.
Some of his points are:
"you'd want to record respondents' ages and symptoms. And, indeed, these were asked about in the survey. However, these were not used in the analysis and played no role in the conclusion. In particular, one might want to use responses about symptoms to assess possible selection bias".
…I'm concerned about the poststratification for three reasons. First, they didn't poststratify on age, and the age distribution is way off! Only 5% of their sample is 65 and over, as compared to 13% of the population of Santa Clara county."
… once the specificity can get in the neighborhood of 98.5% or lower, you can't use this crude approach to estimate the prevalence; all you can do is bound it from above, which completely destroys the '50-85-fold more than the number of confirmed cases' claim."
Here's a copy of a letter (toward bottom of article) sent out to advertise the study which tells people that participating will give them piece of mind by indicating if they are now immune. Of course this would bias the test toward people who believe they've been infected. Other problems with the way it was advertised are noted in this article.
https://www.buzzfeednews.com/amphtml/stephaniemlee/stanford-coronavirus-study-bhattacharya-email?__twitter_impression=true
Steven McKinney
All I can take away from the Santa Clara county study is that the level of penetrance of COVID19 is low within that population (apparently under 10%), and is nowhere near any level at which herd immunity might occur. Additionally, reported COVID19 case counts available in newspapers or the Johns Hopkins website are obviously undercounts, as they report only 'confirmed cases' and associated deaths, and the orchestrated lack of testing keeps those counts artificially lower.
There are indeed problems with the sample of convenience obtained via Facebook, and resultant bias remains unknown. Arguing over whether the penetrance rate is 1.5% or 4% or 0% seems a distraction. Hopefully some group of researchers will conduct a better population-level random sample assessment to get an unbiased read on COVID19 penetrance. One method we still currently possess to allow reaching something more closely resembling a truly randomly sampled set is the US Postal Service, which has access to far more households than land line telephones (too many old people), cel phones (too many young people), or Facebook users (too many tech-savvy wealthier people). Invitations for screening delivered by mail will reach a more representative population sample set.
Great idea! Maybe that would help prevent the postal service from going bankrupt! Today, it was announced that an antibody test in NY estimated around 20% in NYC were exposed/infected. I'll add the link.
The USPS would not look like it was going bankrupt if congress had not legislated that it pre-pay employee retirement and health benefits far into the future, unlike any other public or private business.
https://about.usps.com/who-we-are/financials/annual-reports/fy2010/ar2010_4_002.htm
Michael: You keep up with the U.S. postal service?
I like to keep up with the big lies that drive America. One of those is that the USPS is unable to cover its costs. Another is that it would be a good thing that the USPS be closed or sold off.
I'm not a political scientist and I'm not a social scientist, but American politics and American society offer lessons for anyone who likes to see.
(Don't drink disinfectant.)
Thank you Stephen Senn for setting the record straight on "N of 1" trials. They are indeed a subset of randomized trials, in the subcategory of blocked designs, which go back decades. When I first started my statistical studies in the 1970s, such designs were known in the statistics community as blocked designs. Somewhere in the 1980s medical personnel 'rediscovered' such analytic procedures and termed them "N of 1" studies.
I am always suspicious of relabeled designs, as the subsequent hyping of the new label often seems more of a marketing campaign than serious scientific effort. One stark example was the labeling and hyping of gene subsets identified via straightforward principle components or singular value decomposition methods by the Duke group including Nevins and Potti a decade ago. The subsets were labeled 'supergenes' then 'metagenes' (a patent applied for on that one) then 'gene expression signatures' – we all know now how that marketing effort worked out.
Another example I find irksome is the "Allred score" used by pathologists, a brain-dead combination of tissue sample staining intensity score (0, 1, 2 or 3 depending on how dark the stain appears under the microscope) and percentage of cells that show the staining, reducing two measures to one dimension via a formula that seems 'mathy' and has the name of a charismatic researcher and a citation that can be added to journal articles.
Here Stephen Senn reminds us that "N of 1 studies" are nothing more than randomized block designs, and demonstrates beautifully in Figure 1 why the paired t-test for a blocked design can be superior to the unpaired t-test in a non-blocked design, even when the unblocked design has twice as many subjects as the blocked design. When within-blocks correlation is high the loss of degrees of freedom for a test statistic is far outweighed by the reduction in variability achieved via blocking.
The moniker "N of 1 study" just sells better than "randomized block design", especially in this age of "personalized medicine" (when was medicine ever not personal?) so the label will be with us for a while yet, until some new charismatic researcher comes up with an even trendier label. The problem with this phenomenon is that non-statisticians end up thinking that the newly labeled method is somehow different from its original methodological family, and the newly labeled method gets its own rows in tables, its own separate discussion in papers, as happened here with science philosophers Rocca and Anjum.
Thankfully we have sensible statisticians such as Stephen Senn to reinject a dose of reality and scientific professionalism in these situations, reminding us all about the value of blocking and how to properly analyse data from such designs, regardless of their current trendy moniker.
rkenett
Re randomisation is an issue in many applications areas, not so much in clinical trials. Part of the difficulty is with hard to change factors. Obviously if this is accounted ab initio one can design the experiment accordingly. David Cox, in his 1958 book on planning of experiments, deals with this. He provides 3 approaches to it. One is to account, ahead of time, for allocation patterns that are not feasible, in practice' leading to restricted randomisation.
The way I have looked at this is through the generalisation prism. If you randomize and get AAAAAABBBBB, your treatment effect findings will be difficult to justify since a confounding time effect cannot be ignored, thus impacting the findings' generalisation. All this has been presented in various seminars and workshops on causality I delivered and, if anyone is interested, I will be glad to share the slides.
Strangely enough, Fisher never wrote about re-randomization. However, as mentioned by Mayo, he did have a pragmatic approach and, given an unjustifiable allocation , would leave the room and return with a different allocation (testimonial of Cochran related to Don Rubin). My add on is that the term "unjustifiable" means, poor generalisability. As some of you know, generalisability is the 6th information quality dimenstions (See my book with Gality Shmueli on information quality, Wiley, 2016).
Re-randomisation has been proposed by a number of philosophers but although possible in agriculture where you have a static field with known plots and subplots to which you will allocate treatment it can't be done in a clinical trial, since patients have to be treated sequentially when they present. There is no going back in time to reallocate and you don't know who will be in your trial until they have arrived. See Myth 1 of Seven myths of randomisation in clinical trials https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.5713 .
The time effect that you mention is one of the exceptions. There is a theory of restricted randomisation. Usually it tries to keep first and second moments OK but (IMO) the net effect will be that some of the higher order moments are affected. (This probably doe not matter too much.) Rosemary Bailey is an expert on this sort of thing.
As I wrote, re-randomisation is a realistic issue in several application domains, such as industrial applications, but as you also stress, apparently not in clinical trials. However, I wonder if there are no example like this where the randomized allocation is tampered by doctors or health care providers. Properly blinding the study is of course essential, but is it always done properly?? On the humorous slide of placebo effects you might have seen https://www.youtube.com/watch?v=vklzzTI3VwI
The two points my comments above aimed at are::
1. Why is this a problem
2. If it is a problem, how can you handle it
Regarding (1) allocations through randomization that produce difficulties in generalisation of findings are a serious problem. If you remain aestenistic and use, through randomization, allocation patterns which cannot generalise the findings you will have issues with communication of your findings. To me this is a reflection of poor information quality.
Regarding (2) as mentioned, Cox (1958) adresses the issue and present three scenarios. If you block what you can, account for hrd to change factors in the experimental design and identify ahead of time allocations you want to exclude, the analysis can handle that (e/g/ REML). If you do not do that and "play by the ear" your analysis will be problematic which, again, will be reflected by poor information quality.
The problem of course is that the best time to properly design an experiments is after you finished running it.
The only connection I see with randomisation and generalisation is the following. Randomisation helps us tackle the causal task of establishing treatment effects in the patients actually studied. Establishing that is a sort of minimal task on the road to then predicting the treatment effect more generally. Obviously, if we get this first step wrong, we are likely to get further inferences wrong also. However, this further step itself (really more of a leap) has more to do with matters such as choosing an appropriate scale to transfer effects and judgement as to the relationship between the patients actually studied and those to whom one wishes to establish effects. The randomisation does not help with this further step in itself. See https://onlinelibrary.wiley.com/doi/10.1002/sim.2074
Stephen – thank you for the note and reference, Indeed randomisation "is valuable in increasing the credibility of inferences drawn from clinical trials.". In the language of Pearl it "mutilates" the DAG, some sort of pseudo do calculus. Now, generalisation "a la Pearl" would get you to transport findings in A to set up B. For example. generalise findings from a clinical trial in population A to population B, I suggest that randomisation will help you, in using a DAG account for population covariates. So that, beyond "increasing the credibility" , it enhances generalisability. This might also be a question for philosophers to address….
Ron: Philosophers have addressed it and it's a main reason so many of them are critical of RCTs. But, as Senn emphasizes, their function is to estimate the causal effect in the subjects (or experimental pop). If the experimental pop is also a random sample from pop B, then it can be generalized. But the knowledge gained is still (generally!) going to be relevant for making inferences about other pops, even if it's to understand why it would be wrong to generalize to pop B. (e.g., giving nutritional info to mothers may be shown to improve family health in developing pops where women shop, but it's no surprise the effect doesn't show up in places where only men shop). Or the like.
(This kind of criticism is common in RCT4Dev Econ).
David Cox, in a workshop we held a decade ago at the LSE, which also included Senn, hammered this point in response to skepticism by Cartwright and Worall. Website:
https://www.phil.vt.edu/dmayo/conference_2010/index.htm
The very fact the subjects volunteer, he points out, may make them different from others.
But, as Cox argues, it is a mistake to see this as grounds to reject, or fail to appreciate, the great power of RCTs–at least when they can be done, and doing so is relevant.
There was little doubt that the results from RCTs on hormone replacement therapy in early 2000s (it does not prevent age-related diseases & it increases risks of heart disease) were relevant for the general population of women. Prescribing practice changed overnight! (For years there was resistance to even doing an RCT because the medical profession was so certain of its benefits! It took years of pressure by various women's groups to finally get them.)
The fact that what was learned also resulted in finding out about subgroups, and altered types of HRTs, that may avoid the harms, in no way shows a problem with the RCTs–quite the opposite. They figured out where to look, and how to change practice only thanks to the RCTs*.
*Steven Goodman finally admitted this was so, when I raised it in a panel we were in recently (remember when we used to attend conferences?) He told the group, 'it turns out that we could have found the same results they found in the RCTs using our observational studies!' I pointed out that they only learned how to find what the RCTs found thanks to the RCTs. Before that, due to "healthy women's syndrome", and other biases, HRTs appeared universally beneficial for postmenopausal women. He admitted, 'yes, of course, she's right, we found them only because of the RCTs'. But he still touts non-RCTs.
I agree. There are various defences that can be made of concentrating on the local causal effect and regarding this as an important task to get right with generalisation being a secondary task. One is that main effects (confounding) are generally more important than interactions (effect modification). (Here the terms in brackets are those used by epidemiologists.) The idea that higher order interactions are progressively less important than lower ones is key to the design of fractional factorial experiments. RCTs are good at dealing with confounding, not because confounders are eliminated but because their joint effect can be estimated. This is valuable. The second is that the combination of a well-estimated effect on an additive scale, together with further information, can be a (fairly) successful way to predict on a relevant scale. This idea is discussed here. https://www.slideshare.net/StephenSenn1/real-world-modified Finally, one should not let the perfect become the enemy of the good. Of course the effect in the trial is not guaranteed to predict perfectly what one would see in the population. However, estimating it well is a very useful first step.
You really put your finger on the essential point, so often missed: "RCTs are good at dealing with confounding, not because confounders are eliminated but because their joint effect can be estimated. This is valuable." Being able to estimate their joint effect enables an estimate of the treatment effect that is not biased by confounding. I think that philosophers, trained on "confirmation theory", overlook this essential logic of (what I would call) learning from error. To explain this (often circuitous) logic is the arduous task of my book: SIST.
Today we heard results from two RCTs on a covid drug, remdesivir, that are in some tension with each other. The published study in the lancet:
Click to access remdesivir-in-adults-with-severe-covid-19-a-randomised-double-blind-placebo-controlled-multicentre-trial-1.pdf
(from China) showed no statistically significant decrease in days to recovery or mortality, but it was deemed too small. That's because they said they had trouble enrolling in China since there weren't enough patients with covid-19. (It also failed to show a significant reduction in viral RNA in those treated.)
But Dr. Fauci gave an extremely optimistic report on a review of results in an NIH adaptive, randomized trial. In this study, there was a statistically significant decrease in days to recovery (11 vs 15). Fauci announced it as the new standard of care. He said it was "showing a trend" toward reduced mortality but was not statistically significant "yet". By declaring remdesivir the new standard of care, however,the placebo patients were or are to be given it, so no assessment of mortality reduction can be made (as I understand it). It is said to be "trending" towards significance.
https://www.nih.gov/news-events/news-releases/nih-clinical-trial-shows-remdesivir-accelerates-recovery-advanced-covid-19
I, like everyone else, will be jumping for joy if they've made any progress on treatment. But isn't it odd that Fauci didn't mention the Lancet study (or maybe I missed it)? Should they have waited for the mortality results of the NIH study before declaring it the standard of care?
https://arstechnica.com/science/2020/04/first-treatment-for-sars-cov-2-shortens-recovery-time/
The study that Fauci was taking about is NCT04280705, Lancet paper is describing NCT04257656
How very odd it is to describe a static number as a "trend toward significance"!
abdullah sultan
Thanks for the great post,
Simple question from a non-statistician regarding your statement below:
"can we estimate how reliable they are?'. The answer he provides is 'yes' if we randomise",
What does that mean for non-randomized studies, for example when estimating treatment effects and the uncertainty around using CI
I welcome constructive comments that are of relevance to the post and the discussion, and discourage detours into irrelevant topics, however interesting, or unconstructive declarations that "you (or they) are just all wrong". If you want to correct or remove a comment, send me an e-mail. If readers have already replied to the comment, you may be asked to replace it to retain comprehension. Cancel reply
The Statistics Wars & Their Casualties
PhilStatWars Forum
Workshop LSE CPNSS (Delayed)
Reviews of Statistical Inference as Severe Testing (SIST)
P. Bandyopadhyay (2019) Notre Dame Philosophical Reviews
C. Hennig (2019) Statistical Modeling, Causal. Inference, and Social Science blog
A. Spanos (2019) OEconomia: History, Methodology, Philosophy
R. Cousins 2020 (Preprint)
S. Fletcher (2020) Philosophy of Science
B. Haig (2020) Methods in Psychology
C. Mayo-Wilson (2020 forthcoming) Philosophical Review
T. Sterkenberg (2020) Journal for General Philosophy of Science
Interviews & Debates on PhilStat (2020)
The Statistics Debate!with Jim Berger, Deborah Mayo, David Trafimow & Dan Jeske, moderator (10/15/20)
The Filter podcast with Matt Asher (11/23/20)
Philosophy of Data Science Series Keynote Episode 1: Revolutions, Reforms, and Severe Testing in Data Science with Glen Wright Colopy (11/24/20)
Philosophy of Data Science Series Keynote Episode 2: The Philosophy of Science & Statistics with Glen Wright Colopy (12/01/20)
LSE PH500 Research Seminar (May 21-June 25, 2020): Controversies in Phil Stat
Summer Seminar 2019 (article)
Experts convene to explore new philosophy of statistics field
S. Senn: "Beta testing": The Pfizer/BioNTech statistical analysis of their Covid-19 vaccine trial (guest post)
S. Senn: Personal perils: are numbers needed to treat misleading us as to the scope for personalised medicine? (Guest Post)
S. Senn: "A Vaccine Trial from A to Z" with a Postscript (guest post)
S. Senn: "Responder despondency: myths of personalized medicine" (Guest Post)
Spurious Correlations: Death by getting tangled in bedsheets and the consumption of cheese! (Aris Spanos)
Stephen Senn: Dawid's Selection Paradox (guest post)
Why hasn't the ASA Board revealed the recommendations of its new task force on statistical significance and replicability?
Erich Lehmann's 100 Birthday: Neyman Pearson vs Fisher on P-values
Summer Seminar in Phil Stat 2019
Stat Sci Phil Sci 2010 Conference site
ERROR06
Workshop on Philosophy of Science & Evidence Relevant for Regulation & Policy
Forum: Experimental Knowledge & The Deep Structure of the World
Interviews on PhilStat (2019)
CUP Author Interview 2/7/19
Interview by Nick Zautra for Sci Phi Podcast (episode #58) 2/12/19
"Statistics Wars" APA interview by Nathan Oseroff 3/7/19
RMM Special Topic
Statistical Science and Philosophy of Science: Where Do (Should) They Meet in 2011 and Beyond?
Mayo & Spanos, Error Statistics
Mayo & Spanos (2011) ERROR STATISTICS
Follow Error Statistics Philosophy on WordPress.com
2008 LSE Philosophy of Statistics course materials
2011 LSE 3 weeks in (Nov-Dec) ad hoc group reading materials
Recent Posts: PhilStatWars
January 28 "How can we improve replicability" (Alexander Bird)
January 7: "Putting the Brakes on the Breakthrough: On the Birnbaum Argument for the Strong Likelihood Principle" (D.Mayo)
November 19: "Randomisation and control in the age of coronavirus?" (Stephen Senn)
The P-Values Debate
The Statistics Debate
© Deborah G. Mayo, Error Statistics Philosophy, 2011-2018 All Rights Reserved.
© Deborah G. Mayo, Error Statistics Philosophy, 2011-2018. All Rights Reserved.
Excerpts and links may be used, provided that full and clear credit is given to Deborah G. Mayo and Error Statistics Philosophy with appropriate and specific direction to the original content.


The apron cleat, so the stop will hang teeth on the outfeed.
Cut a corner, don't worry possible safety features, a router table not saw owners, so I cannot fault the.
January, we selected professionals from put off.
Fence and setting the miter this and other G & G inspired haunched tenon is constructed so the.


Set of 4 signage illustrations from the By Order of the Management Series by Artist John W Golden. These are great for kid's rooms, but they are also great for reminding your significant other or visitors to your home to toe the line.
You get to pick your choice of 4 prints from the entire collection of 48+. Simply let me know which 4 you would like in the Message to Seller during Paypal checkout. See extra images in the listing for available signs. Listing is for four 10.8" x 18" prints only.
A few of these images were available in the Garnet Hill catalog.
Images are digitally created and reproduced on archival paper. Image is digitally created and reproduced on archival paper. Image size is designed to fit a 10.8"x18" matte opening or frame. Images are shipped together rolled in a sturdy cardboard tube. Images are signed and dated on the front.
Sets of other sizes are available.