Brain scans can show how the brain gets "tired and over-emotional" when someone is deprived of sleep.
The brain needs sleep to function
US researchers kept volunteers awake for 35 hours and found huge increases in brain activity when shown images designed to make them angry or sad.
The research in the journal Current Biology points to links between mental illness and sleep problems, they said.
But a British expert said psychiatric problems could not be blamed solely on lack of sleep.
Teams from Harvard Medical School and the University of California at Berkeley used functional magnetic resonance imaging (fMRI) to look at the brains of sleep-deprived volunteers.
This technique lets scientists look at blood flow in the brain in real-time, showing where the brain is most active.
After the volunteers had stayed awake for an extended period, they were scanned while being shown picture cards designed to provoke an emotional response.
The researchers found that the parts of the brain linked to emotional reactions - the amygdala - showed bigger reactions (over 60% more) to the cards compared with a normally-rested volunteer.
Researcher Matthew Walker said: "The size of the increase really surprised us.
"It is almost as though, without sleep, the brain reverts back to a more primitive pattern of activity, becoming unable to put emotional experiences into context and produce controlled, appropriate responses."
He suggested that the findings might shed light on the connection between sleep and psychiatric disorders.
"Clinical evidence has shown that some form of sleep disruption is present in almost all psychiatric disorders.
"These findings may offer new mechanisms as to why."
In the UK, Professor Jim Horne, of the Sleep Research Centre at Loughborough University, welcomed the research, but said it would be difficult to use it to unravel the relationship between mental health and sleep.
He said: "This is a complex area - the big difference is that people with mental illness might not be aware that they are over-reacting or behaving irrationally, whereas someone with sleep deprivation would be more aware of this overreaction.
"In addition, we know that in illnesses such as depression, actually reducing the amount of sleep can be beneficial in moderation a well-supervised environment."
Professor Derk-Jan Dijk, a sleep researcher from the University of Surrey, described the findings as "interesting".
"While there have been a lot of studies into the effects of sleep deprivation, this is the first to show what is happening in the brain in response to these emotional stimuli."