Scans of 21 people showed the brain was more attracted to food if breakfast was missed so people had more food at lunch.
Scientists said it made losing weight challenging as missing meals made calorific food even more appealing.
Nutrition experts say breakfast is known to take the edge off appetite.
However, researchers were curious about what happened inside the brain to alter the food people choose to eat.
Twenty one people, who were all normal weight, were shown pictures of calorie packed foods while they were positioned in a magnetic resonance imaging (MRI) machine at Imperial College London.
On one day they were given no breakfast before the scans and on a different day they were fed a large, 730 calorie, breakfast an hour and a half before.
The researchers said skipping breakfast created a "bias" in the brain in favour of high calorie foods.
The results, presented at the Neuroscience 2012 conference, showed the brain changed how it responded to pictures of high calorie foods, but not low calorie foods, when breakfast was skipped.
They showed part of the brain thought to be involved in "food appeal", the orbitofrontal cortex, became more active on an empty stomach.
When the researchers offered the participants lunch at the end of the study, people ate a fifth more calories if breakfast was missed.
Dr Tony Goldstone, from Imperial College London, said: "Through both the participants' MRI results and observations of how much they ate at lunch, we found ample evidence that fasting made people hungrier, and increased the appeal of high calorie foods and the amount people ate.
"One reason it is so difficult to lose weight is because the appeal of high calorie food goes up."
Dr Catherine Hankey, a senior lecturer in nutrition at the University of Glasgow, said research had shown that breakfast "takes the edge off appetite" and that the latest study was an "interesting" insight.
She said breakfast was linked to stable blood sugar levels, which "keeps you on the straight and narrow".
Future studies will investigate how obesity affects the same system in the brain.

According to the Daily Telegraph, the minimum fine is set to double up to £100. Current fines set by councils range between £50 and £80.
Ministers are said to be preparing a "clutter-free legacy for England".
And Communities minister Marcus Jones said those who drop litter would be "hit in the pocket".
He added: "Dropping litter is the kind of anti-social behaviour that really gets people's backs up, and rightly so.
"It's thoughtless, selfish and ruins shared spaces for everyone. Not only that, litter clearance and disposal costs hundreds of millions of pounds for councils every year - money that could be going on vital services.
"If litter louts can't put their rubbish in the bin perhaps we can hit them in the pocket instead."
The Clean for the Queen campaign is a nationwide clean-up operation, ahead of Her Majesty's 90th birthday.
Its chief executive, Adrian Evans, said some people thought it was their "human right" to drop rubbish but has urged the public to challenge offenders.
"The way I have done it in the past is to treat it as though they have made a mistake and something has inadvertently fallen out of their bag or pocket. Sort of, 'Oh no, did you mean to drop that?'"
In 2015, councils in England and Wales called for new powers to tackle the "huge and spiralling" problem of people throwing litter from vehicles.
In London, vehicle owners can be fined if someone drops litter from their car.
Brain scans show that skipping breakfast makes fatty, high calorie foods appear far more attractive later in the day, according to researchers.
[NEXT_CONCEPT]
Penalties for people who drop litter could reach as much as £150 under plans set out by the Department for Communities and Local Government.