A Saudi-led coalition of nine countries has been heavily criticised over the civilian death toll from a bombing campaign in Yemen.
The parliament called on EU foreign policy chief Federica Mogherini to "launch an initiative aimed at imposing an EU arms embargo".
The motion passed by 449 votes to 36.
There were 78 abstentions.
Although the vote is not legally binding, lawmakers hope it will pressure EU governments to agree to an embargo, following a petition of 750,000 European citizens.
Scottish MEP Alyn Smith, who led the motion, said EU-made weapons were being exported to Saudi Arabia "in breach of international law".
Mr Smyth said: "Having grown up in Saudi Arabia I am sensitive to the realities of the Saudis and appreciate that the Saudis have concerns in their neighbourhood.
"But our duty is to the civilians in Yemen, and given widespread and very valid concerns over the conduct of the war by Saudi forces, our call for an EU-wide arms embargo is proportionate and necessary."
Anna MacDonald, director of campaign group Control Arms, said the group welcomed "this first step towards preventing European arms from being used to further human suffering in Yemen and elsewhere".
Nearly 6,000 people have been killed since the Saudi-led coalition entered the conflict in March 2015, almost half of them civilians, according to the United Nations.
The US is the largest international supplier of arms to Saudi Arabia. The UK and France are the main European suppliers, while Germany has also licensed arms exports to the kingdom.
Correction 26 February 2016:  This report has been amended to change incorrect figures given for the European Parliament vote.

The activity of hundreds of genes was altered when people's sleep was cut to less than six hours a day for a week.
Writing in the journal PNAS, the researchers said the results helped explain how poor sleep damaged health.
Heart disease, diabetes, obesity and poor brain function have all been linked to substandard sleep.
What missing hours in bed actually does to alter health, however, is unknown.
So researchers at the University of Surrey analysed the blood of 26 people after they had had plenty of sleep, up to 10 hours each night for a week, and compared the results with samples after a week of fewer than six hours a night.
More than 700 genes were altered by the shift. Each contains the instructions for building a protein, so those that became more active produced more proteins - changing the chemistry of the body.
Sources: Mental Health Foundation and BBC Science
Discover what disturbs your sleep the most
Meanwhile the natural body clock was disturbed - some genes naturally wax and wane in activity through the day, but this effect was dulled by sleep deprivation.
Prof Colin Smith, from the University of Surrey, told the BBC: "There was quite a dramatic change in activity in many different kinds of genes."
Areas such as the immune system and how the body responds to damage and stress were affected.
Prof Smith added: "Clearly sleep is critical to rebuilding the body and maintaining a functional state, all kinds of damage appear to occur - hinting at what may lead to ill health.
"If we can't actually replenish and replace new cells, then that's going to lead to degenerative diseases."
He said many people may be even more sleep deprived in their daily lives than those in the study - suggesting these changes may be common.
Dr Akhilesh Reddy, a specialist in the body clock at the University of Cambridge, said the study was "interesting".
He said the key findings were the effects on inflammation and the immune system as it was possible to see a link between those effects and health problems such as diabetes.
The findings also tie into research attempting to do away with sleep, such as by finding a drug that could eliminate the effects of sleep deprivation.
Dr Reddy said: "We don't know what the switch is that causes all these changes, but theoretically if you could switch it on or off, you might be able to get away without sleep.
"But my feeling is that sleep is fundamentally important to regenerating all cells."
The European Parliament has adopted a resolution calling for an arms embargo on Saudi Arabia over alleged breaches of international law in Yemen.
[NEXT_CONCEPT]
A run of poor sleep can have a potentially profound effect on the internal workings of the human body, say UK researchers.