The benchmark Nikkei 225 closed down 3% at 16,883.19 points - its worst day since March.
The US oil price fell below the symbolic threshold of $50 a barrel for the first time since April 2009.
Meanwhile in London, Brent crude fell as low as $53 a barrel.
That prompted energy shares to fall.
Concerns over Greece's future also weighed on the euro, which slid to a nine-year low against the dollar.
Chinese shares recovered earlier losses after a private survey showed the country's services sector grew at its fastest pace in three months in December.
The Shanghai Composite ended up closing flat at 3,351.45 after falling as much as 1.3% earlier in the day.
In Hong Kong, the benchmark Hang Seng index finished down 1% to 23,485.41, following the global trend.
In Australia, investors also reacted to falling oil prices and eurozone worries, with the benchmark S&P/ASX 200 closing down 1.57% at 5,364.80 points.  Earlier in the day, the index was down as much as 1.9%  - one of its biggest falls in more than a month.
One company's shares saw a positive reaction to falling oil prices, however, with Qantas, Australia's national airline, seeing its share price at a four-year high during the Tuesday session.
Australia's latest trade figures showed the country's deficit widening in November from October to about 950m Australian dollars ($770m). It is the eighth trade deficit in a row.
But analysts said investors largely ignored the numbers, which were smaller than expected and due in part to falling prices for resources such as iron ore.
Tristan K'Nell, head of trading at Quay Equities, said there were a number of issues worldwide causing uncertainty in equity markets.
"Investors [are] watching closely... US interest rates, growth slowdowns in Europe and the Asia Pacific, political uncertainty in Greece, geopolitical issues between Russia and Ukraine and also the Middle East, and the volatility and crashing prices in crude oil and iron ore," he said.
In South Korea, the benchmark Kospi closed down 1.74% to 1,882.45 points, with shares in energy firms leading the declines on oil price worries.

The 28-year-old victim, from Liverpool, required surgery for his injuries following Thursday's attack.
He was travelling on the 20.34 GMT service from Liverpool Lime Street to Birmingham New Street when he challenged the group.
He was attacked after leaving the train at Winsford station, Cheshire, at around 21.10.
Det Sgt Matt Davies, of British Transport Police, said: "Thankfully the victim's injuries are not life-threatening, but he remains in hospital where he has received surgery for his wounds.
"Our investigation into who is behind this attack continues and we have been at the station conducting a forensic examination of the scene, local inquiries and reassurance patrols.
"It is vital we now speak to anyone who was on the 8.34pm train out of Liverpool and witnessed the disturbance in the carriage, as well as anyone else who was in the area of Winsford station at 21:.10 and saw what happened."
The attacker was in his late 20s and of stocky build. It is believed he was part of a group also consisting of four or five women.
He followed the victim before being repeatedly stabbing him in the groin and upper body.
The victim was taken to safety by another passenger from the train.

They are accused of conspiring together, and with others, to sell goods in 2012 that contained wholly, or in part, a mix of beef and horsemeat.
Ulrik Nielsen, Alex Ostler-Beech and Andronicos Sideras will appear at City of London Magistrates' Court on Tuesday, 27 September.
The charges follow an investigation led by the City of London police.
Japan's stocks led falls across Asia on Tuesday, posting their biggest drop in nearly 10 months, as investors worried about falling oil prices and political turmoil in Greece.
[NEXT_CONCEPT]
A man was stabbed in a station car park after challenging a group throwing objects around a train carriage.
[NEXT_CONCEPT]
Three men have been charged with fraud for allegedly conspiring to sell horsemeat as beef.