Chris Coleman's team will play Belgium at the Stade Pierre Mauroy on Friday.
The stadium is less than 10 miles (16km) from the border, with reports suggesting more than 100,000 Belgians could descend on Lille.
But Wales fans in France believe the "away-game" atmosphere could suit the team.
Karen Crocker, a Wales fan from Cardiff, said: "There's going to be far more Belgian fans than Welsh.
"There's lots of Welsh coming over to go in the fan zone but in the stadium we're going to be outnumbered."
But Mark Watkins, 50, also of Cardiff, said it would "definitely" suit Wales having to play like an away team.
"The way we play, we're going to need to ride our luck a bit, but we've got a chance."
He added: "I think we will have a fair chunk of support [in Lille].
"We'll make plenty of noise. I think it will be a fantastic atmosphere.
"We'll try and be the 12th man in this game."
On Tuesday, Belgium midfielder Kevin de Bruyne said: "I can't wait for this match, which will be like a home match for us because it's right on the Belgian border."
And residents of the French city are also keen to back the Belgians.
Solange Baillon, 27, said: "Here in the north of France, we were really part of the same country in the past so we just cheer for each other, every time."
She said people in the city had found Wales fans "more discreet" than England fans when the two sets of supporters were in town for England's 2-1 win in Lens.
Ms Baillon said the people of Lille had been "very happy" to be part of the tournament.
Fellow Lille resident Vincent Houttelitu, 29, said he believed most of France would cheer on Belgium, saying: "We are very close."
Wales fans have begun arriving in the town and collecting their tickets from the voucher exchange at Grand Palais.
Some 4,500 of the tickets at the 50,000 capacity stadium have been allocated to Wales fans with Follow My Team vouchers.
And while many fans in France for the earlier stages have returned home, the Football Supporters' Federation Cymru is expecting tens of thousands of Wales fans to make it to Lille on day trips.
For those without a ticket, the city's fan zone can hold 30,000 people.
Wales will go into the game with a measure of confidence after beating and drawing with Belgium during their qualifying campaign.
Should they win on Friday, the team would face a potential semi-final against either Portugal or Poland in Lyon on 6 July.
Wales fans will be heavily outnumbered in their Euro 2016 quarter-final with Belgium in Lille - with most French fans cheering for the opposition.