Thousands of fans headed for the exits after 74 minutes - the club was formed in 1874 - in protest at the team's plight and owner Randy Lerner.
"I am concentrating on my job," said Frenchman Garde, whose side are bottom of the table, eight points from safety.
"I don't have any comments about the fans. It's something very deep, very old. My job is difficult enough."
Villa fans have directed a lot of anger towards American Lerner, who bought the West Midlands club in 2006.
They are upset at the lack of investment and the decision-making at the club, but the players have also been criticised for a perceived lack of desire.
Aston Villa fan Mark Bricknell, one of the organisers of the protest, told BBC Radio 5 live: "This has been coming for six years.
"We can't say we're happy to be having to do this.
"The club have failed to halt what supporters have seen coming and it's down to total incompetence from the owner."
BBC Radio 5 live football reporter Pat Murphy, who has been covering Villa for more than four decades, added: "The mood is turning ugly.
"I remember the demonstrations against Doug Ellis, but he must seem a great chairman in retrospect.
"This is the worst team I've seen in 41 years of reporting on Villa."
A crowd of 29,755 were at Villa Park on Tuesday to watch the loss to Everton, Villa's 18th league defeat of the season.
In the top flight since 1988, Villa have won just three of their 28 league games so far this season and have only 16 points.
They are not mathematically down yet but face fourth-placed Manchester City and Tottenham - who will go top of the table with a win at West Ham on Wednesday - in their next two games.
Garde, appointed at the start of November, still believes Villa can retain their Premier League status but admits the situation is "getting more difficult".
"We have to stick together," said the 49-year-old former Lyon boss. "If not, we have no chance."
If Villa do go down, Joey Barton - who has been relegated from the Premier League with both Newcastle and QPR - thinks they could struggle in the second tier.
"I feel sorry for manager Remi Garde as the ship had massive holes in it when he took over," the Burnley midfielder, whose are top of the Championship, told BBC Radio 5 live.
"It is not a team equipped for Championship football. The Villa fans are showing discontent and you can't blame them.
"Randy Lerner and the board - I've called them muppets before - and nothing has changed.
"If you don't spend in January and say that you are going to prepare for life in the Championship, then why not start playing some kids from the academy?
"At least they'll show some passion."
Watch Match of the Day on Wednesday on BBC One and online - 22:45-00:15 in England, 23:45-01:15 in Scotland & NI; 23:20-00:50 in Wales.

The 26-year-old collided with Black Cats midfielder Wahbi Khazri.
"It is dislocated so he is out for several weeks," said United boss Louis van Gaal after Saturday's game.
Luke Shaw, Marcos Rojo, Ashley Young, Antonio Valencia, Phil Jones and Guillermo Varela are out - leaving Cameron Borthwick-Jackson and Donald Love as the club's only fit full-backs.
Love, 21, who was on loan at Wigan earlier this season, came off the bench during the Premier League game to make his first-team debut. Saturday was also the first time he had featured in United's first-team squad.
Borthwick-Jackson, 19, made his first-team debut in November.
"We don't have any more full-backs in this club," added Van Gaal. "I have to line up a 13-year-old guy or someone like that."
Darmian joined United from Italian side Torino last summer and started at right-back against Sunderland.
Aston Villa manager Remi Garde refused to criticise supporters for walking out during the 3-1 home defeat by Everton.
[NEXT_CONCEPT]
Matteo Darmian added to Manchester United's full-back problems after dislocating his shoulder at Sunderland.