There are two mechanisms the Xbox provides for you to do this.
It lists the most downloaded games
It lists the highest rated games
Now you have to take both of these metrics with a grain of salt.
People will rate games high
they have only played once, or download games they hear about a lot.
So keep that in mind.
As for the actual steps to do this.
Open your 360 dashboard Scroll up to game marketplace
Once inside scroll right to see highest rated and most downloaded If you're looking for good games
I might suggest:
Castle Crashers Shadow Complex Trials HD Puzzle Quest 2
gaming.stackexchange.com/users/102/tzenes
Every game on Xbox Live Arcade has a trial mode.
I suggest you take advantage of this excellent feature.
Don't bother looking through what games are most popular or most highly rated.
The best possible advice I can give you is to download as many games as you can in the genres
you're interested in, and play through their trials.
If you can't trust your own opinion of the game after having played the trial version, there is little help to be found elsewhere.
gaming.stackexchange.com/users/1037/Christian
Metacritic tags Xbox
Live Arcade games.
Try this search .
It's not perfect - it misses Braid!
- but seems to cover most the good ones I know.
The dashboard also supports community-based star rating of games.
gaming.stackexchange.com/users/80/monorailkitty
There is a new Destination Arcade coming soon.
Looks interesting.
gaming.stackexchange.com/users/40/senloe
Check out Major Nelson's blog , particularly the "LIVE Activity" reports that he publishes weekly to find out what XBL Arcade games people are actively playing.
It changes from week to week, but gems like Trials HD will hang around in the top ten for quite some time.
gaming.stackexchange.com/users/50/ZoogieZork
I've been looking through http://www.xblaratings.com lately.
It has user submitted ratings and reviews for XBLA games.
Granted since it requires and account there isn't MUCH info there sometimes, but it's one resource available.
gaming.stackexchange.com/users/599/JustinP8
With so many games on Xbox Live Marketplace (XBLM), how can I identify the better games or more popular games from the market?
How to find the better games on Xbox Live Marketplace?
gaming.stackexchange.com
<xbox-360><xbox-live>
gaming.stackexchange.com/questions/148
gaming.stackexchange.com/users/71/drye
Every game has a trial mode. You can look at Mjor Nelson's blog to find the most popular games. S2: Don't bother looking through what games are most popular or most highly rated. Every game on Xbox Live Arcade has a trial mode you can use and try out yourself. You could check out  Major Nelson's blog , particularly the "LIVE Activity" reports that he publishes weekly to find out what XBL Arcade games people are actively playing or take a look at  http://www.xblaratings.com.
S2: Don't bother looking through what games are most popular or most highly rated. Every game on Xbox Live Arcade has a trial mode you can use and try out yourself. You could check out  Major Nelson's blog , particularly the "LIVE Activity" reports that he publishes weekly to find out what XBL Arcade games people are actively playing or take a look at  http://www.xblaratings.com.