Most similar Apple TV software for OS X ? I really love the way latest Apple TV front-row works. Its easy, clean and fast.
I've been trying many different Media Center applications for my Mac Mini, but none of them are just like the one Apple TV provides.

I could buy an AppleTV, but I really don't want to have yet another device at home, so I think there should be an application that replicates Apple TV menus/workflow.

Do you know of any?  Plex, XBMC or Boxee are far of what I'm looking for
I'd say go with Plex. I've been using and testing different media center apps for years, and, like you, stuck with Front Row until Lion came out, because nothing else felt quite right. Since then, Plex has won me over, especially with the huge progress made in the last year or so.

In terms of user experience, Plex is the closest thing to Snow Leopard's Front Row. I always disliked the visual clutter of XBMC and Boxee, and even earlier Plex versions. But the current Plex version has a VERY clean interface and works great with the Apple Remote.

It can handle almost any file format (without installing extra codecs), unlike Front Row, and you can have it watch any folder on your machine (or network), not just the ~/Movies, ~/Videos and iTunes.

It automatically downloads artwork and metadata for movies and TV shows in your library based on their filenames, and there are many high-quality channels (plugins for streaming content) that you can install right inside the client.

The channels have to present their data in the same layout as the native Plex library, so the interface is consistent. Plex integrates with iPhoto and iTunes as well, showing them as available channels.

Finally, since Plex is really a two-part setup (client and server, both of which can easily run on the same machine), it's expandable. Once you set up the server, you can use it with the client on that machine, and you can also view your library on other Macs, PCs, iOS devices, Android devices, Google TV, Roku, LG Smart TVs, and DLNA TVs. I think they even released a WebOS client too.

Most of the client apps work with absolutely zero configuration. As long as you're on the same network, all available Plex servers and their libraries just show up. Start playing something, and the server will even transcode the video or audio (in realtime) if the client device doesn't have the necessary codecs to handle it.

Plex isn't perfect, but it's under very active development, and constantly improving. And the UI finally looks and feels natural. 

Also, it's everywhere. Practically every screen in my house has access to my Plex library. Some of the apps (iOS, Android etc.) cost money, which is good for the long-term survival of Plex as a platform, but the Linux (server only), Mac (client + server), Win (client + server) are all free.