"Why should it be The IRS?" Because government costs money. Just like we pay a price for hamburgers, apartments, Broncos tickets, or Chevy Sonics or Lincoln MKSs
Taxes are the price we pay for government.
Have you ever heard of gift taxes? Your hypothetical would simply be a very large gift. Or is that also unfair?
To answer all your questions, "Yes, it is fair".
"What did I do to earn that money? Nothing."
Really, Robert? You are using the argument that because you did not earn that hypothetical money, it should not be taxed?  So, inherited money should not be taxed? Gifted money should not be taxed? Dividends should not be taxed? Interest on savings or investment should not be taxed? 
Is it okay for people who earned their money (that is, worked for it) to be taxed?
Or is it your view that people who worked for their money should not be taxed, and people who didn't work for their money should not be taxed?
Nobody should be taxed?