Question: The subject pretty much speaks for itself, but I am quite confident that if I deploy a standard Web API to an Azure web site, it will work without issue using standard forms authentication.

So I assume an Azure API app has some added extra's, like security and other services. What does an Azure app offer me that a standard one doesn't, and if not using the extras, is there any reason to create an Azure app especially for deployment to Azure?

Answer: 
Azure API Management is a way to publish and consume your Web APIs, it enables you to categorize your APIs, create segments and add pricing for each segment, it also offers many other features like caching, throttling, and format conversion. All of these features can be enabled by some manipulation of a simple XML file.

API Apps allow you to host your APIs, monitor it and easily deploy your Web APIs to Azure.

So, you can assume that it is an addition to Web API
**Web Apps** and **API Apps** are both part of **App Service**. They are both web hosting platforms. And as you said, a Web API will work just fine in a Web App.

Before, API Apps had certain features that Web Apps did not, though that has changed. Web Apps also support e.g. Swagger definitions now.

The difference between the two at present time is the icon. That's what it says in the documentation.