Question: Where can I get polygons (multipolygons) of all countries of the world including the territorial waters of the countries? 

OpenStreetmap displays these areas but I can't find an extract of this subset of the OSM data.

* is there a subset of OSM data available, just containing the territories (incl. water territories) of a country as multipolygons ?
* or, can I find this set of polygons somewhere else?

Answer: 
The country boundaries including the territorial waters are available in the OSM data set.
I found a site where these country shapes can be downloaded: 
<https://wambachers-osm.website/boundaries/>
Take a look at the Global Administrative Areas dataset at <http://www.gadm.org>, it contains at least the boundaries, though I am not so sure about the territorial waters part