Question: Does anyone know how to create a iTunes Playlist that shows only songs in My Library which are part of a complete album in My Library.

Essentially I want to see all my complete albums. You can in Spotify but with Apple Music every song you add to a playlist it assumes you want it added to My Library.

Answer: 
I don't have a full answer for this, but I eliminated about 200 songs from my latest Smart Playlist (e.g., 2017 songs) by adding criteria of Album does not contain text "Single". There's unfortunately no filter for "of" field for total count of tracks, because then you could filter out anything where this is set to 1 (1 of 1 tracks, vs. 1 of 13 on an album, for example).
Apple has released a fix for this. In Settings>Music it lets you toggle between showing playlist songs in library or not.