Can iPhone wake me up if I completely turn it off? On the feature phones I used to have, I could set the alarm clock, then turn the phone "off": the alarm clock would trigger at the set time anyway. Can I do this on iPhone?

By "off" (in quotes, because, we know that from electrical point of view, some parts of the phone stay powered) I mean the state of the phone you get when it is turned on, then you press the power button for a couple of seconds, then a red slider appears saying "slide to power off" and then you slide it.
Playing the Clock alarm sound requires the main application ARM processor/CPU to be running the OS and Clock app.  When you "slide to power off", power to the main CPU, radios and audio amplifier is removed.  So, although the iOS device keeps time (as long as the battery isn't completely dead yet), it can't run the Clock app to play any sound.

When you just "lock" the device (quickly tap the top button and the display turns off), the CPU still occasionally gets power (something like a few times per second), and the audio circuits can be powered and active (to allow background music, etc.), so alarms can go off in that state.