Can you keep an iPhone permanently offline? After doing initial setup and whatnot, will the phone and all preinstalled apps work properly without ever connecting to the internet again? (Assuming basic cellular is active, or not if that makes a difference) 

Specifically, is there any “phone home“ requirement in the default software?
Yes.

You can keep an iPhone permanently offline after doing the initial setup and still use it. You can continue to use the basic telephonic feature, and the stock apps such as Calculator, Camera, Photos etc. which doesn't require an active Internet connection to function.

There's no requirement in iOS, the operating system running on iPhones to phone home, that would hinder its normal operation, in total absence of an Internet connection.

However, any feature, even within the built-in apps, dependant on Internet connectivity would remain unavailable, such as the ability to send and receive iMessages, accessing the App Store to install new apps, syncing contacts, email, calendar etc. to iCloud, using Siri, Apple virtual assistant and even updating the iOS software (in untethered mode).