Should a weak iPhone battery output less than 3.7V? Is there a way to check for weak battery? My battery dies after 2 hours of use but still outputs 3.7V in a multimeter check. Could it still be dead?
The answer is yes it could be week.
What you measuring is the Voltage without LOAD.

In that state even an almost completely empty battery will output the Voltage it was designed for.

You need to bridge it with a resistor (load) and then measure voltage.

Your multimeter does that (LOAD) but only very small kind, like micro amps by bridging the outputs with a resistor of 1 Meg or more so it draws very small current

Try using a LED or if you have resistors of lets say 100 Ohm, use that as load (bridge).