text: Title: `coerce_numbers_to_str` can cause unicode decode errors
  Body: ### Initial Checks
  
  - [X] I confirm that I'm using Pydantic V2
  
  ### Description
  
  Hi! I just upgraded pydantic version and was testing passing a string like `'hi there!\ud835'` which contains an unpaired unicode character to a model with `coerce_numbers_to_str=True` and saw a unicode error. When I changed it to `False` it went away.
  
  I'm not sure whether this should/shouldn't throw an error, but I think the behavior should be more consistent.
  
  
  ### Example Code
  
  ```Python
  from pydantic import BaseModel, ConfigDict
  
  class ModelWithCoercion(BaseModel):
       x: str
       model_config = ConfigDict(coerce_numbers_to_str=True)
  
  class ModelWithoutCoercion(BaseModel):
       x: str
  
  # Ok
  ModelWithoutCoercion(x='hi there!\ud835')
  
  # Also Ok
  ModelWithoutCoercion(x=b'hi there!\ud835')
  
  # Also Ok
  ModelWithCoercion(x=b'hi there!\ud835')
  
  # Error
  ModelWithCoercion(x='hi there!\ud835')
  ```
  
  
  ### Python, Pydantic & OS Version
  
  ```Text
  pydantic version: 2.9.2
          pydantic-core version: 2.23.4
            pydantic-core build: profile=release pgo=false
                   install path: /Users/aagam.dalal/Library/Caches/pypoetry/virtualenvs/egp-api-backend-YxsoG3qx-py3.11/lib/python3.11/site-packages/pydantic
                 python version: 3.11.5 (main, Sep 11 2023, 08:31:25) [Clang 14.0.6 ]
                       platform: macOS-14.3.1-arm64-arm-64bit
               related packages: fastapi-0.115.2 mypy-1.12.0 typing_extensions-4.12.2
                         commit: unknown
  ```
  