Answers.Body: <p>Incomplete UTF8 support is <a href="https://lab.civicrm.org/dev/core/-/blob/5.30/CRM/Core/BAO/CustomField.php#L1619" rel="nofollow noreferrer">hard coded</a> into the core code before 5.33 (<em>thanks @michaelmcandrew!</em>).</p>
  <p>The solution seems to be to re-run the <code>System.utf8conversion</code> routine</p>
  <ul>
  <li>Pre 5.33: do this after any time you create a field/fieldset (including anytime you install an extension that creates a custom field/fielset)</li>
  <li>5.33+ do this once, then the only time it should be needed again is if an extension hard-codes the old UTF8 in place.</li>
  </ul>
  <p>Note that you can use the <a href="https://lab.civicrm.org/extensions/uhoh" rel="nofollow noreferrer">uh-oh</a> extension to help discover whether your db uses the old UTF8 stuff.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-30T15:18:24.220Z
Answers.Text: Note that there is a [PR 18642](https://github.com/civicrm/civicrm-core/pull/18642) to fix this, thanks Seamus!
Answers.UserDisplayName: artfulrobot
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-21T08:35:11.400Z
Answers.Text: That PR is merged, but not in 5.31 so assume it will make it to 5.32
Answers.UserDisplayName: artfulrobot
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-10T16:49:50.213Z
Answers.Text: Actually it looks like it came out in 5.33 (https://civicrm.org/blog/dev-team/civicrm-533-release)
Answers.UserDisplayName: michaelmcandrew
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-30T08:06:45.067Z
Answers.LastActivityDate: 2021-12-13T08:53:51.653Z
Answers.LastEditDate: 2021-12-13T08:53:51.653Z
Answers.UserDisplayName: artfulrobot
Body: <p>There's an API call <code>System.utf8conversion</code> which will change all your tables to MySQL/MariaDB's utf8mb4 format, i.e. full unicode with emoji support, rather than the incomplete utf8 implementation of old.</p>
  <p>I ran this and it worked fine but then found that when I add a custom fieldset, that table is set to the old UTF8 charset.</p>
  
Tags: <custom-fields>
Title: How to use full unicode on custom fields
ThreadText: # How to use full unicode on custom fields
  
  (Asked by: username_0 on 2020-09-30)
  
  There's an API call `System.utf8conversion` which will change all your tables to MySQL/MariaDB's utf8mb4 format, i.e. full unicode with emoji support, rather than the incomplete utf8 implementation of old.
  
  I ran this and it worked fine but then found that when I add a custom fieldset, that table is set to the old UTF8 charset.
  
  **Tags:** custom-fields
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-09-30. Score: 5 | Accepted answer
  
  Incomplete UTF8 support is [hard coded](https://lab.civicrm.org/dev/core/-/blob/5.30/CRM/Core/BAO/CustomField.php#L1619) into the core code before 5.33 (*thanks @username_1!*).
  
  The solution seems to be to re-run the `System.utf8conversion` routine
  
  * Pre 5.33: do this after any time you create a field/fieldset (including anytime you install an extension that creates a custom field/fielset)
  * 5.33+ do this once, then the only time it should be needed again is if an extension hard-codes the old UTF8 in place.
  
  Note that you can use the [uh-oh](https://lab.civicrm.org/extensions/uhoh) extension to help discover whether your db uses the old UTF8 stuff.
  
  > **Comments:**
  > **username_0**: Note that there is a [PR 18642](https://github.com/civicrm/civicrm-core/pull/18642) to fix this, thanks Seamus!  
  > **username_0**: That PR is merged, but not in 5.31 so assume it will make it to 5.32  
  > **username_1**: Actually it looks like it came out in 5.33 (https://civicrm.org/blog/dev-team/civicrm-533-release)  
  
  ---
  
  