user
As the world gets smaller each day, internationalization becomes more and more important.   For handling multiple languages, Unicode is your best friend.

It is a very capable and remarquable tool, but also quite complex compared to older single- and double-byte character encodings.

How well prepared is your programming language for Unicode?

Task

Discuss and demonstrate its unicode awareness and capabilities.

Some suggested topics:

:   * How easy is it to present Unicode strings in source code?
    * Can Unicode literals be written directly, or be part of identifiers/keywords/etc?
    * How well can the language communicate with the rest of the world?
    * Is it good at input/output with Unicode?
    * Is it convenient to manipulate Unicode strings in the language?
    * How broad/deep does the language support Unicode?
    * What encodings (e.g. UTF-8, UTF-16, etc) can be used?
    * Does it support normalization?

Note

This task is a bit unusual in that it encourages general discussion rather than clever coding.

See also

* Unicode variable names
* Terminal control/Display an extended character
assistant
(formerly Perl 6)

Raku programs and strings are all in Unicode and operate at a grapheme abstraction level, which is agnostic to underlying encodings. (These are generally handled at program boundaries.) Opened files default to UTF-8 encoding. All Unicode character properties are in play, so any appropriate characters may be used as parts of identifiers, whitespace, or user-defined operators. For instance:

```
sub prefix:<∛> (\𝐕) { 𝐕 ** ⅓ }
say ∛27;  # prints 3

```

Non-Unicode strings are represented as Buf types rather than Str types, and Unicode operations may not be applied to Buf types without some kind of explicit conversion. Only ASCIIish operations are allowed on buffers.

Raku tracks the Unicode consortium standards releases and is generally up to the latest standard within a few months or so of its release. (currently at 15.0 as of February 2023)

* Supports the normalized forms NFC, NFD, NFKC, and NFKD, and character equivalence as specified in Unicode technical report #15.
* Built-in routines provide access to character classifications (Letter, Numeric. White-space, etc) and sub-classifications: (Letter-lowercase, Letter-uppercase, Numeric-digit, etc.)
* Allows users to use any Unicode character that has a Numeric property **as** that numeric value.
* Provides Unicode aware upper-case, lower-case and fold-case routines.
* Implements the Unicode technical standard #10 collation algorithm, (though not all optional mappings are supported yet).
* Provides built-in routines to access character names, do name-to-character character-to-ordinal and ordinal-to-character conversions.
* Works seamlessly with upper plane and private use plane character codepoints.
* Provides tools to deal with strings that contain invalid Unicode characters.

In general, it tries to make dealing with Unicode "just work".

Raku intends to support Unicode even better than Perl 5, which already does a great job in recent versions of accessing large swaths of Unicode spec. functionality. Raku improves on Perl 5 primarily by offering explicitly typed strings that always know which operations are sensical and which are not.

A very important distinctive characteristic of Raku to keep in mind is that it applies normalization (Unicode NFC form (Normalization Form Canonical)) automatically by default to all strings as showcased and explained on the String comparison page.