Struggling to type a TypeScript function
Your ResolveOrientationFunction type definition,
type ResolveOrientationFunction = <
  T extends HTMLAttributes<HTMLElement>["aria-orientation"],
  U extends HTMLAttributes<HTMLElement>["role"]
>(
  ariaOrientation: T,
  role: U
) => "both" | NonNullable<T>;

is generic in both T, the type of ariaOrientation, and U, the type of role.  It returns a value of type "both" | NonNullable<T>.  So if T is undefined because ariaOrientation is undefined, then the function must return "both" | NonNullable<undefined> which is "both".  But your implementation doesn't do that.  It can instead return "horizontal" or "vertical" depending on role.
So your resolveOrientation function is not a valid ResolveOrientation.

It's not clear that you need the function to be generic at all.  Certainly the U type parameter isn't useful as written, since it has no effect on the return type.  And you don't really want the T type parameter to be reflected directly in the output type either.  It seems like your return type should just be "both" | "vertical" | "horizontal" without reference to T or U.  And if you have a generic function where there's no obvious dependency on the type parameters, then you might not want a generic function in the first place.
If you change the generics to specific types like this:
type AriaOrientation = HTMLAttributes<HTMLElement>["aria-orientation"];
type AriaRole = HTMLAttributes<HTMLElement>["role"]
type ResolveOrientationFunction =
    (ariaOrientation: AriaOrientation, role: AriaRole) => 
    "both" | NonNullable<AriaOrientation>;

Then your function compiles cleanly:
export const resolveOrientation: ResolveOrientationFunction = (ariaOrientation, role) => {
    if (ariaOrientation === undefined) {
        switch (role) {
            case "menubar":
            case "slider":
            case "tablist":
            case "toolbar": {
                return "horizontal";
            }

            case "listbox":
            case "menu":
            case "scrollbar":
            case "tree": {
                return "vertical";
            }
        }
    }

    return ariaOrientation ?? "both";
};

Playground link to code

I've got this TypeScript error and I don't fully understand what's going on:
src/helpers.ts:11:14 - error TS2322: Type '<T extends "horizontal" | "vertical" | undefined, U extends AriaRole | undefined>(ariaOrientation: T, role: U) => "horizontal" | "vertical" | NonNullable<T> | "both"' is not assignable to type 'ResolveOrientationFunction'.
  Type '"horizontal" | "vertical" | NonNullable<T> | "both"' is not assignable to type 'NonNullable<T> | "both"'.
    Type '"horizontal"' is not assignable to type 'NonNullable<T> | "both"'.

Here is my function:
import { type HTMLAttributes } from "react";

type ResolveOrientationFunction = <
  T extends HTMLAttributes<HTMLElement>["aria-orientation"],
  U extends HTMLAttributes<HTMLElement>["role"]
>(
  ariaOrientation: T,
  role: U
) => "both" | NonNullable<T>;

export const resolveOrientation: ResolveOrientationFunction = (ariaOrientation, role) => {
  if (ariaOrientation === undefined) {
    switch (role) {
      case "menubar":
      case "slider":
      case "tablist":
      case "toolbar": {
        return "horizontal";
      }

      case "listbox":
      case "menu":
      case "scrollbar":
      case "tree": {
        return "vertical";
      }
    }
  }

  return ariaOrientation ?? "both";
};

The function is supposed to return "both" | "horizontal" | "vertical".
HTMLAttributes<HTMLElement>["aria-orientation"] is actually "horizontal" | "vertical" | undefined and HTMLAttributes<HTMLElement>["role"] is React.AriaRole | undefined.
I'm actually trying to make this function match the type "both" | NonNullable<HTMLAttributes<HTMLElement>["aria-orientation"]>.
