I now understand how the embedding layers function in your implementation, but in what sense would feeding the quantized values directly “lose complexity on the input presentation”? What sort of considerations for model performance lead you to that conclusion?

Let’s give you a simple example.
If we feeding the value directly, the input channel size is 1 and the first layer map them to size of residual channel, so the parameter size in the first layer is (res_channels, 1, *), it’s actually just one vector; if we use one-hot encoding (or embedding layer), the parameter size of first layer is (res_channels, 256, *).
You can see that, the first method is actually just scale the amplitude of a hidden vector base on the input, but the second method the model will try to learn 256 different hidden vector for each quantized input value, which has much more freedom than the first method.
Also I want to point out why the first method might still work in this case is that our input data is waveform, although being mulaw-quantized, its raw value still represent some kind of information; but if our input is some kinds of discrete data, like discrete tokens in language processing, its category number can be permutaded and have no extra information like amplitude.

Also, in the more efficient training method you proposed, are the dimensions you are assuming for x, (batch, 256, seq_len)? Is it normal to create a one-hot-encoding by having the dimensions be (num_classes, 1) for each encoded vector? I ask because in PyTorch I thought that this would mean that we have 256 channels, each with a value representing the 0,1 class value - or in the case of the output the probability that the next item belongs to each class.

Exactly, or you can just feed (batch, seq_len) with integer type, and map them to size (batch, seq_len, 256) using embedding layer.

The reason I ask the above question is that I am currently using CrossEntropyLoss which was described in the Wavenet paper. According to the PyTorch documentation, this expects input with dimensions (minibatch, num_classes) and a target describing a class index (in my case: [0,255]). Given this, I would think I should make the one-hot-encoded vectors such that the input has dimensions (batch, seq_len, 256) and the output should be (batch, 1, 256) or (batch, 256).

Yes, or if you use the paralle training method I suggested, your output can also be (batch, seq_len, 256) with target as (batch, seq_len).