Hi! Yes, shuffle rearranges the rows of a Dataset. I agree this part of the docs is not very clear (cc @stevhliu).  And if you are interested in shuffling a single column, you can do the following:
import datasets
single_col = <col_name>  # specify the column name here
dset_single_col = dset.remove_columns([col for col in dset.column_names if col != single_col])
dset_single_col_shuffled = dset_single_col.shuffle()
dset_without_single_col = dset.remove_columns([single_col])
dset = datasets.concatenate_datasets([dset_without_single_col, dset_single_col_shuffled], axis=1)