Hi ! Not sure if datasets is able to convert a memory-mapped NumPy array to an Arrow array without bringing the NumPy array in RAM (I haven’t tested).
However the error you’re getting looks more like an integer precision issue because the array you are passing has more than 2147483648 elements, maybe you could try chunking your list of embeddings, then get a dataset object per chunk, and concatenate them to get the dataset with all the embeddings.