Large Language Models, Llm, AI, Deep Learning, Machine Learning.
training data could reduce the speed of calculation and not memory efficient. Solution: Implementing a vector compression based on the count-sketch data structure, Min-Max Hash, and random projection, by mixing random shuffling and random projection to compress the gradient vector. RandomShuffling.