I see so basically the code can pass to the next line of calculation before GPU finishes calculating and I should put sync where ever I either copy something to the GPU or do some calculations in the GPU. I will update the code as you suggested and try again.