title: cudaHostAlloc limitations
body: <p>I have a system running Windows Server 2012. The system has 128 GB of memory and 6 Nvidia K10 GPUs with 8 GB of memory.</p>
  
  <p>I use pinned host memory for all my devices (about 47 GB, 12 * 3.92 GB per device). The reason I want to use pinned memory is that my application runs about 12 - 15 times since I can use asynchronous memory transfers.</p>
  
  <p>I have 3.92 GB memory available on each device. If I use cudaHostAlloc and try to allocate 3.92 GB I get cudaErrorMemoryAllocation when I try to allocate host memory for the second device. I use cudaHostAlloc with the cudaHostAllocDefault default flag.</p>
  
  <p>If call cudaHostAlloc with 3.79 GB (instead of 3.92) then everything works as expected, hence I can allocate 3.79 GB of pinned memory for all 12 devices. </p>
  
  <p>I do not understand this limitation and I have done an attempt using VirtualAlloc, VirtualLock and cuMemHostRegister but I run into the same problem when calling cuMemHostRegister. I have ensured that the working set is enough and the VirtualAlloc, VirtualLock combo works with the specified working set. cudaHostAlloc does not seem to care what working set I use. The default working set works as long as I do not try to use more then 3.79 GB of memory per device with cudaHostAlloc. </p>
  
  <p>Any one have an explanation for this behavior?</p>