CUDAメモリ転送系関数の Async サフィックス有無*1と、実際の同期(synchronous)／非同期(asynchronous)動作は1:1対応しない。Asyncサフィックス無しメモリ転送関数でも、条件によっては非同期動作となる可能性がある。
API synchronization behavior
https://docs.nvidia.com/cuda/cuda-runtime-api/api-sync-behavior.html
The API provides memcpy/memset functions in both synchronous and asynchronous forms, the latter having an "Async" suffix. This is a misnomer as each function may exhibit synchronous or asynchronous behavior depending on the arguments passed to the function.
シングルスレッド処理のホストかつ単一デバイス利用という単純構成では、特に意識しなくとも問題にはならない（たぶん）。一方、ホスト側がマルチスレッド処理であったり、複数GPU間でのデバイス間メモリ転送を行う場合には、非同期動作には十分留意する必要がある。罠すぎる（#＾ω＾）
後者環境では、CUDAストリーム(cudaStream_tやCUstream)とAsyncサフィックス付き関数を組み合わせ、明示的な同期関数(cudaStreamSynchronizeやcuStreamSynchronize)呼び出しによってメモリ転送完了を待機する。*2
Memcpy
In the reference documentation, each memcpy function is categorized as synchronous or asynchronous, corresponding to the definitions below.Synchronous
- All transfers involving Unified Memory regions are fully synchronous with respect to the host.
- For transfers from pageable host memory to device memory, a stream sync is performed before the copy is initiated. The function will return once the pageable buffer has been copied to the staging memory for DMA transfer to device memory, but the DMA to final destination may not have completed.
- For transfers from pinned host memory to device memory, the function is synchronous with respect to the host.
- For transfers from device to either pageable or pinned host memory, the function returns only once the copy has completed.
- For transfers from device memory to device memory, no host-side synchronization is performed.
- For transfers from any host memory to any host memory, the function is fully synchronous with respect to the host.
Asynchronous
https://docs.nvidia.com/cuda/cuda-runtime-api/api-sync-behavior.html
- For transfers from device memory to pageable host memory, the function will return only once the copy has completed.
- For transfers from any host memory to any host memory, the function is fully synchronous with respect to the host.
- For all other transfers, the function is fully asynchronous. If pageable memory must first be staged to pinned memory, this will be handled asynchronously with a worker thread.
関連URL
