Specify GPU-ID for single-GPU simulation on multi-GPU server

[OliverAh]

I want to run a circuit simulation (cudaq.sample(...) & cudaq.get_state(...)) on a server with multiple GPUs. The simulation requires only a single GPU. I want to specify which GPU should be used.

What I have so far:

Following the docs https://nvidia.github.io/cuda-quantum/0.9.0/using/examples/multi_gpu_workflows.html#circuit-batching I managed to use the asynchronous cudaq.sample_async(kernel, qpu_id=3), but the standard cudaq.sample(kernel) always uses GPU 0, and cudaq.sample(kernel, qpu_id=3) throws an error "... unexpected keyword argument qpu_id". See also MWE below. I used nvidida-smi to inspect the memory utilization of the 4 available GPUs, to verify that cudaq.sample_async(kernel, qpu_id=3) indeed loads GPU 3.

Questions:

1. Is it intended that only cudaq.sample_async is able to target a specific GPU?
2. If not, how would I achieve targeting a specific GPU with the standard cudaq.sample?

MWE

import cudaq

cudaq.set_target("nvidia", option="mqpu,fp64")
print('number of gpus found:', cudaq.num_available_gpus())

@cudaq.kernel
def kernel():
    q = cudaq.qvector(2)
    h(q)

shots_count = int(1e6)

# The following line fails with the error: "TypeError: sample() got an unexpected keyword argument 'qpu_id'"
#print(cudaq.sample(kernel, shots_count=shots_count, qpu_id=3))

# The following line works, but always runs on GPU 0
print(cudaq.sample(kernel, shots_count=shots_count))

# The following line works, and respects the qpu_id argument, but is asynchronous.
samples_async = cudaq.sample_async(kernel, shots_count=shots_count, qpu_id=3)
print(samples_async.get())

Prints:

number of gpus found: 4
{ 00:249710 01:249749 10:250246 11:250295 }

{ 00:249710 01:249749 10:250246 11:250295 }

Edit:

Environment

• CUDA-Q Version 0.9.0
• Python Version 3.11.11
• C++ compiler: x86_64-conda-linux-gnu-c++ (installed via micromamba following quick start guide)
• Operating System: Ubuntu 24.04.1 LTS

[mitchdz]

Hi Oliver, thanks for your question!

I do believe that cudaq.sample_async is the only function able to target a specific GPU, but I might be wrong off the top of my head.

If you want to target a specific GPU more generally, I'd recommend setting the CUDA_VISIBLE_DEVICES environment variable to a comma separated list of the GPU device IDs you would like to expose.

[OliverAh]

Hi Mitchell,

thanks for the clarification.
CUDA_VISIBLE_DEVICES set to a single GPU-ID, in combination with cudaq.sample, works to target the specific GPU.
Nevertheless, I think it would be nice to set the GPU-ID within the script. But for now I do not mind sticking to cudaq.sample_async.

[mitchdz]

Great to hear that! If you would like that capability, please feel free to raise an issue with your request and then we can assess adding that feature in.

[arjami25]

Yes, currently only cudaq.sample_async(..., qpu_id=...) supports GPU targeting. The standard cudaq.sample(...) doesn’t accept qpu_id yet. To run on a specific GPU, stick with the async version for now.