Comprehensive Guide to NVIDIA CUDA Runtime CU11 API

The NVIDIA CUDA Runtime CU11 is a foundational framework for developers leveraging GPU acceleration in their applications. With support for parallel computing, the runtime provides numerous APIs designed for high-performance computing, deep learning, and scientific simulations. This guide will introduce key APIs, examples, and an application example encapsulating the power of CUDA Runtime CU11.

Getting Started with CUDA Runtime CU11

Before diving into APIs, ensure you have installed the CUDA Toolkit. You can download it from NVIDIA’s official website. The CUDA Runtime provides a user-friendly, high-level API interface to handle memory allocation, parallel computation, and device management.

Useful APIs in NVIDIA CUDA Runtime CU11

1. cudaMalloc – Allocating Memory on the Device

This API allocates memory on your GPU device for computation.

  cudaError_t cudaMalloc(void **devPtr, size_t size);
  
  // Example
  float *d_array;
  size_t size = 10 * sizeof(float);
  cudaMalloc((void**)&d_array, size);

2. cudaMemcpy – Copying Data Between Host and Device

Transfer data between CPU (host) and GPU (device).

  cudaError_t cudaMemcpy(void *dst, const void *src, size_t count, cudaMemcpyKind kind);
  
  // Example
  float h_array[10] = {0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0};
  cudaMemcpy(d_array, h_array, size, cudaMemcpyHostToDevice);

3. cudaFree – Freeing Device Memory

Release allocated GPU memory once computations are complete.

  cudaError_t cudaFree(void *devPtr);
  
  // Example
  cudaFree(d_array);

4. cudaMemset – Setting Memory

Initialize or reset GPU memory to a particular value.

  cudaError_t cudaMemset(void *devPtr, int value, size_t count);
  
  // Example
  cudaMemset(d_array, 0, size);

5. cudaGetDeviceCount – Query Available Devices

Find out the number of GPUs available on a system.

  cudaError_t cudaGetDeviceCount(int *count);
  
  // Example
  int device_count;
  cudaGetDeviceCount(&device_count);
  printf("Number of CUDA devices: %d\n", device_count);

6. cudaDeviceSynchronize – Synchronize Device

Waits for the device to finish all preceding tasks.

  cudaError_t cudaDeviceSynchronize();
  
  // Example
  cudaDeviceSynchronize();

7. Kernel Launch Syntax

Parallelize computations with a kernel function.

  // Example
  __global__ void addArrays(float *a, float *b, float *result, int n) {
      int idx = threadIdx.x + blockIdx.x * blockDim.x;
      if (idx < n) {
          result[idx] = a[idx] + b[idx];
      }
  }
  
  dim3 blocks(2, 1, 1);
  dim3 threads(512, 1, 1);
  addArrays<<>>(d_a, d_b, d_result, n);

Application Example: Vector Addition

Below is a complete example using the discussed APIs to perform vector addition:

  #include <cuda_runtime.h>
  #include <iostream>

  __global__ void vectorAdd(const float *a, const float *b, float *result, int n) {
      int idx = blockIdx.x * blockDim.x + threadIdx.x;
      if (idx < n) {
          result[idx] = a[idx] + b[idx];
      }
  }

  int main() {
      int n = 1000;
      size_t size = n * sizeof(float);

      // Host memory
      float *h_a = (float*)malloc(size);
      float *h_b = (float*)malloc(size);
      float *h_result = (float*)malloc(size);

      // Initialize input vectors
      for (int i = 0; i < n; i++) {
          h_a[i] = static_cast<float>(i);
          h_b[i] = static_cast<float>(i);
      }

      // Device memory
      float *d_a, *d_b, *d_result;
      cudaMalloc((void**)&d_a, size);
      cudaMalloc((void**)&d_b, size);
      cudaMalloc((void**)&d_result, size);

      // Transfer to device
      cudaMemcpy(d_a, h_a, size, cudaMemcpyHostToDevice);
      cudaMemcpy(d_b, h_b, size, cudaMemcpyHostToDevice);

      // Execute kernel
      int threadsPerBlock = 256;
      int blocksPerGrid = (n + threadsPerBlock - 1) / threadsPerBlock;
      vectorAdd<<>>(d_a, d_b, d_result, n);

      // Copy results back
      cudaMemcpy(h_result, d_result, size, cudaMemcpyDeviceToHost);

      // Check results
      for (int i = 0; i < n; i++) {
          std::cout << h_result[i] << " ";
      }

      // Free memory
      free(h_a);
      free(h_b);
      free(h_result);
      cudaFree(d_a);
      cudaFree(d_b);
      cudaFree(d_result);

      return 0;
  }

Conclusion

The NVIDIA CUDA Runtime CU11 offers an extensive set of APIs to harness the processing power of GPUs for various domains of computation. This guide has covered a selection of these APIs and provided a complete example of vector addition. Explore more APIs in the CUDA documentation and try integrating GPU computing into your next project!