HIP: Heterogenous-computing Interface for Portability: Stream Management

HIP: Heterogenous-computing Interface for Portability

Typedefs | Functions
Stream Management
HIP API

This section describes the stream management functions of HIP runtime API. The following Stream APIs are not (yet) supported in HIP: More...

Collaboration diagram for Stream Management:

## Typedefs
typedef void(* | hipStreamCallback_t) (hipStream_t stream, hipError_t status, void *userData)

## Functions
hipError_t | hipStreamCreate (hipStream_t *stream)
| Create an asynchronous stream. More...
hipError_t | hipStreamCreateWithFlags (hipStream_t *stream, unsigned int flags)
| Create an asynchronous stream. More...
hipError_t | hipStreamCreateWithPriority (hipStream_t *stream, unsigned int flags, int priority)
| Create an asynchronous stream with the specified priority. More...
hipError_t | hipDeviceGetStreamPriorityRange (int *leastPriority, int *greatestPriority)
| Returns numerical values that correspond to the least and greatest stream priority. More...
hipError_t | hipStreamDestroy (hipStream_t stream)
| Destroys the specified stream. More...
hipError_t | hipStreamQuery (hipStream_t stream)
| Return hipSuccess if all of the operations in the specified `stream` have completed, or hipErrorNotReady if not. More...
hipError_t | hipStreamSynchronize (hipStream_t stream)
| Wait for all commands in stream to complete. More...
hipError_t | hipStreamWaitEvent (hipStream_t stream, hipEvent_t event, unsigned int flags)
| Make the specified compute stream wait for an event. More...
hipError_t | hipStreamGetFlags (hipStream_t stream, unsigned int *flags)
| Return flags associated with this stream. More...
hipError_t | hipStreamGetPriority (hipStream_t stream, int *priority)
| Query the priority of a stream. More...
hipError_t | hipExtStreamCreateWithCUMask (hipStream_t *stream, uint32_t cuMaskSize, const uint32_t *cuMask)
| Create an asynchronous stream with the specified CU mask. More...
hipError_t | hipExtStreamGetCUMask (hipStream_t stream, uint32_t cuMaskSize, uint32_t *cuMask)
| Get CU mask associated with an asynchronous stream. More...
hipError_t | hipStreamAddCallback (hipStream_t stream, hipStreamCallback_t callback, void *userData, unsigned int flags)
| Adds a callback to be called on the host after all currently enqueued items in the stream have completed. For each hipStreamAddCallback call, a callback will be executed exactly once. The callback will block later work in the stream until it is finished. More...
hipError_t | hipStreamWaitValue32 (hipStream_t stream, void *ptr, uint32_t value, unsigned int flags, uint32_t mask __dparm(0xFFFFFFFF))
| Enqueues a wait command to the stream.[BETA]. More...
hipError_t | hipStreamWaitValue64 (hipStream_t stream, void *ptr, uint64_t value, unsigned int flags, uint64_t mask __dparm(0xFFFFFFFFFFFFFFFF))
| Enqueues a wait command to the stream.[BETA]. More...
hipError_t | hipStreamWriteValue32 (hipStream_t stream, void *ptr, uint32_t value, unsigned int flags)
| Enqueues a write command to the stream.[BETA]. More...
hipError_t | hipStreamWriteValue64 (hipStream_t stream, void *ptr, uint64_t value, unsigned int flags)
| Enqueues a write command to the stream.[BETA]. More...

## Detailed Description

This section describes the stream management functions of HIP runtime API. The following Stream APIs are not (yet) supported in HIP:

This section describes Stream Memory Wait and Write functions of HIP runtime API.

- hipStreamAttachMemAsync is a nop

## Typedef Documentation

## ◆ hipStreamCallback_t

typedef void(* hipStreamCallback_t) (hipStream_t stream, hipError_t status, void *userData)

Stream CallBack struct

## Function Documentation

## ◆ hipDeviceGetStreamPriorityRange()

hipError_t hipDeviceGetStreamPriorityRange | ( | int * | leastPriority,
|  | int * | greatestPriority
| ) |  |

Returns numerical values that correspond to the least and greatest stream priority.

Parameters

  [in,out] | leastPriority | pointer in which value corresponding to least priority is returned.
  [in,out] | greatestPriority | pointer in which value corresponding to greatest priority is returned.

Returns in *leastPriority and *greatestPriority the numerical values that correspond to the least and greatest stream priority respectively. Stream priorities follow a convention where lower numbers imply greater priorities. The range of meaningful stream priorities is given by [*greatestPriority, *leastPriority]. If the user attempts to create a stream with a priority value that is outside the the meaningful range as specified by this API, the priority is automatically clamped to within the valid range.

## ◆ hipExtStreamCreateWithCUMask()

hipError_t hipExtStreamCreateWithCUMask | ( | hipStream_t * | stream,
|  | uint32_t | cuMaskSize,
|  | const uint32_t * | cuMask
| ) |  |

Create an asynchronous stream with the specified CU mask.

Parameters

  [in,out] | stream | Pointer to new stream
  [in] | cuMaskSize | Size of CU mask bit array passed in.
  [in] | cuMask | Bit-vector representing the CU mask. Each active bit represents using one CU. The first 32 bits represent the first 32 CUs, and so on. If its size is greater than physical CU number (i.e., multiProcessorCount member of hipDeviceProp_t), the extra elements are ignored. It is user's responsibility to make sure the input is meaningful.

Returns
  hipSuccess, #hipErrorInvalidHandle, hipErrorInvalidValue

Create a new asynchronous stream with the specified CU mask. `stream` returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, application must call hipStreamDestroy.

See also
  hipStreamCreate, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

## ◆ hipExtStreamGetCUMask()

hipError_t hipExtStreamGetCUMask | ( | hipStream_t | stream,
|  | uint32_t | cuMaskSize,
|  | uint32_t * | cuMask
| ) |  |

Get CU mask associated with an asynchronous stream.

Parameters

  [in] | stream | stream to be queried
  [in] | cuMaskSize | number of the block of memories (uint32_t *) allocated by user
  [out] | cuMask | Pointer to a pre-allocated block of memories (uint32_t *) in which the stream's CU mask is returned. The CU mask is returned in a chunck of 32 bits where each active bit represents one active CU

Returns
  hipSuccess, #hipErrorInvalidHandle, hipErrorInvalidValue

See also
  hipStreamCreate, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

## ◆ hipStreamAddCallback()

hipError_t hipStreamAddCallback | ( | hipStream_t | stream,
|  | hipStreamCallback_t | callback,
|  | void * | userData,
|  | unsigned int | flags
| ) |  |

Adds a callback to be called on the host after all currently enqueued items in the stream have completed. For each hipStreamAddCallback call, a callback will be executed exactly once. The callback will block later work in the stream until it is finished.

Parameters

  [in] | stream | - Stream to add callback to
  [in] | callback | - The function to call once preceding stream operations are complete
  [in] | userData | - User specified data to be passed to the callback function
  [in] | flags | - Reserved for future use, must be 0

Returns
  hipSuccess, #hipErrorInvalidHandle, hipErrorNotSupported

See also
  hipStreamCreate, hipStreamCreateWithFlags, hipStreamQuery, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy, hipStreamCreateWithPriority

## ◆ hipStreamCreate()

hipError_t hipStreamCreate | ( | hipStream_t * | stream | ) |

Create an asynchronous stream.

Parameters

  [in,out] | stream | Valid pointer to hipStream_t. This function writes the memory with the newly created stream.

Returns
  hipSuccess, hipErrorInvalidValue

Create a new asynchronous stream. `stream` returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, applicaiton must call hipStreamDestroy.

Returns
  hipSuccess, hipErrorInvalidValue

See also
  hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

## ◆ hipStreamCreateWithFlags()

hipError_t hipStreamCreateWithFlags | ( | hipStream_t * | stream,
|  | unsigned int | flags
| ) |  |

Create an asynchronous stream.

Parameters

  [in,out] | stream | Pointer to new stream
  [in] | flags | to control stream creation.

Returns
  hipSuccess, hipErrorInvalidValue

Create a new asynchronous stream. `stream` returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, applicaiton must call hipStreamDestroy. Flags controls behavior of the stream. See hipStreamDefault, hipStreamNonBlocking.

See also
  hipStreamCreate, hipStreamCreateWithPriority, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

## ◆ hipStreamCreateWithPriority()

hipError_t hipStreamCreateWithPriority | ( | hipStream_t * | stream,
|  | unsigned int | flags,
|  | int | priority
| ) |  |

Create an asynchronous stream with the specified priority.

Parameters

  [in,out] | stream | Pointer to new stream
  [in] | flags | to control stream creation.
  [in] | priority | of the stream. Lower numbers represent higher priorities.

Returns
  hipSuccess, hipErrorInvalidValue

Create a new asynchronous stream with the specified priority. `stream` returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, applicaiton must call hipStreamDestroy. Flags controls behavior of the stream. See hipStreamDefault, hipStreamNonBlocking.

See also
  hipStreamCreate, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

## ◆ hipStreamDestroy()

hipError_t hipStreamDestroy | ( | hipStream_t | stream | ) |

Destroys the specified stream.

Parameters

  [in,out] | stream | Valid pointer to hipStream_t. This function writes the memory with the newly created stream.

Returns
  hipSuccess #hipErrorInvalidHandle

Destroys the specified stream.

If commands are still executing on the specified stream, some may complete execution before the queue is deleted.

The queue may be destroyed while some commands are still inflight, or may wait for all commands queued to the stream before destroying it.

See also
  hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamQuery, hipStreamWaitEvent, hipStreamSynchronize

## ◆ hipStreamGetFlags()

hipError_t hipStreamGetFlags | ( | hipStream_t | stream,
|  | unsigned int * | flags
| ) |  |

Return flags associated with this stream.

Parameters

  [in] | stream | stream to be queried
  [in,out] | flags | Pointer to an unsigned integer in which the stream's flags are returned

Returns
  hipSuccess, hipErrorInvalidValue, #hipErrorInvalidHandle
  hipSuccess hipErrorInvalidValue #hipErrorInvalidHandle

Return flags associated with this stream in *`flags`.

See also
  hipStreamCreateWithFlags

## ◆ hipStreamGetPriority()

hipError_t hipStreamGetPriority | ( | hipStream_t | stream,
|  | int * | priority
| ) |  |

Query the priority of a stream.

Parameters

  [in] | stream | stream to be queried
  [in,out] | priority | Pointer to an unsigned integer in which the stream's priority is returned

Returns
  hipSuccess, hipErrorInvalidValue, #hipErrorInvalidHandle
  hipSuccess hipErrorInvalidValue #hipErrorInvalidHandle

Query the priority of a stream. The priority is returned in in priority.

See also
  hipStreamCreateWithFlags

## ◆ hipStreamQuery()

hipError_t hipStreamQuery | ( | hipStream_t | stream | ) |

Return hipSuccess if all of the operations in the specified `stream` have completed, or hipErrorNotReady if not.

Parameters

  [in] | stream | stream to query

Returns
  hipSuccess, hipErrorNotReady, #hipErrorInvalidHandle

This is thread-safe and returns a snapshot of the current state of the queue. However, if other host threads are sending work to the stream, the status may change immediately after the function is called. It is typically used for debug.

See also
  hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamWaitEvent, hipStreamSynchronize, hipStreamDestroy

## ◆ hipStreamSynchronize()

hipError_t hipStreamSynchronize | ( | hipStream_t | stream | ) |

Wait for all commands in stream to complete.

Parameters

  [in] | stream | stream identifier.

Returns
  hipSuccess, #hipErrorInvalidHandle

This command is host-synchronous : the host will block until the specified stream is empty.

This command follows standard null-stream semantics. Specifically, specifying the null stream will cause the command to wait for other streams on the same device to complete all pending operations.

This command honors the hipDeviceLaunchBlocking flag, which controls whether the wait is active or blocking.

See also
  hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamWaitEvent, hipStreamDestroy

## ◆ hipStreamWaitEvent()

hipError_t hipStreamWaitEvent | ( | hipStream_t | stream,
|  | hipEvent_t | event,
|  | unsigned int | flags
| ) |  |

Make the specified compute stream wait for an event.

Parameters

  [in] | stream | stream to make wait.
  [in] | event | event to wait on
  [in] | flags | control operation [must be 0]

Returns
  hipSuccess, #hipErrorInvalidHandle

This function inserts a wait operation into the specified stream. All future work submitted to `stream` will wait until `event` reports completion before beginning execution.

This function only waits for commands in the current stream to complete. Notably,, this function does not impliciy wait for commands in the default stream to complete, even if the specified stream is created with hipStreamNonBlocking = 0.

See also
  hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamSynchronize, hipStreamDestroy

## ◆ hipStreamWaitValue32()

hipError_t hipStreamWaitValue32 | ( | hipStream_t | stream,
|  | void * | ptr,
|  | uint32_t | value,
|  | unsigned int | flags,
|  | uint32_t mask | __dparm0xFFFFFFFF
| ) |  |

Enqueues a wait command to the stream.[BETA].

Parameters

  [in] | stream | - Stream identifier
  [in] | ptr | - Pointer to memory object allocated using 'hipMallocSignalMemory' flag
  [in] | value | - Value to be used in compare operation
  [in] | flags | - Defines the compare operation, supported values are hipStreamWaitValueGte hipStreamWaitValueEq, hipStreamWaitValueAnd and hipStreamWaitValueNor
  [in] | mask | - Mask to be applied on value at memory before it is compared with value, default value is set to enable every bit

Returns
  hipSuccess, hipErrorInvalidValue

Enqueues a wait command to the stream, all operations enqueued on this stream after this, will not execute until the defined wait condition is true.

hipStreamWaitValueGte: waits until *ptr&mask >= value hipStreamWaitValueEq : waits until *ptr&mask == value hipStreamWaitValueAnd: waits until ((*ptr&mask) & value) != 0 hipStreamWaitValueNor: waits until ~((*ptr&mask) | (value&mask)) != 0

Note
  when using 'hipStreamWaitValueNor', mask is applied on both 'value' and '*ptr'.
  Support for hipStreamWaitValue32 can be queried using 'hipDeviceGetAttribute()' and 'hipDeviceAttributeCanUseStreamWaitValue' flag.

@beta This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

See also
  hipExtMallocWithFlags, hipFree, hipStreamWaitValue64, hipStreamWriteValue64, hipStreamWriteValue32, hipDeviceGetAttribute

## ◆ hipStreamWaitValue64()

hipError_t hipStreamWaitValue64 | ( | hipStream_t | stream,
|  | void * | ptr,
|  | uint64_t | value,
|  | unsigned int | flags,
|  | uint64_t mask | __dparm0xFFFFFFFFFFFFFFFF
| ) |  |

Enqueues a wait command to the stream.[BETA].

Parameters

  [in] | stream | - Stream identifier
  [in] | ptr | - Pointer to memory object allocated using 'hipMallocSignalMemory' flag
  [in] | value | - Value to be used in compare operation
  [in] | flags | - Defines the compare operation, supported values are hipStreamWaitValueGte hipStreamWaitValueEq, hipStreamWaitValueAnd and hipStreamWaitValueNor.
  [in] | mask | - Mask to be applied on value at memory before it is compared with value default value is set to enable every bit

Returns
  hipSuccess, hipErrorInvalidValue

Enqueues a wait command to the stream, all operations enqueued on this stream after this, will not execute until the defined wait condition is true.

hipStreamWaitValueGte: waits until *ptr&mask >= value hipStreamWaitValueEq : waits until *ptr&mask == value hipStreamWaitValueAnd: waits until ((*ptr&mask) & value) != 0 hipStreamWaitValueNor: waits until ~((*ptr&mask) | (value&mask)) != 0

Note
  when using 'hipStreamWaitValueNor', mask is applied on both 'value' and '*ptr'.
  Support for hipStreamWaitValue64 can be queried using 'hipDeviceGetAttribute()' and 'hipDeviceAttributeCanUseStreamWaitValue' flag.

@beta This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

See also
  hipExtMallocWithFlags, hipFree, hipStreamWaitValue32, hipStreamWriteValue64, hipStreamWriteValue32, hipDeviceGetAttribute

## ◆ hipStreamWriteValue32()

hipError_t hipStreamWriteValue32 | ( | hipStream_t | stream,
|  | void * | ptr,
|  | uint32_t | value,
|  | unsigned int | flags
| ) |  |

Enqueues a write command to the stream.[BETA].

Parameters

  [in] | stream | - Stream identifier
  [in] | ptr | - Pointer to a GPU accessible memory object
  [in] | value | - Value to be written
  [in] | flags | - reserved, ignored for now, will be used in future releases

Returns
  hipSuccess, hipErrorInvalidValue

Enqueues a write command to the stream, write operation is performed after all earlier commands on this stream have completed the execution.

@beta This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

See also
  hipExtMallocWithFlags, hipFree, hipStreamWriteValue32, hipStreamWaitValue32, hipStreamWaitValue64

## ◆ hipStreamWriteValue64()

hipError_t hipStreamWriteValue64 | ( | hipStream_t | stream,
|  | void * | ptr,
|  | uint64_t | value,
|  | unsigned int | flags
| ) |  |

Enqueues a write command to the stream.[BETA].

Parameters

  [in] | stream | - Stream identifier
  [in] | ptr | - Pointer to a GPU accessible memory object
  [in] | value | - Value to be written
  [in] | flags | - reserved, ignored for now, will be used in future releases

Returns
  hipSuccess, hipErrorInvalidValue

Enqueues a write command to the stream, write operation is performed after all earlier commands on this stream have completed the execution.

@beta This API is marked as beta, meaning, while this is feature complete, it is still open to changes and may have outstanding issues.

See also
  hipExtMallocWithFlags, hipFree, hipStreamWriteValue32, hipStreamWaitValue32, hipStreamWaitValue64