Skip to content

Commit

Permalink
Add flag ROC_SIGNAL_POOL_SIZE_PROFILE and update default value
Browse files Browse the repository at this point in the history
The change is two-fold:

- add a flag to configure the pool size used when profiling,
this allows to clearly configure the two values (profile vs. not-profile).

- change the size of the profile pool down from 4096 which was too large:
the kernel only provides 4094 events for DGPUs, *and* using two command-queues
in OpenCL results in the bug described here:
ROCm/ROCR-Runtime#186

The new default of 1000 has this rationale: it allows up to 4 queues to fit
within the 4094 events provided by the kernel (with a little margin).
  • Loading branch information
preda committed Mar 25, 2024
1 parent 0385836 commit d945d9c
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 4 deletions.
7 changes: 3 additions & 4 deletions rocclr/device/rocm/rocvirtual.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -355,10 +355,9 @@ VirtualGPU::HwQueueTracker::~HwQueueTracker() {

// ================================================================================================
bool VirtualGPU::HwQueueTracker::Create() {
uint kSignalListSize = ROC_SIGNAL_POOL_SIZE;
if (activity_prof::IsEnabled(OP_ID_DISPATCH) || gpu_.profiling_) {
kSignalListSize = !flagIsDefault(ROC_SIGNAL_POOL_SIZE) ? ROC_SIGNAL_POOL_SIZE : 4 * Ki;
}
bool isProfile = activity_prof::IsEnabled(OP_ID_DISPATCH) || gpu_.profiling_;
uint kSignalListSize = isProfile ? ROC_SIGNAL_POOL_SIZE_PROFILE : ROC_SIGNAL_POOL_SIZE;

signal_list_.resize(kSignalListSize);

hsa_agent_t agent = gpu_.gpu_device();
Expand Down
2 changes: 2 additions & 0 deletions rocclr/utils/flags.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,8 @@ release(uint, ROC_AQL_QUEUE_SIZE, 16384, \
"AQL queue size in AQL packets") \
release(uint, ROC_SIGNAL_POOL_SIZE, 32, \
"Initial size of HSA signal pool") \
release(uint, ROC_SIGNAL_POOL_SIZE_PROFILE, 1000, \
"Initial size of HSA signal pool") \
release(uint, DEBUG_CLR_LIMIT_BLIT_WG, 16, \
"Limit the number of workgroups in blit operations") \
release(bool, ROC_SKIP_KERNEL_ARG_COPY, false, \
Expand Down

0 comments on commit d945d9c

Please sign in to comment.