Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable Debug build for native code in spark-rapids-jni #1573

Closed
wants to merge 1 commit into from
Closed

Enable Debug build for native code in spark-rapids-jni #1573

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions build-libcudf.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
<arg value="${cudf.path}/cpp"/>
<arg value="-DBUILD_SHARED_LIBS=OFF"/>
<arg value="-DBUILD_TESTS=OFF"/>
<arg value="-DCMAKE_BUILD_TYPE=${LIBCUDF_BUILD_TYPE}"/>
<arg value="-DCMAKE_CUDA_ARCHITECTURES=${GPU_ARCHS}"/>
<arg value="-DCMAKE_INSTALL_PREFIX=${libcudf.install.path}"/>
<arg value="-DCUDA_STATIC_RUNTIME=ON"/>
Expand Down
7 changes: 7 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,9 @@
<RMM_LOGGING_LEVEL>OFF</RMM_LOGGING_LEVEL>
<SPARK_RAPIDS_JNI_CXX_FLAGS/>
<USE_GDS>OFF</USE_GDS>
<LIBCUDF_BUILD_TYPE>Release</LIBCUDF_BUILD_TYPE>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my previous experience, libcudf as a whole fails to build with BUILD_TYPE=Debug. I have been relying on selective Debug compiling https://github.com/rapidsai/cudf/blob/branch-23.12/CONTRIBUTING.md#device-debug-symbols. After a certain threshold depending on how many files are compiled with debugging symbols, the plugin slow to start or hangs for me.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering now, if I should try the CUDF approach here with spark-rapids-jni code and then maybe file another PR with a similar section in spark-rapids-jni CONTRIBUTING.md if that works with the way we build.

Might make a bit more sense then.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Closing this for now.

<BUILD_TYPE>${LIBCUDF_BUILD_TYPE}</BUILD_TYPE>
<CUDA_FLAGS_DEBUG>-g</CUDA_FLAGS_DEBUG>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intended to be overridden by the user for some use cases? I assume this is only used for Debug builds, and I'm a bit surprised this needs to be specified at all.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's a little weird, but the default flags only adds -g (which enables debugging in host code), you need -G to enable debugging in device code as well.

<BUILD_TESTS>OFF</BUILD_TESTS>
<BUILD_BENCHMARKS>OFF</BUILD_BENCHMARKS>
<BUILD_FAULTINJ>ON</BUILD_FAULTINJ>
Expand Down Expand Up @@ -393,6 +396,7 @@
<property name="RMM_LOGGING_LEVEL" value="${RMM_LOGGING_LEVEL}"/>
<property name="USE_GDS" value="${USE_GDS}"/>
<property name="BUILD_BENCHMARKS" value="${BUILD_BENCHMARKS}"/>
<property name="LIBCUDF_BUILD_TYPE" value="${LIBCUDF_BUILD_TYPE}"/>
<ant antfile="build-libcudf.xml" target="build-libcudf"/>
</target>
</configuration>
Expand All @@ -415,6 +419,7 @@
<arg value="-DBUILD_SHARED_LIBS=OFF"/>
<arg value="-DCUDA_STATIC_RUNTIME=ON"/>
<arg value="-DCUDF_JNI_LIBCUDF_STATIC=ON"/>
<arg value="-DCMAKE_BUILD_TYPE=${LIBCUDF_BUILD_TYPE}"/>
<arg value="-DGPU_ARCHS=${GPU_ARCHS}"/>
<arg value="-DCUDF_USE_PER_THREAD_DEFAULT_STREAM=${CUDF_USE_PER_THREAD_DEFAULT_STREAM}"/>
<arg value="-DRMM_LOGGING_LEVEL=${RMM_LOGGING_LEVEL}"/>
Expand Down Expand Up @@ -443,6 +448,8 @@
failonerror="true"
executable="cmake">
<arg value="${basedir}/src/main/cpp"/>
<arg value="-DCMAKE_BUILD_TYPE=${BUILD_TYPE}"/>
<arg value="-DCMAKE_CUDA_FLAGS_DEBUG=${CUDA_FLAGS_DEBUG}"/>
<arg value="-DGPU_ARCHS=${GPU_ARCHS}"/>
<arg value="-DCUDF_USE_PER_THREAD_DEFAULT_STREAM=${CUDF_USE_PER_THREAD_DEFAULT_STREAM}"/>
<arg value="-DRMM_LOGGING_LEVEL=${RMM_LOGGING_LEVEL}"/>
Expand Down