versions of the libraries

[ghost]

The requirements file does not include the versions of the libraries.
I have 3060Ti and having some problem with creating the .so files.
can someone write the versions of the libraries?

[Owen-Liuyuxuan]

Could you provide and check the nvidia-driver version, the cuda version, and the cuda version that pytorch is compiled on (torch.version.cuda).

[ghost]

driver version:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.103.01 Driver Version: 470.103.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:02:00.0 On | N/A |
| 30% 38C P5 38W / 200W | 288MiB / 7973MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1184 G /usr/lib/xorg/Xorg 102MiB |
| 0 N/A N/A 1476 G /usr/bin/gnome-shell 86MiB |
| 0 N/A N/A 2367 G /usr/lib/firefox/firefox 96MiB |
+-----------------------------------------------------------------------------+

Cuda Version:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Fri_Dec_17_18:16:03_PST_2021
Cuda compilation tools, release 11.6, V11.6.55
Build cuda_11.6.r11.6/compiler.30794723_0

Torch.cuda.version
Python 3.8.0 (default, Nov 6 2019, 21:49:08)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import torch
torch.version.cuda
'11.3'

When I run make.sh in the /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/ I get the error at the below.

bash ./make.sh 
running build_ext
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/utils/cpp_extension.py:782: UserWarning: The detected CUDA version (11.6) has a minor version mismatch with the version that was used to compile PyTorch (11.3). Most likely this shouldn't be a problem.
  warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
building '..deform_conv_ext' extension
Emitting ninja build file /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/3] /usr/local/cuda/bin/nvcc  -DWITH_CUDA -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/TH -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/safa/anaconda3/envs/visionEnv/include/python3.8 -c -c /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/src/cuda/deform_conv_cuda_kernel.cu -o /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/build/temp.linux-x86_64-3.8/src/cuda/deform_conv_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=deform_conv_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++14
FAILED: /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/build/temp.linux-x86_64-3.8/src/cuda/deform_conv_cuda_kernel.o 
/usr/local/cuda/bin/nvcc  -DWITH_CUDA -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/TH -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/safa/anaconda3/envs/visionEnv/include/python3.8 -c -c /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/src/cuda/deform_conv_cuda_kernel.cu -o /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/build/temp.linux-x86_64-3.8/src/cuda/deform_conv_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=deform_conv_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++14
cc1plus: fatal error: cuda_runtime.h: No such file or directory
compilation terminated.
[2/3] c++ -MMD -MF /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/build/temp.linux-x86_64-3.8/src/cuda/deform_conv_cuda.o.d -pthread -B /home/safa/anaconda3/envs/visionEnv/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/cuda/include -fPIC -DWITH_CUDA -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/TH -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/safa/anaconda3/envs/visionEnv/include/python3.8 -c -c /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/src/cuda/deform_conv_cuda.cpp -o /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/build/temp.linux-x86_64-3.8/src/cuda/deform_conv_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=deform_conv_ext -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
[3/3] c++ -MMD -MF /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/build/temp.linux-x86_64-3.8/src/deform_conv_ext.o.d -pthread -B /home/safa/anaconda3/envs/visionEnv/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/cuda/include -fPIC -DWITH_CUDA -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/TH -I/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/safa/anaconda3/envs/visionEnv/include/python3.8 -c -c /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/src/deform_conv_ext.cpp -o /home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/build/temp.linux-x86_64-3.8/src/deform_conv_ext.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=deform_conv_ext -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1717, in _run_ninja_build
    subprocess.run(
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

**The above exception was the direct cause of the following exception:**

Traceback (most recent call last):
  File "setup.py", line 172, in <module>
    setup(
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 735, in build_extensions
    build_ext.build_extensions(self)
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
    _build_ext.build_extension(self, ext)
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
    objects = self.compiler.compile(sources,
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 556, in unix_wrap_ninja_compile
    _write_ninja_file_and_compile_objects(
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1399, in _write_ninja_file_and_compile_objects
    _run_ninja_build(
  File "/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1733, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension

When I uninstall ninja and run make.sh again it compiles without error.

bash ./make.sh 
running build_ext
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/utils/cpp_extension.py:381: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
  warnings.warn(msg.format('we could not find ninja.'))
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/torch/utils/cpp_extension.py:782: UserWarning: The detected CUDA version (11.6) has a minor version mismatch with the version that was used to compile PyTorch (11.3). Most likely this shouldn't be a problem.
  warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
copying build/lib.linux-x86_64-3.8/deform_conv_ext.cpython-38-x86_64-linux-gnu.so -> 

After this compilation when I run the train.sh file I got this error:

`./train.sh ../config/confSafa.py 0 safaTEst
Nnumber of GPUs being 1, will directly launch:
	 python3 train --experiment_name=safaTEst
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/numba/cuda/decorators.py:110: NumbaDeprecationWarning: Eager compilation of device functions is deprecated (this occurs when a signature is provided)
  warn(NumbaDeprecationWarning(msg))
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/numba/cuda/decorators.py:110: NumbaDeprecationWarning: Eager compilation of device functions is deprecated (this occurs when a signature is provided)
  warn(NumbaDeprecationWarning(msg))
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/numba/cuda/decorators.py:110: NumbaDeprecationWarning: Eager compilation of device functions is deprecated (this occurs when a signature is provided)
  warn(NumbaDeprecationWarning(msg))
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/numba/cuda/decorators.py:110: NumbaDeprecationWarning: Eager compilation of device functions is deprecated (this occurs when a signature is provided)
  warn(NumbaDeprecationWarning(msg))
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/numba/cuda/decorators.py:110: NumbaDeprecationWarning: Eager compilation of device functions is deprecated (this occurs when a signature is provided)
  warn(NumbaDeprecationWarning(msg))
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/numba/cuda/decorators.py:110: NumbaDeprecationWarning: Eager compilation of device functions is deprecated (this occurs when a signature is provided)
  warn(NumbaDeprecationWarning(msg))
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/numba/cuda/decorators.py:110: NumbaDeprecationWarning: Eager compilation of device functions is deprecated (this occurs when a signature is provided)
  warn(NumbaDeprecationWarning(msg))
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/numba/cuda/decorators.py:110: NumbaDeprecationWarning: Eager compilation of device functions is deprecated (this occurs when a signature is provided)
  warn(NumbaDeprecationWarning(msg))
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/numba/cuda/decorators.py:110: NumbaDeprecationWarning: Eager compilation of device functions is deprecated (this occurs when a signature is provided)
  warn(NumbaDeprecationWarning(msg))
/home/safa/anaconda3/envs/visionEnv/lib/python3.8/site-packages/numba/cuda/decorators.py:110: NumbaDeprecationWarning: Eager compilation of device functions is deprecated (this occurs when a signature is provided)
  warn(NumbaDeprecationWarning(msg))
Traceback (most recent call last):
  File "../scripts/train.py", line 17, in <module>
    from visualDet3D.networks.utils.registry import DETECTOR_DICT, DATASET_DICT, PIPELINE_DICT
  File "/home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/__init__.py", line 1, in <module>
    from .pipelines import *
  File "/home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/pipelines/__init__.py", line 3, in <module>
    from .evaluators import evaluate_kitti_obj
  File "/home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/pipelines/evaluators.py", line 16, in <module>
    from visualDet3D.data.kitti.utils import write_result_to_file
  File "/home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/data/__init__.py", line 1, in <module>
    from .kitti.dataset import mono_dataset, depth_mono_dataset
  File "/home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/data/kitti/__init__.py", line 1, in <module>
    from .dataset import KittiMonoDataset, KittiMonoTestDataset
  File "/home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/data/kitti/dataset/__init__.py", line 4, in <module>
    from .KM3D_dataset import KittiRTM3DDataset
  File "/home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/data/kitti/dataset/KM3D_dataset.py", line 19, in <module>
    from visualDet3D.networks.utils.rtm3d_utils import gen_hm_radius, project_to_image, gaussian_radius
  File "/home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/utils/rtm3d_utils.py", line 6, in <module>
    from visualDet3D.networks.lib.ops.iou3d.iou3d import boxes_iou3d_gpu
  File "/home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/__init__.py", line 1, in <module>
    from .dcn.deform_conv import ModulatedDeformConvPack, DeformConvPack
  File "/home/safa/PycharmProjects/visualDet3D-1.1/visualDet3D/networks/lib/ops/dcn/deform_conv.py", line 50, in <module>
    from . import deform_conv_ext
ImportError: libcudart.so.10.2: cannot open shared object file: No such file or directory

[Owen-Liuyuxuan]

It seems that your environment contains multiple cuda version. Please clean it up and it is better to keep version number aligned in nvcc / torch.cuda / nvidia-driver.