Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inceptionv3 c++ example inference build trt engine failed; #331

Open
Weili17 opened this issue Mar 2, 2023 · 1 comment
Open

inceptionv3 c++ example inference build trt engine failed; #331

Weili17 opened this issue Mar 2, 2023 · 1 comment

Comments

@Weili17
Copy link

Weili17 commented Mar 2, 2023

**environment:**docker nvcr.io/nvidia/tensorflow:22.06-tf2-py3
Reason: TF-TRT Warning: Engine creation for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 failed. The native segment will be used instead. Reason: NOT_FOUND: No converter for op _FusedBatchNormEx

@Weili17
Copy link
Author

Weili17 commented Mar 2, 2023

environment:docker nvcr.io/nvidia/tensorflow:22.06-tf2-py3
log:
2023-03-02 05:03:37.809625: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0.
2023-03-02 05:03:37.934203: I tensorflow/core/platform/cpu_feature_guard.cc:194] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE3 SSE4.1 SSE4.2 AVX
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-03-02 05:03:39.032139: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 11377 MB memory: -> device: 0, name: Tesla T4, pci bus id: 0000:af:00.0, compute capability: 7.5
2023-03-02 05:03:39.032737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13807 MB memory: -> device: 1, name: Tesla T4, pci bus id: 0000:d8:00.0, compute capability: 7.5
2023-03-02 05:03:39.278224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 11377 MB memory: -> device: 0, name: Tesla T4, pci bus id: 0000:af:00.0, compute capability: 7.5
2023-03-02 05:03:39.278473: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13807 MB memory: -> device: 1, name: Tesla T4, pci bus id: 0000:d8:00.0, compute capability: 7.5
2023-03-02 05:03:42.943295: I tensorflow/compiler/tf2tensorrt/common/utils.cc:104] Linked TensorRT version: 8.2.5
2023-03-02 05:03:42.984223: I tensorflow/compiler/tf2tensorrt/common/utils.cc:106] Loaded TensorRT version: 8.2.5
2023-03-02 05:03:44.790395: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:1055] TF-TRT Warning: Engine creation for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 failed. The native segment will be used instead. Reason: NOT_FOUND: No converter for op _FusedBatchNormEx
2023-03-02 05:03:44.790482: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:888] TF-TRT Warning: Engine retrieval for input shapes: [[1,224,224,3]] failed. Running native segment for PartitionedCall/PartitionedCall/TRTEngineOp_000_000
2023-03-02 05:03:46.008720: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8401
2023-03-02 05:03:47.439435: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 11377 MB memory: -> device: 0, name: Tesla T4, pci bus id: 0000:af:00.0, compute capability: 7.5
2023-03-02 05:03:47.439682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13807 MB memory: -> device: 1, name: Tesla T4, pci bus id: 0000:d8:00.0, compute capability: 7.5
2023-03-02 05:03:47.444964: I tensorflow/examples/image_classification/main.cc:276] malamute (250): 0.575496
2023-03-02 05:03:47.444990: I tensorflow/examples/image_classification/main.cc:276] Saint Bernard (248): 0.399285
2023-03-02 05:03:47.444998: I tensorflow/examples/image_classification/main.cc:276] Eskimo dog (249): 0.0228339
2023-03-02 05:03:47.445006: I tensorflow/examples/image_classification/main.cc:276] Ibizan hound (174): 0.00127912
2023-03-02 05:03:47.445013: I tensorflow/examples/image_classification/main.cc:276] Mexican hairless (269): 0.000520922
2023-03-02 05:03:47.445205: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:888] TF-TRT Warning: Engine retrieval for input shapes: [[1,224,224,3]] failed. Running native segment for PartitionedCall/PartitionedCall/TRTEngineOp_000_000
2023-03-02 05:03:47.538143: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:888] TF-TRT Warning: Engine retrieval for input shapes: [[1,224,224,3]] failed. Running native segment for PartitionedCall/PartitionedCall/TRTEngineOp_000_000
2023-03-02 05:03:47.553149: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:888] TF-TRT Warning: Engine retrieval for input shapes: [[1,224,224,3]] failed. Running native segment for PartitionedCall/PartitionedCall/TRTEngineOp_000_000
2023-03-02 05:03:47.558721: W tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc:888] TF-TRT Warning: Engine retrieval for input shapes: [[1,224,224,3]] failed. Running native segment for PartitionedCall/PartitionedCall/TRTEngineOp_000_000
2023-03-02 05:03:47.857838: I tensorflow/examples/image_classification/main.cc:423] Step 0: 6 ms
2023-03-02 05:03:48.168601: I tensorflow/examples/image_classification/main.cc:423] Step 50: 6 ms
2023-03-02 05:03:48.492328: I tensorflow/examples/image_classification/main.cc:423] Step 100: 6.0198 ms
2023-03-02 05:03:48.805529: I tensorflow/examples/image_classification/main.cc:423] Step 150: 6.01325 ms
2023-03-02 05:03:49.118205: I tensorflow/examples/image_classification/main.cc:423] Step 200: 6.00995 ms
2023-03-02 05:03:49.436369: I tensorflow/examples/image_classification/main.cc:423] Step 250: 6.01195 ms
2023-03-02 05:03:49.751933: I tensorflow/examples/image_classification/main.cc:423] Step 300: 6.01329 ms
2023-03-02 05:03:50.065457: I tensorflow/examples/image_classification/main.cc:423] Step 350: 6.0114 ms
2023-03-02 05:03:50.381459: I tensorflow/examples/image_classification/main.cc:423] Step 400: 6.01247 ms
2023-03-02 05:03:50.698780: I tensorflow/examples/image_classification/main.cc:423] Step 450: 6.01109 ms
2023-03-02 05:03:51.011541: I tensorflow/examples/image_classification/main.cc:423] Step 500: 6.00998 ms
2023-03-02 05:03:51.327086: I tensorflow/examples/image_classification/main.cc:423] Step 550: 6.00907 ms
2023-03-02 05:03:51.643465: I tensorflow/examples/image_classification/main.cc:423] Step 600: 6.00832 ms
2023-03-02 05:03:51.961735: I tensorflow/examples/image_classification/main.cc:423] Step 650: 6.01229 ms
2023-03-02 05:03:52.281288: I tensorflow/examples/image_classification/main.cc:423] Step 700: 6.01712 ms
2023-03-02 05:03:52.601280: I tensorflow/examples/image_classification/main.cc:423] Step 750: 6.01598 ms
2023-03-02 05:03:52.922119: I tensorflow/examples/image_classification/main.cc:423] Step 800: 6.01998 ms
2023-03-02 05:03:53.240485: I tensorflow/examples/image_classification/main.cc:423] Step 850: 6.01998 ms
2023-03-02 05:03:53.558849: I tensorflow/examples/image_classification/main.cc:423] Step 900: 6.01887 ms
2023-03-02 05:03:53.875391: I tensorflow/examples/image_classification/main.cc:423] Step 950: 6.01788 ms
2023-03-02 05:03:54.186272: I tensorflow/examples/image_classification/main.cc:426] Throughput: 166.196 images/s

@Weili17 Weili17 changed the title inceptionv3 c++ example build trt engine failed; inceptionv3 c++ example build trt engine failed;Reason:TF-TRT Warning: Engine creation for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 failed. The native segment will be used instead. Reason: NOT_FOUND: No converter for op _FusedBatchNormEx Mar 2, 2023
@Weili17 Weili17 changed the title inceptionv3 c++ example build trt engine failed;Reason:TF-TRT Warning: Engine creation for PartitionedCall/PartitionedCall/TRTEngineOp_000_000 failed. The native segment will be used instead. Reason: NOT_FOUND: No converter for op _FusedBatchNormEx inceptionv3 c++ example inference build trt engine failed; Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant