Yolov5+tensorrt+vs2019+windows部署

Yolov5+tensorrt+vs2019+windows部署

1.Tensorrt介绍

TensorRT是nvidia家的一款高性能深度学习推理SDK。此SDK包含深度学习推理优化器和运行环境，可为深度学习推理应用提供低延迟和高吞吐量。在推理过程中，基于TensorRT的应用程序比仅仅使用CPU作为平台的应用程序要快40倍。

 2.基于Yolov5/Yolov7/Yolov8的Tensort 部署

关键点：Tensorrt，CUDA，CUDNN版本必须对应，否则很有可能报错！！！

这里按照Yolov5进行展示

本文使用的Tensorrt版本、CUDA、CUDNN版本：

TensorRT-8.0.1.6.Windows10.x86_64.cuda-11.3.cudnn8.2

源码下载Github：GitHub - wang-xinyu/tensorrtx: Implementation of popular deep learning networks with TensorRT network definition API

 2.1使用Cmake进行编译

提供

cmake_minimum_required(VERSION 3.10)project(yolov5)add_definitions(-std=c++11)
add_definitions(-DAPI_EXPORTS)
option(CUDA_USE_STATIC_CUDA_RUNTIME OFF)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_BUILD_TYPE Debug)# TODO(Call for PR): make cmake compatible with Windows
set(CMAKE_CUDA_COMPILER /usr/local/cuda/bin/nvcc)
enable_language(CUDA)# include and link dirs of cuda and tensorrt, you need adapt them if yours are different
# cuda
include_directories(C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.2/include/)
link_directories(C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.2/lib/x64/)
# tensorrt
# TODO(Call for PR): make TRT path configurable from command line
include_directories(D:/TensorRT-8.2.3.0/include/)
link_directories(D:/TensorRT-8.2.3.0/lib/)include_directories(${PROJECT_SOURCE_DIR}/src/)
include_directories(${PROJECT_SOURCE_DIR}/plugin/)
file(GLOB_RECURSE SRCS ${PROJECT_SOURCE_DIR}/src/*.cpp ${PROJECT_SOURCE_DIR}/src/*.cu)
file(GLOB_RECURSE PLUGIN_SRCS ${PROJECT_SOURCE_DIR}/plugin/*.cu)add_library(myplugins SHARED ${PLUGIN_SRCS})
target_link_libraries(myplugins nvinfer cudart)find_package(OpenCV)
include_directories(${OpenCV_INCLUDE_DIRS})add_executable(yolov5_det yolov5_det.cpp ${SRCS})
target_link_libraries(yolov5_det nvinfer)
target_link_libraries(yolov5_det cudart)
target_link_libraries(yolov5_det myplugins)
target_link_libraries(yolov5_det ${OpenCV_LIBS})add_executable(yolov5_cls yolov5_cls.cpp ${SRCS})
target_link_libraries(yolov5_cls nvinfer)
target_link_libraries(yolov5_cls cudart)
target_link_libraries(yolov5_cls myplugins)
target_link_libraries(yolov5_cls ${OpenCV_LIBS})add_executable(yolov5_seg yolov5_seg.cpp ${SRCS})
target_link_libraries(yolov5_seg nvinfer)
target_link_libraries(yolov5_seg cudart)
target_link_libraries(yolov5_seg myplugins)
target_link_libraries(yolov5_seg ${OpenCV_LIBS})

2.2  编译通过后打开VS工程，可能存在的报错

2.2.1  无法打开包括文件: “dirent.h”: No such file or directory

解决方案：dirent/dirent.h at master · tronkko/dirent · GitHub

 2.2.2  identifier 'kIgnoreThresh' is undefined in device code ，located in config.h, line 31

// The bboxes whose confidence is lower than kIgnoreThresh will be ignored in yololayer plugin.
const static float kIgnoreThresh = 0.1f;

 更改为：

constexpr static float kIgnoreThresh = 0.1f;

2.2.3  std::min/std::max conflict issue （STL中的min/max在windows上的冲突问题）

解决方案：可以在头文件中添加#define NOMINMAX，解决问题

2.3  编译通过后，如何生成.wts和.engine

运行：（gen_wts.py需要放在v5代码下才能运行成功）

python gen_wts.py  -w yolov5s.pt -o  yolov5s.wts  -t detect

生成.engine，在Release路径下

 -s yolov5s.ine s

2.4 预测图片

 -ine  images/

2.5 下一步封装成dll，方便调用。

源码下载地址：