Skip to content

ci: Ship trtexec in the QA image when built on an inference-runtime base #9014

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
devin-ai-integration wants to merge 6 commits into
base: main
Choose a base branch
from
+8 −0
Open

ci: Ship trtexec in the QA image when built on an inference-runtime base #9014

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
66ea70e
ci: Ship trtexec in the QA image when built on an inference-runtime base
nv-rinig Oct 12, 2026
5d6f4d6
ci: Also ship TensorRT builder resource libraries for trtexec
nv-rinig Oct 12, 2026
0f4e58b
ci: Copy trtexec and builder resources directly from cibase
nv-rinig Oct 12, 2026
960aa53
ci: Install trtexec builder resources next to libnvinfer
nv-rinig Oct 12, 2026
d267df1
ci: Trim trtexec install to the minimal copy
nv-rinig Oct 13, 2026
0a03881
ci: Rely on the base image's trtexec symlink
nv-rinig Oct 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Dockerfile.QA
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,14 @@ ENV GOPATH /root/go
ENV PATH $PATH:/usr/local/go/bin:$GOPATH/bin
RUN go install google.golang.org/protobuf/cmd/protoc-gen-go@latest && \
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

# The CUDA inference-runtime base ships the TensorRT runtime libraries and the
# /usr/src/tensorrt/bin/trtexec symlink, but not trtexec or the builder resource
# libraries it dlopens from libnvinfer's directory to build engines.
COPY --from=cibase /usr/bin/trtexec /usr/bin/trtexec
COPY --from=cibase /usr/lib/*-linux-gnu/libnvinfer_builder_resource*.so.*.*.* /tmp/trt_builder_resource/
RUN mv /tmp/trt_builder_resource/* /usr/lib/*-linux-gnu/

Comment on lines +382 to +389

@mc-nv mc-nv Oct 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really think that we need to go with small changes.
Devin overcomplicating the solution IMO.

Suggested change
# The CUDA inference-runtime base ships the TensorRT runtime libraries but not
# trtexec or the builder resource libraries it dlopens to build engines. The
# builder resources must sit next to libnvinfer. QA tests call trtexec through
# the devel image's /usr/src/tensorrt/bin symlink.
COPY --from=cibase /usr/bin/trtexec /usr/bin/trtexec
COPY --from=cibase /usr/lib/*-linux-gnu/libnvinfer_builder_resource*.so.*.*.* /tmp/trt_builder_resource/
RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
TRT_LIB_DIR=/usr/lib/aarch64-linux-gnu; \
else \
TRT_LIB_DIR=/usr/lib/x86_64-linux-gnu; \
fi && \
mv /tmp/trt_builder_resource/* ${TRT_LIB_DIR}/ && \
rm -rf /tmp/trt_builder_resource && \
ls ${TRT_LIB_DIR}/libnvinfer.so.* > /dev/null && \
mkdir -p /usr/src/tensorrt/bin && \
ln -sf /usr/bin/trtexec /usr/src/tensorrt/bin/trtexec && \
ldconfig && \
! ldd /usr/bin/trtexec | grep 'not found'
# The CUDA inference-runtime base ships the TensorRT runtime libraries but not
# trtexec or the builder resource libraries it dlopens to build engines.
COPY --from=cibase /usr/bin/trtexec /usr/bin/trtexec

@devin-ai-integration devin-ai-integration Bot Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed it should be smaller. d267df1 trims it to two COPYs and a three-command RUN. The TARGETPLATFORM branch, the /tmp checks and the ldd check are gone. The mv uses /usr/lib/*-linux-gnu/, so it works on both x86 and sbsa.

The single COPY line on its own isn't enough, though. CI has already shown that:

  • Binary only (rev 1, pipeline 73321996, job 483227906): trtexec runs but can't create a builder: Unable to load library: libnvinfer_builder_resource_sm80.so.11.4.0. Those libraries aren't in inference-runtime.
  • Builder resources outside libnvinfer's directory (rev 3, pipeline 73373859, job 483674704): same error, even with an ld.so.conf.d entry. They have to be next to libnvinfer, which is why the mv is needed (a COPY destination can't take the arch-specific directory).
  • The tests call /usr/src/tensorrt/bin/trtexec (L0_device_memory_tracker, L0_memory_growth, L0_perf_*). Without the symlink they fail with the original No such file or directory. The alternative would be changing those test.sh files to call /usr/bin/trtexec directly. Let me know if you'd prefer that.

d267df1 is being validated in pipeline 73388065 (L0 DMT only).

@mc-nv mc-nv Oct 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a broken link that should recover after the above change

$ docker run --rm -it gitlab-master.nvidia.com:5005/dl/dgx/cuda:13.4-inference-runtime-ubuntu24.04--26.10 ls 
lrwxrwxrwx 1 root root 16 Oct  9 07:53 /usr/src/tensorrt/bin/trtexec -> /usr/bin/trtexec

@devin-ai-integration devin-ai-integration Bot Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that makes things simpler. Since the base image already has the symlink, copying /usr/bin/trtexec back fixes the path, and my mkdir/ln -s step was redundant. It would also have broken the build, because ln -s without -f can't replace an existing link. 0a03881 drops it. What remains is the trtexec COPY plus the builder-resource COPY and a one-line mv that puts the builder resources next to libnvinfer:

COPY --from=cibase /usr/bin/trtexec /usr/bin/trtexec
COPY --from=cibase /usr/lib/*-linux-gnu/libnvinfer_builder_resource*.so.*.*.* /tmp/trt_builder_resource/
RUN mv /tmp/trt_builder_resource/* /usr/lib/*-linux-gnu/

The devel builds in pipeline 73388065 hadn't started when I pushed, so that pipeline validates 0a03881 (L0 DMT only).

@devin-ai-integration devin-ai-integration Bot Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0a03881 passed: all 11 jobs in pipeline 73388065 succeeded, including L0_device_memory_tracker--base (job 483799087). The py3-devel log shows the two COPYs and the single mv, and trtexec built all three engines (vgg19, resnet50, resnet152). That covers the QA-image approach on A100; the DGX-Spark variants haven't run yet. Whether to keep this or move to model generation is your call (see the top-level thread).

# CI expects tests in /opt/tritonserver/qa. The triton-server (1000)
# user should own all artifacts in case CI is run using triton-server
# user.
Expand Down
Loading