Skip to content

Reap the log aggregator's tail processes - #350

Merged
Sayan- merged 2 commits into
mainfrom
hypeship/wrapper-tail-wait
Aug 25, 2026
Merged

Reap the log aggregator's tail processes#350
Sayan- merged 2 commits into
mainfrom
hypeship/wrapper-tail-wait

Conversation

@Sayan-

@Sayan- Sayan- commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

tailFile starts tail -n +1 -F <path> and reads its stdout, but never calls Wait. When that tail exits, nothing collects its status, so it stays a zombie.

The wrapper is pid 1 in both the container (ENTRYPOINT) and the unikernel (Kraftfile cmd), so there is no other init to clean up after it. On a long-lived instance whose services restart repeatedly, these accumulate for the life of the instance and each one holds a pid slot.

A clean scan ends exactly when tail closes its stdout, which is when it has exited, so that is where the wait belongs. A scan that ends on a read error instead is a different case: bufio.Scanner also stops on ErrTooLong, and there tail is still alive, so waiting would park the goroutine forever. Kill it first in that path.

Testing

go build, go vet, gofmt and go test -race pass on server/cmd/wrapper.

No test added here. tail -F does not exit on its own and tailFile keeps no handle to the process, so there is no seam to drive it from a test without reshaping the function. The reaper PR stacked on this one carries tests that cover the same defect class directly.


Note

Low Risk
Localized change to log tailing cleanup in the wrapper; no auth, data, or API surface impact.

Overview
tailFile in the wrapper supervisord log aggregator previously started tail -F and read lines but never Wait() on the child. Because the wrapper runs as pid 1 in container and unikernel, exited tails became zombies that could pile up when services restart.

After the scan loop, the change always **Wait()**s to reap the process. If scanner.Err() is set (e.g. a line exceeds the 1MB scanner limit while tail is still running), it **Kill()**s tail first so Wait() does not block forever.

Reviewed by Cursor Bugbot for commit 20a0f55. Bugbot is set up for automated code reviews on this repo. Configure here.

@Sayan-
Sayan- marked this pull request as ready for review August 24, 2026 20:54
tailFile started tail -F and never waited on it. The wrapper runs as pid 1
in both the container and the unikernel, so every exited tail stayed a
zombie for the life of the instance.

A scan that ends on a read error rather than EOF leaves tail alive, so kill
it before waiting instead of parking the goroutine forever.
@Sayan-
Sayan- force-pushed the hypeship/wrapper-tail-wait branch from 47c35bf to cf73a4d Compare August 24, 2026 21:00
@Sayan-
Sayan- marked this pull request as draft August 24, 2026 21:01
@Sayan-
Sayan- marked this pull request as ready for review August 24, 2026 21:10
@Sayan-
Sayan- marked this pull request as draft August 24, 2026 21:24
@Sayan-
Sayan- marked this pull request as ready for review August 24, 2026 21:27

@masnwilliams masnwilliams left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed — looks good. the cleanup follows os/exec's pipe lifecycle correctly, handles scanner failure without blocking, and ensures the tail process is always reaped.

@Sayan-
Sayan- merged commit 818d0f3 into main Aug 25, 2026
11 checks passed
@Sayan-
Sayan- deleted the hypeship/wrapper-tail-wait branch August 25, 2026 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants