Conversation
The context cache is named for what it holds, so two builds carrying the same context meet on one name, and two builds carrying no context at all meet on the name of nothing, which is the common case on a machine running several builds at once. Both wrote the same tar, unpacked it to the same directory, and deleted that tar on the way out, so one build removed the file another was still writing to and the build that lost failed with the tar it had just created not existing. Stream into a file of this receive's own and unpack into a directory of its own, then move the finished tree under the shared name. A tree is already there when another receive got its first, and it holds the same content under the same name, so it stands and this copy is given up. The test fails without this, with the error the builds reported.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The context cache is named for what it holds, so two builds carrying the same context meet on one name, and two builds carrying no context at all meet on the name of nothing, which is the common case on a machine running several builds at once. Both wrote the same tar, unpacked it to the same directory, and deleted that tar on the way out, so one build removed the file another was still writing to, and the build that lost failed with the tar it had just created not existing.
Stream into a file of this receive's own and unpack into a directory of its own, then move the finished tree under the shared name. A tree is already there when another receive got its first, and it holds the same content under the same name, so it stands and this copy is given up.
Fixes #92.
Motivation and Context
The cache key is content, which is the right key: two builds carrying identical context should share the unpacked tree rather than each unpacking their own. What did not follow from that key was the working state. The tar being streamed and the directory being unpacked into were both named by the key as well, so the shared name covered not just the finished artifact but every intermediate step, and concurrent receives collided on all of them.
Builds with no context at all are the case that makes this easy to hit, since they all derive the same name.
Testing
TestReceiver_Receive_ConcurrentSameContentdrives eight concurrent receives of identical content and asserts each one gets a complete tree.mainwithout the fix, it fails with the error the builds reported:go build ./...,go vet ./..., andgo test ./pkg/...are clean.Type of Change