[bot-detection] Add custom JS fingerprinting capabilities - #4657
Conversation
buixor
commented
Sep 4, 2026
buixor
commented
- add challenge js support to allow "custom" detection shipped via the hub.
- said detection run on a configurable budget (defaults to 500ms)
- ensure logs contain custom detections and scores
|
@buixor: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.
DetailsI am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
|
@buixor: There are no area labels on this PR. You can add as many areas as you see fit.
DetailsI am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
There was a problem hiding this comment.
🟡 Changes recommended
The custom-hook timeout implementation can allow timed-out hooks to keep mutating the fingerprint concurrently with signing, risking signature/payload mismatches and undermining the budget guarantee.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds hub-distributed “custom JS” fingerprinting hooks to the appsec challenge flow, including a bounded execution budget and server-side persistence of browser-reported custom detection outputs so rules/logs can attribute scores to those detections.
Changes:
- Add support for loading and serving hub-managed
challenge/custom.js, plus a configurablecustom_js_timeoutbudget injected into the challenge page. - Extend the challenge fingerprint model/proto to carry a bounded
custommap (typed union) and trim it to keep sealed cookies within size limits. - Improve appsec logging by attaching request score totals + breakdowns to challenge submit hook logs.
File summaries
| File | Description |
|---|---|
| pkg/exprhelpers/helpers.go | Adds challenge-js data type and skips expr indexing for those files. |
| pkg/appsec/waf_helpers.go | Adds withRequestScore helper and includes score fields in challenge submit hook logs. |
| pkg/appsec/score_log_test.go | Tests withRequestScore behavior. |
| pkg/appsec/challenge/testdata/realistic_fingerprint.json | Adds realistic fixture used for cookie headroom/size assertions. |
| pkg/appsec/challenge/pb/fingerprint.proto | Adds custom map and custom value union types to fingerprint proto. |
| pkg/appsec/challenge/pb/fingerprint.pb.go | Regenerated protobuf output for the new custom fields/types. |
| pkg/appsec/challenge/js/README.md | Documents where hub-shipped detection scripts live vs build pipeline. |
| pkg/appsec/challenge/js/obfuscate/obfuscate.js | Reserves new custom-detection sentinel during obfuscation. |
| pkg/appsec/challenge/js/challenge_code.js | Updates generated/obfuscated challenge bundle to run custom hooks with a budget. |
| pkg/appsec/challenge/fingerprint.go | Adds Custom + CustomDropped fields and sanitization on JSON unmarshal. |
| pkg/appsec/challenge/fingerprint_proto.go | Adds custom map proto conversions; stops sealing URL into cookies. |
| pkg/appsec/challenge/fingerprint_helpers.go | Adds HasCustom and CustomKeys helpers for rules/logging. |
| pkg/appsec/challenge/fingerprint_custom.go | Implements custom value decoding, caps, sanitization, and cookie-budget trimming. |
| pkg/appsec/challenge/fingerprint_custom_test.go | Unit tests for custom value decode/marshal, caps, sanitization, and expr access. |
| pkg/appsec/challenge/fingerprint_custom_proto_test.go | Tests custom proto round-trip + seal-time trimming/headroom invariants. |
| pkg/appsec/challenge/customjs_timeout_test.go | Tests default/configured timeout behavior and JS injection contract. |
| pkg/appsec/challenge/customjs_test.go | Tests sentinel survival, script-tag behavior, and end-to-end submission survival. |
| pkg/appsec/challenge/crypto.go | Trims custom map before sealing to avoid cookie-size failures. |
| pkg/appsec/challenge/config.go | Adds custom_js_timeout config plumbed into runtime options. |
| pkg/appsec/challenge/challenge.js | Adds custom hook runner to mutate/enrich fingerprint before signing. |
| pkg/appsec/challenge/challenge.html.tmpl | Injects _cjsT and conditionally includes custom.js script tag. |
| pkg/appsec/challenge/challenge.go | Serves custom.js, injects timeout/versioning, logs truncations, adds config/options. |
| pkg/appsec/challenge/challenge_test.go | Adjusts URL expectations (dropped from cookie) and adds oversized-URL seal test. |
| pkg/appsec/challenge_customjs.go | Loads and concatenates hub-provided challenge JS data files. |
| pkg/appsec/challenge_customjs_test.go | Tests custom JS loading/concatenation, traversal defense, and config parsing. |
| pkg/appsec/appsec.go | Routes ChallengeCustomJSPath to serve custom JS when configured. |
| pkg/acquisition/modules/appsec/config.go | Loads custom JS from hub data dir into the challenge runtime options. |
Review details
Files not reviewed (2)
- pkg/appsec/challenge/js/challenge_code.js: Generated file
- pkg/appsec/challenge/pb/fingerprint.pb.go: Generated file
- Files reviewed: 25/29 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #4657 +/- ##
==========================================
- Coverage 64.70% 58.24% -6.47%
==========================================
Files 522 525 +3
Lines 39862 40263 +401
==========================================
- Hits 25794 23450 -2344
- Misses 11668 14467 +2799
+ Partials 2400 2346 -54
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
/kind feature |
github-actions
Bot
added
kind/feature
area/appsec
and removed
needs/kind
needs/area
labels
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| `//go:embed`. The pipeline is only exercised when someone intentionally | ||
| changes the JS sources. | ||
|
|
||
| Writing a detection script that ships through the hub is a different job from this build |
| ChallengePowWorkerPath = "/crowdsec-internal/challenge/pow-worker.js" | ||
| ChallengeFPScannerPath = "/crowdsec-internal/challenge/fpscanner.js" | ||
| // ChallengeCustomJSPath serves the custom detection script; see WithCustomJS. | ||
| ChallengeCustomJSPath = "/crowdsec-internal/challenge/custom.js" |
There was a problem hiding this comment.
we need to add nonce markers in the custom.js to ensure it has been correctly loaded and interpreted by the browser.
| logger: logger, | ||
| } | ||
|
|
||
| if challengeRuntime.customJS != "" { |
There was a problem hiding this comment.
verify it's only done once at loadd
| // LoadCustomJS concatenates the declared scripts in order, so a base detection | ||
| // bundle and a site-specific fix compose. Read failures are logged, not fatal: | ||
| // bot detection has to survive a data file that hasn't downloaded yet. | ||
| func (wc *AppsecConfig) LoadCustomJS(dataDir string) string { |
