-
Notifications
You must be signed in to change notification settings - Fork 31
detector service for breakage reporting #2052
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+672
−24
Merged
Changes from 15 commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
5e9d02f
feat: web interference detection prototype
madblex 3abdc05
detector prototype v2
jdorweiler f241a6d
init and service
jdorweiler 5653ae3
update readme
jdorweiler be56f76
update readme
jdorweiler e2d8317
update readme
jdorweiler f5abfad
add auto run
jdorweiler 8096a9c
drop global domains option
jdorweiler cfe918c
update readme
jdorweiler 45b6115
remove old file
jdorweiler 6f45c6e
pr review changes, drop yt detector
jdorweiler 443628a
fix type erros
jdorweiler 13eea37
remove unused'
jdorweiler 9f1aa4a
remove service and init"
jdorweiler 4adc3a0
fix test
jdorweiler b626f15
pass config
jdorweiler f239af5
get settings from webInterference
jdorweiler 37c2dc2
comment auto-run
jdorweiler 5973a92
revert features
jdorweiler 0c9694b
revert features
jdorweiler 45b03aa
remove cache, refresh'
jdorweiler a4a3673
update readme
jdorweiler f38a4f6
update features.js
jdorweiler 93bb05d
fix getFeature
jdorweiler 279c192
detection tests
jdorweiler 00c1396
add px test
jdorweiler 7d8619c
lint
jdorweiler File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,190 @@ | ||
| # Web Interference Detection | ||
|
|
||
| This directory contains web interference detection functionality for content-scope-scripts. Detectors identify CAPTCHAs, fraud warnings, and other interference patterns to support breakage reporting and PIR automation. | ||
|
|
||
| ## Architecture | ||
|
|
||
| The system uses a **ContentFeature** wrapper with simple detection utilities: | ||
|
|
||
| - **`WebInterferenceDetection`** - ContentFeature that auto-runs detectors on page load | ||
| - **Detection utilities** - Pure functions (`runBotDetection`, `runFraudDetection`) with module-level caching | ||
| - **Direct imports** - Other features (breakage reporting, PIR) import detection functions directly | ||
|
|
||
|
|
||
| ## Directory Layout | ||
|
|
||
| ``` | ||
| detectors/ | ||
| ├── detections/ | ||
| │ ├── bot-detection.js # CAPTCHA/bot detection utility | ||
| │ └── fraud-detection.js # fraud/phishing warning utility | ||
| ├── utils/ | ||
| │ └── detection-utils.js # DOM helpers (selectors, text matching, visibility) | ||
| └── default-config.js # fallback detector settings | ||
| ``` | ||
|
|
||
| ## How It Works | ||
|
|
||
| ### 1. Initialization | ||
|
|
||
| The `WebInterferenceDetection` ContentFeature runs detectors automatically: | ||
|
|
||
| 1. Feature loads via standard content-scope-features lifecycle | ||
| 2. `init()` method schedules detectors to run after `autoRunDelayMs` (default: 100ms) | ||
| 3. Each detector runs once and caches results in module scope | ||
| 4. Other features can import and call detection functions to get cached results | ||
|
|
||
| ### 2. Configuration | ||
|
|
||
| Detectors are configured via `privacy-configuration/features/web-interference-detection.json`: | ||
|
|
||
| ```json | ||
| { | ||
| "state": "enabled", | ||
| "settings": { | ||
| "autoRunDelayMs": 100, | ||
| "interferenceTypes": { | ||
| "botDetection": { | ||
| "hcaptcha": { | ||
| "state": "enabled", | ||
| "vendor": "hcaptcha", | ||
| "selectors": [".h-captcha"], | ||
| "windowProperties": ["hcaptcha"] | ||
| } | ||
| }, | ||
| "fraudDetection": { | ||
| "phishingWarning": { | ||
| "state": "enabled", | ||
| "type": "phishing", | ||
| "selectors": [".warning-banner"] | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| **Domain-specific configuration** using `conditionalChanges`: | ||
|
|
||
| ```json | ||
| { | ||
| "settings": { | ||
| "conditionalChanges": [ | ||
| { | ||
| "condition": { | ||
| "urlPattern": "https://*.example.com/*" | ||
| }, | ||
| "patchSettings": [ | ||
| { | ||
| "op": "add", | ||
| "path": "/interferenceTypes/customDetector", | ||
| "value": { "state": "enabled", "selectors": [".custom"] } | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| The framework automatically applies conditional changes based on the current URL before passing settings to the feature. | ||
|
|
||
| ### 3. Using Detection Results | ||
|
|
||
| **Internal features** (same content script context): | ||
|
|
||
| ```javascript | ||
| import { runBotDetection, runFraudDetection } from '../detectors/detections/bot-detection.js'; | ||
|
|
||
| // Get cached results from auto-run | ||
| const botData = runBotDetection(); | ||
| const fraudData = runFraudDetection(); | ||
| ``` | ||
|
|
||
| **External:** | ||
|
|
||
| ```javascript | ||
| // Via messaging | ||
| this.messaging.request('detectInterference', { | ||
| types: ['botDetection', 'fraudDetection'] | ||
| }); | ||
| ``` | ||
|
|
||
| ## Adding New Detectors | ||
|
|
||
| 1. **Create detection utility** in `detections/`: | ||
|
|
||
| ```javascript | ||
| // detections/my-detector.js | ||
| let cachedResult = null; | ||
|
|
||
| export function runMyDetection(config = {}, options = {}) { | ||
| if (cachedResult && !options.refresh) return cachedResult; | ||
|
|
||
| // Run detection logic | ||
| const detected = checkSelectors(config.selectors); | ||
|
|
||
| cachedResult = { | ||
| detected, | ||
| type: 'myDetector', | ||
| timestamp: Date.now(), | ||
| }; | ||
|
|
||
| return cachedResult; | ||
| } | ||
| ``` | ||
|
|
||
| 2. **Add to WebInterferenceDetection feature**: | ||
|
|
||
| ```javascript | ||
| // features/web-interference-detection.js | ||
| import { runMyDetection } from '../detectors/detections/my-detector.js'; | ||
|
|
||
| init(args) { | ||
| const settings = this.getFeatureSetting('interferenceTypes'); | ||
|
|
||
| setTimeout(() => { | ||
| if (settings?.myDetector) { | ||
| runMyDetection(settings.myDetector); | ||
| } | ||
| }, autoRunDelayMs); | ||
| } | ||
| ``` | ||
|
|
||
| 3. **Add config** to `web-interference-detection.json`: | ||
|
|
||
| ```json | ||
| { | ||
| "settings": { | ||
| "interferenceTypes": { | ||
| "myDetector": { | ||
| "state": "enabled", | ||
| "selectors": [".my-selector"] | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Caching Strategy | ||
|
|
||
| - **Module-level cache**: Each detector uses a simple variable (`let cachedResult = null`) | ||
| - **Automatic**: First call runs detection and caches, subsequent calls return cached result | ||
| - **Per-tab**: Each browser tab has its own cache (separate content script instance) | ||
| - **Lifetime**: Cache persists for page lifetime, cleared on navigation | ||
| - **Refresh option**: Callers can force fresh detection with `{ refresh: true }` | ||
|
|
||
| **Examples:** | ||
| ```javascript | ||
| // Get cached result (fast) | ||
| const data = runBotDetection(config); | ||
|
|
||
| // Force fresh scan (slower, bypasses cache) | ||
| const freshData = runBotDetection(config, { refresh: true }); | ||
|
|
||
| // Via messaging (native layer) | ||
| messaging.request('detectInterference', { | ||
| types: ['botDetection'], | ||
| refresh: true // Optional: force rescan | ||
| }); | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| export const DEFAULT_DETECTOR_SETTINGS = Object.freeze({ | ||
| botDetection: { | ||
| cloudflareTurnstile: { | ||
| state: 'enabled', | ||
| vendor: 'cloudflare', | ||
| selectors: ['.cf-turnstile', 'script[src*="challenges.cloudflare.com"]'], | ||
| windowProperties: ['turnstile'], | ||
| statusSelectors: [ | ||
| { | ||
| status: 'solved', | ||
| selectors: ['[data-state="success"]'], | ||
| }, | ||
| { | ||
| status: 'failed', | ||
| selectors: ['[data-state="error"]'], | ||
| }, | ||
| ], | ||
| }, | ||
| cloudflareChallengePage: { | ||
| state: 'enabled', | ||
| vendor: 'cloudflare', | ||
| selectors: ['#challenge-form', '.cf-browser-verification', '#cf-wrapper', 'script[src*="challenges.cloudflare.com"]'], | ||
| windowProperties: ['_cf_chl_opt', '__CF$cv$params', 'cfjsd'], | ||
| }, | ||
| hcaptcha: { | ||
| state: 'enabled', | ||
| vendor: 'hcaptcha', | ||
| selectors: [ | ||
| '.h-captcha', | ||
| '[data-hcaptcha-widget-id]', | ||
| 'script[src*="hcaptcha.com"]', | ||
| 'script[src*="assets.hcaptcha.com"]', | ||
| ], | ||
| windowProperties: ['hcaptcha'], | ||
| }, | ||
| }, | ||
| fraudDetection: { | ||
| phishingWarning: { | ||
| state: 'enabled', | ||
| type: 'phishing', | ||
| selectors: ['.warning-banner', '#security-alert'], | ||
| textPatterns: ['suspicious.*activity', 'unusual.*login', 'verify.*account'], | ||
| textSources: ['innerText'], | ||
| }, | ||
| accountSuspension: { | ||
| state: 'enabled', | ||
| type: 'suspension', | ||
| selectors: ['.account-suspended', '#suspension-notice'], | ||
| textPatterns: ['account.*suspended', 'access.*restricted'], | ||
| textSources: ['innerText'], | ||
| }, | ||
| }, | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| import { checkSelectors, checkWindowProperties, matchesSelectors, matchesTextPatterns } from '../utils/detection-utils.js'; | ||
|
|
||
| // Cache result to avoid redundant DOM scans | ||
| let cachedResult = null; | ||
|
|
||
| /** | ||
| * Run bot detection and cache results. | ||
| * @param {Record<string, any>} config | ||
| * @param {Object} [options] | ||
| * @param {boolean} [options.refresh] - Force fresh detection, bypassing cache | ||
| */ | ||
| export function runBotDetection(config = {}, options = {}) { | ||
| if (cachedResult && !options.refresh) return cachedResult; | ||
| const results = Object.entries(config) | ||
| .filter(([_, challengeConfig]) => challengeConfig?.state === 'enabled') | ||
| .map(([challengeId, challengeConfig]) => { | ||
| const detected = checkSelectors(challengeConfig.selectors) || checkWindowProperties(challengeConfig.windowProperties || []); | ||
| if (!detected) { | ||
| return null; | ||
| } | ||
|
|
||
| const challengeStatus = findStatus(challengeConfig.statusSelectors); | ||
| return { | ||
| detected: true, | ||
| vendor: challengeConfig.vendor, | ||
| challengeType: challengeId, | ||
| challengeStatus, | ||
| }; | ||
| }) | ||
| .filter(Boolean); | ||
|
|
||
| // Cache and return | ||
| cachedResult = { | ||
| detected: results.length > 0, | ||
| type: 'botDetection', | ||
| results, | ||
| timestamp: Date.now(), | ||
| }; | ||
|
|
||
| return cachedResult; | ||
| } | ||
|
|
||
| function findStatus(statusSelectors) { | ||
| if (!Array.isArray(statusSelectors)) { | ||
| return null; | ||
| } | ||
|
|
||
| const match = statusSelectors.find((statusConfig) => { | ||
| const { selectors, textPatterns, textSources } = statusConfig; | ||
| return matchesSelectors(selectors) || matchesTextPatterns(document.body, textPatterns, textSources); | ||
| }); | ||
|
|
||
| return match?.status ?? null; | ||
| } | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| import { checkSelectorsWithVisibility, checkTextPatterns } from '../utils/detection-utils.js'; | ||
|
|
||
| // Cache result to avoid redundant DOM scans | ||
| let cachedResult = null; | ||
|
|
||
| /** | ||
| * Run fraud detection and cache results. | ||
| * @param {Record<string, any>} config | ||
| * @param {Object} [options] | ||
| * @param {boolean} [options.refresh] - Force fresh detection, bypassing cache | ||
| */ | ||
| export function runFraudDetection(config = {}, options = {}) { | ||
| if (cachedResult && !options.refresh) return cachedResult; | ||
| const results = Object.entries(config) | ||
| .filter(([_, alertConfig]) => alertConfig?.state === 'enabled') | ||
| .map(([alertId, alertConfig]) => { | ||
| const detected = | ||
| checkSelectorsWithVisibility(alertConfig.selectors) || | ||
| checkTextPatterns(alertConfig.textPatterns, alertConfig.textSources); | ||
| if (!detected) { | ||
| return null; | ||
| } | ||
|
|
||
| return { | ||
| detected: true, | ||
| alertId, | ||
| category: alertConfig.type, | ||
| }; | ||
| }) | ||
| .filter(Boolean); | ||
|
|
||
| // Cache and return | ||
| cachedResult = { | ||
| detected: results.length > 0, | ||
| type: 'fraudDetection', | ||
| results, | ||
| timestamp: Date.now(), | ||
| }; | ||
|
|
||
| return cachedResult; | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.