-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Labels
Category: EnhancementNew feature or requestNew feature or requestComponent: Utility: Web ScrapingIssues related to Web Scraping utilityIssues related to Web Scraping utility
Milestone
Description
Summary
Sometimes users want to track content or resources from a website that has an invalid, expired, or self-signed certificate, but Playwright, which the Web Scraper component internally relies on, fails for untrusted certificates by default.
"XXX" tracker failed to check for content changes
There was an error while checking content: Failed to load page "https://xxxx.xx/": page.goto: net::ERR_CERT_DATE_INVALID at https://xxxx.xx/ Call log: - navigating to "https://xxxx.xx/", waiting until "load"
We should introduce an option to ignore such errors if the user explicitly opts into this behavior.
const context = await browser.newContext({
extraHTTPHeaders: options.headers,
bypassCSP: false,
userAgent: options.userAgent,
ignoreHTTPSErrors: options.ignoreHTTPSErrors <---- provided by tracker
});
const page = await context.newPage();
Metadata
Metadata
Assignees
Labels
Category: EnhancementNew feature or requestNew feature or requestComponent: Utility: Web ScrapingIssues related to Web Scraping utilityIssues related to Web Scraping utility
Type
Projects
Status
📋 Not started