Skip to content

Add support for XHTML mimetypes #205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .eslintrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"overrides":
[
{
"extends": ["./scripts/eslintrc.esm.js"],
"files": ["./lib/**/*.js", "./test/**/*.js"]
},
{
"extends": ["./scripts/eslintrc.cjs.js"],
"files": ["./scripts/**/*.js"]
}
],
"root": true
}
25 changes: 25 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: tests
on: [push, pull_request]

jobs:
test:

runs-on: ${{ matrix.os }}
strategy:
matrix:
node-version: [12, 13]
os: [ubuntu-latest]

steps:
- uses: actions/checkout@v1
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- name: npm install, build, and test
run: |
npm install
npm run ci
env:
CI: true
COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }}
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
coverage/
lib-cjs/
node_modules/
.nyc_output/
package-lock.json
9 changes: 0 additions & 9 deletions .travis.yml

This file was deleted.

472 changes: 287 additions & 185 deletions README.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion bin/blc
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
#!/usr/bin/env node

new (require("../lib/cli"))().input();
require("../lib-cjs/cli")();
28 changes: 23 additions & 5 deletions changelog.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,21 @@
* 0.8.0
* dropped support for Node versions below 12.0
* API changes
* `linkObj` renamed to `Link`, and schema changed
* options added: `includedKeywords`, `includePage`, `retryHeadCodes`
* options changed: `acceptedSchemes`, `excludeLinksToSamePage`, `excludedSchemes`
* options renamed: `cacheExpiryTime`→`cacheMaxAge`, `customFilter`→`includeLink`, `retry405Head`→`retryHeadFail`
* CLI options added: `--include`, `--verbosity`
* CLI options removed: `--verbose`
* added [WHATWG URL specification](https://url.spec.whatwg.org)-compliance
* added support for HTTP basic auth, compression, proxies
* added support for file:// protocol
* added missing tags/attributes
* added support for pausing/resuming the CLI
* added progress bar and OS notification to the CLI
* refactored to ES2015+
* test suite improvements
* bug fixes
* 0.7.8 fix for Node.js v9
* 0.7.7 added `--host-requests`, `--requests` CLI options
* 0.7.6 bug fix
Expand All @@ -10,11 +28,11 @@
* added `SiteChecker`
* methods added: `numPages()`, `numSites()`
* methods removed: `numActiveItems()`
* methods renamed: `length()`->`numQueuedLinks()`
* methods renamed: `length()``numQueuedLinks()`
* options added: `honorRobotExclusions`
* options removed: `excludeResponseData`
* handlers added: `html`, `robots`
* handlers renamed: `item`->`page`
* handlers renamed: `item``page`
* CLI options added: `--follow`, `--recursive`, `--user-agent`
* linkObj added: `brokenReason`, `excludedReason`, `html.location`
* linkObj removed: `error`, `http.redirects`, `http.statusCode`
Expand Down Expand Up @@ -52,9 +70,9 @@
* API change
* CLI options
* options added: `excludeExternalLinks`, `excludeResponseData`, `maxSockets`
* options renamed: `maxSockets`->`maxSocketsPerHost`
* options renamed: `maxSockets``maxSocketsPerHost`
* linkObj added: `http`
* linkObj moved: `response`->`http.response`
* linkObj moved: `response``http.response`
* linkObj changed: `internal` and `samePage` now compares the base URL (ignoring `<base>`) with links that may have `<base>` applied
* switched from [request](https://npmjs.com/request) to [bhttp](https://npmjs.com/bhttp)
* 0.4.3 added `rateLimit` option, cleanup
Expand All @@ -68,7 +86,7 @@
* linkObj added: `html.selector`
* 0.3.0
* options added: `maxSockets`
* options renamed: `site`->`base`
* options renamed: `site``base`
* `<base>` supported
* requesting links now only downloads the response header
* faster test suite
Expand Down
Loading