Packages from crwlr
| Package | Status | |
|---|---|---|
| crwlr/crawler | Web crawling and scraping library. |  |
| crwlr/crawler-ext-browser | Extension for the crwlr/crawler package containing steps utilizing a headless browser. |  |
| crwlr/crwl-ext-browser | Extension configurations for integration of crwlr/crawler-ext-browser into the crwl.io app. |  |
| crwlr/crwl-extension-utils | Utils for extension packages for the crwl.io app. |  |
| crwlr/html-2-text | Convert HTML to formatted plain text. |  |
| crwlr/query-string | A library for convenient handling of query strings used in HTTP requests. |  |
| crwlr/robots-txt | Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping |  |
| crwlr/schema-org | Extract schema.org structured data from HTML documents. |  |
| crwlr/url | Swiss Army knife for URLs. |  |
| crwlr/utils | Utilities that are needed in multiple crawler packages. |  |