batch_crawl |
Batch Crawl |
Parameters<super::params::BatchCrawlParams> |
Crawl multiple seed URLs concurrently. Returns crawl results for all seeds. |
batch_scrape |
Batch Scrape |
Parameters<super::params::BatchScrapeParams> |
Scrape multiple URLs concurrently. Returns results for all URLs. |
crawl |
Crawl Website |
Parameters<super::params::CrawlParams> |
Crawl a website following links. Returns content for all discovered pages up to max_depth/max_pages. |
download |
Download Document |
Parameters<super::params::DownloadParams> |
Download a document from a URL. Returns metadata about the downloaded file. |
generate_citations |
Generate Citations |
Parameters<super::params::GenerateCitationsParams> |
Convert markdown links into numbered citations with an appended reference list. |
get_version |
Get Version |
Parameters<super::params::EmptyParams> |
Get the current crawlberg library version. |
interact |
Interact |
Parameters<super::params::InteractParams> |
Execute browser actions on a page. Actions may mutate page or application state. |
map |
Map Website |
Parameters<super::params::MapParams> |
Discover all pages on a website via links and sitemaps. Returns a list of discovered URLs. |
scrape |
Scrape URL |
Parameters<super::params::ScrapeParams> |
Scrape a single URL and extract content as markdown or JSON. Returns page content, metadata, links, and images. |
