C API Reference

C API Reference v1.1.4

Functions

cberg_generate_citations()

Convert markdown links to numbered citations.

[Example](https://example.com) becomes Example[1] with [1]: <https://example.com> in the reference list. Images ![alt](url) are preserved unchanged.

Signature:

CbergCitationResult* cberg_generate_citations(const char* markdown);

Example:

CbergCitationResult *result = cberg_generate_citations("value");

Parameters:

Name	Type	Required	Description
`markdown`	`const char*`	Yes	The markdown

Returns: CbergCitationResult

cberg_create_engine()

Create a new crawl engine with the given configuration.

If config is NULL, uses CrawlConfig.default(). Returns an error if the configuration is invalid.

Signature:

CbergCrawlEngineHandle* cberg_create_engine(CbergCrawlConfig config);

Example:

CbergCrawlEngineHandle *result = cberg_create_engine(NULL);

Parameters:

Name	Type	Required	Description
`config`	`CbergCrawlConfig*`	No	The configuration options

Returns: CbergCrawlEngineHandle

Errors: Returns NULL on error.

cberg_scrape()

Scrape a single URL, returning extracted page data.

Signature:

CbergScrapeResult* cberg_scrape(CbergCrawlEngineHandle engine, const char* url);

Example:

CbergScrapeResult *result = cberg_scrape(NULL, "value");

Parameters:

Name	Type	Required	Description
`engine`	`CbergCrawlEngineHandle`	Yes	The crawl engine handle
`url`	`const char*`	Yes	The URL to fetch

Returns: CbergScrapeResult

Errors: Returns NULL on error.

cberg_crawl()

Crawl a website starting from url, following links up to the configured depth.

Signature:

CbergCrawlResult* cberg_crawl(CbergCrawlEngineHandle engine, const char* url);

Example:

CbergCrawlResult *result = cberg_crawl(NULL, "value");

Parameters:

Name	Type	Required	Description
`engine`	`CbergCrawlEngineHandle`	Yes	The crawl engine handle
`url`	`const char*`	Yes	The URL to fetch

Returns: CbergCrawlResult

Errors: Returns NULL on error.

cberg_map_urls()

Discover all pages on a website by following links and sitemaps.

Signature:

CbergMapResult* cberg_map_urls(CbergCrawlEngineHandle engine, const char* url);

Example:

CbergMapResult *result = cberg_map_urls(NULL, "value");

Parameters:

Name	Type	Required	Description
`engine`	`CbergCrawlEngineHandle`	Yes	The crawl engine handle
`url`	`const char*`	Yes	The URL to fetch

Returns: CbergMapResult

Errors: Returns NULL on error.

cberg_interact()

Execute browser actions on a single page.

Signature:

CbergInteractionResult* cberg_interact(CbergCrawlEngineHandle engine, const char* url, CbergPageAction* actions);

Example:

CbergInteractionResult *result = cberg_interact(NULL, "value", NULL);

Parameters:

Name	Type	Required	Description
`engine`	`CbergCrawlEngineHandle`	Yes	The crawl engine handle
`url`	`const char*`	Yes	The URL to fetch
`actions`	`CbergPageAction*`	Yes	The actions

Returns: CbergInteractionResult

Errors: Returns NULL on error.

cberg_batch_scrape()

Scrape multiple URLs concurrently.

Signature:

CbergBatchScrapeResults* cberg_batch_scrape(CbergCrawlEngineHandle engine, const char** urls);

Example:

CbergBatchScrapeResults *result = cberg_batch_scrape(NULL, NULL);

Parameters:

Name	Type	Required	Description
`engine`	`CbergCrawlEngineHandle`	Yes	The crawl engine handle
`urls`	`const char**`	Yes	The urls

Returns: CbergBatchScrapeResults

Errors: Returns NULL on error.

cberg_batch_crawl()

Crawl multiple seed URLs concurrently, each following links to configured depth.

Signature:

CbergBatchCrawlResults* cberg_batch_crawl(CbergCrawlEngineHandle engine, const char** urls);

Example:

CbergBatchCrawlResults *result = cberg_batch_crawl(NULL, NULL);

Parameters:

Name	Type	Required	Description
`engine`	`CbergCrawlEngineHandle`	Yes	The crawl engine handle
`urls`	`const char**`	Yes	The urls

Returns: CbergBatchCrawlResults

Errors: Returns NULL on error.

Types

CbergActionResult

Result from a single page action execution.

Field	Type	Default	Description
`action_index`	`uintptr_t`	—	Zero-based index of the action in the sequence.
`action_type`	`const char*`	—	The type of action that was executed.
`success`	`bool`	—	Whether the action completed successfully.
`data`	`void**`	`NULL`	Action-specific return data (screenshot bytes, JS return value, scraped HTML).
`error`	`const char**`	`NULL`	Error message if the action failed.

CbergArticleMetadata

Article metadata extracted from article:* Open Graph tags.

Field	Type	Default	Description
`published_time`	`const char**`	`NULL`	The article publication time.
`modified_time`	`const char**`	`NULL`	The article modification time.
`author`	`const char**`	`NULL`	The article author.
`section`	`const char**`	`NULL`	The article section.
`tags`	`const char**`	`NULL`	The article tags.

CbergBatchCrawlResult

Result from a single URL in a batch crawl operation.

Field	Type	Default	Description
`url`	`const char*`	—	The seed URL that was crawled.
`result`	`CbergCrawlResult*`	`NULL`	The crawl result, if successful.
`error`	`const char**`	`NULL`	The error message, if the crawl failed.

CbergBatchCrawlResults

Aggregate result of a batch crawl, exposing per-URL results plus precomputed counts.

The counts are derived once at construction so every binding language can read them as plain integer fields without re-iterating the results vector.

Field	Type	Default	Description
`results`	`CbergBatchCrawlResult*`	`NULL`	Per-URL crawl results, in the order seed URLs were submitted.
`total_count`	`uintptr_t`	—	Total number of seed URLs in the batch (equal to `results.len()`).
`completed_count`	`uintptr_t`	—	Number of seed URLs whose crawl succeeded (`error` is `NULL`).
`failed_count`	`uintptr_t`	—	Number of seed URLs whose crawl failed (`error` is `Some`).

CbergBatchCrawlStreamRequest

Request to begin a multi-URL streaming crawl.

Wraps a set of seed URLs for delivery through the streaming-adapter binding surface. Required as a struct because alef’s streaming adapter requires a named request type — primitives are not supported.

Field	Type	Default	Description
`urls`	`const char**`	`NULL`	The seed URLs to crawl. Each URL is followed independently up to the engine’s configured depth.

CbergBatchScrapeResult

Result from a single URL in a batch scrape operation.

Field	Type	Default	Description
`url`	`const char*`	—	The URL that was scraped.
`result`	`CbergScrapeResult*`	`NULL`	The scrape result, if successful.
`error`	`const char**`	`NULL`	The error message, if the scrape failed.

CbergBatchScrapeResults

Aggregate result of a batch scrape, exposing per-URL results plus precomputed counts.

The counts are derived once at construction so every binding language can read them as plain integer fields without re-iterating the results vector.

Field	Type	Default	Description
`results`	`CbergBatchScrapeResult*`	`NULL`	Per-URL scrape results, in the order URLs were submitted.
`total_count`	`uintptr_t`	—	Total number of URLs in the batch (equal to `results.len()`).
`completed_count`	`uintptr_t`	—	Number of URLs whose scrape succeeded (`error` is `NULL`).
`failed_count`	`uintptr_t`	—	Number of URLs whose scrape failed (`error` is `Some`).

CbergBrowserConfig

Browser fallback configuration.

Field	Type	Default	Description
`mode`	`CbergBrowserMode`	`CBERG_CBERG_AUTO`	When to use the headless browser fallback.
`backend`	`CbergBrowserBackend`	`CBERG_CBERG_CHROMIUMOXIDE`	Browser backend used to render JavaScript-heavy pages.
`endpoint`	`const char**`	`NULL`	CDP WebSocket endpoint for connecting to an external browser instance.
`timeout`	`uint64_t`	`30000ms`	Timeout for browser page load and rendering (in milliseconds when serialized).
`wait`	`CbergBrowserWait`	`CBERG_CBERG_NETWORK_IDLE`	Wait strategy after browser navigation.
`wait_selector`	`const char**`	`NULL`	CSS selector to wait for when `wait` is `Selector`.
`extra_wait`	`uint64_t*`	`NULL`	Extra time to wait after the wait condition is met.
`proxy`	`CbergProxyConfig*`	`NULL`	Proxy for browser fetches. Overrides `CrawlConfig.proxy` when set. Native backend supports http/https only (no SOCKS5).
`block_url_patterns`	`const char**`	`NULL`	URL patterns to block before the network request fires. Supports `*` wildcards. Useful for skipping ads/analytics/large images. Honored by `BrowserBackend.Native`; chromiumoxide ignores this field today.
`eval_script`	`const char**`	`NULL`	JavaScript snippet evaluated after navigation completes. Scraping captures the native backend result in `ScrapeResult.browser.eval_result`. Interactions run this script before page actions on both browser backends but do not include the script result in `InteractionResult`.
`robots_user_agent`	`const char**`	`NULL`	User-agent used when fetching robots.txt. Defaults to `BrowserConfig.user_agent` (or crawlberg’s default) if unset. Native only.
`capture_network_events`	`bool`	`false`	Capture the full network event stream into the result. Default false (only the document event is captured). Native only.
`session_affinity`	`bool`	`true`	Enable session affinity: reuse chromiumoxide Pages for same-domain requests so cookies + fingerprint + solved challenges persist. Default: true. When false, each request gets a fresh Page.

Methods

cberg_default()

Signature:

CbergBrowserConfig cberg_default();

Example:

CbergBrowserConfig *result = cberg_default();

Returns: CbergBrowserConfig

CbergBrowserExtras

Browser-specific extras populated when the native browser backend was used.

Available on ScrapeResult.browser when BrowserBackend.Native handled the request.

Field	Type	Default	Description
`eval_result`	`void**`	`NULL`	Return value of `BrowserConfig.eval_script`, if provided.
`network_events`	`CbergResponseMeta*`	`NULL`	Network events captured during page navigation (only populated when `BrowserConfig.capture_network_events` is true).
`cookies`	`CbergCookieInfo*`	`NULL`	All non-expired cookies present in the browser’s cookie jar after navigation completes (includes both prior cookies and server Set-Cookie).

CbergCitationReference

A single numbered reference in a citation list — produced by the citation extractor when content uses inline [N]-style markers.

Field	Type	Default	Description
`index`	`uintptr_t`	—	1-based reference number as it appears in the source text.
`url`	`const char*`	—	Resolved absolute URL for this reference.
`text`	`const char*`	—	Human-readable anchor text or title for the reference.

CbergCitationResult

Result of citation conversion.

Field	Type	Default	Description
`content`	`const char*`	—	Markdown with links replaced by numbered citations.
`references`	`CbergCitationReference*`	`NULL`	Numbered reference list: (index, url, text).

CbergContentConfig

Content extraction and conversion configuration.

Controls how HTML is converted to the output format. Uses html-to-markdown-rs as the conversion engine for all formats (markdown, plain text, djot).

Field	Type	Default	Description
`output_format`	`const char*`	`"markdown"`	Output format: `"markdown"` (default), `"plain"`, `"djot"`.
`preprocessing_preset`	`const char*`	`"standard"`	Preprocessing aggressiveness: `"minimal"`, `"standard"` (default), `"aggressive"`. - Minimal: only scripts/styles removed. - Standard: also removes nav, nav-hinted headers/footers/asides, forms. - Aggressive: removes all footers/asides unconditionally.
`remove_navigation`	`bool`	`true`	Remove navigation elements (nav, breadcrumbs, menus). Default: `true`.
`remove_forms`	`bool`	`true`	Remove form elements. Default: `true`.
`strip_tags`	`const char**`	`NULL`	HTML tag names to strip (render children only, remove the tag wrapper). Default: `\["noscript"\]`.
`preserve_tags`	`const char**`	`NULL`	HTML tag names to preserve as raw HTML in output.
`exclude_selectors`	`const char**`	`NULL`	CSS selectors for elements to exclude entirely (element + all content). Unlike `strip_tags` (which removes the wrapper but keeps children), excluded elements and all descendants are dropped. Supports CSS selectors: `.class`, `#id`, `\[attribute\]`, compound selectors. Example: `\[".cookie-banner", "#ad-container", "\[role='complementary'\]"\]`
`skip_images`	`bool`	`false`	Skip image elements in output. Default: `false`.
`max_depth`	`uintptr_t*`	`NULL`	Max DOM traversal depth. Prevents stack overflow on deeply nested HTML.
`wrap`	`bool`	`false`	Enable line wrapping. Default: `false`.
`wrap_width`	`uintptr_t`	`80`	Wrap width when `wrap` is enabled. Default: `80`.
`include_document_structure`	`bool`	`true`	Include document structure tree in output. Default: `true`.

Methods

cberg_default()

Signature:

CbergContentConfig cberg_default();

Example:

CbergContentConfig *result = cberg_default();

Returns: CbergContentConfig

CbergCookieInfo

Information about an HTTP cookie received from a response.

Field	Type	Default	Description
`name`	`const char*`	—	The cookie name.
`value`	`const char*`	—	The cookie value.
`domain`	`const char**`	`NULL`	The cookie domain, if specified.
`path`	`const char**`	`NULL`	The cookie path, if specified.

CbergCrawlConfig

Configuration for crawl, scrape, and map operations.

Field	Type	Default	Description
`max_depth`	`uintptr_t*`	`NULL`	Maximum crawl depth (number of link hops from the start URL).
`max_pages`	`uintptr_t*`	`NULL`	Maximum number of pages to crawl.
`max_links_per_page`	`uintptr_t*`	`NULL`	Maximum links enqueued from a single page. Defaults to 10000. Bounds the work one hostile or pathological page can create; links past the cap are dropped and a warning is logged.
`max_concurrent`	`uintptr_t*`	`NULL`	Maximum number of concurrent requests.
`respect_robots_txt`	`bool`	`false`	Whether to respect robots.txt directives.
`soft_http_errors`	`bool`	`false`	When true, HTTP-level error responses (404 NotFound, 403 Forbidden, WAF blocks) are surfaced as `ScrapeResult` records with the matching `status_code` rather than raised as `CrawlError`. Default `false` preserves the historical throw-on-error contract for direct fetches. Independently of this flag, 404s reached at the end of a redirect chain are always surfaced softly — the user opted into redirect-following, so receiving a 404 there is part of the normal flow rather than an unexpected error.
`user_agent`	`const char**`	`NULL`	Custom user-agent string.
`stay_on_domain`	`bool`	`false`	Whether to restrict crawling to the same domain.
`allow_subdomains`	`bool`	`false`	Whether to allow subdomains when `stay_on_domain` is true.
`include_paths`	`const char**`	`NULL`	Regex patterns for paths to include during crawling.
`exclude_paths`	`const char**`	`NULL`	Regex patterns for paths to exclude during crawling.
`custom_headers`	`void*`	`NULL`	Custom HTTP headers to send with each request.
`request_timeout`	`uint64_t`	`30000ms`	Timeout for individual HTTP requests (in milliseconds when serialized).
`rate_limit_ms`	`uint64_t*`	`NULL`	Per-domain rate limit in milliseconds. When set, enforces a minimum delay between requests to the same domain. Defaults to 200ms when `NULL`.
`max_redirects`	`uintptr_t`	`10`	Maximum number of redirects to follow.
`retry_count`	`uintptr_t`	`0`	Number of retry attempts for failed requests.
`retry_codes`	`uint16_t*`	`NULL`	HTTP status codes that should trigger a retry.
`cookies_enabled`	`bool`	`false`	Whether to enable cookie handling.
`auth`	`CbergAuthConfig*`	`NULL`	Authentication configuration.
`max_body_size`	`uintptr_t*`	`NULL`	Maximum response body size in bytes.
`remove_tags`	`const char**`	`NULL`	CSS selectors for tags to remove from HTML before processing.
`content`	`CbergContentConfig`	—	Content extraction and conversion configuration.
`map_limit`	`uintptr_t*`	`NULL`	Maximum number of URLs to return from a map operation.
`map_search`	`const char**`	`NULL`	Search filter for map results (case-insensitive substring match on URLs).
`download_assets`	`bool`	`false`	Whether to download assets (CSS, JS, images, etc.) from the page.
`asset_types`	`CbergAssetCategory*`	`NULL`	Filter for asset categories to download.
`max_asset_size`	`uintptr_t*`	`NULL`	Maximum size in bytes for individual asset downloads.
`browser`	`CbergBrowserConfig`	—	Browser configuration.
`proxy`	`CbergProxyConfig*`	`NULL`	Proxy configuration for HTTP requests.
`user_agents`	`const char**`	`NULL`	List of user-agent strings for rotation. If non-empty, overrides `user_agent`.
`capture_screenshot`	`bool`	`false`	Whether to capture a screenshot when using the browser. Only supported by `scrape()` with `BrowserBackend.Chromiumoxide` and `BrowserMode.Always` or `Stealth`. A screenshot is 100–500 KB of PNG per page, so `crawl()` does not carry screenshots in `CrawlPageResult`/`CrawlResult` at all — a multi-thousand-page crawl holding one per page in memory is not a safe default. Setting this with any other configuration (a different backend, `BrowserMode.Auto`/`Never`, or during `crawl()`) has no effect and logs a warning rather than silently doing nothing.
`follow_document_urls`	`bool`	`false`	Re-enqueue discovered `LinkType.Document` URLs into the crawl frontier so the crawl follows links from document pages (PDFs, etc.) as it would from HTML pages. Default: `false` (documents terminate at materialisation).
`document_url_depth`	`uint32_t*`	`NULL`	Maximum document-depth (from the seed URL through document links only) when `follow_document_urls` is true. `NULL` means inherit `max_depth`. Independent of `max_depth`: a document URL is enqueued only if BOTH the outer `max_depth` and (if set) `document_url_depth` permit it.
`download_documents`	`bool`	`true`	Whether to download non-HTML documents (PDF, DOCX, images, code, etc.) instead of skipping them. Defaults to `true` — unlike `download_assets` and `capture_screenshot`, which default to `false`.
`document_max_size`	`uintptr_t*`	`52428800`	Maximum size in bytes for document downloads. Defaults to 50 MB.
`document_mime_types`	`const char**`	`NULL`	Allowlist of MIME types to download. If empty, uses built-in defaults.
`document_output_dir`	`const char**`	`NULL`	Directory to stream downloaded document bytes into instead of holding them in memory on `DownloadedDocument.content`. When set, `content` is left empty and `DownloadedDocument.content_path` is populated with `<dir>/<content_hash>.<ext>`. `NULL` (default) preserves today’s in-memory-only behavior. Has no effect on wasm32, which has no filesystem — use `document_content_encoding` there instead.
`document_content_encoding`	`CbergDocumentContentEncoding*`	`NULL`	Opt-in encoding that duplicates `DownloadedDocument.content` into a serializable field for language bindings that need the bytes in-memory (`content` itself is `alef(skip)`ed). `NULL` (default) means no encoding is produced. Independent of `document_output_dir` — set both to get a file on disk and an in-memory copy.
`warc_output`	`const char**`	`NULL`	Path to write WARC output. If `NULL`, WARC output is disabled.
`browser_profile`	`const char**`	`NULL`	Named browser profile for persistent sessions (cookies, localStorage). Chromiumoxide backend only. The native backend runs an in-process JavaScript engine with no Chrome process and therefore no profile directory, so this is ignored there and logs a warning. It is also ignored — with a warning — when a shared browser pool is in use (the pool launches before any per-crawl config exists) or when connecting to an external CDP endpoint whose process crawlberg does not own.
`save_browser_profile`	`bool`	`false`	Whether to save changes back to the browser profile on exit.
`ssrf`	`CbergSsrfPolicy`	—	SSRF policy for outbound network requests. Default: deny private networks, allow http/https only, max 5 redirects. `deny_private`, `allowlist` and `max_redirects` are exposed to all language bindings. `scheme_allowlist` stays Rust-only — see `SsrfPolicy`. wasm32 (including Node.js): `deny_private` does not stop hostname-based requests. There is no DNS resolution on this target, so only a literal IP host is checked against the policy — a domain name is always permitted, regardless of `deny_private`. Under Node, where `fetch` enforces no CORS, this means a service embedding the wasm binding can be driven to internal hosts by domain name even with `deny_private = true`. Enforce egress restrictions at the network layer for that deployment target; do not rely on this field. See `crawlberg.net.validate_url`.
`ssrf_deny_private_explicit`	`bool*`	`NULL`	Pins `SsrfPolicy.deny_private` to a caller-chosen value, bypassing the `CRAWLBERG_ALLOW_PRIVATE_NETWORK` operator override entirely for this config. `ssrf.deny_private` is a plain, always-serialized `bool`: several alef-generated bindings construct `SsrfPolicy.default()` (hardcoding `deny_private: true`) whenever their caller never touches SSRF settings at all, so `true` on that field alone cannot distinguish “the caller wants private networks denied” from “the binding’s own structural default landed on `true`”. The environment variable exists precisely to resolve that ambiguity in the common case by treating any `true` as inconclusive and deferring to the operator. Set this field when that default-deferral is wrong for your call — e.g. a test that must prove `deny_private: true` still denies even while the operator has set `CRAWLBERG_ALLOW_PRIVATE_NETWORK` suite-wide for every other call. `NULL` (default) preserves today’s behavior: the environment variable may still flip `ssrf.deny_private` to `false`. `Some(value)` pins `ssrf.deny_private` to `value` and the environment variable is not consulted for this config.

Methods

cberg_default()

Signature:

CbergCrawlConfig cberg_default();

Example:

CbergCrawlConfig *result = cberg_default();

Returns: CbergCrawlConfig

cberg_validate()

Validate the configuration, returning an error if any values are invalid.

Signature:

void cberg_validate();

Example:

cberg_validate(instance);

Returns: No return value.

Errors: Returns NULL on error.

CbergCrawlEngineHandle

Opaque handle to a configured crawl engine.

Constructed via create_engine with an optional CrawlConfig. Default implementations for all pluggable components are used internally.

CbergCrawlPageResult

The result of crawling a single page during a crawl operation.

Field	Type	Default	Description
`url`	`const char*`	—	The original URL of the page.
`normalized_url`	`const char*`	—	The normalized URL of the page.
`status_code`	`uint16_t`	—	The HTTP status code of the response.
`content_type`	`const char*`	—	The Content-Type header value.
`html`	`const char*`	—	The HTML body of the response.
`body_size`	`uintptr_t`	—	The size of the response body in bytes.
`metadata`	`CbergPageMetadata`	—	Extracted metadata from the page.
`links`	`CbergLinkInfo*`	`NULL`	Links found on the page.
`images`	`CbergImageInfo*`	`NULL`	Images found on the page.
`feeds`	`CbergFeedInfo*`	`NULL`	Feed links found on the page.
`json_ld`	`CbergJsonLdEntry*`	`NULL`	JSON-LD entries found on the page.
`depth`	`uintptr_t`	—	The depth of this page from the start URL.
`stayed_on_domain`	`bool`	—	Whether this page is on the same domain as the start URL.
`was_skipped`	`bool`	—	Whether this page was skipped (binary or PDF content).
`is_pdf`	`bool`	—	Whether the content is a PDF.
`detected_charset`	`const char**`	`NULL`	The detected character set encoding.
`markdown`	`CbergMarkdownResult*`	`NULL`	Markdown conversion of the page content.
`extracted_data`	`void**`	`NULL`	Structured data extracted by LLM. Populated when extraction is configured.
`extraction_meta`	`CbergExtractionMeta*`	`NULL`	Metadata about the LLM extraction pass (cost, tokens, model).
`downloaded_document`	`CbergDownloadedDocument*`	`NULL`	Downloaded non-HTML document (PDF, DOCX, image, code, etc.).
`browser_used`	`bool`	—	Whether the browser fallback was used to fetch this page.

CbergCrawlResult

The result of a multi-page crawl operation.

Field	Type	Default	Description
`pages`	`CbergCrawlPageResult*`	`NULL`	The list of crawled pages.
`final_url`	`const char*`	—	The final URL after following redirects.
`redirect_count`	`uintptr_t`	—	The number of redirects followed.
`was_skipped`	`bool`	—	Whether any page was skipped during crawling.
`error`	`const char**`	`NULL`	An error message, if the crawl encountered an issue.
`cookies`	`CbergCookieInfo*`	`NULL`	Cookies collected during the crawl.
`stayed_on_domain`	`bool`	—	Whether all crawled pages stayed on the same domain as the start URL.
`browser_used`	`bool`	—	Whether the browser fallback was used for any page in this crawl.

Methods

cberg_unique_normalized_urls()

Returns the count of unique normalized URLs encountered during crawling.

Computed from pages (not the deprecated normalized_urls field) so it is correct across every binding that reconstructs CrawlResult from pages alone. In streaming mode pages is empty, so this returns 0 on the opaque-handle (C/Go/C#/Zig/Dart) path where it previously counted streamed pages — a known, accepted cost of making the other ten binding families correct.

Signature:

uintptr_t cberg_unique_normalized_urls();

Example:

uintptr_t result = cberg_unique_normalized_urls(instance);

Returns: uintptr_t

CbergCrawlStreamRequest

Request to begin a single-URL streaming crawl.

Wraps a single seed URL for delivery through the streaming-adapter binding surface. Required as a struct because alef’s streaming adapter requires a named request type — primitives are not supported.

Field	Type	Default	Description
`url`	`const char*`	—	The seed URL to crawl.

CbergDownloadedAsset

A downloaded asset from a page.

Field	Type	Default	Description
`url`	`const char*`	—	The original URL of the asset.
`content_hash`	`const char*`	—	The SHA-256 content hash of the asset.
`mime_type`	`const char**`	`NULL`	The MIME type from the Content-Type header.
`size`	`uintptr_t`	—	The size of the asset in bytes.
`asset_category`	`CbergAssetCategory`	`CBERG_CBERG_IMAGE`	The category of the asset.
`html_tag`	`const char**`	`NULL`	The HTML tag that referenced this asset (e.g., “link”, “script”, “img”).

CbergDownloadedDocument

A downloaded non-HTML document (PDF, DOCX, image, code file, etc.).

When the crawler encounters non-HTML content and download_documents is enabled, it downloads the raw bytes and populates this struct instead of skipping the resource.

Field	Type	Default	Description
`url`	`const char*`	—	The URL the document was fetched from.
`mime_type`	`const char*`	—	The MIME type from the Content-Type header.
`size`	`uintptr_t`	—	Size of the document in bytes.
`filename`	`const char**`	`NULL`	Filename extracted from Content-Disposition or URL path.
`content_hash`	`const char*`	—	SHA-256 hex digest of the content.
`headers`	`void*`	`NULL`	Selected response headers.
`truncated`	`bool`	—	True when `content` (or the file at `content_path`) was truncated to `document_max_size`; `size` still reports the original, untruncated length.
`content_path`	`const char**`	`NULL`	Filesystem path the document was streamed to when `document_output_dir` was set. `content` is empty in memory when this is populated.
`content_base64`	`const char**`	`NULL`	Base64-encoded copy of `content`, populated only when `document_content_encoding` was set to `Base64`.

CbergExtractionMeta

Metadata about an LLM extraction pass.

Field	Type	Default	Description
`cost`	`double*`	`NULL`	Estimated cost of the LLM call in USD.
`prompt_tokens`	`uint64_t*`	`NULL`	Number of prompt (input) tokens consumed.
`completion_tokens`	`uint64_t*`	`NULL`	Number of completion (output) tokens generated.
`model`	`const char**`	`NULL`	The model identifier used for extraction.
`chunks_processed`	`uintptr_t`	—	Number of content chunks sent to the LLM.

CbergFaviconInfo

Information about a favicon or icon link.

Field	Type	Default	Description
`url`	`const char*`	—	The icon URL.
`rel`	`const char*`	—	The `rel` attribute (e.g., “icon”, “apple-touch-icon”).
`sizes`	`const char**`	`NULL`	The `sizes` attribute, if present.
`mime_type`	`const char**`	`NULL`	The MIME type, if present.

CbergFeedInfo

Information about a feed link found on a page.

Field	Type	Default	Description
`url`	`const char*`	—	The feed URL.
`title`	`const char**`	`NULL`	The feed title, if present.
`feed_type`	`CbergFeedType`	`CBERG_CBERG_RSS`	The type of feed.

CbergHeadingInfo

A heading element extracted from the page.

Field	Type	Default	Description
`level`	`uint8_t`	—	The heading level (1-6).
`text`	`const char*`	—	The heading text content.

CbergHreflangEntry

An hreflang alternate link entry.

Field	Type	Default	Description
`lang`	`const char*`	—	The language code (e.g., “en”, “fr”, “x-default”).
`url`	`const char*`	—	The URL for this language variant.

CbergImageInfo

Information about an image found on a page.

Field	Type	Default	Description
`url`	`const char*`	—	The image URL.
`alt`	`const char**`	`NULL`	The alt text, if present.
`width`	`uint32_t*`	`NULL`	The width attribute, if present and parseable.
`height`	`uint32_t*`	`NULL`	The height attribute, if present and parseable.
`source`	`CbergImageSource`	`CBERG_CBERG_IMG`	The source of the image reference.

CbergInteractionResult

Result of executing a sequence of page interaction actions.

Field	Type	Default	Description
`action_results`	`CbergActionResult*`	`NULL`	Results from each executed action.
`final_html`	`const char*`	—	Final page HTML after all actions completed.
`final_url`	`const char*`	—	Final page URL (may have changed due to navigation).
`screenshot_base64`	`const char**`	`NULL`	Base64-encoded PNG screenshot taken after all actions. Populated only when a `PageAction.Screenshot` action actually ran, so callers that never request a screenshot do not pay the encoding cost.

CbergJsonLdEntry

A JSON-LD structured data entry found on a page.

Field	Type	Default	Description
`schema_type`	`const char*`	—	The `@type` value from the JSON-LD object.
`name`	`const char**`	`NULL`	The `name` value, if present.
`raw`	`const char*`	—	The raw JSON-LD string.

CbergLinkInfo

Information about a link found on a page.

Field	Type	Default	Description
`url`	`const char*`	—	The resolved URL of the link.
`text`	`const char*`	—	The visible text of the link.
`link_type`	`CbergLinkType`	`CBERG_CBERG_INTERNAL`	The classification of the link.
`rel`	`const char**`	`NULL`	The `rel` attribute value, if present.
`nofollow`	`bool`	—	Whether the link has `rel="nofollow"`.

CbergMapResult

The result of a map operation, containing discovered URLs.

Field	Type	Default	Description
`urls`	`CbergSitemapUrl*`	`NULL`	The list of discovered URLs.

CbergMarkdownResult

Rich markdown conversion result from HTML processing.

Field	Type	Default	Description
`content`	`const char*`	—	Converted markdown text.
`document_structure`	`void**`	`NULL`	Structured document tree with semantic nodes.
`tables`	`void**`	`NULL`	Extracted tables with structured cell data.
`warnings`	`const char**`	`NULL`	Non-fatal processing warnings.
`citations`	`bool`	—	Whether citation conversion was applied and produced at least one reference. `true` when the markdown contained inline links that were converted to numbered citation references. The converted content (with `\[N\]` markers) is available in `content`; the full reference list is accessible via `generate_citations` if needed separately.
`fit_content`	`const char**`	`NULL`	Content-filtered markdown optimized for LLM consumption.

CbergPageMetadata

Metadata extracted from an HTML page’s <meta> tags and <title> element.

Field	Type	Default	Description
`title`	`const char**`	`NULL`	The page title from the `<title>` element.
`description`	`const char**`	`NULL`	The meta description.
`canonical_url`	`const char**`	`NULL`	The canonical URL from `<link rel="canonical">`.
`keywords`	`const char**`	`NULL`	Keywords from `<meta name="keywords">`.
`author`	`const char**`	`NULL`	Author from `<meta name="author">`.
`viewport`	`const char**`	`NULL`	Viewport content from `<meta name="viewport">`.
`theme_color`	`const char**`	`NULL`	Theme color from `<meta name="theme-color">`.
`generator`	`const char**`	`NULL`	Generator from `<meta name="generator">`.
`robots`	`const char**`	`NULL`	Robots content from `<meta name="robots">`.
`html_lang`	`const char**`	`NULL`	The `lang` attribute from the `<html>` element.
`html_dir`	`const char**`	`NULL`	The `dir` attribute from the `<html>` element.
`og_title`	`const char**`	`NULL`	Open Graph title.
`og_type`	`const char**`	`NULL`	Open Graph type.
`og_image`	`const char**`	`NULL`	Open Graph image URL.
`og_description`	`const char**`	`NULL`	Open Graph description.
`og_url`	`const char**`	`NULL`	Open Graph URL.
`og_site_name`	`const char**`	`NULL`	Open Graph site name.
`og_locale`	`const char**`	`NULL`	Open Graph locale.
`og_video`	`const char**`	`NULL`	Open Graph video URL.
`og_audio`	`const char**`	`NULL`	Open Graph audio URL.
`og_locale_alternates`	`const char***`	`NULL`	Open Graph locale alternates.
`twitter_card`	`const char**`	`NULL`	Twitter card type.
`twitter_title`	`const char**`	`NULL`	Twitter title.
`twitter_description`	`const char**`	`NULL`	Twitter description.
`twitter_image`	`const char**`	`NULL`	Twitter image URL.
`twitter_site`	`const char**`	`NULL`	Twitter site handle.
`twitter_creator`	`const char**`	`NULL`	Twitter creator handle.
`dc_title`	`const char**`	`NULL`	Dublin Core title.
`dc_creator`	`const char**`	`NULL`	Dublin Core creator.
`dc_subject`	`const char**`	`NULL`	Dublin Core subject.
`dc_description`	`const char**`	`NULL`	Dublin Core description.
`dc_publisher`	`const char**`	`NULL`	Dublin Core publisher.
`dc_date`	`const char**`	`NULL`	Dublin Core date.
`dc_type`	`const char**`	`NULL`	Dublin Core type.
`dc_format`	`const char**`	`NULL`	Dublin Core format.
`dc_identifier`	`const char**`	`NULL`	Dublin Core identifier.
`dc_language`	`const char**`	`NULL`	Dublin Core language.
`dc_rights`	`const char**`	`NULL`	Dublin Core rights.
`article`	`CbergArticleMetadata*`	`NULL`	Article metadata from `article:*` Open Graph tags.
`hreflangs`	`CbergHreflangEntry**`	`NULL`	Hreflang alternate links.
`favicons`	`CbergFaviconInfo**`	`NULL`	Favicon and icon links.
`headings`	`CbergHeadingInfo**`	`NULL`	Heading elements (h1-h6).
`word_count`	`uintptr_t*`	`NULL`	Computed word count of the page body text.

CbergProxyConfig

Proxy configuration for HTTP requests.

Field	Type	Default	Description
`url`	`const char*`	—	Proxy URL (e.g. “http://proxy:8080", “socks5://proxy:1080”).
`username`	`const char**`	`NULL`	Optional username for proxy authentication.
`password`	`const char**`	`NULL`	Optional password for proxy authentication.

CbergResponseMeta

Response metadata extracted from HTTP headers.

Field	Type	Default	Description
`etag`	`const char**`	`NULL`	The ETag header value.
`last_modified`	`const char**`	`NULL`	The Last-Modified header value.
`cache_control`	`const char**`	`NULL`	The Cache-Control header value.
`server`	`const char**`	`NULL`	The Server header value.
`x_powered_by`	`const char**`	`NULL`	The X-Powered-By header value.
`content_language`	`const char**`	`NULL`	The Content-Language header value.
`content_encoding`	`const char**`	`NULL`	The Content-Encoding header value.

CbergScrapeResult

The result of a single-page scrape operation.

Field	Type	Default	Description
`status_code`	`uint16_t`	—	The HTTP status code of the response.
`final_url`	`const char*`	—	The final URL after following all redirects.
`content_type`	`const char*`	—	The Content-Type header value.
`html`	`const char*`	—	The HTML body of the response.
`body_size`	`uintptr_t`	—	The size of the response body in bytes.
`metadata`	`CbergPageMetadata`	—	Extracted metadata from the page.
`links`	`CbergLinkInfo*`	`NULL`	Links found on the page.
`images`	`CbergImageInfo*`	`NULL`	Images found on the page.
`feeds`	`CbergFeedInfo*`	`NULL`	Feed links found on the page.
`json_ld`	`CbergJsonLdEntry*`	`NULL`	JSON-LD entries found on the page.
`is_allowed`	`bool`	—	Whether the URL is allowed by robots.txt.
`crawl_delay`	`uint64_t*`	`NULL`	The crawl delay from robots.txt, in seconds.
`noindex_detected`	`bool`	—	Whether a noindex directive was detected.
`nofollow_detected`	`bool`	—	Whether a nofollow directive was detected.
`x_robots_tag`	`const char**`	`NULL`	The X-Robots-Tag header value, if present.
`is_pdf`	`bool`	—	Whether the content is a PDF.
`was_skipped`	`bool`	—	Whether the page was skipped (binary or PDF content).
`detected_charset`	`const char**`	`NULL`	The detected character set encoding.
`auth_header_sent`	`bool`	—	Whether an authentication header was sent with the request.
`response_meta`	`CbergResponseMeta*`	`NULL`	Response metadata extracted from HTTP headers.
`assets`	`CbergDownloadedAsset*`	`NULL`	Downloaded assets from the page.
`js_render_hint`	`bool`	—	Whether the page content suggests JavaScript rendering is needed.
`browser_used`	`bool`	—	Whether the browser fallback was used to fetch this page.
`markdown`	`CbergMarkdownResult*`	`NULL`	Markdown conversion of the page content.
`extracted_data`	`void**`	`NULL`	Structured data extracted by LLM. Populated when extraction is configured.
`extraction_meta`	`CbergExtractionMeta*`	`NULL`	Metadata about the LLM extraction pass (cost, tokens, model).
`screenshot_base64`	`const char**`	`NULL`	Base64-encoded PNG screenshot of the page. Populated only when `CrawlConfig.capture_screenshot` was enabled for this request, so callers that never requested a screenshot do not pay the encoding cost.
`downloaded_document`	`CbergDownloadedDocument*`	`NULL`	Downloaded non-HTML document (PDF, DOCX, image, code, etc.).
`browser`	`CbergBrowserExtras*`	`NULL`	Browser-specific extras (eval result, network events, cookies). Only populated when `BrowserBackend.Native` was used for this request.

CbergSitemapUrl

A URL entry from a sitemap.

Field	Type	Default	Description
`url`	`const char*`	—	The URL.
`lastmod`	`const char**`	`NULL`	The last modification date, if present.
`changefreq`	`const char**`	`NULL`	The change frequency, if present.
`priority`	`const char**`	`NULL`	The priority, if present.

CbergSsrfPolicy

SSRF policy configuration.

Field	Type	Default	Description
`deny_private`	`bool`	`true`	If true, reject URLs that resolve to private/metadata IP ranges.
`allowlist`	`CbergHostMatcher*`	`NULL`	Hostnames and IP ranges permitted regardless of `deny_private`. The allowlist is an override of `deny_private`, not an intersection with it. Precedence, in order: 1. `deny_private == false` permits everything; the allowlist is not consulted. 2. A hostname matching an `Exact` or `Suffix` entry is permitted immediately, before DNS resolution — so the deny-list is never applied to it. This trusts the host string: a name that resolves into private space is still permitted. 3. A literal or resolved IP inside a `Cidr` entry is permitted even though it is in the default deny-list. 4. Otherwise the default deny-list decides. An empty allowlist therefore denies nothing by itself — it simply leaves `deny_private` and the deny-list in sole control.
`max_redirects`	`uint8_t`	`5`	Maximum number of HTTP redirects to follow during validation.

Methods

cberg_default()

Signature:

CbergSsrfPolicy cberg_default();

Example:

CbergSsrfPolicy *result = cberg_default();

Returns: CbergSsrfPolicy

cberg_from_env()

Create a policy from environment variables.

On native platforms, reads CRAWLBERG_ALLOW_PRIVATE_NETWORK — if set to “1” or “true” (case-insensitive), sets deny_private = false. Otherwise, defaults to deny_private = true.

On wasm32 targets (browser/Node.js), environment variables are not accessible to the compiled module. Defaults to deny_private = false because:

Outbound requests in a browser go through the fetch API, which enforces its own network policies.
Rust-side SSRF checking is unenforceable and redundant in a wasm32 context.
For testing and localhost access, the host’s network sandbox is the enforcing boundary.

Node.js caveat: deny_private (whatever its value) has no effect on hostname-based requests under wasm32. There is no DNS resolution on this target, so validate_url only ever checks a literal IP host; a domain name falls straight through to Ok(()). In a browser this is covered by same-origin/CORS. Node’s fetch enforces no CORS, so a Node service embedding this wasm module can be driven to internal hosts by domain name even though deny_private = true. Do not rely on this policy to stop that in Node — enforce egress restrictions (network policy, firewall, proxy allowlist) outside the process.

Signature:

CbergSsrfPolicy cberg_from_env();

Example:

CbergSsrfPolicy *result = cberg_from_env();

Returns: CbergSsrfPolicy

Enums

CbergBrowserMode

When to use the headless browser fallback.

Value	Description
`CBERG_AUTO`	Automatically detect when JS rendering is needed and fall back to browser.
`CBERG_ALWAYS`	Always use the browser for every request.
`CBERG_NEVER`	Never use the browser fallback.
`CBERG_STEALTH`	Always use the browser with all stealth surfaces enabled. Behaves like `Always` for escalation purposes (every request is routed through the browser tier), but additionally enables: - browser JavaScript stealth patches - native-backend TLS fingerprint spoofing - stealth-aware default user-agent when no explicit UA is set - 1920×1080 viewport override Use this instead of setting the now-removed `BrowserConfig.stealth` boolean field.

CbergBrowserWait

Wait strategy for browser page rendering.

Value	Description
`CBERG_NETWORK_IDLE`	Wait until network activity is idle.
`CBERG_SELECTOR`	Wait for a specific CSS selector to appear in the DOM.
`CBERG_FIXED`	Wait for a fixed duration after navigation.

CbergBrowserBackend

Browser backend used for JavaScript rendering.

Value	Description
`CBERG_CHROMIUMOXIDE`	Existing Chromium/CDP backend powered by chromiumoxide.
`CBERG_NATIVE`	Crawlberg-owned native browser backend derived from Obscura.

CbergDocumentContentEncoding

Opt-in encoding applied to a downloaded document’s bytes for callers who need the content available in a serializable field rather than reading it from disk.

NULL (the CrawlConfig.document_content_encoding default) produces neither — unlike screenshots, base64-encoding a document by default would duplicate an already up-to-document_max_size buffer (50 MB default) in memory per document.

Value	Description
`CBERG_BASE64`	Populate `DownloadedDocument.content_base64` with a base64-encoded copy.

CbergAuthConfig

Authentication configuration.

Value	Description
`CBERG_BASIC`	HTTP Basic authentication. — Fields: `username`: `const char`, `password`: `const char`
`CBERG_BEARER`	Bearer token authentication. — Fields: `token`: `const char*`
`CBERG_HEADER`	Custom authentication header. — Fields: `name`: `const char`, `value`: `const char`

CbergLinkType

The classification of a link.

Value	Description
`CBERG_INTERNAL`	A link to the same domain.
`CBERG_EXTERNAL`	A link to a different domain.
`CBERG_ANCHOR`	A fragment-only link (e.g., `#section`).
`CBERG_DOCUMENT`	A link to a downloadable document (PDF, DOC, etc.).

CbergImageSource

The source of an image reference.

Value	Description
`CBERG_IMG`	An `<img>` tag.
`CBERG_PICTURE_SOURCE`	A `<source>` tag inside `<picture>`.
`CBERG_OG_IMAGE`	An `og:image` meta tag.
`CBERG_TWITTER_IMAGE`	A `twitter:image` meta tag.

CbergFeedType

The type of a feed (RSS, Atom, or JSON Feed).

Value	Description
`CBERG_RSS`	RSS feed.
`CBERG_ATOM`	Atom feed.
`CBERG_JSON_FEED`	JSON Feed.

CbergAssetCategory

The category of a downloaded asset.

Value	Description
`CBERG_DOCUMENT`	A document file (PDF, DOC, etc.).
`CBERG_IMAGE`	An image file.
`CBERG_AUDIO`	An audio file.
`CBERG_VIDEO`	A video file.
`CBERG_FONT`	A font file.
`CBERG_STYLESHEET`	A CSS stylesheet.
`CBERG_SCRIPT`	A JavaScript file.
`CBERG_ARCHIVE`	An archive file (ZIP, TAR, etc.).
`CBERG_DATA`	A data file (JSON, XML, CSV, etc.).
`CBERG_OTHER`	An unrecognized asset type.

CbergCrawlEvent

An event emitted during a streaming crawl operation.

Not available on wasm32 targets — streaming requires native concurrency primitives (tokio channels, JoinSet) that are not supported on wasm32.

Delivered to bindings through each target’s native streaming idiom.

Value	Description
`CBERG_PAGE`	A single page has been crawled. — Fields: `result`: `CbergCrawlPageResult`
`CBERG_ERROR`	An error occurred while crawling a URL. — Fields: `url`: `const char`, `error`: `const char`
`CBERG_COMPLETE`	The crawl has completed. — Fields: `pages_crawled`: `uintptr_t`

CbergPageAction

A single page interaction action.

Actions are serialized with a type tag using camelCase naming, except ExecuteJs which is explicitly renamed to "executeJs".

Value	Description
`CBERG_CLICK`	Click on an element matching the given CSS selector. — Fields: `selector`: `const char*`
`CBERG_TYPE_TEXT`	Type text into an element matching the given CSS selector. — Fields: `selector`: `const char`, `text`: `const char`
`CBERG_PRESS`	Press a keyboard key (e.g. “Enter”, “Tab”, “Escape”). — Fields: `key`: `const char*`
`CBERG_SCROLL`	Scroll the page or a specific element. — Fields: `direction`: `CbergScrollDirection`, `selector`: `const char*`, `amount`: `int64_t`
`CBERG_WAIT`	Wait for a duration or for an element to appear. — Fields: `milliseconds`: `int64_t`, `selector`: `const char*`
`CBERG_SCREENSHOT`	Take a screenshot of the current page. — Fields: `full_page`: `bool`
`CBERG_EXECUTE_JS`	Execute arbitrary JavaScript in the page context. Safety: The script runs with full page privileges in the browser context. Only execute scripts from trusted sources. — Fields: `script`: `const char*`
`CBERG_SCRAPE`	Scrape the current page HTML.

CbergScrollDirection

Direction for a scroll action.

Value	Description
`CBERG_UP`	Scroll upward.
`CBERG_DOWN`	Scroll downward.

CbergHostMatcher

Hostname/IP allowlist matcher for SSRF policy.

Serializes as an internally-tagged object so each variant is distinguishable on the wire and round-trips losslessly:

{"type": "exact",  "value": "api.example.com"}
{"type": "suffix", "value": ".example.com"}
{"type": "cidr",   "value": "10.0.0.0/8"}

A bare JSON string is still accepted on deserialization and resolves to Exact, preserving configs written against the previous untagged representation.

Exact: HostMatcher.Exact

Value	Description
`CBERG_EXACT`	Exact hostname match (case-insensitive). — Fields: `value`: `const char*`
`CBERG_SUFFIX`	Suffix match: “.xberg.io” matches “api.xberg.io” and “xberg.io”. — Fields: `value`: `const char*`
`CBERG_CIDR`	CIDR match: “10.0.0.0/8” matches IP addresses in that range. — Fields: `value`: `const char*`

Errors

CbergCrawlError

Errors that can occur during crawling, scraping, or mapping operations.

Variant	Description
`CBERG_NOT_FOUND`	The requested page was not found (HTTP 404).
`CBERG_UNAUTHORIZED`	The request was unauthorized (HTTP 401).
`CBERG_FORBIDDEN`	The request was forbidden (HTTP 403).
`CBERG_WAF_BLOCKED`	The request was blocked by a WAF or bot protection (HTTP 403 with WAF indicators). `vendor` is the lowercase identifier of the detected WAF (e.g. “cloudflare”, “datadome”). When the engine cannot identify the vendor, it uses “unknown”. `message` is the freeform description for logs and human readers. The stable error tag remains `forbidden: waf/blocked: MESSAGE` so existing log-grep patterns and cross-language bindings continue to work; vendor is surfaced separately for structured consumers.
`CBERG_TIMEOUT`	The request timed out.
`CBERG_RATE_LIMITED`	The request was rate-limited (HTTP 429).
`CBERG_SERVER_ERROR`	A server error occurred (HTTP 5xx).
`CBERG_BAD_GATEWAY`	A bad gateway error occurred (HTTP 502).
`CBERG_GONE`	The resource is permanently gone (HTTP 410).
`CBERG_CONNECTION`	A connection error occurred.
`CBERG_DNS`	A DNS resolution error occurred.
`CBERG_SSL`	An SSL/TLS error occurred.
`CBERG_DATA_LOSS`	Data was lost or truncated during transfer.
`CBERG_BROWSER_ERROR`	The browser failed to launch, connect, or navigate.
`CBERG_BROWSER_TIMEOUT`	The browser page load or rendering timed out.
`CBERG_INVALID_CONFIG`	The provided configuration is invalid.
`CBERG_UNSUPPORTED`	The requested capability is not supported by the active backend or build.
`CBERG_SSRF_POLICY_VIOLATION`	A URL was rejected by SSRF policy (private IP, metadata, disallowed scheme, etc).
`CBERG_OTHER`	An unclassified error occurred.

CbergSsrfError

SSRF validation error.

Variant	Description
`CBERG_DENIED_BY_POLICY`	URL denied by SSRF policy: private IP, metadata IP, etc.
`CBERG_NOT_ON_ALLOWLIST`	Host not on allowlist when an allowlist is configured.
`CBERG_INVALID_CIDR`	Allowlist entry is not a parseable CIDR block.
`CBERG_DNS_RESOLUTION_FAILED`	DNS resolution failed for hostname.
`CBERG_INVALID_URL`	Invalid URL format.
`CBERG_DISALLOWED_SCHEME`	URL scheme not in allowlist (e.g., `ftp://` when only `http`/`https` allowed).
`CBERG_TOO_MANY_REDIRECTS`	Too many HTTP redirects encountered during validation.