datapulse_scrape_bulk
datapulse:scrapeRead-onlySubmit up to 250 URLs for scraping per request. Asynchronous: the answer is a receipt, not scrape results.
Description
Submit up to 250 URLs for scraping per request. Asynchronous: the answer is a receipt, not scrape results.
Scrapes are public by design: each is a fetch of a public web page, kept in one cache shared by all users, so every result is readable by every user.
LOWER PRIORITY: bulk work runs behind interactive submissions, can take hours during large background runs, and pauses while the extraction model and its overflow capacity are both unavailable. A long-pending URL has not failed, and resubmitting the batch does not make it run sooner. When someone is waiting, datapulse_scrape_submit (wait=true) per URL or datapulse_domain_overview is faster; one URL from a batch is rushed with datapulse_scrape_submit(url=…, force=true, wait=true).
Each item is {url, force?}. Receipt: accepted, cached, rejected (the API’s counts over the URLs sent); sent (index, url as sent, url_hash) for every URL sent; rejected_urls (refused by the API); rejected_local (refused before the request: a malformed item, blank, not a web URL, credentials; NOT counted in rejected); duplicates (index, first_index; sent once); errors. accepted + cached + rejected + len(rejected_local) + len(duplicates) equals the number of URLs. When no item can be sent, nothing is sent and the call is an error whose text is this receipt: every item in rejected_local, the counts 0, the other lists [], and a note.
Results: sent[].url_hash is the key for every URL sent, accepted or cached. datapulse_scrape_bulk_results(url_hashes=[sent[].url_hash, …]) reads them all in one call; its counts.by_status (pending, processing, completed, failed, over every result found) shows how far the batch has got, so the same call repeated with backoff polls the batch.
Reference: datapulse_help(topic="scraping")
Parameters
| Parameter | Type | Description |
|---|---|---|
urlsrequired | object[] | Array of URLs to scrape. Maximum 250 per request. Min items: 1Max items: 250 |
urls[].urlrequired | string | The URL to scrape. Min length: 1Max length: 2048 |
urls[].force | boolean | Bypass 90-day cache. |
Input schema (JSON)
{
"additionalProperties": false,
"properties": {
"urls": {
"description": "Array of URLs to scrape. Maximum 250 per request.",
"items": {
"additionalProperties": false,
"properties": {
"force": {
"description": "Bypass 90-day cache.",
"type": "boolean"
},
"url": {
"description": "The URL to scrape.",
"maxLength": 2048,
"minLength": 1,
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
},
"maxItems": 250,
"minItems": 1,
"type": "array"
}
},
"required": [
"urls"
],
"type": "object"
}Generated from the live server (DataPulse MCP v0.9-341-g27163c2) on October 20, 2026.