MCP documentation menu

datapulse_scrape_submit

OtherScope: datapulse:scrapeRead-only

Scrape one URL: render it in a headless browser and extract entity data with an LLM.

Description

Scrape one URL: render it in a headless browser and extract entity data with an LLM.

Scrapes are public by design: each is a fetch of a public web page, kept in one cache shared by all users, so every result is readable by every user.

Interactive work: single submissions are processed ahead of datapulse_scrape_bulk. For domain research, datapulse_domain_overview combines scrape, DNS and RDAP in one call; this tool covers specific URL paths.

wait=true blocks until the scrape finishes (wait_timeout, default and max 50 s) and returns the result row. A scrape still running at the timeout answers with its in_flight receipt (url_hash) and a note, not an error. Without wait=true the answer is a receipt {status, job_id, domain_hash, url_hash} with status queued or in_flight; the row is read later with datapulse_scrape_result(url_hash=…), and a scrape takes 5-30 s.

In either mode a stored row comes back inline (a success within 90 days: status “cached” with cached=true; a failure within 7 days keeps “failed”). Only a scrape that ran for this call reports “completed”. force=true scrapes again. A returned row has datapulse_scrape_result’s fields without the DOM; link_data selects its link structures.

The URL is fetched as given (www. and scheme kept; no scheme tries https, then http). Storage is keyed on url_hash, which every scheme and www. variant shares, so the row’s url says which variant was fetched. URLs with credentials or an IP address as host are refused.

Reference: datapulse_help(topic="scraping")

Parameters

ParameterTypeDescription
url
required
string
The URL to scrape, fetched as given (www. and scheme kept; without a scheme, https then http).
Min length: 1Max length: 2048
forceboolean
Scrape again even when a result from the last 90 days is stored. Default false.
Default: false
link_datastring
Which link structures a returned row carries: none, domains (link_domains; the default), links (the links array, 300 records per page) or both.
Default: "domains"Max length: 7
nonedomainslinksboth
waitboolean
Wait for the scrape to finish and return the result row. Default false.
Default: false
wait_timeoutinteger
Max seconds to wait in sync mode. Default 50, max 50 (larger values are capped). Only used when wait=true.
Default: 50
Input schema (JSON)
{
  "additionalProperties": false,
  "properties": {
    "force": {
      "default": false,
      "description": "Scrape again even when a result from the last 90 days is stored. Default false.",
      "type": "boolean"
    },
    "link_data": {
      "default": "domains",
      "description": "Which link structures a returned row carries: none, domains (link_domains; the default), links (the links array, 300 records per page) or both.",
      "enum": [
        "none",
        "domains",
        "links",
        "both"
      ],
      "maxLength": 7,
      "type": "string"
    },
    "url": {
      "description": "The URL to scrape, fetched as given (www. and scheme kept; without a scheme, https then http).",
      "maxLength": 2048,
      "minLength": 1,
      "type": "string"
    },
    "wait": {
      "default": false,
      "description": "Wait for the scrape to finish and return the result row. Default false.",
      "type": "boolean"
    },
    "wait_timeout": {
      "default": 50,
      "description": "Max seconds to wait in sync mode. Default 50, max 50 (larger values are capped). Only used when wait=true.",
      "type": "integer"
    }
  },
  "required": [
    "url"
  ],
  "type": "object"
}

Generated from the live server (DataPulse MCP v0.9-341-g27163c2) on October 20, 2026.