Scrapes and extracts structured data from any web page. Below is the code example.
### Python ### from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 client = ApifyClient("<YOUR_API_TOKEN>") # Prepare the Actor input run_input = { "endpoint": "scrape", "url": "https://www.amazon.com/Cordless-Variable-Position-Masterworks-MW316/dp/B07CR1GPBQ/", "fields": { "name": "", "rating": "", "price": "", "brand": "", "key_selling_points": [] } } # Run the Actor and wait for it to finish run = client.actor("zeeb0t/web-scraping-api---scrape-any-website").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) ### JavaScript ### import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '<YOUR_API_TOKEN>' with your token // Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 const client = new ApifyClient({ token: '<YOUR_API_TOKEN>', }); // Prepare Actor input const input = { "endpoint": "scrape", "url": "https://www.amazon.com/Cordless-Variable-Position-Masterworks-MW316/dp/B07CR1GPBQ/", "fields": { "name": "", "rating": "", "price": "", "brand": "", "key_selling_points": [] } }; // Run the Actor and wait for it to finish const run = await client.actor("zeeb0t/web-scraping-api---scrape-any-website").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); ### HTTP ### # Set API token # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 API_TOKEN=<YOUR_API_TOKEN> # Prepare Actor input cat > input.json << 'EOF' { "endpoint": "scrape", "url": "https://www.amazon.com/Cordless-Variable-Position-Masterworks-MW316/dp/B07CR1GPBQ/", "fields": { "name": "", "rating": "", "price": "", "brand": "", "key_selling_points": [] } } EOF # Run the Actor using an HTTP API # See the full API reference at https://docs.apify.com/api/v2 curl "https://api.apify.com/v2/acts/zeeb0t~web-scraping-api---scrape-any-website/run-sync-get-dataset-items?token=$API_TOKEN" \ -X POST \ -d @input.json \ -H 'Content-Type: application/json' ### MCP ### # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 { "mcpServers": { "web-scraping-api-by-instantapi-ai": { "command": "npx", "args": [ "-y", "mcp-remote", "https://web-scraping-api-by-instantapi-ai.help-052.workers.dev/sse", "--header", "Authorization:${AUTH_HEADER}" ], "env": { "AUTH_HEADER": "Bearer <YOUR_API_TOKEN>" } } } }
{ "scrape": < The populated JSON object that matches the structure you provided. >, "markdown": "< Markdown of the page which can be optionally saved for further analysis. >", "html": "< HTML of the page which can be optionally saved for further analysis. >" }
Scrapes and extracts links matching a description from any web page. Below is the code example.
### Python ### from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 client = ApifyClient("<YOUR_API_TOKEN>") # Prepare the Actor input run_input = { "endpoint": "links", "url": "https://www.ikea.com/au/en/cat/quilt-cover-sets-10680/?page=3", "description": "individual product urls" } # Run the Actor and wait for it to finish run = client.actor("zeeb0t/web-scraping-api---scrape-any-website").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) ### JavaScript ### import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '<YOUR_API_TOKEN>' with your token // Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 const client = new ApifyClient({ token: '<YOUR_API_TOKEN>', }); // Prepare Actor input const input = { "endpoint": "links", "url": "https://www.ikea.com/au/en/cat/quilt-cover-sets-10680/?page=3", "description": "individual product urls" }; // Run the Actor and wait for it to finish const run = await client.actor("zeeb0t/web-scraping-api---scrape-any-website").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); ### HTTP ### # Set API token # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 API_TOKEN=<YOUR_API_TOKEN> # Prepare Actor input cat > input.json << 'EOF' { "endpoint": "links", "url": "https://www.ikea.com/au/en/cat/quilt-cover-sets-10680/?page=3", "description": "individual product urls" } EOF # Run the Actor using an HTTP API # See the full API reference at https://docs.apify.com/api/v2 curl "https://api.apify.com/v2/acts/zeeb0t~web-scraping-api---scrape-any-website/run-sync-get-dataset-items?token=$API_TOKEN" \ -X POST \ -d @input.json \ -H 'Content-Type: application/json' ### MCP ### # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 { "mcpServers": { "web-scraping-api-by-instantapi-ai": { "command": "npx", "args": [ "-y", "mcp-remote", "https://web-scraping-api-by-instantapi-ai.help-052.workers.dev/sse", "--header", "Authorization:${AUTH_HEADER}" ], "env": { "AUTH_HEADER": "Bearer <YOUR_API_TOKEN>" } } } }
{ "links": [< An array of URLs that match the description you provided. >], "markdown": "< Markdown of the page which can be optionally saved for further analysis. >", "html": "< HTML of the page which can be optionally saved for further analysis. >" }
Scrapes and extracts the 'next page' links from any web page with pagination. Below is the code example.
### Python ### from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 client = ApifyClient("<YOUR_API_TOKEN>") # Prepare the Actor input run_input = { "endpoint": "next", "url": "https://www.ikea.com/au/en/cat/quilt-cover-sets-10680/" } # Run the Actor and wait for it to finish run = client.actor("zeeb0t/web-scraping-api---scrape-any-website").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) ### JavaScript ### import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '<YOUR_API_TOKEN>' with your token // Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 const client = new ApifyClient({ token: '<YOUR_API_TOKEN>', }); // Prepare Actor input const input = { "endpoint": "next", "url": "https://www.ikea.com/au/en/cat/quilt-cover-sets-10680/" }; // Run the Actor and wait for it to finish const run = await client.actor("zeeb0t/web-scraping-api---scrape-any-website").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); ### HTTP ### # Set API token # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 API_TOKEN=<YOUR_API_TOKEN> # Prepare Actor input cat > input.json << 'EOF' { "endpoint": "next", "url": "https://www.ikea.com/au/en/cat/quilt-cover-sets-10680/" } EOF # Run the Actor using an HTTP API # See the full API reference at https://docs.apify.com/api/v2 curl "https://api.apify.com/v2/acts/zeeb0t~web-scraping-api---scrape-any-website/run-sync-get-dataset-items?token=$API_TOKEN" \ -X POST \ -d @input.json \ -H 'Content-Type: application/json' ### MCP ### # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 { "mcpServers": { "web-scraping-api-by-instantapi-ai": { "command": "npx", "args": [ "-y", "mcp-remote", "https://web-scraping-api-by-instantapi-ai.help-052.workers.dev/sse", "--header", "Authorization:${AUTH_HEADER}" ], "env": { "AUTH_HEADER": "Bearer <YOUR_API_TOKEN>" } } } }
{ "next": [< An array of all matched 'next page' URLs. >], "markdown": "< Markdown of the page which can be optionally saved for further analysis. >", "html": "< HTML of the page which can be optionally saved for further analysis. >" }
Scrapes and extracts relevant URLs from Google search results pages. Below is the code example.
### Python ### from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 client = ApifyClient("<YOUR_API_TOKEN>") # Prepare the Actor input run_input = { "endpoint": "search", "google_domain": "www.google.com", "query": "AVID POWER 20V MAX Lithium Ion Cordless Drill Set", "page": 1 } # Run the Actor and wait for it to finish run = client.actor("zeeb0t/web-scraping-api---scrape-any-website").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) ### JavaScript ### import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '<YOUR_API_TOKEN>' with your token // Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 const client = new ApifyClient({ token: '<YOUR_API_TOKEN>', }); // Prepare Actor input const input = { "endpoint": "search", "google_domain": "www.google.com", "query": "AVID POWER 20V MAX Lithium Ion Cordless Drill Set", "page": 1 }; // Run the Actor and wait for it to finish const run = await client.actor("zeeb0t/web-scraping-api---scrape-any-website").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); ### HTTP ### # Set API token # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 API_TOKEN=<YOUR_API_TOKEN> # Prepare Actor input cat > input.json << 'EOF' { "endpoint": "search", "google_domain": "www.google.com", "query": "AVID POWER 20V MAX Lithium Ion Cordless Drill Set", "page": 1 } EOF # Run the Actor using an HTTP API # See the full API reference at https://docs.apify.com/api/v2 curl "https://api.apify.com/v2/acts/zeeb0t~web-scraping-api---scrape-any-website/run-sync-get-dataset-items?token=$API_TOKEN" \ -X POST \ -d @input.json \ -H 'Content-Type: application/json' ### MCP ### # Replace '<YOUR_API_TOKEN>' with your token. # Sign up for your own API token here: https://web.instantapi.ai/#pricing-03-254921 { "mcpServers": { "web-scraping-api-by-instantapi-ai": { "command": "npx", "args": [ "-y", "mcp-remote", "https://web-scraping-api-by-instantapi-ai.help-052.workers.dev/sse", "--header", "Authorization:${AUTH_HEADER}" ], "env": { "AUTH_HEADER": "Bearer <YOUR_API_TOKEN>" } } } }
{ "search": [< An array of relevant URLs for any of the promoted and organic search results. >], "markdown": "< Markdown of the page which can be optionally saved for further analysis. >", "html": "< HTML of the page which can be optionally saved for further analysis. >" }
How to handle the error response code and message that can be returned.
{ "error": true, "reason": "Missing required parameters. Please check and try again with required parameters." }