Overview

Query Parameters

PARAMETER	TYPE	DEFAULT	DESCRIPTION
apiKey	string	Get your free api key
url	string		The URL you want to scrape
render	boolean	false	Render the JavaScript on the page with a headless browser.
config.window_width	number	1920	Set the width of the browser
config.window_height	number	1080	Set the height of the browser
screenshot	boolean	false	This parameter is used to take a screenshot of the visible area once the API processing is complete
screenshot_full_page	boolean	false	This parameter is used to take a screenshot of the entire website when the API processing is complete
instructions	object[]		Give instructions to scrape automate, to do stuff such as clicking, waiting, fillup input fields etc...
proxy_type	string		Select the proxy type for better anonymity, performance and speed
proxy_country	string		Select the geographical location of your proxy, which country should the proxy come from.
workflow_id	string		This parameter is used to reuse configurations using api workflow!
webhook_url	string		This parameter is used for setting a webhook where the response will be sent to.

Getting Started

To start using our api, you need 2 required parameters. First is the apiKey, you can get the apiKey after setting up your account. The other required parameter is the url parameter, this is the website you want to scrape.

API key

All the requests you make are authorized by using apiKey, this parameter is required to allow access to our API. You can get your apikey after creating an account on ScrapeAutomate.

Axios

const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Curl

curl --request GET \
--url 'https://api.scrapeautomate.com/scraper?apiKey=<exampleToken>'

Target URL

The URL is the page you want to scrape. It needs to be encoded. Some HTTP clients will do that automatically for you.

Axios

const axios = require('axios').default;
const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    url: 'https://example.com/',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Curl

curl --request GET \
--url 'https://api.scrapeautomate.com/scraper?url=https%3A%2F%2Fexample.com%2F'

Render

The render parameter is to enable javascript rendering and opening a browser so you can scrape. Render can be either true or false. Enabling render allows you to do many other things such as wait for a selector, take screenshots, give javascript instructions to the scraper, and more! Enabling render cost more credits than having it disabled.

Axios

const axios = require('axios').default;
const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    render: 'true',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Curl

curl --request GET \
  --url 'https://api.scrapeautomate.com/scraper?render=true'

Full Process