Skip to main content

Output Types

Plain HTML​

By default, the scraper API returns plain HTML without requiring any additional parameters. This does not require javascript rendering to be enabled! Here are some examples:

Request​


const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'false',
    url: 'https://example.com/',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Screenshot​

Full Page Screenshot​

To capture a full-page screenshot, set the screenshot_full_page parameter to true. This captures the entire length of the web page. This requires for render to be enabled!


const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'true',
    url: 'https://example.com/',
    screenshot_full_page: 'true',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Viewport Screenshot​

To capture a screenshot of the current viewport or how much the current user can see, set the screenshot parameter to true. This captures only the visible part of the web page within the viewport. This requires for render to be enabled!


const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'true',
    url: 'https://example.com/',
    screenshot: 'true',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Webhook​

If you would like for your responses to be sent to a webhook rather then directly responding with the response then you can use the webhook webhook_url property. You can use this on addition to screenshots or html. To use this just pass your webhook_url in the query parameters and the response from the api will be sent to that webhook. Here is an demo:

const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'true',
    url: 'https://example.com/',
    screenshot: 'true',
    webhook_url: "https://webhook.site/7132edb4-8590-4bc0-9055-30e2153936d1"
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Now what happens is that when you hit that API it will automatically send the response to that api rather then responding with the response data to person who requested it, but you will still get a response saying weather or not if we were able to successfully hit the webhook like this:

{
  "status": "ok",
  "message": "Webhook sent successfully"
}

This also works with api workflows, just make sure to pass it with the query parameters when either editing or creating the webhook!