Skip to main content

Javascript Rendering

Overview​

In this section, you will learn how to use JavaScript rendering in our scraper API. This includes waiting for specific elements, adjusting window dimensions, and interacting with elements.

Window Width​

To specify the width of the window of the website you want to scrape data from, you can use the config.window_width property. By default the value is set to 1920.


const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'true',
    url: 'https://example.com/',
    config: '{"window_width":1920}',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Window Height​

To specify the height of the window, you can use the config.window_height property. By default the value is set to 1080.


const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'true',
    url: 'https://example.com/',
    config: '{"window_height":1080}',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Wait for a specific selector​

If you would like to wait for a specific element based on a selector you can use wait inside instructions. This requires for render to be true.


const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'true',
    url: 'https://example.com/',
    instructions: '[{"wait":".test"}]',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Delay​

To add delays inbetween instructions for example to wait 1 second before clicking on an element you can use the delay instruction. Delay is taken as millisecond.


const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'true',
    url: 'https://example.com/',
    instructions: '[{"delay":1000},{"click":".submit"}]'
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Click on an element​

To click on an element, use the click property with the desired selector.


const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'true',
    url: 'https://example.com/',
    instructions: '[{"click":".search-submit"}]',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Fill in an input field​

To fill in an input field you can push an object with the fill property inside the instructions parameter.

{
  selector: string; // The valid input field css selector
  text: string; // The value text you want to input in that field
  delay: number; // The number in ms of type speed
}

const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'true',
    url: 'https://example.com/',
    instructions: '[{"fill":{"selector":".search-field","text":"foo bar","delay":1000}}]',
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}

Block resources​

If you would like to block resources such as stylesheets, images, media, javascript files, fonts, etc from loading while doing automation either to make your scraping faster or to stop loading unnessecary things from loading you can use the block_resources query parameter which allows you to block any kind of resources from appearing. If you like to block multiple resources from loading, you need to sepearate them with an , make sure to add the empty space otherwise it might not work! here is an demo:

const axios = require('axios').default;

const options = {
  method: 'GET',
  url: 'https://api.scrapeautomate.com/scraper',
  params: {
    apiKey: '<exampleToken>',
    render: 'false',
    url: 'https://example.com/',
    block_resources: 'image, stylesheet, font, texttrack'
  },
};

try {
  const { data } = await axios.request(options);
  console.log(data);
} catch (error) {
  console.error(error);
}