During the automation of multiple tasks on my job and personal projects, i decided to move on Puppeteer instead of the old school PhantomJS. One of the most usual problems with pages that contain a lot of content, because of the ads, images etc. is the load time, an exception is thrown (specifically the TimeoutError) after a page takes more than 30000ms (30 seconds) to load totally.
To solve this problem, you will have 2 options, either to increase this timeout in the configuration or remove it at all. Personally, i prefer to remove the limit as i know that the pages that i work with will end up loading someday.
In this article, i'll explain you briefly 2 ways to bypass this limitation.
A. Globally on the tab
The option that i prefer, as i browse multiple pages in the same tab, is to remove the timeout limit on the tab that i use to browse. For example, to remove the limit you should add:
await page.setDefaultNavigationTimeout(0); The setDefaultNavigationTimeout method available on a created page of Puppeteer allows you to define the timeout of the tab and expects as first argument, the value in milliseconds. A value of 0 means an unlimited amount of time. The following snippet shows how you can do it in a real example:
// Require puppeteer
const puppeteer = require('puppeteer');
(async () => {
// Create an instance of the chrome browser
// But disable headless mode !
const browser = await puppeteer.launch({
headless: false
});
// Create a new page
const page = await browser.newPage();
// Configure the navigation timeout
await page.setDefaultNavigationTimeout(0);
// Navigate to some website e.g Our Code World
await page.goto('http://ourcodeworld.com');
// Do your stuff
// ...
})();B. Specifically on the current page
Alternatively, for specifical pages in case that you handle multiple pages on different variables, you should be able to specify the limit on the context as an option in the configuration object of the page.goto method:
await page.goto('https://ourcodeworld.com', {
waitUntil: 'load',
// Remove the timeout
timeout: 0
});The following snippet shows how to do it in a real example:
// Require puppeteer
const puppeteer = require('puppeteer');
(async () => {
// Create an instance of the chrome browser
// But disable headless mode !
const browser = await puppeteer.launch({
headless: false
});
// Create a new page
const page = await browser.newPage();
// Configure the navigation timeout
await page.goto('https://ourcodeworld.com', {
waitUntil: 'load',
// Remove the timeout
timeout: 0
});
// Navigate to some website e.g Our Code World
await page.goto('http://ourcodeworld.com');
// Do your stuff
// ...
})();Happy coding !
