I have a Puppeteer scraping script that successfully fetches my target data. Now, I want to post the data in my Google Apps Script web app and read it with doPost()
in GAS. I expect to see the data in my Google Sheets file but instead I get the following error.
TimeoutError: Navigation timeout of 30000 ms exceeded at /Users/path/to/node_modules/puppeteer/lib/LifecycleWatcher.js:142:21 -- ASYNC -- at Frame. (/Users/path/to/node_modules/puppeteer/lib/helper.js:111:15) at Page.goto (/Users/path/to/node_modules/puppeteer/lib/Page.js:675:49) at Page. (/Users/path/to/node_modules/puppeteer/lib/helper.js:112:23) at /Users/path/to/proto.js:215:37 at processTicksAndRejections (internal/process/task_queues.js:93:5) { name: 'TimeoutError' }
What am I doing wrong?
puppeteer.js const pagePost = await browser.newPage();
...
postUrl = 'https://script.google.com/macros/s/abcdexyz123/exec' // GAS endpoint, tested and working
itemsAsCsv = // csv data
...
// Allows you to intercept a request; must appear before
// your first page.goto()
await pagePost.setRequestInterception(true);
// Request intercept handler... will be triggered with
// each page.goto() statement
pagePost.on('request', interceptedRequest => {
// Here, is where you change the request method and
// add your post data
var data = {
method: POST,
postData: itemsAsCsv,
};
// Request modified... finish sending!
interceptedRequest.continue(data);
});
// Navigate, trigger the intercept, and resolve the response
try {
const response = await pagePost.goto(postUrl);
const responseBody = await response.text();
console.log(responseBody);
} catch(e) {
console.log(e)
}
// // Close the browser - done!
// await browser.close();
// end POST
await browser.close();