I would like to load random list of user-agents from my default location path , for example: 'agents.json' instead of adding direct and only 1 user-agent.
agents.json
["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36", "Mozilla/5.0 (iPad; CPU OS 13_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/79.0.3945.73 Mobile/15E148 Safari/604.1"]My code:
const puppeteer = require('../core/puppeteer');const referers = require('../core/referers.json');const referers = require('../core/agents.json');const viewVideosInBatch = async ({ targetUrls, durationInSeconds, port }) => {let browser;try {browser = await puppeteer.getBrowserInstance(port);const randomReferer = referers[Math.floor(Math.random() * referers.length)];const randomAgents = agents[Math.floor(Math.random() * agents.length)];const page = await browser.newPage();browser = await page.setUserAgent({ agents: randomAgents });page.setDefaultTimeout(PAGE_DEFAULT_TIMEOUT * 1000);page.on('error', handlePageCrash(page));page.on('pageerror', handlePageCrash(page));page.setExtraHTTPHeaders({ referer: randomReferer });await page.setViewport({width: 640,height: 480,deviceScaleFactor: 1,});const ipAddr = await getCurrentIP(page);const targetUrlsForAction = _take(_shuffle(targetUrls), VIEW_ACTION_COUNT);await watchVideosInSequence(page, ipAddr, targetUrlsForAction, durationInSeconds);await page.close();} catch (error) {logger.warn('Entire view action in a batch failed.');logger.debug(error);} finally {await browser.close();}};ERROR I'm getting:
(node:1) UnhandledPromiseRejectionWarning: Error: Protocol error (Network.setUserAgentOverride): Target closed.view_4 | at /app/node_modules/puppeteer/lib/Connection.js:183:56view_4 | at new Promise (<anonymous>)view_4 | at CDPSession.send (/app/node_modules/puppeteer/lib/Connection.js:182:12)view_4 | at next (/app/node_modules/puppeteer-extra-plugin-stealth/evasions/sourceurl/index.js:30:43)view_4 | at CDPSession.send (/app/node_modules/puppeteer-extra-plugin-stealth/evasions/sourceurl/index.js:46:18)view_4 | at Plugin.onPageCreated (/app/node_modules/puppeteer-extra-plugin-stealth/evasions/user-agent-override/index.js:75:18)view_4 | at runMicrotasks (<anonymous>)view_4 | at processTicksAndRejections (internal/process/task_queues.js:97:5)view_4 | at async Plugin._onTargetCreated (/app/node_modules/puppeteer-extra-plugin/dist/index.cjs.js:491:17)view_4 | (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 79)Best Answer
You seem to redeclare browser when you set the UA, which you shouldn't. Also you don't need to set it as an object: as page.setUserAgent accept a string: page.setUserAgent(userAgent).
await page.setUserAgent(randomAgent); will be just fine.
const browser = await puppeteer.getBrowserInstance(port);const randomReferer = referers[Math.floor(Math.random() * referers.length)];const randomAgent = agents[Math.floor(Math.random() * agents.length)]; // made it singular instead of pluralconst page = await browser.newPage();await page.setUserAgent(randomAgent); // like thispage.setDefaultTimeout(PAGE_DEFAULT_TIMEOUT * 1000);page.on('error', handlePageCrash(page));page.on('pageerror', handlePageCrash(page));page.setExtraHTTPHeaders({ referer: randomReferer });Note: I think it is only a mistake in your post, but you also have two const named referers, this one should be the agents:
const referers = require('../core/agents.json');