How to Hide Cookie Banners with Puppeteer

Let's face it, cookie consent banners are annoying, especially when you're trying to take clean screenshots with Puppeteer. In this guide, I'll show you how to automatically handle these pesky banners and get perfect screenshots every time.

Installing Required Dependencies

Before we dive into the code examples, you'll need to install the necessary packages:

# Install Puppeteer
npm install puppeteer

# Install AdBlocker for Puppeteer
npm install @cliqz/adblocker-puppeteer

# Install AutoConsent (optional, for Method 2)
npm install @duckduckgo/autoconsent

Method 1: Using AdBlocker Filter Lists

One of the most efficient ways to handle cookie banners is by leveraging existing filter lists designed for ad blockers. The @cliqz/adblocker-puppeteer package provides an excellent solution:

const puppeteer = require('puppeteer');
const { PuppeteerBlocker } = require('@cliqz/adblocker-puppeteer');

(async () => {
	const browser = await puppeteer.launch();
	try {
		const page = await browser.newPage();

		// Initialize the blocker with cookie banner filter list
		const blocker = await PuppeteerBlocker.fromLists(fetch, [
			'https://secure.fanboy.co.nz/fanboy-cookiemonster.txt',
		]);

		// Enable blocking in the page
		await blocker.enableBlockingInPage(page);

		await page.goto('https://example.com');
		await page.screenshot({ path: 'screenshot.png' });
	} catch (error) {
		console.error('Error:', error);
	} finally {
		await browser.close();
	}
})();

This approach is particularly effective because:

• It uses community-maintained filter lists
• It's regularly updated with new cookie banner patterns
• It's lightweight and efficient
• It works across multiple websites

Method 2: Using AutoConsent

For more complex scenarios, we can combine the filter list approach with AutoConsent, a tool specifically designed to handle cookie consent dialogs:

const puppeteer = require('puppeteer');
const { PuppeteerBlocker } = require('@cliqz/adblocker-puppeteer');
const autoconsent = require('@duckduckgo/autoconsent');

(async () => {
	const browser = await puppeteer.launch();
	try {
		const page = await browser.newPage();

		// Initialize blocker
		const blocker = await PuppeteerBlocker.fromLists(fetch, [
			'https://secure.fanboy.co.nz/fanboy-cookiemonster.txt',
		]);
		await blocker.enableBlockingInPage(page);

		// Inject AutoConsent
		await page.evaluateOnNewDocument(autoconsent.script);

		await page.goto('https://example.com');
		await page.screenshot({ path: 'screenshot.png' });
	} catch (error) {
		console.error('Error:', error);
	} finally {
		await browser.close();
	}
})();

Method 3: Custom Cookie Banner Handler

For maximum control, you can implement a custom handler that combines multiple approaches:

const puppeteer = require('puppeteer');

async function handleCookieBanners(page) {
	// Common cookie banner selectors
	const selectors = [
		"[id*='cookie']",
		"[class*='cookie']",
		"[id*='consent']",
		"[class*='consent']",
		"[id*='gdpr']",
		"[class*='gdpr']",
		'.modal__overlay',
		'.cookie-banner',
		'.consent-banner',
	];

	// Common accept button texts
	const acceptTexts = [
		'Accept',
		'Accept All',
		'Accept Cookies',
		'Allow All',
		'I agree',
		'Got it',
		'Continue',
	];

	// Try to click accept buttons
	await page.evaluate((texts) => {
		const buttons = Array.from(
			document.querySelectorAll('button, a, [role="button"]')
		);
		const acceptButton = buttons.find((button) =>
			texts.includes(button.innerText.trim())
		);
		if (acceptButton) acceptButton.click();
	}, acceptTexts);

	// Hide any remaining banners
	await page.evaluate((selectors) => {
		document
			.querySelectorAll(selectors.join(', '))
			.forEach((el) => (el.style.display = 'none'));
	}, selectors);
}

(async () => {
	const browser = await puppeteer.launch();
	try {
		const page = await browser.newPage();
		await page.goto('https://example.com');

		// Handle cookie banners
		await handleCookieBanners(page);

		// Take screenshot
		await page.screenshot({ path: 'screenshot.png' });
	} catch (error) {
		console.error('Error:', error);
	} finally {
		await browser.close();
	}
})();

Best Practices

When implementing cookie banner handling, consider these best practices:

1. Wait for Page Load: Ensure the page is fully loaded before handling banners
2. Multiple Approaches: Combine different methods for better coverage
3. Error Handling: Implement proper error handling for failed banner removal
4. Testing: Test across different websites and banner types
5. Maintenance: Regularly update selectors and patterns

Alternative: Using CaptureKit Screenshot API

If you're looking for a more reliable solution without managing browser instances, consider using CaptureKit's Screenshot API:

curl "https://api.capturekit.dev/capture?url=https://example.com&remove_cookie_banners=true&access_key=YOUR_ACCESS_KEY"

Benefits of using CaptureKit:

• Built-in cookie banner handling
• No need to manage browser instances
• Consistent results across different websites
• Regular updates to handle new banner types

Why Cookie Banners Matter

Now that we've covered the solutions, let's understand why handling cookie banners is crucial for your screenshot automation:

• Content Visibility: Banners often cover critical content, making your screenshots incomplete
• Consistency: Different banner states can lead to inconsistent results across captures
• Layout Issues: Banners can trigger different page layouts or behaviors
• Professional Quality: Clean screenshots without banners look more professional
• Automation Reliability: Banners can interfere with automated testing and monitoring

Conclusion

Handling cookie banners is crucial for producing clean, professional screenshots. By combining filter lists, AutoConsent, and custom handlers, you can effectively manage most cookie banner scenarios. For production use cases, consider using CaptureKit's API to eliminate the complexity of browser automation.

Happy capturing! 📸

Related Posts

If you found this post useful, you might also enjoy these related Puppeteer tutorials:

• How to Take Full Page Screenshots with Puppeteer
• How to Block Requests with Puppeteer
• How to Take Screenshots with Puppeteer