The usefulness of headless browsing seems to come up frequently in my projects, although it is rarely a specified requirement upfront. Mostly this is because automating boring browser tasks tends to save both time and nerves, but also because there is an npm package that is both a pleasure to use, and simply just works very well.
After Chrome got it’s headless browsing mode out-of-the box, Google’s Puppeteer has quickly become the De Facto choice for all headless browsing projects. Its main advantage is that it removes the need to interact with Chrome DevTools directly, and it provides a high-level API compared to other headless browsers like PhantomJS and CasperJS.
You can literally be up and running with something useful in 15 minutes!
const puppeteer = require(’puppeteer’);
puppeteer.launch().then(async browser => {
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800 })
await page.goto(’https://www.google.com’);
await page.screenshot({ path: ‘myscreenshot.png’, fullPage: true });
await browser.close();
});
Some obvious ways to utilize Puppeteer easily spring to mind, but there is a lot more to do than general web-scraping.
Optimize heavy single-page applications
Surprisingly, heavy sites can benefit from adding a headless browser and utilizing server-side rendering of dynamic pages. Puppeteer makes the implementation of this very trivial.
For more on this, see Eric Bidelman’s article with some more optimization tips on using Puppeteer for server-side rendering and commentary on showing the viability of this approach.
Easy PDF exports
The screenshot functionality of Puppeteer is probably one of the more common feature you stumble upon in tutorials and various examples, but this simple functionality can really be a lifesaver in certain cases.
Providing PDF exports of data is a very typical requirement in many applications, but creating the actual file is not always a pleasant experience. You can choose to implement the document with a dedicated API such as PDFKit and maintain that, or choose to create a template for your export as a web page and then convert it to a PDF. The latter is a good choice if you are exporting data that you already happen to have in a form, and this is where Puppeteer comes in.
…
await page.goto(’https://www.google.com’);
await page.pdf({ path: ’myexport.pdf’, format: ’A4’, printBackground: true })
await browser.close();
The export is achieved with very few changes to the previous example, and issues such as paging are handled automatically. Naturally, you also have the option of manipulating the newly created files before serving them to the user, if needed.
Unrestricted testing
Where Puppeteer really shines is testing. Obviously, one can actually create scripts to do specific manipulations of elements, filling forms and testing for specific issues. But there is quite a bit more!
Need to ensure that two different implementations provide the same result on the screen?
Once again, it’s Puppeteer’s screenshotting to the rescue! Simply take two screenshots and use any pixel by pixel comparison library (such as pixelmatch ) to achieve the end result.
Maybe you are already using a test framework like Jest?
Adding Puppeteer into the mix can provide you with a lot of the functionality of dedicated UI testing tools like Selenium, but without the configurations hassle. Puppeteer can fill-in forms, click on buttons and do complex manipulation where needed in a very simple and concise way. See jest-puppeteer for examples.
You can also choose to set Chrome to run in “headful” mode to actually see the Puppeteer performing the actions.
const browser = await puppeteer.launch({headless: false}); // default is true
Maybe you want to create a customized testing report for your own needs?
Jest already provides a lot of data, and Puppeteer can create a quick web-page on the fly to display the content as you wish In fact, throw in Lighthouse and you have a full, robust audit of your web-page with very little effort.
And of course, an npm package already exists to do this.
Interested in a job?
Come work with Puppeteer and other cool tech with us.
