Miki Lombardi

Automate everything via NodeJS and Puppeteer

How do you automate tasks on sites without a public API? See how NodeJS and Puppeteer can scrape data, publish content, and run continuous performance tests.

Automate everything via NodeJS and Puppeteer
#1about 2 minutes

An introduction to the Puppeteer library for Node.js

Puppeteer provides a high-level API to control headless or full Chrome instances through the DevTools Protocol.

#2about 2 minutes

The history and evolution of web automation tools

Web automation has evolved from cross-language frameworks like Selenium to JavaScript-based tools like PhantomJS and now Puppeteer.

#3about 3 minutes

Understanding Puppeteer's API and common capabilities

Puppeteer's readable, action-based API can be used for scraping, testing, generating PDFs, and analyzing performance.

#4about 3 minutes

Comparing Puppeteer with alternatives like Playwright

Playwright is a strong alternative for cross-browser end-to-end testing, while Puppeteer is more of a general-purpose automation tool.

#5about 4 minutes

Exploring common Puppeteer use cases for web applications

Puppeteer is well-suited for scraping dynamic sites, automating UI and visual regression tests, and building custom development utilities.

#6about 7 minutes

Case study on scraping Skyscanner for cheap flights

A personal project demonstrates how to automate flight searches by splitting code into modular functions and intercepting API responses.

#7about 5 minutes

Automating podcast uploads from YouTube to Anchor.fm

A GitHub Action uses Puppeteer to automate converting YouTube videos to audio and publishing them on Anchor.fm, saving significant manual effort.

#8about 7 minutes

Automating Lighthouse performance tests with authentication

A professional setup uses Puppeteer to handle authentication and run Lighthouse tests on protected pages, sending metrics to Prometheus and Grafana.

#9about 9 minutes

Q&A on Puppeteer vs Cypress and cloud deployment

The discussion covers when to choose Puppeteer over Cypress, running it in the cloud using containers instead of serverless, and its use in visual testing.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Automating browser workflows with AI-powered tools
03:32 MIN

Automating browser workflows with AI-powered tools

WeAreDevelopers LIVE: Scammer Payback with Python, Grok Goes Unhinged, The Future of Chromium and mo

Presenting live web scraping demos at a developer conference
01:57 MIN

Presenting live web scraping demos at a developer conference

Tech with Tim at WeAreDevelopers World Congress 2024

Navigating the complexities of modern web scraping
03:28 MIN

Navigating the complexities of modern web scraping

How to scrape modern websites to feed AI agents

Automating reactive performance testing in your CI/CD pipeline
11:21 MIN

Automating reactive performance testing in your CI/CD pipeline

Frontend Performance Testing in practice

Applying automation across multiple platforms
10:33 MIN

Applying automation across multiple platforms

How to achieve web automation with UiPath

Key takeaways for automated performance testing
02:44 MIN

Key takeaways for automated performance testing

Measure and improve frontend performance by using test automation

How AI is changing test generation and maintenance
04:01 MIN

How AI is changing test generation and maintenance

The 2025 State of JavaScript Testing

Automating accessibility checks in the CI/CD pipeline
03:44 MIN

Automating accessibility checks in the CI/CD pipeline

Accessibility Test Driven Development

Featured Partners

Related Videos

See all videos
1, 2, 3... Fastify!
42:08

1, 2, 3... Fastify!

Matteo Collina

One-click-to-production: Test and automate your application
31:15

One-click-to-production: Test and automate your application

Bartosz Pietrucha

From clicks to cribs - How to find your dream home with web scraping
26:08

From clicks to cribs - How to find your dream home with web scraping

Alexander Lichter

Measure and improve frontend performance by using test automation
25:38

Measure and improve frontend performance by using test automation

Ramona Schwering

Frontend Performance Testing in practice
36:50

Frontend Performance Testing in practice

Jonas Kröger

Livecoding with AI
31:39

Livecoding with AI

Rainer Stropek

The Lean Web
46:46

The Lean Web

Chris Ferdinandi

The 2025 State of JavaScript Testing
28:33

The 2025 State of JavaScript Testing

Daniel Afonso

Related Articles

View all articles
Chris Heilmann
Dev Digest 138 - Are you secure about this?
Hello there! This is the 2nd "out of the can" edition of 3 as I am on vacation in Greece eating lovely things on the beach. So, fewer news, but lots of great resources. Many around the topic of security. Enjoy! News and ArticlesGoogle Pixel phones t...
Dev Digest 138 - Are you secure about this?
Chris Heilmann
All the videos of Halfstack London 2024!
Last month was Halfstack London, a conference about the web, JavaScript and half a dozen other things. We were there to deliver a talk, but also to record all the sessions and we're happy to share them with you. It took a bit as we had to wait for th...
All the videos of Halfstack London 2024!

From learning to earning

Jobs that call for the skills explored in this talk.