GoogleBot Crawler Now Uses HTTP/2 to Index the Web, Let us discuss how this affects our Back-end?

The Backend Engineering Show with Hussein Nasser

Sep 18, 2020•22 min

--:--

Listen in podcast apps:

Apple Podcasts

Spotify

Download

Listen to this episode in Metacast mobile app

Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more

Episode description

Google Bot is now attempting to use HTTP/2 to crawl the web

Ever since mainstream browsers started supporting the next major revision of HTTP, HTTP/2 or h2 for short, web professionals asked us whether Googlebot can crawl over the upgraded, more modern version of the protocol.

Today we're announcing that starting mid November 2020, Googlebot will support crawling over HTTP/2 for select sites.

Article

https://webmasters.googleblog.com/2020/09/googlebot-will-soon-speak-http2.html

* Intro 0:00

* What is a Crawler 1:06

* Current Crawler uses h1 2:00

* Crawler now uses h2 4:40

* Less connections, slightly high CPU usage 6:30

* Opting out of h2 crawling 9:00

* FAQ 11:40

Http/2 playlist 0;50 https://www.youtube.com/playlist?list=PLQnljOFTspQWbBegaU790WhH7gNKcMAl-

http/2 not cheap 7;00 https://www.youtube.com/watch?v=GriONb4EfPY

lucid chart 9;50 https://www.youtube.com/watch?v=gejfT1h6LBo

h2c smuggling 18:30 https://www.youtube.com/watch?v=B2VEQ3jFq6Q

For the best experience, listen in Metacast app for iOS or Android