[{"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://clickbankuniversity.com/go/jvtools/img/728x90.png", "src": "https://clickbankuniversity.com/go/jvtools/img/728x90.png", "alt_text": "Revealed: The Secrets our Clients Used to Earn $3 Billion"}, null, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f1ea778d9.11038093.png", "src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f1ea778d9.11038093.png", "formatted_filename": "f ea d"}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f1f55c475.72629357.png", "src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f1f55c475.72629357.png", "formatted_filename": "f f c", "alt_text": "Related image"}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f20d4af52.25966653.png", "src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f20d4af52.25966653.png", "formatted_filename": "f d af"}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f21c27ae5.19548523.png", "src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f21c27ae5.19548523.png", "formatted_filename": "f c ae", "alt_text": "Image result"}, null, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f24aba301.18832721.jpg", "src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f24aba301.18832721.jpg", "formatted_filename": "f aba", "alt_text": "adamaudette - I don't always JavaScript, but when I do, I know google can crawl the dom and dynamically generated HTML"}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f2538c510.34140097.jpg", "src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f2538c510.34140097.jpg", "formatted_filename": "f c", "alt_text": "\"It's not just Google understanding your JavaScript. It's also about the speed.\" -DOM - \"It's not just about Google understanding your Javascript. it's also about your perceived latency.\" -DOM"}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f25a2a532.46891061.png", "src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f25a2a532.46891061.png", "formatted_filename": "f a a", "alt_text": "progressive page rendering"}, null, null, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f2aebdf52.52120655.png", "src": "https://d2v4zi8pl64nxt.cloudfront.net/javascript-seo/59482f2aebdf52.52120655.png", "formatted_filename": "f aebdf"}, null, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "//longtailpro.postaffiliatepro.com/accounts/default1/4ym941157gb/aff0f98b.png", "src": "http://longtailpro.postaffiliatepro.com/accounts/default1/4ym941157gb/aff0f98b.png", "formatted_filename": "aff f b", "rendered_width": 728, "rendered_height": 90}, null, null, null, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://pbs.twimg.com/profile_images/613798779423182848/YJ1z7f17_normal.jpg", "src": "https://pbs.twimg.com/profile_images/613798779423182848/YJ1z7f17_normal.jpg", "formatted_filename": "YJ z f normal", "alt_text": "sspencer", "rendered_width": 48, "rendered_height": 48}, null, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://pbs.twimg.com/profile_images/1036703857798205444/yMxm8Wiu_normal.jpg", "src": "https://pbs.twimg.com/profile_images/1036703857798205444/yMxm8Wiu_normal.jpg", "formatted_filename": "yMxm Wiu normal", "alt_text": "Br11Token", "rendered_width": 48, "rendered_height": 48}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://pbs.twimg.com/profile_images/978868719752540160/3galapVe_normal.jpg", "src": "https://pbs.twimg.com/profile_images/978868719752540160/3galapVe_normal.jpg", "formatted_filename": "galapVe normal", "alt_text": "Aoxford23", "rendered_width": 48, "rendered_height": 48}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://pbs.twimg.com/profile_images/961069405433024512/CtDRJin__normal.jpg", "src": "https://pbs.twimg.com/profile_images/961069405433024512/CtDRJin__normal.jpg", "formatted_filename": "CtDRJin normal", "alt_text": "SaHondow", "rendered_width": 48, "rendered_height": 48}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://pbs.twimg.com/profile_images/923743319804596225/Y4RPlFVh_normal.jpg", "src": "https://pbs.twimg.com/profile_images/923743319804596225/Y4RPlFVh_normal.jpg", "formatted_filename": "Y RPlFVh normal", "alt_text": "OrtizDYST", "rendered_width": 48, "rendered_height": 48}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://pbs.twimg.com/profile_images/910125954181120000/h9GyLOk9_normal.jpg", "src": "https://pbs.twimg.com/profile_images/910125954181120000/h9GyLOk9_normal.jpg", "formatted_filename": "h GyLOk normal", "alt_text": "1MDHASIBULISLAM", "rendered_width": 48, "rendered_height": 48}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://pbs.twimg.com/profile_images/1022068980859764736/KrMsN4bS_normal.jpg", "src": "https://pbs.twimg.com/profile_images/1022068980859764736/KrMsN4bS_normal.jpg", "formatted_filename": "KrMsN bS normal", "alt_text": "Chels_fitnessuk", "rendered_width": 48, "rendered_height": 48}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://pbs.twimg.com/profile_images/871085504019738624/rhpVFMcs_normal.jpg", "src": "https://pbs.twimg.com/profile_images/871085504019738624/rhpVFMcs_normal.jpg", "formatted_filename": "rhpVFMcs normal", "alt_text": "WorldStarPR", "rendered_width": 48, "rendered_height": 48}, null, {"document_url": "https://www.goodtoseo.com/javascript-amp-seo-making-your-bot-experience-as-good-as-your-user-experience/", "unformatted_src": "https://pbs.twimg.com/profile_images/993021619055484928/mjRPG9TZ_normal.jpg", "src": "https://pbs.twimg.com/profile_images/993021619055484928/mjRPG9TZ_normal.jpg", "formatted_filename": "mjRPG TZ normal", "alt_text": "dgursh", "rendered_width": 48, "rendered_height": 48}, null]

Posted by alexis-sanders

Understanding JavaScript and its potential impact on search performance is a core skillset of the modern SEO professional. If search engines can’t crawl a site or can’t parse and understand the content, nothing is going to get indexed and the site is not going to rank.

The most important questions for an SEO relating to JavaScript: Can search engines see the content and grasp the website experience? If not, what solutions can be leveraged to fix this?


What is JavaScript?

When creating a modern web page, there are three major components:

Learn more about webpage development and how to code basic JavaScript. Image sources: 1, 2, 3

JavaScript is either placed in the HTML document within <script> tags (i.e., it is embedded in the HTML) or linked/referenced. There are currently a plethora of JavaScript libraries and frameworks, including jQuery, AngularJS, ReactJS, EmberJS, etc.

JavaScript libraries and frameworks: What is AJAX?

AJAX, or Asynchronous JavaScript and XML, is a set of web development techniques combining JavaScript and XML that allows web applications to communicate with a server in the background without interfering with the current page. Asynchronous means that other functions or lines of code can run while the async script is running. XML used to be the primary language to pass data; however, the term AJAX is used for all types of data transfers (including JSON; I guess “AJAJ” doesn’t sound as clean as “AJAX” [pun intended]).

A common use of AJAX is to update the content or layout of a webpage without initiating a full page refresh. Normally, when a page loads, all the assets on the page must be requested and fetched from the server and then rendered on the page. However, with AJAX, only the assets that differ between pages need to be loaded, which improves the user experience as they do not have to refresh the entire page.

One can think of AJAX as mini server calls. A good example of AJAX in action is Google Maps. The page updates without a full page reload (i.e., mini server calls are being used to load content as the user navigates). 
What is the Document Object Model (DOM)?

As an SEO professional, you need to understand what the DOM is, because it’s what Google is using to analyze and understand webpages.

The DOM is what you see when you “Inspect Element” in a browser. Simply put, you can think of the DOM as the steps the browser takes after receiving the HTML document to render the page.

The first thing the browser receives is the HTML document. After that, it will start parsing the content within this document and fetch additional resources, such as images, CSS, and JavaScript files.

The DOM is what forms from this parsing of information and resources. One can think of it as a structured, organized version of the webpage’s code.

Nowadays the DOM is often very different from the initial HTML document, due to what’s collectively called dynamic HTML. Dynamic HTML is the ability for a page to change its content depending on user input, environmental conditions (e.g. time of day), and other variables, leveraging HTML, CSS, and JavaScript.

Simple example with a <title> tag that is populated through JavaScript:

What is headless browsing?

Headless browsing is simply the action of fetching webpages without the user interface. It is important to understand because Google, and now Baidu, leverage headless browsing to gain a better understanding of the user’s experience and the content of webpages.

PhantomJS and Zombie.js are scripted headless browsers, typically used for automating web interaction for testing purposes, and rendering static HTML snapshots for initial requests (pre-rendering).

Why can JavaScript be challenging for SEO? (and how to fix issues)

There are three (3) primary reasons to be concerned about JavaScript on your site: Are bots able to find URLs and understand your site’s architecture? There are two important elements here:

Why is blocking JavaScript such a big deal?

If search engines are blocked from crawling JavaScript, they will not be receiving your site’s full experience. This means search engines are not seeing what the end user is seeing. This can reduce your site’s appeal to search engines and could eventually be considered cloaking (if the intent is indeed malicious).

Fetch as Google and TechnicalSEO.com’s robots.txt and Fetch and Render testing tools can help to identify resources that Googlebot is blocked from.

The easiest way to solve this problem is through providing search engines access to the resources they need to understand your user experience.

!!! Important note: Work with your development team to determine which files should and should not be accessible to search engines.


Internal linking should be implemented with regular anchor tags within the HTML or the DOM (using an HTML tag) versus leveraging JavaScript functions to allow the user to traverse the site.

Essentially: Don’t use JavaScript’s onclick events as a replacement for internal linking. While end URLs might be found and crawled (through strings in JavaScript code or XML sitemaps), they won’t be associated with the global navigation of the site.

Internal linking is a strong signal to search engines regarding the site’s architecture and importance of pages. In fact, internal links are so strong that they can (in certain situations) override “SEO hints” such as canonical tags.

URL structure

Historically, JavaScript-based websites (aka “AJAX sites”) were using fragment identifiers (#) within URLs.

READ ALSO  Social media automation: Oxymoron or godsend? Search engines have been shown to employ headless browsing to render the DOM to gain a better understanding of the user’s experience and the content on page. That is to say, Google can process some JavaScript and uses the DOM (instead of the HTML document).

At the same time, there are situations where search engines struggle to comprehend JavaScript. Nobody wants a Hulu situation to happen to their site or a client’s site. It is crucial to understand how bots are interacting with your onsite content. When you aren’t sure, test.

Assuming we’re talking about a search engine bot that executes JavaScript, there are a few important elements for search engines to be able to obtain content:

How to make sure Google and other search engines can get your content


The most popular solution to resolving JavaScript is probably not resolving anything (grab a coffee and let Google work its algorithmic brilliance). Providing Google with the same experience as searchers is Google’s preferred scenario.

Google first announced being able to “better understand the web (i.e., JavaScript)” in May 2014. Industry experts suggested that Google could crawl JavaScript way before this announcement. The iPullRank team offered two great pieces on this in 2011: Googlebot is Chrome and How smart are Googlebots? (thank you, Josh and Mike). Adam Audette’s Google can crawl JavaScript and leverages the DOM in 2015 confirmed. Therefore, if you can see your content in the DOM, chances are your content is being parsed by Google. Recently, Barry Goralewicz performed a cool experiment testing a combination of various JavaScript libraries and frameworks to determine how Google interacts with the pages (e.g., are they indexing URL/content? How does GSC interact? Etc.). It ultimately showed that Google is able to interact with many forms of JavaScript and highlighted certain frameworks as perhaps more challenging. John Mueller even started a JavaScript search group (from what I’ve read, it’s fairly therapeutic).

All of these studies are amazing and help SEOs understand when to be concerned and take a proactive role. However, before you determine that sitting back is the right solution for your site, I recommend being actively cautious by experimenting with small section Think: Jim Collin’s “bullets, then cannonballs” philosophy from his book Great by Choice:

“A bullet is an empirical test aimed at learning what works and meets three criteria: a bullet must be low-cost, low-risk, and low-distraction… 10Xers use bullets to empirically validate what will actually work. Based on that empirical validation, they then concentrate their resources to fire a cannonball, enabling large returns from concentrated bets.”

Consider testing and reviewing through the following:

READ ALSO  The 2018 guide to free SEO training courses online

After you’ve tested all this, what if something’s not working and search engines and bots are struggling to index and obtain your content? Perhaps you’re concerned about alternative search engines (DuckDuckGo, Facebook, LinkedIn, etc.), or maybe you’re leveraging meta information that needs to be parsed by other bots, such as Twitter summary cards or Facebook Open Graph tags. If any of this is identified in testing or presents itself as a concern, an HTML snapshot may be the only decision.


What are HTmL snapshots?

HTML snapshots are a fully rendered page (as one might see in the DOM) that can be returned to search engine bots (think: a static HTML version of the DOM).

Google introduced HTML snapshots 2009, deprecated (but still supported) them in 2015, and awkwardly mentioned them as an element to “avoid” in late 2016. HTML snapshots are a contentious topic with Google. However, they’re important to understand, because in certain situations they’re necessary.

If search engines (or sites like Facebook) cannot grasp your JavaScript, it’s better to return an HTML snapshot than not to have your content indexed and understood at all. Ideally, your site would leverage some form of user-agent detection on the server side and return the HTML snapshot to the bot.

At the same time, one must recognize that Google wants the same experience as the user (i.e., only provide Google with an HTML snapshot if the tests are dire and the JavaScript search group cannot provide support for your situation).


When considering HTML snapshots, you must consider that Google has deprecated this AJAX recommendation. Although Google technically still supports it, Google recommends avoiding it. Yes, Google changed its mind and now want to receive the same experience as the user. This direction makes sense, as it allows the bot to receive an experience more true to the user experience.

A second consideration factor relates to the risk of cloaking. If the HTML snapshots are found to not represent the experience on the page, it’s considered a cloaking risk. Straight from the source:

“The HTML snapshot must contain the same content as the end user would see in a browser. If this is not the case, it may be considered cloaking.”
– Google Developer AJAX Crawling FAQs


Despite the considerations, HTML snapshots have powerful advantages: Site latency

When browsers receive an HTML document and create the DOM (although there is some level of pre-scanning), most resources are loaded as they appear within the HTML document. This means that if you have a huge file toward the top of your HTML document, a browser will load that immense file first.

The concept of Google’s critical rendering path is to load what the user needs as soon as possible, which can be translated to → “get everything above-the-fold in front of the user, ASAP.”

Critical Rendering Path – Optimized Rendering Loads Progressively ASAP: 
However, if you have unnecessary resources or JavaScript files clogging up the page’s ability to load, you get “render-blocking JavaScript.” Meaning: your JavaScript is blocking the page’s potential to appear as if it’s loading faster (also called: perceived latency). Render-blocking JavaScript – Solutions

If you analyze your page speed results (through tools like Page Speed Insights Tool, WebPageTest.org, CatchPoint, etc.) and determine that there is a render-blocking JavaScript issue, here are three potential solutions: !!! Important note: It’s important to understand that scripts must be arranged in order of precedence. Scripts that are used to load the above-the-fold content must be prioritized and should not be deferred. Also, any script that references another file can only be used after the referenced file has loaded. Make sure to work closely with your development team to confirm that there are no interruptions to the user’s experience.

Read more: Google Developer’s Speed Documentation

TL;DR – Moral of the story

Crawlers and search engines will do their best to crawl, execute, and interpret your JavaScript, but it is not guaranteed. Make sure your content is crawlable, obtainable, and isn’t developing site latency obstructions. The key = every situation demands testing. Based on the results, evaluate potential solutions. Thanks: Thank you Max Prin (@maxxeight) for reviewing this content piece and sharing your knowledge, insight, and wisdom. It wouldn’t be the same without you. ampamp, Bot, Experience, Good, Javascript, making, seo, User


