What is Screaming Frog? A detailed guide to using it

Screaming Frog is an SEO tool well loved by online marketers. It is very powerful, and used properly it can improve your site's SEO strength and ranking. In this article, MONA Media will walk SEOers through using Screaming Frog in detail for an SEO audit.

What is Screaming Frog?

Screaming Frog is a full SEO tools suite that lets you carry out tasks related to website SEO optimisation. The software works like a “SEO spider”, also known as the search bots of search engines (such as Google, Bing and so on).

This way, you collect the assessments of search engines about your website. Every detail that is not optimised or that hurts SEO performance is listed and easy to find with filters. Using this data, you simply fix those errors until the counter is back to 0.

Of course, Screaming Frog SEO Spider also integrates many other in-depth tools and applications to improve website SEO, advanced features aimed at experienced online marketers. Let's go through this SEO tool in detail in the next section.

The interface of the SEO tool Screaming Frog

UI (User Interface) has a very intuitive interface, and you can get the overall picture within a few minutes. You will immediately notice the address bar where you enter the URL of the website you want to audit. Above the address bar is the Menu bar – which holds the software's main menus.

Right below the address bar is a tab bar, where each tab is an aspect you need to pay attention to in a SEO. They are:

• Internal and External link: The site's internal links and external links
• URL: URLs of the pages in the website
• Meta keywords: see the page's keywords
• Meta description: see the page's description
• Content: statistics on the content of each page
• …

There are many different tabs so you can audit the website thoroughly and in detail, as shown below:

Note: Below the tab bar is a detail window listing every crawled URL with all its metrics.

In addition, there is another tab bar at the bottom of the window Screaming Frog that lets you look at different aspects of each page. The tab content also changes with the URL you click in the detail window above.

On the right column, you will see the filter (Filter) in the upper half and the statistics window in the lower half. Both change depending on the tab you choose on the main tab bar. When auditing a website, the filter is a very handy tool for isolating problems and fixing them quickly.

 → See also:

• The 9 most common types of content and how to do content marketing effectively
• What does a complete SEO-friendly post structure include?
• The fastest way to get your site to the top of Google

How to use Screaming Frog to audit a website

1. Customise the website crawl

You need to select the items you want to crawl. Doing this makes the research results as accurate as possible. Let's go through the steps:

Step 1: To customise the types of data to crawl, go to Configuration on the toolbar.

Step 2: Then choose Spider, and the search bot customisation window will appear.

Step 3: Here you can select or deselect any of the items you want under the groups Resource Links (Images, CSS, etc.) and Page Links (Internal Hyperlinks, External Links, Canonicals, etc.).

-> See also: The most detailed glossary of common SEO terms

2. Audit link

Also in the Spider Configuration window, you can audit your website's links as well. This takes 4 steps:

Step 1: In the Crawl Behavioursection, choose [Crawl All Subdomains].

Step 2: If you also want to check nofollow links, tick the 2 boxes [Follow Internal “nofollow”] and [Follow External “nofollow”].

Step 3: Click [OK] and start crawling a site. When the results are in, click the [Export] button next to the List/Table View button.

Step 4: Choose the export file type you want, such as .csv or an Excel file (.xlsx), then click OK.

With just these simple steps, you have a list of backlink pointing to the website you are auditing.

3. Check response codes

To check response codes, you need to crawl the site first. The same goes for all the features of Screaming Frog covered below. So from here on, assume you already have the crawl results before carrying out any task.

Switch to the tab Response Codes on the tab bar above. Screaming Frog will list the response code and status of every link on your website.

Then use the filter just under the tab bar, or in the right-hand column, to filter response codes quickly. Links with 2xx codes are working normally. 3xx, 4xx and 5xx codes signal problems and need your attention.

Specifically, these codes mean:

• Code 301: Permanent Redirect
• Code 302: Temporary Redirect.
• Code 404: URL not found, meaning the URL points to a page that does not exist
• Code 500: Server Error, usually from overload, bandwidth running out, CPU overflow and so on
• Code 501: Unavailable: the website's server is temporarily out of service

-> Related reading:

• Causes and fixes for the 502 Bad Gateway error
• How to fix the 504 Gateway Timeout error quickly

4. Check URLs

Optimize URL are an important part of SEO. So Screaming Frog also helps you check the URLs on your website. Here is how:

Step 1: To check your website's URLs, especially the homepage and other key pages, switch to the [URL] tab as shown.

Step 2: The link detail window changes little. But the filter panel shows many kinds of bad URLs that hurt SEO. These include:

• Uppercase, Multiple Slashes and Containing Space: URLs should not contain capital letters, more than one / in a row, or spaces (which computers turn into % when shown in a URL).
• Underscores: Underscores (_) are not allowed in URLs.
• Non ASCII characters: Characters outside the ASCII set (such as “đ”, “â”, “ớ”), which make URLs harder for computers to read.
• Over 115 characters: URLs longer than 115 characters, which research suggests hurt SEO and make URLs harder for users to remember.

-> Read next: The fastest way to submit a URL to Google

5. Check title tags

The title tag is one of the most important components HTML (HTML elements) on a website. Search bots use the title tag to identify a page's content and main keyword. That makes checking title tags one of the first steps of any website audit.

Switch to the [Title] tab and you'll see the titles of every page on your website. Keep using the filter to check for problems with your title tags. Specifically:

• Missing: Missing title tag, so search bots cannot identify the page's title
• Over 60 characters or Over 554 pixels: The recommended maximum title length is 60 characters (or 554 pixels).
• Below 30 characters or Below 200 pixels: The recommended minimum title length is 30 characters (or 200 pixels).
• Multiple: A page with more than 1 title tag will hurt SEO performance.

6. Check H1 tags

After the title tag, the H1 tag is also an HTML element to watch. The H1 tag doesn't affect SEO as much as the title tag, and some pages may not need one.

However, it is the H1, not the title, that is the headline shown on the page and seen directly by readers. So it also helps optimise user experience. Specifically:

• Missing: Missing H1: usually the homepage, image galleries or functional pages without much content.
• Duplicate: H1 tags should be unique across the whole site; duplicates are not good.
• Over 70 characters: Unlike titles, the best H1 length is 70 characters (and it isn't measured in pixels).

7. Check meta descriptions

Another factor that affects your website's SEO is the meta description. The meta description is shown in search results, can attract clicks and contains your target keywords.

The Meta Description tab in Screaming Frog lists each page's description on your website with related figures. You can click each row to see the full descriptions. Using the filter helps you quickly find the meta descriptions that need work. Specifically:

• Missing: Don't leave any page's description empty, because, as noted, they have a real effect on the page's SEO
• Duplicate: Duplicate descriptions are not ideal, but are acceptable on pages where the content matters little
• Over 155 characters or Over 1005 pixels: A description is naturally longer than a title; the recommended maximum is 155 characters (or 1005 pixels)
• Below 70 characters or Below 400 pixels: The minimum length for a meta description to work for SEO is 70 characters (or 400 pixels)

8. Preview snippets on search engine results pages (SERPs)

Related to auditing elements like the title and meta tags, Screaming Frog also lets you preview a page's snippet. A snippet is the information card shown for each page on search results pages (search engine result page – SERP).

Every Rich snippets which usually includes the page's title, description and URL. Previewing snippets lets you adjust the content and length of these tags so all the details you want (keywords, main message, hook and so on) are displayed.

To preview the snippet of any page, click that row and switch to the [SERP Snippet] tab on the tab bar at the bottom of the Screaming Frogwindow. The snippet appears in the window above the tab bar, and below it is an edit window where you can change the title and description straight away.

-> See also:

• The 5 best free rich snippet plugins for WordPress
• Yoast SEO: a very effective tool for optimising snippets

9. Check images

For SEOs, images are also an effective on-page SEO tool. By adding keywords to Alt text tags, you can “stuffing” keywords in without hurting the coherence of your site content.

In this step, switch to the [Image] tab to see the changes needed for the images on your website. Here is how to read the figures:

• Over 100 KB: Keep each image in your library under 100 KB to cut page load time.
• Missing Alt Text: Don't leave any Alt Text empty, because it is a powerful hidden SEO tool.
• Alt Over 100 Characters: On the other hand, don't let Alt Text run too long either; 100 characters at most.

Above is a detailed answer to the question of what What is Screaming Frog and a beginner's guide to using Screaming Frog. In practice, it is an extremely comprehensive SEO toolset, especially in the paid version. It helps you handle every SEO issue, and in particular website auditing. If you find the tool fairly complex and want a better solution, you can use SEO services of MONA Media. Contact us today for advice, SEO services quote in full detail.

-> You might also like:

• What does SEO-friendly mean? What is SEO-friendly web design and what does it do?
• Top 10 free tools that help with online marketing today