26 August, 2026

Is AI ignoring your website? How to check if ChatGPT and Gemini can read it

At MONA, we review dozens of websites every month for clients who use our SEO and web services. Over the past six months we've kept running into a new kind of problem across every industry: the site still ranks on Google and customers still pour in, but AI assistants like ChatGPT and Gemini can't read a single word on it. The site owner has no idea. The cause is usually a few lines of old settings left behind by a previous web team, not anything anyone did on purpose.

This is becoming a real money problem. Every month, thousands of people stop typing into Google and ask ChatGPT, Gemini or Perplexity directly, with very buying-minded questions: "which water purifier brand should I buy" or "who is a reputable web design company in Ho Chi Minh City." AI reads back through the websites it has collected and suggests a few names to the person asking. Your website has to be readable by AI to have a chance of being named. If it's blocked, you're invisible to a whole new layer of customers. This article shows you how to check in 5 minutes, and how to fix it if you're affected.

🔎 FREE TOOL

Is AI overlooking your website?

Paste your web address and in 5 minutes you'll know whether ChatGPT, Gemini and Perplexity can read you, and which bots are being blocked.

Check for free →
Free · No sign-up · We don't store your website address
customers ask AI assistants instead of searching Google
More and more customers ask AI directly instead of typing into Google.

How AI "reads" your website

Before ChatGPT or Gemini names a website in an answer to a customer, that AI company's systems must have already read the site's content, just as Google has to crawl a page before it can rank it. It sounds technical, but the idea is simple. This reading is done by automated programs called bot (or crawlers). Each AI company has its own bot: OpenAI has GPTBot, Anthropic has ClaudeBot, Perplexity has PerplexityBot, and Google has a switch called Google-Extended for Gemini. The bot visits your website, downloads the content, and the AI company uses that content to answer users.

MONA bear illustrating how AI bots read website content
Each AI company has its own bot that collects web content before answering.

This works exactly the way Googlebot has read the web to rank search results for over 20 years. The only difference is that there's now a whole new set of bots from AI companies. The key point we want you to remember: your website decides which bots to let in and which to block. That permission is written in a file called robots.txt placed at the root of your website. Every technical team knows this file. To see your own, type tenmien.com/robots.txt into your browser and it appears right away. It's very easy to try. But most websites set that file up before AI existed, so they accidentally leave out or wrongly block the new AI bots. More than half of the sites we've checked recently had exactly this problem.

Why AI overlooks your website even though no one meant it to

Because we review websites for clients all the time, we've collected the three most common situations:

One, blocking everything in the name of "safety." Many web teams worry that unfamiliar bots will slow down the server, so they block everything that isn't Google. A single Disallow line written too broadly can cut off GPTBot, ClaudeBot and PerplexityBot all at once. The site still ranks on Google as normal and the owner sees nothing unusual, but AI can't read a word. This is the case we see most often.

Two, reusing an old configuration. A website moves from one platform to another and the new team carries the old robots.txt over unchanged. A file written 4 to 5 years ago has no GPTBot or Google-Extended in it, because those bots didn't exist yet.

Three, setting the wrong blocking tag. Besides robots.txt there is also the noindex tag and the X-Robots-Tagheader. A page that gets this tag by mistake tells every bot "don't store me," AI bots included.

MONA bear spotting robots.txt wrongly blocking AI bots
Just a few old rules in robots.txt are enough to block a whole set of AI bots by mistake.

What do all three have in common? The website owner doesn't know they are blocking. Like the sample result in MONA's tool: a website scoring 68 points, with 8 of the 12 key AI bots blocked, all because of 2 badly written lines. The only way to know whether you're affected is to check directly. You can't guess. For all three cases, just paste your site into the tool and you'll see everything.

How to check your website yourself in 5 minutes

You don't need to read code. We built a free tool so anyone can check for themselves: paste your website address into the box, click one button, and see the result. Quick and simple.

👉 Check your website here with no sign-up, no storage of the address you enter, and results in a few seconds.

MONA bear using MONA's AI website checker
Paste your web address into the tool and you get a score and a bot list right away.

The tool returns three easy-to-read things. First, a score from 0 to 100: the higher the score, the more open your website is to AI, and a low score means important bots are being blocked. Second, a list of each bot with its status, allowed or blocked (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more), grouped so you can see at a glance who can read you and who can't. Third, a to-do list ranked by importance, with a snippet of settings you can copy and hand straight to your technical team.

MONA bear reading the detailed score breakdown of the AI checker
The result splits into 3 score parts and comes with a list of things to fix.

If your website scores high and no bots are blocked, good news. The foundation for getting named by AI is in place. If some bots are blocked, the section below shows how to open them up.

The AI bots we recommend you let in

Not every bot deserves your attention. This is the group MONA advises clients to open first, because these are the assistants actually answering Vietnamese users and citing sources:

  • GPTBot and OAI-SearchBot (OpenAI): these two decide whether your website gets into ChatGPT and ChatGPT Search. ChatGPT is a tool many Vietnamese people use to ask about shopping and get advice.
  • ClaudeBot (Anthropic): for Claude, an assistant that's increasingly popular among office workers and businesses.
  • PerplexityBot (Perplexity): Perplexity answers with clear source citations, so once its bot is in, you have a chance of your name appearing with a link back to your site.
  • Google-Extended : this isn't a bot, but a switch that lets Gemini use your content. Opening it doesn't touch your Google Search ranking, and for most websites it's fine to leave open.
  • Bytespider (ByteDance/TikTok): this one matters especially for the Vietnamese market because it's tied to the TikTok ecosystem, where a lot of younger customers are.
MONA bear opening the door for important AI bots to enter the website
The group of AI assistants answering Vietnamese customers is the one to open first.
MONA bear explaining how TikTok's Bytespider bot reads websites
TikTok's Bytespider also reads the web, and is worth noting for the Vietnamese market.

To understand what each bot does, who runs it, and whether to allow or block it, see the Bot library : we've written a Vietnamese profile for each one.

How to fix robots.txt to open the door to AI

If the tool reports blocked bots, opening them up is fairly simple and takes under 2 minutes. Add a few lines to the robots.txt file at the root of your website. We've prepared the snippet below. Hand it to your technical person to paste in and you're done:

# Allow AI assistants to read the website User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / Sitemap: https://ten-mien-cua-ban.com/sitemap.xml
MONA bear showing how to edit robots.txt to open up to AI bots
A few added lines in robots.txt open the door for AI assistants to read your site.

One thing we have to say plainly so you don't expect the wrong thing. Opening the door to AI is only the first step. It lets AI see your content, but it doesn't guarantee AI will name you when customers ask. Whether you get named depends on how your content is written and whether other sources mention your website. That's the story of the final section.

AI can read you now, so how do you get AI to name you

This is where many people misunderstand. After opening robots.txt, customers still ask ChatGPT for a "reputable web company" and don't see your name. Opening the door is only the necessary condition. The sufficient condition is that your website content answers exactly what customers ask, and that enough outside sources (press, forums, business profiles) mention your brand for AI to trust and cite it.

MONA bear helping a brand get named by AI when customers ask
Being read by AI is the first step. Being named by AI is what brings customers.

At MONA we call this area GEO, meaning optimizing to be read and cited by AI tools, rather than only optimizing to rank on Google. To get there, you have two more things to do. One, review your content so it answers customers' questions directly. See what checklist GEO we've published. Two, measure how often AI currently mentions your brand compared with your competitors, which we do for you in MONA's GEO service . Paste your site into the tool to measure your current visibility, or leave your number and MONA's GEO team will call you back within 1 business day.

MONA bear comparing before and after being named by AI
Before GEO optimization, AI names your competitor. Afterward, AI names you.

FAQ

Does blocking AI bots hurt my Google ranking?

Not at all. AI bots (GPTBot, Google-Extended and others) are separate from the Googlebot that ranks search results. You can block AI bots without changing your Google ranking, and likewise, allowing AI bots doesn't affect it. Because they're separate, site owners can easily block them by mistake and not notice for months.

Does MONA's checker store the website address I enter?

We store nothing. The tool only reads your website's public configuration files (robots.txt and a few tags) and returns the result right away. You don't need to sign up or leave an email.

Should I allow all AI bots or only some?

For most businesses that want customers to find them through AI, it makes sense to allow the assistants that answer users (ChatGPT, Claude, Perplexity, Gemini). Most sites can simply allow them all. Only consider blocking if you have a specific reason around content copyright.

What is llms.txt, and do I need it?

It's a file that describes your website for AI to read, placed at the root of your site. No major AI company has confirmed it must read this file. But it takes 5 minutes to add and does no harm, so treat it as an early preparation step. MONA's tool also checks whether your website already has this file.

If the check shows blocked bots, can I fix it myself?

Yes, if you have a technical person managing your website. Just paste the snippet above into your robots.txt file. If another company manages your site or you're not comfortable with it, MONA can review and fix it for you.

Many business owners have opened robots.txt themselves, added llms.txt and read a few articles on GEO, yet AI still stays silent and doesn't name them. That's because letting AI read you and getting AI to cite you are two different things, and the second needs the right content and strong enough outside sources. That's the work MONA does every day: we examine your website, measure whether AI is naming you or your competitors, then map out the path for your name to appear in ChatGPT and Gemini answers.

⚡ CHECK NOW

Don't let your website stay invisible to AI

Check for free whether AI bots are overlooking your site, with a list of things to fix right away. And if you want AI to name you when customers ask, MONA's GEO team will measure it for you and map out the path.

Check your website →
Or call 1900 636 648 and MONA's GEO team will advise you for free.
MONA has a podcast: listen instead of readingEpisodes on AI, automation & SEO for Vietnamese businesses
Nghe ngay
About the author

CEO & Founder · The MONA

Founder of The MONA, who built a services company of 60+ staff with 14,000+ projects delivered. Khánh Hùng shares how he thinks about building systems that run themselves, automating a business, and the real lessons from delivering projects at MONA.

Views 353
Rate this article 4,6/5 · 10 ratings

Professional website
design services

Own a website with a beautiful, 100% exclusive design and a lifetime warranty, with the ability
to expand features flexibly as your business grows, starting today!

Contact Mona

    MONA có riêng một Người "Bạn Thân" cho bạn - Người Account sẽ đồng hành, hỗ trợ, hướng dẫn, đặt đồ ăn cho bạn mãi mãi, từ đây về sau!
    MONA will never sell your information or send you spam
    The MONA team advising on an online education management system
    Hỏi đáp giáo dục 4.0
    Tạo cuộc hẹn miễn phí với MONA để giải đáp và tư vấn mọi thắc mắc về giải pháp số hoá ngành giáo dục
    Thời lượng cuộc hẹn
    45 Phút
    Ngày và giờ
    Thứ 2, ngày 25 tháng 12, 2023
    [9:30 - 10:15]

      Chọn ngày và giờ
      Khung giờ
      Back
      Hãy cho MONA biết bạn là ai
      0:00