Have You Checked if Ghost.org’s API is Blocking Google Bots from Accessing?

5 min readFeb 12, 2025

After joining the SEO Team in 2024, I realized that the company’s blog system wasn’t very user-friendly. The reason was that it was too expensive, costing $500 per month, yet its features and multi-account support were not practical or aligned with our future content growth direction. After discussing this with the CEO, we decided to seamlessly migrate from the original blog system to Ghost.org, which I had previously loved using.

Ghost （Since Medium doesn’t support HTML formatting, I have listed the advantages of Ghost.）

Advantages:

• Simple and intuitive user interface.

• Focused entirely on content creation.

• Built-in SEO features for better ranking.

• Fast loading speeds for improved user experience.

• Multi-user and permission management for team collaboration.

• Flexible customization options.

• Built-in membership and subscription systems.

• Open-source(ghost api) and free to use.

• Strong focus on data privacy.

Disadvantages:

Limited themes and plugins compared to other platforms.
Requires technical knowledge for self-hosting.
Not as feature-rich for advanced e-commerce needs.

Challenges with Article Indexing After Migration to Ghost

However, the company’s new website has been facing a persistent “couldn’t fetch” issue in the sitemap, which was affecting the indexing process. After migrating to Ghost, I hoped that the situation would improve, but I noticed that the issue persisted. Article indexing became more challenging, and despite using best practices, some of our content was not being indexed properly. This was particularly evident in non-English-speaking countries, where in some cases, articles were not indexed at all, resulting in zero visibility. The migration seemed to trigger unforeseen challenges in how search engines interacted with our site, making it harder for our content to be discovered in those regions.

During our research and development (R&D) troubleshooting, we uncovered a significant issue that had been affecting the indexing of our non-English blogs. We found that these pages were consistently returning a 301 error, which was preventing the search engine bots from properly accessing and indexing the content. Unfortunately, I did not save many screenshots at the time, but I can explain what we observed. When we clicked on the test URL, we noticed that the page being crawled by the bot was completely empty, even though the blog article appeared fine on the frontend for regular users. This discrepancy between how the page displayed to users and how it was being read by search engine bots was a clear indicator of an underlying issue.

This problem had been a persistent frustration for us for about a month, with little success in resolving it. We initially couldn’t pinpoint the root cause, but after thorough investigation, we were able to connect the dots. Finally, after some time and effort, we discovered the root cause of the issue, which had been impacting the visibility of our non-English content. Identifying and fixing this issue was a breakthrough moment for the team, as it allowed us to restore the proper indexing of our articles and ensure they would be discovered by search engines.

Maybe the Impact of Google’s JavaScript Rendering on SSR Websites

This issue might not have been encountered before 2024, but after 2024, Google’s crawler underwent significant changes in its approach to indexing content. Specifically, Google’s crawler began loading and executing JavaScript code before processing the content of the page. While this shift in behavior was meant to improve the crawler’s ability to index dynamic content, it created unforeseen problems for some SSR (Server-Side Rendered) websites’ blogs. These sites, unlike traditional static blogs, did not rely on the standard blog links provided by Ghost for indexing. Instead, they called article information dynamically through the Ghost API, which led to issues with indexing.

The Ghost API is a powerful tool that allows developers to interact with Ghost’s content management system to fetch and display articles on a website. However, because the crawler was executing JavaScript before it attempted to fetch content from the Ghost API, it sometimes failed to retrieve the article data properly. As a result, the search engines were unable to index the full content of these non-English blog articles, and visibility in search results was significantly impacted. This issue highlighted the complexity of using the Ghost API with SSR websites and showed that changes in how crawlers process JavaScript can have a direct effect on how effectively content is indexed.

The technical explanation is that when the crawler accesses the webpage, it first executes the JavaScript code, which delays or prevents the successful retrieval of article data via the Ghost API. As a result, the API request for the article content is blocked or fails. However, from the end user’s perspective, there are no visible issues, as the blog articles render correctly on the front end due to the client-side JavaScript rendering the content as intended.

If you’re encountering similar issues, we recommend following these steps for troubleshooting:

1. Check if the same error appears in Google Search Console (GSC).

2. Look for empty article content in GSC’s crawler screenshot.

3. Verify that your robots.txt file is set up correctly, with the following content:

User-agent: *
Disallow: /
Allow: /ghost/api/
Allow: /content/images/

Since SSR websites’ blogs do not use the blog links provided by Ghost, make sure that the relevant paths are allowed for proper access when using the robots.txt file. After checking these three steps, the issue should be resolved.

After struggling with the Ghost API issue for a month, we finally fixed it. I hope this can inspire others who are facing similar problems, and I look forward to more people connecting with me to share insights.

Now, our blog is finally accessible. Below are the links to all our multilingual blogs.

Of course, you may be wondering, Ghost.org doesn’t natively support multilingual content blogs, but we managed to implement this feature. How did we do it? In the future, I’ll be updating an article explaining how to solve the multilingual blog issue using the Ghost API.

Here are the links for all language versions of the TrustDice Blog:

• Taiwannees: https://trustdice.win/zh/blog

• Russian: https://trustdice.win/ru/blog

• Korean: https://trustdice.win/ko/blog

• Japanese: https://trustdice.win/ja/blog

• German: https://trustdice.win/de/blog

• Turkish: https://trustdice.win/tr/blog

• Spanish: https://trustdice.win/es/blog

• Portuguese: https://trustdice.win/pt/blog

• French: https://trustdice.win/fr/blog

• Italian: https://trustdice.win/it/blog

• Swedish: https://trustdice.win/sv/blog