Frequently Asked Questions


When will a project be recrawled?
When you subscribed to a premium plan, you can re-crawl your project at any time. Additionally, you can let Seobility automatically crawl every project and specify a wait time in hours between each crawlings. Users of the free account have to wait 24 hours after each crawling until they can start a new crawling. An automatic crawling is also only available for paid accounts.
Do I have to wait until the crawling has been finished?
No, you can view the most tests and analyzes even when crawling is running. Please note that tests that need aggregated data from the whole website are only available when the crawling is finished.
How long does it take until the crawling is finished?
The duration of the crawling depends on the number of retrieved sub pages as well as the load time of your website. If the load time is good and there are not too many sub pages, the crawling takes only a few minutes.
What are the reasons why a crawling cannot be completed?
The following problems are the most common reasons, why a crawling cannot be finished:
  • The domain is not connected (typo?)
  • The provided URL redirects to another location on a different domain
  • The home page could not be retrieved (404, timeout, etc.)
  • Seobility bot is blocked in robots.txt (User-Agent: * or Seobility)
  • The web server stops responding during the crawling (timeout, Too many requests etc.)

If you experience problems crawling your site, just contact our Support Team.
IP address and user agent of Seobility bot.

The Seobility bot has the follwing details:

IP address: 88.198.114.16
User Agent: Seobility

Unless your site is protected by firewall or similar, please create a rule to permit the access for Seobility.

If you have any questions about Seobility or about our plans and pricing please contact us at support@seobility.net