{"id":2286,"date":"2012-04-17T10:58:12","date_gmt":"2012-04-17T14:58:12","guid":{"rendered":"http:\/\/acro.net\/blog\/?p=2286"},"modified":"2012-04-17T10:59:44","modified_gmt":"2012-04-17T14:59:44","slug":"why-on-earth-does-google-want-to-ftp-into-your-server","status":"publish","type":"post","link":"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/","title":{"rendered":"Why on earth does Google want to FTP into your server?"},"content":{"rendered":"<p>Recently I discovered that Google bots were being blocked after visiting my server.<\/p>\n<p>After examining the log files, it appeared that Google bots were not just visiting, they were attempting to <strong>FTP<\/strong> into the server, something that perplexed me.<\/p>\n<p>After several such blocks, an entire <strong>C class<\/strong> of Google&#8217;s network would be blocked, due to the rules of my firewall.<\/p>\n<p>The logs look like this:<\/p>\n<blockquote><p>66.249.71.13 # lfd: (ftpd) Failed FTP login from 66.249.71.13 (US\/United States\/crawl-66-249-71-13.googlebot.com): 1 in the last 300 secs &#8211; Tue Apr 17 04:58:43 2012<\/p>\n<p>66.249.71.238 # lfd: (ftpd) Failed FTP login from 66.249.71.238 (US\/United States\/crawl-66-249-71-238.googlebot.com): 1 in the last 300 secs &#8211; Tue Apr 17 05:03:23 2012<\/p>\n<p>66.249.71.73 # lfd: (ftpd) Failed FTP login from 66.249.71.73 (US\/United States\/crawl-66-249-71-73.googlebot.com): 1 in the last 300 secs &#8211; Tue Apr 17 05:34:46 2012<\/p>\n<p>66.249.71.0\/24 # lfd: (NETBLOCK) 66.249.71.0\/24 has had more than 4 blocks in the last 86400 secs &#8211; Tue Apr 17 05:34:36 2012<\/p><\/blockquote>\n<p>As I have anonymous FTP turned off, I resorted to researching Google about its own snooping behavior. I found out it&#8217;s not uncommon, as <a href=\"http:\/\/www.webmasterworld.com\/search_engine_spiders\/4386547.htm\" target=\"_blank\"><strong>others reported<\/strong><\/a> the same behavior by <a href=\"https:\/\/groups.google.com\/a\/googleproductforums.com\/forum\/?fromgroups#!topic\/websearch\/-UUwPKFQHBA\" target=\"_blank\"><strong>Google bots attempting to FTP into servers.<\/strong><\/a><\/p>\n<p><a href=\"http:\/\/groups.google.com\/a\/googleproductforums.com\/forum\/#!topic\/webmasters\/6934m4Bf8Qw\" target=\"_blank\"><strong>A response<\/strong><\/a> to a similar question was given by a <a href=\"https:\/\/profiles.google.com\/johnmu.com\/about\" target=\"_blank\"><strong>Google technician from Switzerland<\/strong><\/a>, who stated:<\/p>\n<blockquote>\n<div><em>&#8220;When we find links to FTP content, we&#8217;ll generally attempt to crawl those URLs. If they&#8217;re publicly accessible and return normal content, we may choose to index them as well. While it&#8217;s not that common, there are occasionally queries where a file on an FTP server is a good result. [&#8230;] If you wish to block crawling of a public FTP server, you can use the robots.txt file just as you would on a normal website. If your FTP server isn&#8217;t publicly accessible, then you wouldn&#8217;t need to do anything specific to prevent that content from being indexed (as it can&#8217;t be accessed).&#8221;<\/em><\/div>\n<\/blockquote>\n<p>The problem is, that there is <strong>no public FTP<\/strong> on my server and thus the reasoning behind visits from Google bots, along with the above method of blocking Google bots do not apply.<\/p>\n<p>I resorted to checking my logs frequently and removing entries where Google had been blocked, manually. It seems that Google keeps on trying to unlock a virtual door with no keyhole to it.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently I discovered that Google bots were being blocked after visiting my server. After examining the log files, it appeared that Google bots were not just visiting, they were attempting to FTP into the server, something that perplexed me. After several such blocks, an entire C class of Google&#8217;s network would be blocked, due to [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[9,193,10,37,68,1189,799,798,800],"class_list":["post-2286","post","type-post","status-publish","format-standard","hentry","category-domains","tag-acro","tag-acronet","tag-acroplex","tag-acroplex-llc","tag-acroplexcom","tag-domains","tag-google-attempts-ftp-to-my-server","tag-google-tries-to-ftp","tag-googlebots-ftp","entry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Why on earth does Google want to FTP into your server? - Acro.net - A Domain Investing Blog by Theo Develegas<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Why on earth does Google want to FTP into your server? - Acro.net - A Domain Investing Blog by Theo Develegas\" \/>\n<meta property=\"og:description\" content=\"Recently I discovered that Google bots were being blocked after visiting my server. After examining the log files, it appeared that Google bots were not just visiting, they were attempting to FTP into the server, something that perplexed me. After several such blocks, an entire C class of Google&#8217;s network would be blocked, due to [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/\" \/>\n<meta property=\"og:site_name\" content=\"Acro.net - A Domain Investing Blog by Theo Develegas\" \/>\n<meta property=\"article:published_time\" content=\"2012-04-17T14:58:12+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2012-04-17T14:59:44+00:00\" \/>\n<meta name=\"author\" content=\"Theo Develegas\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Theo Develegas\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/why-on-earth-does-google-want-to-ftp-into-your-server\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/why-on-earth-does-google-want-to-ftp-into-your-server\\\/\"},\"author\":{\"name\":\"Theo Develegas\",\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/#\\\/schema\\\/person\\\/9c9625f061a0e603a87f5bf0f6f781fe\"},\"headline\":\"Why on earth does Google want to FTP into your server?\",\"datePublished\":\"2012-04-17T14:58:12+00:00\",\"dateModified\":\"2012-04-17T14:59:44+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/why-on-earth-does-google-want-to-ftp-into-your-server\\\/\"},\"wordCount\":386,\"commentCount\":4,\"keywords\":[\"Acro\",\"Acro.net\",\"Acroplex\",\"Acroplex LLC\",\"Acroplex.com\",\"Domains\",\"Google attempts FTP to my server\",\"Google tries to FTP\",\"Googlebots FTP\"],\"articleSection\":[\"Domains\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/acro.net\\\/blog\\\/why-on-earth-does-google-want-to-ftp-into-your-server\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/why-on-earth-does-google-want-to-ftp-into-your-server\\\/\",\"url\":\"https:\\\/\\\/acro.net\\\/blog\\\/why-on-earth-does-google-want-to-ftp-into-your-server\\\/\",\"name\":\"Why on earth does Google want to FTP into your server? - Acro.net - A Domain Investing Blog by Theo Develegas\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/#website\"},\"datePublished\":\"2012-04-17T14:58:12+00:00\",\"dateModified\":\"2012-04-17T14:59:44+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/#\\\/schema\\\/person\\\/9c9625f061a0e603a87f5bf0f6f781fe\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/why-on-earth-does-google-want-to-ftp-into-your-server\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/acro.net\\\/blog\\\/why-on-earth-does-google-want-to-ftp-into-your-server\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/why-on-earth-does-google-want-to-ftp-into-your-server\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/acro.net\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Why on earth does Google want to FTP into your server?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/acro.net\\\/blog\\\/\",\"name\":\"Acro.net - A Domain Investing Blog by Theo Develegas\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/acro.net\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/acro.net\\\/blog\\\/#\\\/schema\\\/person\\\/9c9625f061a0e603a87f5bf0f6f781fe\",\"name\":\"Theo Develegas\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/6794630c371bee89f2b833c1f4b777d9ba75767b217c8fce2cfd6e6d7d90960d?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/6794630c371bee89f2b833c1f4b777d9ba75767b217c8fce2cfd6e6d7d90960d?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/6794630c371bee89f2b833c1f4b777d9ba75767b217c8fce2cfd6e6d7d90960d?s=96&d=mm&r=g\",\"caption\":\"Theo Develegas\"},\"description\":\"Theo Develegas - News and opinions on domain name investing, brand development, design, and the occasional rant or two about life's challenges. Founder of Acroplex LLC.\",\"sameAs\":[\"https:\\\/\\\/acro.net\",\"https:\\\/\\\/x.com\\\/acroplex\"],\"url\":\"https:\\\/\\\/acro.net\\\/blog\\\/author\\\/admin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Why on earth does Google want to FTP into your server? - Acro.net - A Domain Investing Blog by Theo Develegas","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/","og_locale":"en_US","og_type":"article","og_title":"Why on earth does Google want to FTP into your server? - Acro.net - A Domain Investing Blog by Theo Develegas","og_description":"Recently I discovered that Google bots were being blocked after visiting my server. After examining the log files, it appeared that Google bots were not just visiting, they were attempting to FTP into the server, something that perplexed me. After several such blocks, an entire C class of Google&#8217;s network would be blocked, due to [&hellip;]","og_url":"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/","og_site_name":"Acro.net - A Domain Investing Blog by Theo Develegas","article_published_time":"2012-04-17T14:58:12+00:00","article_modified_time":"2012-04-17T14:59:44+00:00","author":"Theo Develegas","twitter_misc":{"Written by":"Theo Develegas","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/#article","isPartOf":{"@id":"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/"},"author":{"name":"Theo Develegas","@id":"https:\/\/acro.net\/blog\/#\/schema\/person\/9c9625f061a0e603a87f5bf0f6f781fe"},"headline":"Why on earth does Google want to FTP into your server?","datePublished":"2012-04-17T14:58:12+00:00","dateModified":"2012-04-17T14:59:44+00:00","mainEntityOfPage":{"@id":"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/"},"wordCount":386,"commentCount":4,"keywords":["Acro","Acro.net","Acroplex","Acroplex LLC","Acroplex.com","Domains","Google attempts FTP to my server","Google tries to FTP","Googlebots FTP"],"articleSection":["Domains"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/","url":"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/","name":"Why on earth does Google want to FTP into your server? - Acro.net - A Domain Investing Blog by Theo Develegas","isPartOf":{"@id":"https:\/\/acro.net\/blog\/#website"},"datePublished":"2012-04-17T14:58:12+00:00","dateModified":"2012-04-17T14:59:44+00:00","author":{"@id":"https:\/\/acro.net\/blog\/#\/schema\/person\/9c9625f061a0e603a87f5bf0f6f781fe"},"breadcrumb":{"@id":"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/acro.net\/blog\/why-on-earth-does-google-want-to-ftp-into-your-server\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/acro.net\/blog\/"},{"@type":"ListItem","position":2,"name":"Why on earth does Google want to FTP into your server?"}]},{"@type":"WebSite","@id":"https:\/\/acro.net\/blog\/#website","url":"https:\/\/acro.net\/blog\/","name":"Acro.net - A Domain Investing Blog by Theo Develegas","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/acro.net\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/acro.net\/blog\/#\/schema\/person\/9c9625f061a0e603a87f5bf0f6f781fe","name":"Theo Develegas","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/6794630c371bee89f2b833c1f4b777d9ba75767b217c8fce2cfd6e6d7d90960d?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/6794630c371bee89f2b833c1f4b777d9ba75767b217c8fce2cfd6e6d7d90960d?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/6794630c371bee89f2b833c1f4b777d9ba75767b217c8fce2cfd6e6d7d90960d?s=96&d=mm&r=g","caption":"Theo Develegas"},"description":"Theo Develegas - News and opinions on domain name investing, brand development, design, and the occasional rant or two about life's challenges. Founder of Acroplex LLC.","sameAs":["https:\/\/acro.net","https:\/\/x.com\/acroplex"],"url":"https:\/\/acro.net\/blog\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/acro.net\/blog\/wp-json\/wp\/v2\/posts\/2286","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/acro.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/acro.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/acro.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/acro.net\/blog\/wp-json\/wp\/v2\/comments?post=2286"}],"version-history":[{"count":0,"href":"https:\/\/acro.net\/blog\/wp-json\/wp\/v2\/posts\/2286\/revisions"}],"wp:attachment":[{"href":"https:\/\/acro.net\/blog\/wp-json\/wp\/v2\/media?parent=2286"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/acro.net\/blog\/wp-json\/wp\/v2\/categories?post=2286"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/acro.net\/blog\/wp-json\/wp\/v2\/tags?post=2286"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}