Is Apple building a search engine?
Apple has published a document (see bellow) pointing to a new web crawler.
The crawler seems to be using the 184.108.40.206 IP block that is owned solely by Apple and have been doing the rounds since late last year.
Not sure why they would want to compete with Google or Bing search, but I have a feeling it could be to bolster their Siri, Spotlight products, maybe even Safari search?The document reads…
User-agent strings will contain “Applebot” together with additional agent information. For example:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1)
If robots instructions don’t mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions.
If you have questions or concerns, please contact us at: Apple-NOC “at” apple.com
It can be found here… http://support.apple.com/en-us/HT204683