Apple updated its Applebot documentation over the weekend to explicitly state that “Site rules for Applebot-Extended are not considered in ranking for Search,” decoupling its AI training crawler from the ranking algorithm that determines visibility in Spotlight, Siri, and Safari search results.
As spotted by Search Engine Roundtable, the clarification mirrors how Google separates Googlebot-Extended from traditional search crawling and arrives just before iOS 27 reaches the public in the coming weeks. It establishes clear rules for a two-tier content-use model that web publishers are increasingly demanding as generative AI training raises concerns about content licensing and consent.
What Applebot-Extended Actually Does
Applebot-Extended does not crawl webpages itself, instead, it determines how Apple can use data already crawled by the standard Applebot user agent to train foundation models powering Apple Intelligence, Services, and Developer Tools. The two systems operate independently.
A publisher who disallows Applebot-Extended in their robots.txt file prevents Apple from using their content to train generative AI models, but their site remains fully eligible for indexing and ranking in Apple Search. Webpages can still appear in results on Spotlight, Siri, and Safari even if Applebot-Extended is blocked. Conversely, blocking standard Applebot would remove content from search entirely, while still potentially allowing Applebot-Extended to process already-cached data for AI purposes.
Granular Publisher Control Over Content Use
Publishers now gain independent toggles for search visibility and AI training consent. A site owner can:
- Allow both: Content ranks normally in Apple Search and trains generative AI models
- Block Applebot-Extended only: Content ranks in Apple Search but opts out of AI model training
- Block Applebot: Content excluded from search entirely, regardless of Applebot-Extended status
Publishers concerned about unauthorized use of their content to train AI can now maintain search visibility without sacrificing that concern. Apple added technical controls in its June 8, 2026 documentation update to support this, including nosnippet meta tags, crawl delay settings, and X-Robots-Tag headers.
How Apple Search Actually Ranks Results
Apple Search considers various factors when ranking web results, with no pre-determined weighting assigned to any single factor. The documentation clarifies only that Applebot-Extended restrictions do not feed into the ranking process, meaning a publisher’s AI training choice has zero impact on search placement.
Timing Tied to iOS 27 and Apple Intelligence Launch
The documentation update comes intentionally before iOS 27’s public release, which brings Apple Intelligence features that depend on Applebot-Extended data to improve Siri responses and native AI capabilities across devices. Apple is clarifying the rules ahead of that launch so publishers understand exactly what data collection powers new system features and how to control it.
This move positions Apple’s emerging search infrastructure as distinct from Google’s in one meaningful way: Apple is being explicit about separating search ranking from AI training data collection, reducing friction for privacy-conscious publishers and those wary of unrestricted AI model training on proprietary content.
