1. Crawl
Search engines use their own web crawlers or search engine spiders to scour billions of pages on the internet and find new and updated content, such as web pages, videos, PDF files and images. Search engine spiders fetch a few websites and follow the links on these pages to find new URLs.

2. Index
In this phase, search engines store and organize the content based on the keywords, data, recency and user engagement.The new URLs found in the crawling process are indexed in a massive database called Caffeine to be retrieved when the content on the URL matches an online user’s search query.In this phase, search engines store and organize the content based on the keywords, data, recency and user engagement.The new URLs found in the crawling process are indexed in a massive database called Caffeine to be retrieved when the content on the URL matches an online user’s search query.