Contributors:
Amol Ghemud Published: October 21, 2021
In This Article
Share On:
It’s important that searchers have access to up-to-date information. However, one of the most significant issues for website owners is having search engines promptly detect and assess their most recent website improvements.
New URLs might take days, if not weeks, to be discovered and indexed by search engines, resulting in a loss of potential traffic, consumers, and sales.
Microsoft has unveiled a new index, which marks a fundamental shift in how search engines discover and index material. Search Engines IndexNow is open-source and accessible to all search engines that participate. Index Now tells all participating search engines when the material is published or changed on a website, significantly reducing indexing time.
Because search engines don’t scan every URL every day, it can take days or weeks for search engines to notice that the content has changed if you don’t use IndexNow. With IndexNow, search engines can quickly identify “URLs that have changed,” allowing them to prioritise crawling for these URLs and limiting organic crawling for new content discovery.
IndexNow is available under the Creative Commons Attribution-ShareAlike License and is supported by Microsoft Bing and Yandex.
SEO is the Conversion Booster
Our SEO checklist lays out the essentials of optimizing a website, thus aiding in driving more traffic and enhancing conversions.
When a search engine has finished crawling a page, what happens next? The technique by which search engines organise material prior to a search in order to provide super-fast responses to queries is known as indexing.
Search engines would have to filter through individual sites for keywords and themes to find relevant material, which would take a long time. Inverted indexes, also known as reverse indexes, are used by search engines like Google.
The technique by which search engines organize material prior to a search in order to provide super-fast responses to queries is known as indexing. Search engines would have to comb through individual sites for keywords and themes to find relevant material, which would take a long time.
What is IndexNow?
Microsoft has introduced a new index that represents a significant change in the way search engines discover and index content. IndexNow is open source and available to all participating search engines.
An index is updated when a website’s material is published or altered. Webmasters and website owners can use IndexNow to notify search engines about content changes on their sites.
IndexNow converts content discovery into a push technique, informing CMS search engines like WordPress when new or updated content has been uploaded.
Microsoft and Yandex collaborated to produce IndexNow. All search engines can use the protocol because it is open source.
All participating search engines will be notified when a search engine receives the notification of a new or changed web page.
Search engines can get data from web pages in two ways: pull and push.
When a search engine crawler visits a site to request web pages, the data is “pulled” from the server by the crawler. This is how search engines functioned in the past.
Reducing server load improves server performance without adding to the already heavy load of redundant web pages that search engines have.
Search Engines IndexNow shifts content discovery to a push mechanism, in which a CMS such as WordPress notifies search engines when the material is uploaded or modified. Publishers get an added advantage here since they don’t have to wait for search engines to crawl and discover their content.
Search engines are alerted and immediately visit the site to retrieve the new or updated pages.
This is what Microsoft had to say about the IndexNew Launch:
“…historically one of the biggest pain points for website owners has been to have search engines quickly discover and consider their latest website changes.
It can take days or even weeks for new URLs to be discovered and indexed in search engines, resulting in loss of potential traffic, customers and even sales.
…once search engines are notified of updates they quickly crawl and reflect website changes in their index and search results.”
The Brains Behind Index Now
Microsoft and Yandex have launched IndexNow, a new protocol that any search engine can use to allow site owners to have their pages and content indexed by the search engine in real-time.
Presently, Microsoft Bing and Yandex are the only two search engines that are fully committed to the initiative, but others are invited to join.
Furthermore, every search engine that adopts this protocol should acknowledge that submitted URLs would be immediately shared with all other participating search engines, removing the need to submit to both Bing and Yandex, as well as other search engines if this protocol becomes more generally accepted.
Benefits Of IndexNow
Because the expanding demands on the server are decreased by IndexNow, search engines no longer need to undertake exploratory crawls to see if website pages have been updated.
Also, you can enter up to 10,000 URLs per post, alternating between http and https if necessary.
Reduced server load allows the server to operate more efficiently without introducing redundant web-site load. It eventually benefits the entire world by reducing the energy demands of crawling and indexing, as well as assisting in the reduction of global warming.
“IndexNow is an initiative towards a more efficient Internet,” says Bing. Website owners send a clear signal to search engines, indicating that certain URLs should be crawled first, reducing the need for exploratory crawls to see if the content has changed… Search engines plan to limit the crawling of websites that use IndexNow in the future.
Even large corporations have adopted the new protocol. Microsoft has stated that it plans to migrate large companies such as eBay, Linkedin, MSN, GitHub, and Bizapedia to IndexNow from the Bing URL submission API. The benefits of the new approach are evident, so the other larger corporations are expected to follow.
Smaller businesses may be enticed to adopt the new standard in order to stay competitive and improve their rankings.
How does it work?
Three ways to send URLs are recommended:
1. IndexNow SEO Integration
IndexNow, OnCrawl, and Botify are the first three options.
2. API-based direct submission
Another option is to submit an API request directly. Developers can use APIs to submit IndexNow URLs automatically. According to the announcement,
Using our online key generation tool, generate a protocol-compliant key.
Store the key in a text file at the root of your website titled after the key’s value.
When your URLs are added, modified, or deleted, start submitting them. Per API call, you can submit a single URL or a group of URLs.
Sending a simple HTTP request with the URL modified and your key is all it takes to submit one URL.
3. Integration with content management systems (CMS) and content delivery networks (CDNs)
However, the third option involves interacting with IndexNow via a function that is built into the core of any CMS used by a publisher. The third option is to go the traditional route.
Wix and Duda are already aware of the benefits that their users enjoy, and they plan to provide them with the new feature. Cloudflare and Akamai both support the new IndexNow protocol.
In Conclusion
The way search engines identify newly published or updated information has changed dramatically, and IndexNow is a major concern. Because bots visit their online pages regularly, publishers benefit from faster indexing and lower server loads.
The use of major search engines has changed the way publishers and consumers interact. Even though major search engines have adopted IndexNow, it remains unclear whether Google will be participating in this initiative.
In the end, it benefits the world by lowering the energy requirements for crawling and indexing while also contributing to the reduction of global warming pressures.
FAQ
What would happen after I submit my URL to IndexNow?
If search engines approve your URL, search engines will attempt crawling it to get the latest content quickly based on their crawl scheduling logic and crawl quota for your site.
What is the maximum URL request per day one can submit to IndexNow?
You can submit up to 10,000 URLs per post, mixing http and https URLs if needed.
I submitted an URL, but I don’t see the URL indexed?
Using IndexNow ensures that search engines are aware of your website changes. Using IndexNow does not guarantee that web pages will be crawled or indexed by search engines. It may take time for the change to reflect in search engines.
I have a small website that has few web pages. Should I use IndexNow?
Yes, if you want search engines to discover content as soon as it’s changed then you should use IndexNow. You will not have to wait many hours or worse weeks to see your changes on search engines.
Can I submit the same URL multiple times a day?
Avoid submitting the same URL many times a day. If pages are edited often, then it is advised to wait 10 minutes between edits before notifying search engines. If pages are updated constantly (examples: time in Waimea, Weather in Tokyo), it’s preferable to not use IndexNow for every change.
I have a sitemap, do I need IndexNow?
Yes, when sitemaps are an easy way for webmasters to inform search engines about all pages on their sites that are available for crawling, sitemaps are visited by Search Engines infrequently. With IndexNow, webmasters ”don’t” have to wait for search engines to discover and crawl sitemaps but can directly notify search engines of new content.
Amol has helped catalyse business growth with his strategic & data-driven methodologies. With a decade of experience in the field of marketing, he has donned multiple hats, from channel optimization, data analytics and creative brand positioning to growth engineering and sales.
Digital Marketing Bootcamp
Leave a Reply