Implement IndexNow integration for improved search engine indexing
The IndexNow integration enhances the pulumi.com search engine presence by instantly notifying search engines (Bing and Yandex) about new and updated content, eliminating the typical waiting period for discovery and indexing. This means users searching for Pulumi resources will immediately find the most current documentation, blog posts, and tutorials rather than outdated cached versions. The feature works seamlessly within the existing build pipeline, requiring minimal overhead while maximizing content visibility, improving SEO performance, and creating a better overall user experience by ensuring the latest content is readily discoverable.
Changes:
- Add IndexNow API implementation in scripts/search/indexnow.js
- Create shell wrapper in scripts/search/submit-indexnow.sh
- Integrate with build pipeline in ci-push.sh
- Add Makefile target for manual submission
- Include API key in GitHub workflow environment
- Add test scripts to verify API functionality
The implementation successfully:
- Integrates naturally with your existing build and deployment workflow
- Uses the same patterns and approach as other search-related functionality
- Reuses existing sitemap to determine which URLs to submit
- Maintains a cache to avoid resubmitting unchanged URLs
- Includes error handling and logging
- Has a test mode for local validation
Contingent on the IndexNow API key (33134703c43349ddb227d769745f33cc) being added to our ESC secrets management system.
Note to follow after merge, need to make sure: - The key file is accessible at https://www.pulumi.com/indexnow.txt - The script is running successfully during deployments
Your site preview for commit 5f94157f is ready! :tada:
http://www-testing-pulumi-docs-origin-pr-14997-5f94157f.s3-website.us-west-2.amazonaws.com.