First things first: What is a sitemap? Sitemaps are xml-files containing structured data about the pages of the website. Each page has an entry similar to this one: https://startupnamecheck.com 2020-03-06T20:31:03+00:00 0.9 monthly < > url < > loc </ > loc < > lastmod </ > lastmod < > priority </ > priority < > changefreq </ > changefreq </ > url What are sitemaps good for? Sitemaps are helpers for search-engines to discover all relevant pages and content on a website. While there are also sitemaps for images, the focus here is on web-pages only. How can I generate a sitemap? A sitemap can be created in various ways. If you are using a framework such as you can create these on the fly or whenever you publish or update your content. Laravel After some experiments and checking several solutions on GitHub I've not found the solution I was looking for: A simple, permanent crawler of the actual website. It considers ` ` robots tags as well as canonicals and of course the ` ` tag. noindex article:modified_time Ignores JavaScript as Google does mostly. This allows it to run much faster than executing a headless browser only to access a pure HTML5/CSS3 page. My solution for sitemaps on the fly As mentioned, after some research I haven't found what I had in mind. So, being a developer at heart, I've opted to build my own solution. It's heavily reliant on , a crawler package for PHP. Besides this, the package is using some regex to identify the most interesting parts of the website. Other values, such as ` ` are guessed by the depth within the website (nesting level). More detail can also be found on the GitHub repo for . PHP Spider priority Laravel-Sitemaps How can I get this package? The package is distributed using and can be installed using: composer composer require bringyourownideas/laravel-sitemap This will automatically configure the required Laravel ServiceProvider. If you opted out of package discovery you can install it manually using: php artisan vendor:publish --provider= "BringYourOwnIdeas\LaravelSitemap\SitemapServiceProvider" How to use the package? The package registers an ` `-command called ` `. This triggers a crawl of your site and writing out of the sitemap in the public-directory. For convenience, you can add this to your deployment steps. artisan generate:sitemap Regular updates of the sitemap If you'd like to run updates of the ` ` regularly, you can add a new line in ` ` in the ` ` function: sitemap.xml app/Console/Kernel.php schedule { $schedule->command( )->daily(); $schedule->command( )->daily()->at( ); } /** * Define the application's command schedule. * * \Illuminate\Console\Scheduling\Schedule $schedule * void */ @param @return protected function schedule (Schedule $schedule) 'generate:sitemap' // ...or with a defined time... 'generate:sitemap' '02:50' Summary & Feedback If you've got issues please raise an . To stay updated please subscribe to my . More information can also be found in the BYOI article around the . Thanks for reading - I hope you like the sitemap crawler :) issue on GitHub newsletter Laravel Sitemap Generator Previously published at https://peterthaleikis.com/posts/how-to-generate-a-laravel-sitemap-dynamically/
