robots icon indicating copy to clipboard operation
robots copied to clipboard

[Question] How to pass multiple options?

Open kwiat1990 opened this issue 2 years ago • 5 comments

Hey, I have multiple options for different bots, which look like this:

 User-agent: Googlebot
    Disallow: /nogooglebot/

    User-agent: *
    Allow: /

    Sitemap: `${process.env.CMS_URL}/sitemap/index.xml`

I wonder how I could mirror this file with the robots plugin? I have tried using an array for the rules but it returns only the last entry:

// config
robots: {
    rules: [
      {
        UserAgent: "Googlebot",
        Disallow: "/nogooglebot/",
      },
      {
        UserAgent: "*",
        Allow: "/",
      },
      { 
        Sitemap: `${process.env.STRAPI_URL}/sitemap/index.xml`
      },
    ],
  },

// robots.txt page:
User-agent: *
Disallow: 

kwiat1990 avatar Apr 07 '23 07:04 kwiat1990

robots: [
    {
        UserAgent: 'Googlebot',
        Disallow: [
            '/nogooglebot/'
        ]
    },
    {
        UserAgent: '*',
        Allow: '/',
    },
]

abyshake-anand avatar Apr 17 '23 13:04 abyshake-anand

@abyshake-anand the output on the robot.txt is the same and contains only the last object:

User-agent: *
Disallow: 

kwiat1990 avatar Apr 17 '23 13:04 kwiat1990

Can you share your complete config that relates to robots? This is the output I get when I use the following config.

modules: [
  // ....
  '@nuxtjs/robots',
  // ....
],
robots: [
  {
    UserAgent: 'Googlebot',
    Disallow: ['/user', '/admin'],
  },
  {
    UserAgent: '*',
    Disallow: '/admin',
  },
],
image

abyshake-anand avatar Apr 17 '23 14:04 abyshake-anand

I have tried with the very same config but got the output you can see in my previous message. I’ve also restarted dev server and even change user agent in Chrome to googlebot but it didn’t make any difference at all.

kwiat1990 avatar Apr 17 '23 17:04 kwiat1990

Sorry for messing the PR history.

yauri-io avatar Aug 11 '23 10:08 yauri-io