Hi guys,
I have 5 web servers with the same URL but I want google not to spider the content of one of the servers. So if googlebot comes around, it should only reach servers 1 through 4.
I'm thinking I can use content matching based on user-agent to only allow googlebot to reach those 4 servers, but I'm not having any luck figuring it out.
Can anyone help?
