User-agent: JSpiderUnitTest Disallow: /testcases/specific/robotstxt/disallowedFolder2 Disallow: /testcases/specific/robotstxt/disallowedResource2.html User-agent: JSpider Disallow: /testcases/specific/robotstxt/disallowedFolder1 Disallow: /testcases/specific/robotstxt/disallowedResource1.html Disallow: /samplesite/forbidden User-Agent: Trusted Allow: / User-Agent: * Disallow: /testcases/specific/robotstxt/disallowedFolder1 Disallow: /testcases/specific/robotstxt/disallowedResource1.html DisAllow: /samplesite/forbidden