# Bit of a hack: generally allow robots everything, except for # things in ecore identifying a 'lastnode_id' to stop the site # being spidered too heavily, but allow indexing of 'prime' # pages. User-agent: * Disallow: /ecore/index.pl?lastnode_id= User-agent: * Disallow: /index.pl?lastnode_id= # User-agent: msnbot # Disallow: /