.Google.com confines robots.txt help to 4 industries, clarifying its standpoint on in need of support instructions.Google merely assists four certain robots.txt industries.In need of support instructions in robots.txt will definitely be neglected.Take into consideration auditing your robots.txt files in light of this improve.