You can either - block the directory by robots or just ensure all contents are 404. If you have blocked by robots.txt to get the directory removed, then when it is removed then I would probably unblock it so the 404's can be seen and the reason is that removing stuff just 'hides' it in the search results and contents can still be crawled - so once it has gone from public view - allow the urls to be crawled so they can never come back. |