The unveiled threat to religious liberty

So, if we are to ban the face veil worn by some Muslims, then why shouldn’t we ban all forms of attire that isolate their wearers from the mainstream?