"We wouldn't have to take your rights away if you'd just stop exercising them."