Ben Affleck's ex-nanny has filed a restraining order against an alleged admirer she says took things too far ... claiming he's continued to try and contact her even though she asked him to stop -- and ...