The alert won't be removed from the system. It will stop firing since the created email recipient will no longer be with the company.

Please submit a support case with the Alert Name that needs to be deleted.