Be sure to set your email notification address to a valid one in your scripts. The sample cluster scripts have the following default, which will not work. Just replace user@somewhere.edu with your real email address.
# -- BEGIN USER DIRECTIVE --
# Send notifications to the following address
#$ -M user@somewhere.edu
# -- END USER DIRECTIVE --
It seems that last week we stopped receiving email notifications when our cluster jobs are finished. Has something changed such that this is no longer possible, or can we update our scripts somehow to fix this?
In the process of doing an update to the cluster image last week i made a changed that enabled a service i needed, but simultaneously broke email. I rebooted about half the nodes before i realized that email didn't work. I've since fixed it and email currently works on the 2nd half ... i just need to reboot the others and then everything will be back to normal again.