We (OpenCraft) have updated our periodic build mechanize to send an email to the open-edx-btr-notifications@googlegroups.com mailing list when an installation is failing. We will try to send a test email soon, so do not forget to subscribe to it. 
You can find more information on the BTR-8 ticket.