Unable to send an email to multiple email addresses

[watarumohawk]

ISSUE SUMMARY

I can not send an email to multiple email addresses by using notifications-client even though the log says “Notifications Emailer.Successfully sent email”.
So it makes me think that notifications-client doesn't treat multiple email addresses correctly.

EXPECTED BEHAVIOR

You can send an email to multiple email addresses.

ACTUAL BEHAVIOR

You can not send an email to multiple email addresses.

STEPS TO REPRODUCE

1. Install Postfix
2. Confirm you can send an email to 1 email address by using Postfix

$ echo "This is the body of the email" | mail -s "This is the subject line" youraddress@yahoo.com

3. Confirm you can send an email to multiple email addresses by using Postfix

Both white space and comma as delimiter work as expected.

$ echo "This is the body of the email" | mail -s "Space - This is the subject line" youraddress@yahoo.com youraddress2@yahoo.com

$ echo "This is the body of the email" | mail -s "Comma - This is the subject line" youraddress@yahoo.com, youraddress2@yahoo.com

4. Confirm you can send an email to 1 email address by using notifications-client.

Edit the bash script which is in /var/vcap/jobs/send-email/bin/run in mysql instance, and run the edited script like this:

/var/vcap/packages/notifications-client/bin/notifications-client \
--uaaDomain="uaa.YOURDOMAIN.com" \
--notificationsDomain="notifications.YOURDOMAIN.com" \
--uaaAdminClientUsername="admin" \
--uaaAdminClientSecret="YOUR-ADMIN-SECRET" \
--uaaClientUsername="notifications" \
--uaaClientSecret="YOUR-UAA-SECRET" \
--toAddress="youraddress@yahoo.com" \
--subject="SINGLE EMAIL - [YOURDOMAIN.com][cluster_identifier] P-Mysql Alert 100" \
--bodyHTML="{alert-code 100}<br /> Hello, just wanted to let you know that the MySQL node/cluster has gone down and has " \
--kindID="$p-mysql" \
--skipSSLCertVerify="true" \
>>/var/vcap/sys/log/mysql/notifications-client.combined.log 2>/var/vcap/sys/log/mysql/notifications-client.combined.log

5. Confirm you can not send an email to multiple email addresses by using notifications-client.

Both white space and comma as delimiter don't work.

Command:

/var/vcap/packages/notifications-client/bin/notifications-client \
--uaaDomain="uaa.YOURDOMAIN.com" \
--notificationsDomain="notifications.YOURDOMAIN.com" \
--uaaAdminClientUsername="admin" \
--uaaAdminClientSecret="YOUR-ADMIN-SECRET" \
--uaaClientUsername="notifications" \
--uaaClientSecret="YOUR-UAA-SECRET" \
--toAddress="youraddress@yahoo.com, youraddress2@yahoo.com" \
--subject="COMMA - [YOURDOMAIN.com][cluster_identifier] P-Mysql Alert 100" \
--bodyHTML="{alert-code 100}<br /> Hello, just wanted to let you know that the MySQL node/cluster has gone down and has " \
--kindID="$p-mysql" \
--skipSSLCertVerify="true" \
>>/var/vcap/sys/log/mysql/notifications-client.combined.log 2>/var/vcap/sys/log/mysql/notifications-client.combined.log

Log:

{"timestamp":"1557985472.742089748","source":"Notifications Emailer","message":"Notifications Emailer.Running with config","log_level":1,"data":{"kindID":"-mysql","notificationsDomain":"notifications.YOURDOMAIN.com","subject":"COMMA - [YOURDOMAIN.com][cluster_identifier] P-Mysql Alert 100","toAddress":"youraddress@yahoo.com, youraddress2@yahoo.com","uaaAdminClientUsername":"admin","uaaClientUsername":"notifications","uaaDomain":"uaa.YOURDOMAIN.com"}}
{"timestamp":"1557985472.742255211","source":"Notifications Emailer","message":"Notifications Emailer.adminUAACONFIG:","log_level":1,"data":{"SkipVerification":true,"UAAEndpoint":"https://uaa.YOURDOMAIN.com","clientName":"admin"}}
{"timestamp":"1557985472.742285728","source":"Notifications Emailer","message":"Notifications Emailer.HTTP Request timeout in seconds","log_level":0,"data":{"value":0}}
{"timestamp":"1557985472.742307663","source":"Notifications Emailer","message":"Notifications Emailer.uaa-client.started-fetching-token","log_level":0,"data":{"endpoint":"https://uaa.YOURDOMAIN.com/oauth/token","force-update":false,"session":"2"}}
{"timestamp":"1557985472.742568731","source":"Notifications Emailer","message":"Notifications Emailer.uaa-client.fetch-token-from-uaa-start","log_level":1,"data":{"endpoint":{"Scheme":"https","Opaque":"","User":null,"Host":"uaa.YOURDOMAIN.com","Path":"/oauth/token","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"session":"3"}}
{"timestamp":"1557985472.792995930","source":"Notifications Emailer","message":"Notifications Emailer.uaa-client.fetch-token-from-uaa-end","log_level":1,"data":{"session":"3","status-code":200}}
{"timestamp":"1557985472.793133974","source":"Notifications Emailer","message":"Notifications Emailer.uaa-client.successfully-fetched-token","log_level":0,"data":{"session":"2"}}
{"timestamp":"1557985472.793151140","source":"Notifications Emailer","message":"Notifications Emailer.caching-token","log_level":0,"data":{}}
{"timestamp":"1557985472.914425850","source":"Notifications Emailer","message":"Notifications Emailer.HTTP Request timeout in seconds","log_level":0,"data":{"value":0}}
{"timestamp":"1557985472.914503574","source":"Notifications Emailer","message":"Notifications Emailer.uaa-client.started-fetching-token","log_level":0,"data":{"endpoint":"https://uaa.YOURDOMAIN.com/oauth/token","force-update":true,"session":"5"}}
{"timestamp":"1557985472.914726496","source":"Notifications Emailer","message":"Notifications Emailer.uaa-client.fetch-token-from-uaa-start","log_level":1,"data":{"endpoint":{"Scheme":"https","Opaque":"","User":null,"Host":"uaa.YOURDOMAIN.com","Path":"/oauth/token","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"session":"6"}}
{"timestamp":"1557985472.937147617","source":"Notifications Emailer","message":"Notifications Emailer.uaa-client.fetch-token-from-uaa-end","log_level":1,"data":{"session":"6","status-code":200}}
{"timestamp":"1557985472.937236071","source":"Notifications Emailer","message":"Notifications Emailer.uaa-client.successfully-fetched-token","log_level":0,"data":{"session":"5"}}
{"timestamp":"1557985472.937263727","source":"Notifications Emailer","message":"Notifications Emailer.caching-token","log_level":0,"data":{}}
{"timestamp":"1557985472.937336683","source":"Notifications Emailer","message":"Notifications Emailer.Making request to notifications","log_level":0,"data":{"method":"POST","url":{"Scheme":"https","Opaque":"","User":null,"Host":"notifications.YOURDOMAIN.com","Path":"/emails","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""}}}
{"timestamp":"1557985472.993559599","source":"Notifications Emailer","message":"Notifications Emailer.Successfully sent email","log_level":1,"data":{}}

NOTES

• Comma, semicolon, and white space as delimiter don't work when I use multiple email addresses.
• Gmail address (@gmail.com) might not receive your test email because of their spam filter.

[osis]

This is a really detailed and thoroughly documented issue/feature. Thanks @watarumohawk!