How to stop Magento's emails from bouncing

Recently I noticed some emails sent by Magento keep bouncing back. The mail server dropped me a few "Undelivered Mail Returned to Sender" messages. This is very undesirable, since this means that some order confirmations and/or invoices are not being delivered to the customer. After some investigation I discovered a Magento setting under System > Configuration > Advanced > System > Mail Sending Settings > "Set Return-Path". Set this value to "Yes" to make the Return-Path the same as the sender. Alternatively, you can set it to "Specified" and specify a "Return-Path Email" where all bounced emails get sent to. This can come in handy if you want to have those emails checked with priority for example, you could sent bounced mails to a priority mailbox instead.

Not specifying a Return-Path in Magento (ie. setting "Set Return-Path" to "No") will most likely get your emails blocked by a lot of spam filters.