postal icon indicating copy to clipboard operation
postal copied to clipboard

Attachments are not displayed/retrieved in some sent emails

Open erdemolcay opened this issue 8 years ago • 19 comments

I am having a problem with attachments in emails I send via Postal API. Specifically, recipients using Outlook (tested with Outlook 2013 and Outlook 2016) can't retrieve e-mail attachments.

Here are the results for emails I sent to different services:

  • Office 365: -- Webmail: Attachment displayed. -- Outlook: No attachment.

  • Yandex: -- Webmail: No attachment. -- Outlook: No attachment.

  • Gmail (G-Suite): -- Webmail: Attachment displayed. -- Outlook: No attachment.

Here is the header of the test email I sent:

received: from api (portal.avrupasafak.com [10.11.10.18]) by Postal with HTTP; Sun, 11 Jun 2017 10:14:43 +0000
date: Sun, 11 Jun 2017 13:14:43 +0300
from: "Sender name" <sender email>
to: recipient email
message-id: <[email protected]>
subject: test
mime-version: 1.0
content-type: multipart/alternative; boundary="--==_mimepart_593d1813bad83_67d336b2002467ac"; charset=UTF-8
content-transfer-encoding: 7bit
dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=avrupasafak.com; s=postal-l18PfP; t=1497176083; bh=R77Icyn7Tg6+HtTNYXqJX2OX71cewJzveVRyUHMb6cw=; h=date:from:to:message-id:subject; b=BWK8ADUgSLGS8MdneI5AB/OaUDdEVBCx6/UPl5AXmzrci4+Tm6cTa5EfNge29GebTvdmivgB7mUmpZe35AkqW0QrLVt6oqO1xQCKXePqb0yC/R3vFsVkuQG91ORwIISUFCuDHqHdBFM9RgbrLljQpgtlpke5GeFK2EdD2/O9Mrc=
x-postal-msgid: mnEIPBw9ec9I

I think the problem might be related to the header of the e-mail. When I send the same e-mail with other services (eg SendGrid) the content-type field in the header is set to "multipart/mixed". But in Postal this field is "multipart/alternative". As far as I know, when an attachment is added to an email in the form of "multipart/alternative", the main content type must be "multipart/mixed".

erdemolcay avatar Jun 12 '17 11:06 erdemolcay

We aren't experiencing attachment issues in Outlook but related to https://github.com/atech/postal/issues/106, you could try including both types of body to see if that temporarily remedies the issue?

willpower232 avatar Jun 12 '17 12:06 willpower232

We aren't experiencing attachment issues in Outlook but related to #106, you could try including both types of body to see if that temporarily remedies the issue?

I've already tried it. Same result.

What is main content-type field in header set on emails with attachments you successfully send to Outlook? Could it be a problem with my Postal installation?

erdemolcay avatar Jun 12 '17 12:06 erdemolcay

Looks same as yours.

Content-Type: multipart/alternative;
	boundary="--==_mimepart_592ee59ac11ca_5f462e06b981157f1"; charset="UTF-8"
Content-Transfer-Encoding: 7bit

If it helps, I am not setting any additional headers through the API.

Can you definitely see both HTML and plain versions in the postal screen? Which API client are you using?

willpower232 avatar Jun 12 '17 13:06 willpower232

I can see both HTML and plain text versions of my outgoing email in the Postal web interface.

I'm using the PHP library for Postal:

require 'vendor/autoload.php';

$client = new Postal\Client('https://postal.avrupasafak.com', 'Postal API key');
$message = new Postal\SendMessage($client);
$message->to('recipient email');
$message->from('sender email');
$message->subject('Test');
$message->plainBody('Hello world!');
$message->htmlBody('<p>Hello world!</p>');
$message->attach('textmessage.txt', 'text/plain', 'Hello world!');
$result = $message->send();

The result is the same when I use PHP Curl:

$post_data = array(
  'to' => array('recipient email'),
  'from' => 'Sender Name <sender email>',
  'subject' => 'test',
  'html_body' => 'test',
  'plain_body' => 'test',
  'attachments' => array(
    array(
      'name' => 'postal-send-attachment-test.txt',
      'content_type' => 'text/plain',
      'data' => base64_encode(file_get_contents('/root/postal-send-attachment-test.txt')),
    ),
  ),
);
$post_data = json_encode($post_data);

$post_header = array(
  'X-Server-API-Key: Postal API key',
  'Content-Type: application/json',
  'Content-Length: ' . strlen($post_data)
);

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"https://postal.avrupasafak.com/api/v1/send/message");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $post_header);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output = curl_exec($ch);
curl_close($ch);

erdemolcay avatar Jun 13 '17 09:06 erdemolcay

Interesting, when was the last time you ran postal auto-upgrade?

willpower232 avatar Jun 13 '17 12:06 willpower232

I just ran postal auto-upgrade. The result is the same, attachments still aren't displayed in Outlook.

erdemolcay avatar Jun 13 '17 14:06 erdemolcay

Very disturbing. You aren't running some kind of filtering on your email accounts per chance? Or local email scanning antivirus? At this stage I'm not sure what can be responsible :-/

willpower232 avatar Jun 13 '17 19:06 willpower232

Very disturbing. You aren't running some kind of filtering on your email accounts per chance? Or local email scanning antivirus? At this stage I'm not sure what can be responsible :-/

I'm not using any kind of filtering, antivirus etc.

erdemolcay avatar Jun 28 '17 07:06 erdemolcay

Very odd indeed. We're getting random reports of people not getting their attachment but the attachment is showing in the Postal interface.

We haven't been able to replicate it across Office 365, Hotmail, or Gmail when sending hundreds of messages.

Can you confirm if you are able to see the attachment in the Postal interface?

willpower232 avatar Jul 12 '17 15:07 willpower232

Yes, I can see the attachment in the Postal interface.

erdemolcay avatar Jul 17 '17 06:07 erdemolcay

If you want to check yourself, I can provide web and SSH access to our server. Thank you.

erdemolcay avatar Jul 18 '17 06:07 erdemolcay

We installed Postal on another server and we did the same tests here. The results are the same as my first comment. Postal version: 1.0.0-f1bee5644b-stable

erdemolcay avatar Aug 06 '17 09:08 erdemolcay

I am still not able to replicate this behaviour however one extra test could be to set postal up on another server using a copy from github rather than the current update method, see here https://github.com/atech/postal/commit/2a606320bead2b6d5d98408227833102f803ee90#diff-b43de171c710873f5067c6d50d2cfb99R29

willpower232 avatar Aug 06 '17 10:08 willpower232

I've been experiencing this issue as well.

I did some more digging and eventually stumbled upon this issue in the Ruby mail library: https://github.com/mikel/mail/issues/118

And indeed, when only sending a plaintext body, attachments seem to appear in Outlook. As soon as you send a html body the attachment disappears. This is only on Windows by the way, Outlook on Mac OS seems to work just fine.

alexroosenstein avatar Sep 15 '17 14:09 alexroosenstein

Quick update, the following snippet seems to work. Please keep in mind, I only tested this on a development node (nor did I test this extensively), so if anyone else could test this as well, that'd be great.

This is a replacement for https://github.com/atech/postal/blob/67d0f6514d674ebe823052f0cde0f2931f87e315/app/models/outgoing_message_prototype.rb#L151-L186

  def raw_message
    @raw_message ||= begin
      mail = Mail.new
      if @custom_headers.is_a?(Hash)
        @custom_headers.each { |key, value| mail[key.to_s] = value.to_s }
      end
      mail.to = self.to_addresses.join(', ') if self.to_addresses.present?
      mail.cc = self.cc_addresses.join(', ') if self.cc_addresses.present?
      mail.from = @from
      mail.sender = @sender
      mail.subject = @subject
      mail.reply_to = @reply_to
      if @html_body.blank? && attachments.empty?
        mail.body = @plain_body
      else
        text_part = nil
        html_part = nil

        if !@plain_body.blank?
          text_part = Mail::Part.new
          text_part.body = @plain_body
        end
        if !@html_body.blank?
          html_part = Mail::Part.new
          html_part.content_type = "text/html; charset=UTF-8"
          html_part.body = @html_body
        end
        if !attachments.empty?
          mail.part :content_type => "multipart/alternative"  do |p|
            p.text_part = text_part
            p.html_part = html_part
          end
          attachments.each do |attachment|
            mail.attachments[attachment[:name]] = {
              :mime_type => attachment[:content_type],
              :content => attachment[:data]
            }
          end
        else
          mail.text_part = text_part
          mail.html_part = html_part
        end
      end
      mail.header['Received'] = "from #{@source_type} (#{self.resolved_hostname} [#{@ip}]) by Postal with HTTP; #{Time.now.utc.rfc2822.to_s}"
      mail.message_id = "<#{@message_id}>"
      mail.to_s
    end
  end

alexroosenstein avatar Sep 15 '17 16:09 alexroosenstein

Test this and add a pull request if it is an improvement.

On Sep 15, 2017 12:39 PM, "alexroosenstein" [email protected] wrote:

Quick update, the following snippet seems to work. Please keep in mind, I only tested this on a development node (nor did I test this extensively), so if anyone else could test this as well, that'd be great.

This is a replacement for https://github.com/atech/postal/blob/ 67d0f6514d674ebe823052f0cde0f2931f87e315/app/models/ outgoing_message_prototype.rb#L151-L186

def raw_message @raw_message ||= begin mail = Mail.new if @custom_headers.is_a?(Hash) @custom_headers.each { |key, value| mail[key.to_s] = value.to_s } end mail.to = self.to_addresses.join(', ') if self.to_addresses.present? mail.cc = self.cc_addresses.join(', ') if self.cc_addresses.present? mail.from = @from mail.sender = @sender mail.subject = @subject mail.reply_to = @reply_to if @html_body.blank? && attachments.empty? mail.body = @plain_body else text_part = nil html_part = nil

    if !@plain_body.blank?
      text_part = Mail::Part.new
      text_part.body = @plain_body
    end
    if !@html_body.blank?
      html_part = Mail::Part.new
      html_part.content_type = "text/html; charset=UTF-8"
      html_part.body = @html_body
    end
    if !attachments.empty?
      mail.part :content_type => "multipart/alternative"  do |p|
        p.text_part = text_part
        p.html_part = html_part
      end
      attachments.each do |attachment|
        mail.attachments[attachment[:name]] = {
          :mime_type => attachment[:content_type],
          :content => attachment[:data]
        }
      end
    else
      mail.text_part = text_part
      mail.html_part = html_part
    end
  end
  mail.header['Received'] = "from #{@source_type}

(#{self.resolved_hostname} [#{@ip}]) by Postal with HTTP; #{Time.now.utc.rfc2822.to_s}" mail.message_id = "<#{@message_id}>" mail.to_s end end

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/atech/postal/issues/228#issuecomment-329834754, or mute the thread https://github.com/notifications/unsubscribe-auth/AMMA5weB8LtRrnBG7HICnwpsu0IApRkWks5siqg2gaJpZM4N3Ap3 .

shopglobal avatar Sep 15 '17 19:09 shopglobal

@alexroosenstein I am unable to replicate your issue with Outlook 2016 on Windows 10 (or hotmail.com or gmail.com or office365.com web interfaces) so I am not sure what is going on.

I can confirm that your modification to outgoing_message_prototype.rb doesn't break anything so if it fixes things for you, make a pull request.

FWIW, some people may not have https://github.com/atech/postal/pull/267 yet as its not coming through the update file from atech

willpower232 avatar Sep 17 '17 09:09 willpower232

to replicate the issue easily, you can download the .eml file of the email from the postal interface...

The fix provided by @alexroosenstein worked for me

alexander-rieder avatar Jun 26 '19 11:06 alexander-rieder

This may be related to #375 and #376 and probably conflicts quite badly...

willpower232 avatar Jun 26 '19 14:06 willpower232

Please can you upgrade to the latest version of Postal and confirm whether this issue still exists? If so, please open a new issue for triage. Be sure to include any relevant log data as well as details about your environment as prompted by the new issue template.

adamcooke avatar Mar 06 '24 09:03 adamcooke