thunderbird
SMTP: "HELO can't be [127.0.0.1]"
Checklist
- [X] I have used the search function to see if someone else has already submitted the same bug report.
- [X] I will describe the problem with as much detail as possible.
App version
6.202
Where did you get the app from?
Google Play
Android version
11
Device model
No response
Steps to reproduce
Some mail providers seem to forbid SMTP connections when the HELO hostname is 127.0.0.1.
I found the history to this change originating in #2793 (changed in PR #3798).
The question came back in issue #4428.
I have read theses issues and understand that it has been changed so for privacy reasons, and agree that sending the actual real IP or hostname would be detrimental to privacy. However, I believe we can try to find another value that won't be blocked by mail providers. This could potentially be an opt-in option in each SMTP server settings. I'd be ready to make a PR if that is something you would be willing to accept. We could have a select option with values like 127.0.0.1, 192.168.0.1, 10.0.0.1 and a final value of "Send actual IP address (not recommended)". This way the user still is in control of what is sent and can still use the app even if the mail provider denies privacy conserving values (most providers won't be willing to change their server configuration on user requests).
Expected behavior
Being able to send mail to servers denying the 127.0.0.1 HELO hostname
Actual behavior
Connection to the SMTP server rejected due to a forbidden value of 127.0.0.1 in the HELO message
Logs
No response
Yes disappointing no one is addressing this. My hosting service is one that does not allow a local host address for HELO.
Seeing an old forum post where someone was insisting K9 had it right and the rest of the world needed to change got me thinking and reading RFCs (aaahhhrrgghh). As far as I can see RFC 5321 from 2008 is still current. Quoting part 4.1.3.-Address Literals: "Sometimes a host is not known to the domain name system and communication (and, in particular, communication to report and repair the error) is blocked. To bypass this barrier, a special literal form of the address is allowed as an alternative to a domain name. For IPv4 addresses, this form uses four small decimal integers separated by dots and enclosed by brackets such as [123.255.37.2]..."
127.0.0.1 is a local host address, not the address of the host. SMTP servers that reject this have the right to do so. But this is where it gets tricky.
Thunderbird plays by the book, it uses the IP4 address of the client. Yes that is what the RFC wanted but who wants an IP address off their local LAN subnet sent to the world, it's meaningless, and a minor security weakness. To its credit, T-bird has an advanced config setting where you can set HELO whatever you want (and I have).
My suggestion is that K9 should follow T-bird, use the local IP4 address but provide a setting to change it.
But if you are going to use a non-compliant value then either a fixed fictitious private address, e.g. 192.168.1.254, or any resolvable fqdn like k9mail.app would work better than the local host address. Given where Google/Yahoo are going with SPF and DKIM there is also a risk they too will start blocking the local host address.
The old RFC is really the problem here as in a NAT world this parameter is pointless.
rfc 5321 is in reference to smtp -- machine-to-machine/port 25 mail transfer. a more appropriate rfc for discussion of mail submission standards is 6409 - "message submission for mail". as documented, the MTA (message transfer agent) is free to refuse a message for any number of reasons (and not liking the ipnumber on the HELO/EHLO, while not explicitly mentioned, could be one), however, rejecting or bouncing is always referenced as the subsequent handling that should take place. dropping the message on the floor/into the trash (or explicitly sending it to /dev/null as one provider does) is not a documented option in the specification.
Hi njeyaakili.
You are correct, 6409 does say how an MSA should handle message rejection, and I bet you are correct that many fail to do this as described too!
But that isn't the subject of this bug. This bug is about how k9 incorrectly populates the HELO command and the need to fix that.
6409 is based on 5321, with bits added on. Quote 3.1 "The protocol used is ESMTP [SMTP-MTA], with additional restrictions or allowances as specified here." "SMTP-MTA" is a normative reference to RFC 5321 (see references section 12.1). As HELO format is not one of these restrictions/allowances it remains as defined by 5321.
As a comparison I had a look at how FairEmail populates HELO. It uses an fqdn dummy.faircode.eu, which if you browse to it https://dummy.faircode.eu gets you a brief description of why it is used for HELO and how FairEmail has settings to modify it.
Strictly speaking this is not 5321 compliant either, not identifying the host, but it gets the job done better than a local host IP.
This issue was raised for K9 a long time ago. Yet Thunderbird Android still has the issue. Any progress?
Surely if using the Thunderbird name the Android version should align with the desktop version?
At the moment some users will find Thunderbird desktop can send mail but Thunderbird Android can't as their ISP blocks using the local host address.
Thunderbird for Android is going to be amazing, but it isn't the magic pill to fix all the issues and be just like Desktop. Thunderbird for Android is based on K-9 Mail, so it inherits both the good and the missing. We haven't had a chance to focus on this, but if someone would like to take a stab at it I'm sure we can provide some guidance and potentially setup something similar.
I'd like to avoid too many options and preferences, maybe a good first step would be for someone to propose how this could look to remain compatible in a way that we could provide sensible defaults. Maybe there is a way to detect the scenario and only include the dummy helo domain if 127.0.01 is rejected. Maybe there is another IP we could use instead? I'm sure some of you have done some research into this area.
Thanks for that kewisch. Your point about too many options is well made. My keep it simple suggestion, do what the desktop version does and use the device IP address. That is RFC compliant (though not perfect), and would stop those ISPs who block the local host address.
I'm not a dev - is this hard or trivial?
Thanks for your work Kewish, but I have the same issue on my main email provider, it works perfectly with Thunderbird Desktop, I agree with mutaunt
I suspect this will be somewhere between hard and trivial. Actually changing the IP address isn't a big deal, it is more putting thought into what is the "right thing" to do, and making sure all corner cases are covered. If there are UI options then that would make it a bit more difficult. If there is complex logic or the need to retry that would also make it more difficult to implemet.
If Thunderbird desktop uses the device IP, I think that would be a reasonable first step. I understand the desire to limit config options, so changing the IP from loopback to the device IP seems like a fair compromise to me, at least for now. It would at least (hopefully) get folks moving who use hosting providers that block the loopback, because right now we don't have any other option but to use a different app. I fought with Hostgator's support team, but they are not budging.
We use a loopback address specifically so we don't leak the internal IP address of the device. We also don't want to have to contact an external service to get the device's public IP address just so we can use it with the EHLO command (servers that actually care about the IP address get it from the connection, not the EHLO argument).
That pretty much leaves us with sending static data. So far we've learned that some providers don't accept "localhost", others don't accept "127.0.0.1". I think FairEmail's approach of using a fully-qualified domain name is a good next approach to try. I suggest we use ehlo.thunderbird.net.
Ref.: K9 android mail app - Authentication Failed Change HELO greeting #4428
I have been using K9 for years on multiple Android phones. This saga has been running for quite too long. Most email service providers are now filtering out 127.0.0.1 for several years already and for very good reasons.
I read through multiple older posts on this subject that this issue has existed on K-9 versions that immediately followed v5.6. This has make a LOT of people unhappy about K-9, forcing them to move to other email apps.
As now being under the Mozilla Thunderbird umbrella (which I am also a fervent desktop program user), and that the desktop apps is not using the localhost 127.0.0.1, and just working great (using ISP allocated address), it is perhaps time to make a move and open to the community recommendations. I read long time well known SørenR (SorenRR) arguments and suggestions, and I must concur with him.
I read last cketti's post from Nov 2024, and I must admit that this is showing signs of openness, in contrast with older posts when still under the original K-9 project. I also understand the original arguments using 127.0.0.1 for "anonymity", but this proves not working with several public email services.
As an alternate option and combining previous proposals, why not leaving the end user decide what he wants to expose as "origin" and have a configurable parameter (with some default values in a list) on the account outbound email config panel? With proper documentation, user will be able to figure out.
I am just wishing that a correction is being made in a timely manner as I really enjoy the application otherwise.
I'm opened for beta testing.
Thank you
I've asked for the website to be created so we could use ehlo.thunderbird.net. I believe we can simply make it a redirect to a support page, as long as the hostname resolves correctly.
Could folks who are running into this issue answer with the mail server they use? This would help for testing.
My service provider is HostGator. Happy to help test any potential updates!
We have the infrastructure set up now. This should now be a good first bug to switch to ehlo.thunderbird.net instead of [127.0.0.1]. If it is more complex than a few lines, we might want to use a feature flag in case we need to turn it off again.
https://ehlo.thunderbird.net
Hello! Found my way here after just reading the article about ehlo.thunderbird.net. From the wording, I thought it was already implemented. I'm just finding out it's for now a "dummy" page.
My server puts in my actual external IP (I use a mobile network) and port in the Received header, so my country/city/provider can all be determined.
This is what the header looks like:
Received: from [<external IP>] (port=XXXXX helo=[127.0.0.1]) by server.hostname.net
I've been looking for ways to get around it. I'm glad this is in the pipeline. Any idea as to when we'll see this working?
As soon as someone picks up this issue and sends a pull request, plus the time required to move it through beta and release. I didn't realize the SUMO article was discoverable, maybe we should put a banner at the top.
My server puts in my actual external IP (I use a mobile network) and port in the
Receivedheader, so my country/city/provider can all be determined. This is what the header looks like:Received: from [<external IP>] (port=XXXXX helo=[127.0.0.1]) by server.hostname.netI've been looking for ways to get around it. I'm glad this is in the pipeline. Any idea as to when we'll see this working?
Re-reading this, it isn't something we can change. You need to reach out to your provider in this case if they additionally put your real IP address in there. The fix here will just cover the case where the provider uses the EHLO hostname in the Received header and won't accept 127.0.0.1.
Note for someone to start on this, I believe it is just going here and returning ehlo.thunderbird.net. Maybe use a private const to reduce complexity of yet another method. https://github.com/thunderbird/thunderbird-android/blob/ff40998e1ebcb45411c28a97b0246c6d40ea4a99/mail/protocols/smtp/src/main/java/com/fsck/k9/mail/transport/smtp/SmtpTransport.kt#L103-L104
Maybe a test to adjust.
Re-reading this, it isn't something we can change. You need to reach out to your provider in this case if they additionally put your real IP address in there. The fix here will just cover the case where the provider uses the EHLO hostname in the
Receivedheader and won't accept 127.0.0.1.
I see your point. Though, I conjectured it did that because the client sent the loopback address. Maybe if it was a hostname, it'll just resolve that?
It uses Exim, so I glanced through the source a bit. Didn't have time to study it because there was a lot going on. But I did notice some conditionals based on whether the HELO had a hostname, IP, "localhost" or the loopback.
I haven't tested in advance. But I'll make a follow-up if I do before this is implemented.
Maybe if it was a hostname, it'll just resolve that?
I just tested this out. So my hosting provider divulges the mail client's public IP regardless of whatever value is used for EHLO. So if I need to hide my IP, I'll have to proxy the connection.
Not to be a dampener 🫣, but maybe in the future, we could have an option to specify what to use for EHLO? That's just considering the fact that some servers (like mine) attach the client's HELO value in the mail headers, like I showed above, hence "leaking" the client to the receiver, effectively undoing the "Hide mail client" option already present.
It's not a problem for me currently, but just for consideration. Would love to help out, but I'm yet to be at that point.
If we ever have something like about:config then I could imagine this being configurable, but I wouldn't want to expose that in the general preferences. What I think we could do is use [127.0.0.1] if the "Hide mail client" option is on. This still isn't 100% ideal, but I think it would be close enough. Maybe you could open a new ticket for that? @yantarou would you be willing to look into that once created?
@kewisch @emamoah
What I think we could do is use [127.0.0.1] if the "Hide mail client" option is on. This still isn't 100% ideal, but I think it would be close enough. Maybe you could open a new ticket for that? @yantarou would you be willing to look into that once created?
Willing, yes. (But I won't promise.)
Also, I'm not monitoring this repo's issues/tickets. So if someone creates an issue for that please ping me in there with a reference to this conversation.
