How does email forwarding works?

Discuss Advanced use case
1

I searched in the docs and forum but didn’t find info on what really does the “e-mail forwarding” functionality accessible from the SSO.

What I would expect to happen:

Let’s say my yunohost account is something@yuno.tld, I put as forward address me@otherserver.tld.
Now I expect that all incoming mails to something@yuno.tld will be forwarded to my other email address me@otherserver.tld and removed from something@yuno.tld inbox.

What happens instead:

  • If I open the roundcube of yuno.tld, mails are still there, old ones (ok) but also newly received ones (from after adding the forward).
  • I’m not receiving any forwarded mail in me@otherserver.tld inbox.

Maybe the first point is not a bug but a feature, and mails are not removed after being forwarded. Is there a way to have them removed (in order not to occupy unnecessary space on yuno.tld disk)?

For the second point, maybe there is something wrong in my setup but I don’t really understand what.

The specific setup:

The two servers yuno.tld and otherserver.tld are in fact two vms in a same machine. What I’m trying to achieve is to receive admin emails of yuno.tld in the inbox of otherserver.tld admin, so I don’t have to check multiple inboxes to see problems.
At the moment yuno.tld cannot receive emails from the outside, since all the mail traffic coming to the main machine is forwarded to otherserver.tld (the main machine has only one IP so I don’t think I can have multiple mail servers accessible from the outside). But I would expect the server to still be able to send emails, especially to a neighbor VM.

If anyone has any idea or insights on how forwarding normally works or on this specific setup.
Thanks :slight_smile:

1 Like
2

Trying to investigate the issue, I found that in /var/log/mail.log (of yuno.tld server), I seem to have the following log every time a mail arrives to the admin user of yuno.tld and should be forwarded:

Sep 13 03:35:52 yuno postfix/pickup[6703]: BF48E484574: uid=0 from=<root@yuno.tld>
Sep 13 03:35:52 yuno postfix/cleanup[13281]: BF48E484574: message-id=<20210913013552.BF48E484574@yuno.tld>
Sep 13 03:35:52 yuno postfix/qmgr[6986]: BF48E484574: from=<root@yuno.tld>, size=3803, nrcpt=2 (queue active)
Sep 13 03:35:54 yuno dovecot: lda(me@otherserver.tld)<13289><lQiQNPiqPmHpMwAA7MJUCg>: sieve: msgid=<20210913013552.BF48E484574@yuno.tld>: stored mail into mailbox 'INBOX'
Sep 13 03:35:54 yuno postfix/pipe[13285]: BF48E484574: to=<me@otherserver.tld>, orig_to=<root@yuno.tld>, relay=dovecot, delay=1.6, delays=0.2/0.01/0/1.4, dsn=2.0.0, status=sent (delivered via dovecot service (lda(me@otherserver.tld,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission d))
Sep 13 03:35:54 yuno dovecot: lda(something@yuno.tld)<13290><HXmTNPiqPmHqMwAA7MJUCg>: sieve: msgid=<20210913013552.BF48E484574@yuno.tld>: stored mail into mailbox 'INBOX'
Sep 13 03:35:54 yuno postfix/pipe[13286]: BF48E484574: to=<something@yuno.tld>, orig_to=<root@yuno.tld>, relay=dovecot, delay=1.8, delays=0.2/0.01/0/1.6, dsn=2.0.0, status=sent (delivered via dovecot service (lda(something@yuno.tld,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permis))
Sep 13 03:35:54 yuno postfix/qmgr[6986]: BF48E484574: removed

something is administrator of yuno.tld and
me is administrator of otherserver.tld

Apart from Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permis, I don’t see anything that looks like an error. And from what I found on internet, I don’t see what the relation could be between that error and the forwarding issue.
On the side of otherserver.tld, there are no related logs: not much logs at that time, and no log whatsoever mentioning yuno.tld.

My knowledge of dovecot and postfix is close to zero, so any help deciphering those logs, or suggestions of direction towards which to inquire are quite appreciated.
:slight_smile:

3

The normal behaviour of forward feature is to resend the mail to the new address but keep it on the yunohost mail box. Like forward on gmail or other kind of mail providers.

My hypothesis on why your email is not received on the second VM

  • your config is not correct and the mail is refused by the antispam feature of the destination server
  • you have “added” the domain of destination on your yunohost, so the yunohost think to be the destination machine
  • you use a router doing hairpining, so when you try to send an email to the destination machine, it send it to the router…

What about your mail diagsnosis result ?
Have you tested to send email to mail-tester.com ?
*

4

Thanks a lot for the explanations and help :slight_smile:

Exploring what you proposed, this is where I’m getting to:

  • you have “added” the domain of destination on your yunohost, so the yunohost think to be the destination machine

Indeed that was the case, I removed it and now the log messages are different, so there is some improvement.

  • you use a router doing hairpining, so when you try to send an email to the destination machine, it send it to the router…

The server is not behind a home router, so I doubt that this should be relevant.

  • your config is not correct and the mail is refused by the antispam feature of the destination server

I wonder if there could be something related to that.
The destination server otherserver.tld is a yunohost server.
I tried sending a mail from something@yuno.tld to me@otherserver.tld, and after a while in yuno.tld server, I’m getting the following log (in /var/log/mail.log):

Sep 16 12:15:08 yuno postfix/smtp[30721]: connect to otherserver.tld[XXX.XXX.XXX.XXX]:25: Connection timed out
Sep 16 12:15:09 yuno postfix/smtp[30721]: BBA0648454A: to=<me@otherserver.tld>, relay=none, delay=30, delays=0.17/0/30/0, dsn=4.4.1, status=deferred (connect to otherserver.tld[XXX.XXX.XXX.XXX]:25: Connection timed out)

On otherserver.tld I’m still not getting any related log.

I also tried sending a mail to mail-tester from yuno.tld. I obviously get a pretty bad result, since the server DNS and other things are not properly configured to receive mails. Could it be that otherserver.tld refuses them for that reason? Maybe what I’m saying is pretty dumb and the answer is obviously yes. But I was expecting to at least see some logs in the receiving yunohost server, when it refuses a mail.

Here are the things that mail-tester complains about:

message rating
[SPF] yuno.tld does not allow your server XXX.XXX.XXX.XXX to use something@yuno.tld -1
Your DKIM signature is not valid -3
You do not have a DMARC record orange ok
Your reverse DNS does not match with your sending domain. orange ok
We didn’t find a mail server (MX Record) behind your domain name yuno.tld. -3
Your hostname yuno.tld is assigned to a server. green ok

And spam assassin gives a rating of -0.5 with the following details:

rating code message
-0.1 DKIM_INVALID DKIM or DK signature exists, but is not valid
-0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid This rule is automatically applied if your email contains a DKIM signature but other positive rules will also be added if your DKIM signature is valid. See immediately below.
-0.32 KHOP_HELO_FCRDNS Relay HELO differs from its IP’s reverse DNS
-0.001 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
-0.001 SPF_NONE SPF: sender does not publish an SPF Record

From those reports I’m seeing that even if I can’t resolve all issues, some I can. I didn’t expect to have to do all this setup just for internal exchanges of emails, but maybe I still have to.

5

I suggest you to adapt your spf and your dkim to allow your server to send mail.

You are right, you should see some info in log on your destination server. Have you check the /var/log/mail.err ?

6

I’ll adapt the spf and dkim and see how things go.

On the log sides, I had actually checked /var/log/mail.err and it doesn’t contain anything related to refusing mails. Actually, the only log it seems to contain is a reccurring log that looks like the following (it seems to happen to most if not all accounts in the server):

Sep 15 16:22:48 yunohostserver dovecot: lda(someaccount@yunohostserver.tld)<19297><tymbLLgBQmFhSwAAXydDQw>: Error: sieve: msgid=<bf08a61dbd180365d821f7892.c11afa9702.20210915140329.15070b0a90.1225e5d1@mail234.atl271.mcdlv.net>: failed to store into mailbox 'Junk': Mailbox doesn't exist: Junk

I wonder if yunohost has something misconfigured about the junk mailbox for this to happen recurrently. Maybe I should report it to yunohost bug tracker?

7

So now with proper MX, spf and dkim entries in the DNS mail-tester gives a score of 10/10.
I also tested sending mails to the outside world, and that works.
But the problem persists and emails are not being sent to the other vm.
I’m realizing the problem is really specific to my particular setup, and doesn’t seem related to yunohost at all, sorry for that.

If someone still has some ideas on what could cause the issue, I’m really thankful for any advice.
To summarize the setup is the following:

  • main machine with one IP
  • two vms both running yunohost
  • all incoming traffic for port 25, 587, 993 in the main machine is forwarded to VM1
  • VM1 server has fully working mailing stack that is used daily by users
  • VM2’s emails aren’t used except for receiving internal admin messages
  • VM2 is not able to send mail to VM1

If I can’t make it work I can also fallback on forwarding emails to an external account, since that seems to work.

Thanks for any advice, and thanks @ljf for the suggestions you gave me :slight_smile:

8

In VM1, try:

dig +short A DOMAIN_VM2
ping IP_VM2
traceroute IP_VM2
9

From inside VM1:

  • the dig command you gave doesn’t return the private IP of VM2, it returns the public IP of all vms (which is the one of the main server) - don’t know if that’s normal
  • I can ping the public IP
  • traceroute <PUBLIC_IP> just gives 30 lines with * * *
  • I can also ping the private IP of VM2
  • traceroute <VM2_PRIVATE_IP> just gives 30 lines with * * *
10

But VM1 is able to send mail to VM2 ?

I think i have misunderstood your text:
In VM2 try:

dig +short A DOMAIN_VM1
ping PRIVATE_IP_VM1
traceroute PRIVATE_IP_VM1
11

Well I’m not expecting VM2 to be able to receive mails (at least from the outside) because the traffic coming to mail ports of the host are forwarded to VM1.
I don’t know if there is a way to have multiple mail servers on the same IP, it would be nice, but not knowing how to do that, this is the setup I have at the moment.
I just tested sending a mail from VM1 to VM2, just in case, and I get the same log than when I was doing it the other way around: “connection time out”.

If I do the commands from VM2 towards VM1, I get basically the same results as I was saying in previous post:

  • dig gives me the public IP of the server
  • ping to private VM1 IP works fine
  • traceroute to private VM1 IP: 30 lines of * * *
12

SO it’s your problem. To send an email from vm1 to vm2 you should be able to resolve in vm1 the vm2 domain name as it’s private IP. Otherwise you just try to send a mail from vm1 to vm1…

SO i suggest you to edit your vm1 /etc/hosts file with:

PRIVATE_IP_VM DOMAIN_NAME_VM
13

Hey @ljf, thanks for trying to help me figure the issue out, I appreciate a lot.

I think there’s been a misunderstanding, I’m not expecting to be able to send mails from VM1 to VM2. I’m only expecting to be able to send from VM2 to VM1.
Since VM2 resolves VM1 domain’s IP with the server’s public IP, and the host will redirect traffic to VM1, I would imagine it should work.

Still I tried to add VM1 local IP in VM2 /etc/hosts. Then I restarted postfix. But I’m still getting the same log:

Sep 17 09:57:53 VM2 postfix/smtp[14035]: connect to vm1domain.tld[XXX.XXX.XXX.XXX]:25: Connection timed out
Sep 17 09:57:53 VM2 postfix/smtp[14035]: 6772E4845AB: to=<vm1user@vm1domain.tld>, relay=none, delay=4575, delays=4545/0.03/30/0, dsn=4.4.1, status=deferred (connect to vm1domain.tld[XXX.XXX.XXX.XXX]:25: Connection timed out)

The strange thing is that XXX.XXX.XXX.XXX is still the public IP.
Also I don’t understand the Connection timed out thing, since ping is working (both on the private and public IP).
I checked fail2ban logs in VM1, just in case, but the private IP of VM2 is not banned.

Can’t really understand what’s going on.

14

From VM2 try to contact the mail server VM1 directly:

$ telnet DOMAIN1 25

Puis tape:

EHLO DOMAIN2

Tu devrais recevoir un trucs comme:

250-hostname.domainname.tld
250-AUTH=LOGIN CRAM-MD5 PLAIN
250-AUTH LOGIN CRAM-MD5 PLAIN
250-STARTTLS
250-PIPELINING
250 8BITMIME
1 Like
15

This is what I’m getting:

$ telnet vm1domain.tld 25
Trying 192.168.XXX.XXX...    <-- telnet resolves to VM1 private IP
Connected to vm1domain.tld.
Escape character is '^]'.
220 vm1domain.tld Service ready
EHLO vm2domain.tld
250-vm1domain.tld
250-PIPELINING
250-SIZE 35914708
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250 CHUNKING
16

So your mail servers can speak together, you should see some log in /var/log/mail.log when you try to send mail !

You could try to send an email in command line from vm2

echo test | mail -s test root@vm1domain.tld

I don’t know how you tested to send mail ? via roundcube ?

17

Yes I was testing sending mail using roundcube.

So I just did the following test:

  • ran sudo tail /var/log/mail.log -f in VM1
  • ran sudo tail /var/log/mail.log -f in VM2
  • leaving the two previous terminal opened, I ran from VM2 echo test | mail -s test root@vm1domain.tld

What I get:

  • no log at all in VM1
  • the following log, again, in VM2 (the second part after a few minutes)
Sep 17 16:33:36 VM2 postfix/pickup[1430]: 7F3CD4845B8: uid=1007 from=<admin@vm2domain.tld>
Sep 17 16:33:36 VM2 postfix/cleanup[5880]: 7F3CD4845B8: message-id=<20210917143336.7F3CD4845B8@vm2domain.tld>
Sep 17 16:33:36 VM2 postfix/qmgr[7654]: 7F3CD4845B8: from=<admin@vm2domain.tld>, size=349, nrcpt=1 (queue active)
Sep 17 16:34:06 VM2 postfix/smtp[5884]: connect to vm1domain.tld[PUBLIC_IP]:25: Connection timed out
Sep 17 16:34:06 VM2 postfix/smtp[5884]: 7F3CD4845B8: to=<root@vm1domain.tld>, relay=none, delay=30, delays=0.16/0/30/0, dsn=4.4.1, status=deferred (connect to vm1domain.tld[PUBLIC_IP]:25: Connection timed out)

I also tried again the telnet command, then closed it, and I can see those logs in VM1:

Sep 17 16:43:50 VM1 postfix/smtpd[12295]: connect from unknown[VM2_LOCAL_IP]
Sep 17 16:44:17 VM1 postfix/smtpd[12295]: lost connection after EHLO from unknown[VM2_LOCAL_IP]
Sep 17 16:44:17 VM1 postfix/smtpd[12295]: disconnect from unknown[VM2_LOCAL_IP] ehlo=1 commands=1

So yep indeed they can communicate, but somehow with postfix they don’t manage to communicate. I’m quite confused.

Edit: I also checked /var/log/mail.errof both vms, still nothing related.

18

Have you tried to wait 30 minutes to check if the mail is arrived ? Yunohost has a greylisting mechanism sofirst email need to be resent 30 minutes later.

19

Well, I’ve been sending many test emails since yesterday, I didn’t receive anything yet.
Looking once more at the logs in VM1, I didn’t seem to see related messages 30 minutes after, but I found the following line:

Sep 17 16:49:42 VM1 postfix/anvil[12300]: statistics: max connection rate 1/60s for (smtp:VM2_PRIVATE_IP) at Sep 17 16:42:16

that’s the only log I found that seem to refer to VM2. Could it be related?

20

Thanks a lot @ljf for all your efforts in trying to find a solution to the issue. And sorry for creating an infinite thread.

Not having clues on where the problem may come from, I decided to forward mails to an external email address. At least that works fine.

Thanks again, I very much appreciated your efforts and advise :slight_smile:

1 Like