Moving mails by functions.quarantine.inc.php to inbox failed

[vigeland]

Contribution guidelines

• I've read the contribution guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... I understand that not following the below instructions will result in immediate closure and/or deletion of my issue.
• ... I have understood that this bug report is dedicated for bugs, and not for support-related inquiries.
• ... I have understood that answers are voluntary and community-driven, and not commercial support.
• ... I have verified that my issue has not been already answered in the past. I also checked previous issues.

Description

In some special cases functions.quarantine.inc.php quick_release does not work
I am not a php programmer, but debugging the file I am able to say 
$detail_row['msg'] = preg_replace('/^X-Spam-Flag: (.*)/', 'X-Pre-Release-Spam-Flag $1', $detail_row['msg']); 
sometimes not work for me. 
Dumping $detail_row['msg'] before and after call, show the same "X-Spam-Flag: YES" so moving the mail to inbox does not work. The mail is sent twice to junk folder.
It occurs for one user on every mail.

Using the last docker release. 2024-08a / x86

Logs:

X-Spam-Score: 8.5 (++++++++)
X-Spam-Flag: YES
X-Spam-Report: Action: add header

and 

X-Spam-Score: 8.5 (++++++++)
X-Spam-Flag: YES
X-Spam-Report: Action: add header

Steps to reproduce:

-

Which branch are you using?

master

Which architecture are you using?

x86

Operating System:

Debian 6.1.90-1

Server/VM specifications:

8 GB / 4 core

Is Apparmor, SELinux or similar active?

no

Virtualization technology:

KVM

Docker version:

26.1.2

docker-compose version or docker compose version:

docker-compose version 1.29.2

mailcow version:

2024-08a

Reverse proxy:

ngnix

Logs of git diff:

-

Logs of iptables -L -vn:

-

Logs of ip6tables -L -vn:

-

Logs of iptables -L -vn -t nat:

-

Logs of ip6tables -L -vn -t nat:

-

DNS check:

-

[vigeland]

m (PCRE_MULTILINE)
By default, PCRE treats the subject string as consisting of a single "line" of characters (even if it actually contains several newlines). The "start of line" metacharacter (^) matches only at the start of the string, while the "end of line" metacharacter ($) matches only at the end of the string, or before a terminating newline (unless D modifier is set). This is the same as Perl. When this modifier is set, the "start of line" and "end of line" constructs match immediately following or immediately before any newline in the subject string, respectively, as well as at the very start and end. This is equivalent to Perl's /m modifier. If there are no "\n" characters in a subject string, or no occurrences of ^ or $ in a pattern, setting this modifier has no effect.

'/^X-Spam-Flag: (.)/' , a mail has newlines.
So the Parameter have '/^X-Spam-Flag: (.)/m' ?

[vigeland]

I had change
$detail_row['msg'] = preg_replace('/^X-Spam-Flag: (.)/', 'X-Pre-Release-Spam-Flag $1', $detail_row['msg']);
to
$detail_row['msg'] = preg_replace('/^X-Spam-Flag: (.)/m', 'X-Pre-Release-Spam-Flag: $1', $detail_row['msg']);

Now X-Spam-Flag: is replaced and the mail is copied to inbox. From my point of view, may be a incomming mail has a X-Spam-Flag and respamd tag the mail as spam so X-Spam-Flag is in the mail. The preg_replace has never worked, but there is no problem with mails that are not tagged as X-Spam. I check mail from an other user, the mails in the junk folder has no X-Spam-Flag, so the moving works. I don't own exactly how mallow works.

I think, it is a bug.

[vigeland]

A other thing.
Make it sense to do a quick_release if a user move a mail to inbox , without learning as ham ?
qhandler.twig alway build a quick_release link without learning as ham.
I saw no possibility to setup a release with ham learning.

[mkuron]

I'm not familiar with the quarantine code, so I am going to unassign this ticket from myself.

[vigeland]

From my point of view, the first step, independent of the source.
to a php programmer :
$detail_row['msg'] = preg_replace('/^X-Spam-Flag: (.)/', 'X-Pre-Release-Spam-Flag $1', $detail_row['msg']);
Could the call be a right one, if the $detail_row['msg'] hold a mail with CR / CR+NL as a string and it is need to replace X-Spam-Flag: YES at line eg. 46 in the mail.
If the answer No then
$detail_row['msg'] = preg_replace('/^X-Spam-Flag: (.)/m', 'X-Pre-Release-Spam-Flag: $1', $detail_row['msg']);
do the call the right think ?
Is the answer Yes: we had to change two lines in functions.quarantine.inc.php.

[mrclschstr]

I seem to have stumbled across the same problem: I'm usually very meticulous about sorting out my spam, and I'm very conscientious about moving spam that arrives in my Inbox to the Spam folder so that it will be treated as spam in the future. Similarly, I move emails that have been mistakenly treated as spam back to my Inbox.

Unfortunately, with the current version (or maybe even earlier versions?), I can no longer move emails from the Spam folder back to the Inbox. Immediately after moving it from the Spam folder, the email is displayed in the Spam folder again and is missing in the Inbox. At the same time, I cannot find any errors in the container logs. I am relatively sure that this workflow worked in the past... Is this a bug or does it work as intended? Do I have to take the detour via quarantine in the Mailcow UI in the future?

EDIT: The workflow should be valid according to the documentation:

Rspamd learns mail as spam or ham when you move a message in or out of the junk folder to any mailbox besides trash. This is achieved by using the Sieve plugin "sieve_imapsieve" and parser scripts.

[vigeland]

take a look , with the soga webgui, in the raw maildata.
If it contains "X-Spam-Flag: YES" in one line, so maybe the same problem.
Sieve sort these mail into the Spam folder.

so the problem could be that X-Spam-Flag: YES in some times are now in the mails.

[mrclschstr]

If it contains "X-Spam-Flag: YES" in one line, so maybe the same problem.

Yes, I can confirm.

[milkmaker]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[vigeland]

No php developer here?

Question again:
From my point of view, the first step, independent of the source.
to a php programmer :
$detail_row['msg'] = preg_replace('/^X-Spam-Flag: (.)/', 'X-Pre-Release-Spam-Flag $1', $detail_row['msg']);
Could the call be a right one, if the $detail_row['msg'] hold a mail with CR / CR+NL as a string and it is need to replace X-Spam-Flag: YES at line eg. 46 in the mail.
If the answer No then
$detail_row['msg'] = preg_replace('/^X-Spam-Flag: (.)/m', 'X-Pre-Release-Spam-Flag: $1', $detail_row['msg']);
do the call the right think ?
Is the answer Yes: we had to change two lines in functions.quarantine.inc.php.