Bug 11813

Summary:	Monitoring sends crappy mails
Product:	Infrastructure	Reporter:	Peter Müller <peter.mueller>
Component:	Monitoring	Assignee:	Peter Müller <peter.mueller>
Status:	CLOSED FIXED	QA Contact:	Michael Tremer <michael.tremer>
Severity:	Major Usability
Priority:	Will only affect a few users
Version:	unspecified
Hardware:	unspecified
OS:	Unspecified
See Also:	https://bugzilla.ipfire.org/show_bug.cgi?id=11902

Description Peter Müller 2018-08-06 22:56:04 UTC

The mail alerts sent by the monitoring systems have some really poor spam score here:

X-Spam-Status: Yes, score=10.28
X-Rspamd-Server: mx-nbg.link38.eu
Authentication-Results: mx-nbg.link38.eu;
	dkim=pass header.d=ipfire.org;
	dmarc=pass (policy=none) header.from=ipfire.org
X-Spamd-Result: default: False [10.28 / 11.00];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 TO_NEEDS_ENCODING(1.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_DN_ALL(0.00)[];
	 DKIM_TRACE(0.00)[ipfire.org:+];
	 RCVD_IN_DNSWL_MED(-2.00)[1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.5.2.0.0.3.8.1.7.0.7.4.0.1.0.0.2.list.dnswl.org : 127.0.9.2];
	 FORGED_RECIPIENTS_FORWARDING(0.00)[];
	 TO_DOM_EQ_FROM_DOM(0.00)[];
	 PREVIOUSLY_DELIVERED(0.00)[peter.mueller@ipfire.org];
	 FORGED_SENDER(0.00)[monitoring@ipfire.org,icinga@monitoring01.i.ipfire.org];
	 FORGED_RECIPIENTS(0.00)[peter.mueller@ipfire.org,peter.mueller@link38.eu];
	 FORWARDED(0.00)[peter.mueller@ipfire.org];
	 R_DKIM_ALLOW(-0.20)[ipfire.org];
	 FROM_NEQ_ENVFROM(0.00)[monitoring@ipfire.org,icinga@monitoring01.i.ipfire.org];
	 SPAM_FLAG(5.00)[];
	 MX_MISSING(3.50)[requested record is not found];
	 DMARC_POLICY_ALLOW(-0.25)[ipfire.org,none];
	 MX_INVALID(0.50)[greylisted];
	 URL_IN_SUBJECT(0.40)[mail01.i.ipfire.org];
	 FROM_HAS_DN(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 R_SPF_NA(0.00)[];
	 RCVD_TLS_LAST(0.00)[];
	 IP_SCORE(-3.57)[ip: (-9.33), ipnet: 2001:470::/32(-4.69), asn: 6939(-3.75), country: US(-0.10)];
	 ARC_NA(0.00)[];
	 MISSING_MIME_VERSION(2.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 GREYLIST(0.00)[pass,meta];
	 ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US];
	 RCPT_COUNT_ONE(0.00)[1];
	 FORGED_SENDER_FORWARDING(0.00)[]
X-Spam-Level: **********
X-Spam: Yes

Could you adjust the sending script behind this so they are RFC-compliant and have some external parsable MX record (maybe set up an alias <monitoring@ipfire.org>)? Thanks.

Comment 1 Peter Müller 2018-09-23 10:53:39 UTC

Some rspamd symbols of a recent mail:

[Metric: default]
Action: reject
Spam: true
Score: 15.85 / 11.00
Symbol: ARC_NA (0.00)
Symbol: BROKEN_CONTENT_TYPE (1.50)
Symbol: DKIM_TRACE (0.00)[ipfire.org:+]
Symbol: DMARC_POLICY_ALLOW (-0.25)[ipfire.org, none]
Symbol: FROM_HAS_DN (0.00)
Symbol: FROM_NEQ_ENVFROM (0.00)[monitoring@ipfire.org, icinga@monitoring01.i.ipfire.org]
Symbol: MIME_GOOD (-0.10)[text/plain]
Symbol: MISSING_MIME_VERSION (2.00)
Symbol: MX_INVALID (0.50)[greylisted]
Symbol: MX_MISSING (3.50)[requested record is not found]
Symbol: RCPT_COUNT_ONE (0.00)[1]
Symbol: RCVD_COUNT_FIVE (0.00)[5]
Symbol: RCVD_NO_TLS_LAST (0.00)
Symbol: R_DKIM_ALLOW (-0.20)[ipfire.org]
Symbol: R_MISSING_CHARSET (2.50)
Symbol: R_SPF_NA (0.00)
Symbol: SPAM_FLAG (5.00)
Symbol: TO_DN_ALL (0.00)
Symbol: TO_DOM_EQ_FROM_DOM (0.00)
Symbol: TO_NEEDS_ENCODING (1.00)
Symbol: URL_IN_SUBJECT (0.40)[git01.ipfire.org]
Message-ID: 20180923022552.F238B110A292@monitoring01.i.ipfire.org

I will try to find out which system generates these and have a look at the script myself.

Comment 2 Peter Müller 2018-09-23 11:36:25 UTC

Issues with content encoding headers should be fixed now:

(snip from /etc/icinga2/scripts/send-notification on monitoring host)

mail_header+="Content-Type: text/plain; charset=utf-8\n"
mail_header+="Content-Transfer-Encoding: 8bit\n"
mail_header+="MIME-Version: 1.0\n"

To still needs encoding sometimes (base64?) and the message ID needs to be changed so it provides valid MX data. I will care about this.

Comment 3 Peter Müller 2018-10-16 19:09:20 UTC

Corrected FORGED_SENDER , testing...

Comment 4 Peter Müller 2018-10-17 17:00:09 UTC

Rewriting the Message-ID hat nothing to do with some poor reputation (see #11902).

I adjusted the notification script @ /etc/icinga2/scripts/send-notification to make sure the envelope sender is equal to the MIME sender. That way, SPF and MX lookup failures are avoided as well as some symbols like FORGED_SENDER .

rspamd status of recent monitoring mails is now:

X-Spamd-Result: default: False [-5.96 / 11.00];
	 ARC_NA(0.00)[];
	 FORGED_RECIPIENTS_FORWARDING(0.00)[];
	 R_DKIM_ALLOW(-0.20)[ipfire.org];
	 URL_IN_SUBJECT(0.40)[web04.ipfire.org];
	 FROM_HAS_DN(0.00)[];
	 FORWARDED(0.00)[peter.mueller@ipfire.org];
	 R_SPF_ALLOW(-0.20)[+ip4:81.3.27.42];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_GOOD(-0.10)[text/plain];
	 PREVIOUSLY_DELIVERED(0.00)[peter.mueller@ipfire.org];
	 RCPT_COUNT_ONE(0.00)[1];
	 RCVD_COUNT_THREE(0.00)[3];
	 IP_SCORE(-3.40)[ip: (-8.91), ipnet: 81.3.0.0/18(-4.45), asn: 24679(-3.56), country: DE(-0.09)];
	 TO_DN_ALL(0.00)[];
	 DKIM_TRACE(0.00)[ipfire.org:+];
	 MX_GOOD(-0.01)[cached: mail01.ipfire.org];
	 DMARC_POLICY_ALLOW(-0.25)[ipfire.org,none];
	 RCVD_IN_DNSWL_MED(-0.20)[42.27.3.81.list.dnswl.org : 127.0.9.2];
	 TO_NEEDS_ENCODING(1.00)[];
	 FROM_EQ_ENVFROM(0.00)[];
	 RCVD_TLS_LAST(0.00)[];
	 ASN(0.00)[asn:24679, ipnet:81.3.0.0/18, country:DE];
	 FORGED_RECIPIENTS(0.00)[peter.mueller@ipfire.org,peter.mueller@link38.eu];
	 MID_RHS_MATCH_FROM(0.00)[];
	 TO_DOM_EQ_FROM_DOM(0.00)[]
X-Spam-Status: No, score=-5.96
X-Rspamd-Server: mx-nbg.link38.eu

Rejecting some mails falsely is now very unlikely.

Closing this ticket, but we need to make sure our infrastructure always sends mails with equal envelope and MIME sender, using some domain (e.g., @ipfire.org) with at least valid MX records.