Bug 11813 - Monitoring sends crappy mails
Summary: Monitoring sends crappy mails
Status: CLOSED FIXED
Alias: None
Product: Infrastructure
Classification: Unclassified
Component: Monitoring (show other bugs)
Version: unspecified
Hardware: unspecified Unspecified
: Will only affect a few users Major Usability
Assignee: Peter Müller
QA Contact: Michael Tremer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-08-06 22:56 UTC by Peter Müller
Modified: 2018-10-17 17:00 UTC (History)
0 users

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Müller 2018-08-06 22:56:04 UTC 
The mail alerts sent by the monitoring systems have some really poor spam score here:

X-Spam-Status: Yes, score=10.28
X-Rspamd-Server: mx-nbg.link38.eu
Authentication-Results: mx-nbg.link38.eu;
	dkim=pass header.d=ipfire.org;
	dmarc=pass (policy=none) header.from=ipfire.org
X-Spamd-Result: default: False [10.28 / 11.00];
	 BROKEN_CONTENT_TYPE(1.50)[];
	 TO_NEEDS_ENCODING(1.00)[];
	 R_MISSING_CHARSET(2.50)[];
	 TO_DN_ALL(0.00)[];
	 DKIM_TRACE(0.00)[ipfire.org:+];
	 RCVD_IN_DNSWL_MED(-2.00)[1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.5.2.0.0.3.8.1.7.0.7.4.0.1.0.0.2.list.dnswl.org : 127.0.9.2];
	 FORGED_RECIPIENTS_FORWARDING(0.00)[];
	 TO_DOM_EQ_FROM_DOM(0.00)[];
	 PREVIOUSLY_DELIVERED(0.00)[peter.mueller@ipfire.org];
	 FORGED_SENDER(0.00)[monitoring@ipfire.org,icinga@monitoring01.i.ipfire.org];
	 FORGED_RECIPIENTS(0.00)[peter.mueller@ipfire.org,peter.mueller@link38.eu];
	 FORWARDED(0.00)[peter.mueller@ipfire.org];
	 R_DKIM_ALLOW(-0.20)[ipfire.org];
	 FROM_NEQ_ENVFROM(0.00)[monitoring@ipfire.org,icinga@monitoring01.i.ipfire.org];
	 SPAM_FLAG(5.00)[];
	 MX_MISSING(3.50)[requested record is not found];
	 DMARC_POLICY_ALLOW(-0.25)[ipfire.org,none];
	 MX_INVALID(0.50)[greylisted];
	 URL_IN_SUBJECT(0.40)[mail01.i.ipfire.org];
	 FROM_HAS_DN(0.00)[];
	 RCVD_COUNT_THREE(0.00)[3];
	 R_SPF_NA(0.00)[];
	 RCVD_TLS_LAST(0.00)[];
	 IP_SCORE(-3.57)[ip: (-9.33), ipnet: 2001:470::/32(-4.69), asn: 6939(-3.75), country: US(-0.10)];
	 ARC_NA(0.00)[];
	 MISSING_MIME_VERSION(2.00)[];
	 MIME_GOOD(-0.10)[text/plain];
	 GREYLIST(0.00)[pass,meta];
	 ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US];
	 RCPT_COUNT_ONE(0.00)[1];
	 FORGED_SENDER_FORWARDING(0.00)[]
X-Spam-Level: **********
X-Spam: Yes

Could you adjust the sending script behind this so they are RFC-compliant and have some external parsable MX record (maybe set up an alias <monitoring@ipfire.org>)? Thanks.
Comment 1 Peter Müller 2018-09-23 10:53:39 UTC 
Some rspamd symbols of a recent mail:

[Metric: default]
Action: reject
Spam: true
Score: 15.85 / 11.00
Symbol: ARC_NA (0.00)
Symbol: BROKEN_CONTENT_TYPE (1.50)
Symbol: DKIM_TRACE (0.00)[ipfire.org:+]
Symbol: DMARC_POLICY_ALLOW (-0.25)[ipfire.org, none]
Symbol: FROM_HAS_DN (0.00)
Symbol: FROM_NEQ_ENVFROM (0.00)[monitoring@ipfire.org, icinga@monitoring01.i.ipfire.org]
Symbol: MIME_GOOD (-0.10)[text/plain]
Symbol: MISSING_MIME_VERSION (2.00)
Symbol: MX_INVALID (0.50)[greylisted]
Symbol: MX_MISSING (3.50)[requested record is not found]
Symbol: RCPT_COUNT_ONE (0.00)[1]
Symbol: RCVD_COUNT_FIVE (0.00)[5]
Symbol: RCVD_NO_TLS_LAST (0.00)
Symbol: R_DKIM_ALLOW (-0.20)[ipfire.org]
Symbol: R_MISSING_CHARSET (2.50)
Symbol: R_SPF_NA (0.00)
Symbol: SPAM_FLAG (5.00)
Symbol: TO_DN_ALL (0.00)
Symbol: TO_DOM_EQ_FROM_DOM (0.00)
Symbol: TO_NEEDS_ENCODING (1.00)
Symbol: URL_IN_SUBJECT (0.40)[git01.ipfire.org]
Message-ID: 20180923022552.F238B110A292@monitoring01.i.ipfire.org

I will try to find out which system generates these and have a look at the script myself.
Comment 2 Peter Müller 2018-09-23 11:36:25 UTC 
Issues with content encoding headers should be fixed now:

(snip from /etc/icinga2/scripts/send-notification on monitoring host)

mail_header+="Content-Type: text/plain; charset=utf-8\n"
mail_header+="Content-Transfer-Encoding: 8bit\n"
mail_header+="MIME-Version: 1.0\n"

To still needs encoding sometimes (base64?) and the message ID needs to be changed so it provides valid MX data. I will care about this.
Comment 3 Peter Müller 2018-10-16 19:09:20 UTC 
Corrected FORGED_SENDER , testing...
Comment 4 Peter Müller 2018-10-17 17:00:09 UTC 
Rewriting the Message-ID hat nothing to do with some poor reputation (see #11902).

I adjusted the notification script @ /etc/icinga2/scripts/send-notification to make sure the envelope sender is equal to the MIME sender. That way, SPF and MX lookup failures are avoided as well as some symbols like FORGED_SENDER .

rspamd status of recent monitoring mails is now:

X-Spamd-Result: default: False [-5.96 / 11.00];
	 ARC_NA(0.00)[];
	 FORGED_RECIPIENTS_FORWARDING(0.00)[];
	 R_DKIM_ALLOW(-0.20)[ipfire.org];
	 URL_IN_SUBJECT(0.40)[web04.ipfire.org];
	 FROM_HAS_DN(0.00)[];
	 FORWARDED(0.00)[peter.mueller@ipfire.org];
	 R_SPF_ALLOW(-0.20)[+ip4:81.3.27.42];
	 BAYES_HAM(-3.00)[100.00%];
	 MIME_GOOD(-0.10)[text/plain];
	 PREVIOUSLY_DELIVERED(0.00)[peter.mueller@ipfire.org];
	 RCPT_COUNT_ONE(0.00)[1];
	 RCVD_COUNT_THREE(0.00)[3];
	 IP_SCORE(-3.40)[ip: (-8.91), ipnet: 81.3.0.0/18(-4.45), asn: 24679(-3.56), country: DE(-0.09)];
	 TO_DN_ALL(0.00)[];
	 DKIM_TRACE(0.00)[ipfire.org:+];
	 MX_GOOD(-0.01)[cached: mail01.ipfire.org];
	 DMARC_POLICY_ALLOW(-0.25)[ipfire.org,none];
	 RCVD_IN_DNSWL_MED(-0.20)[42.27.3.81.list.dnswl.org : 127.0.9.2];
	 TO_NEEDS_ENCODING(1.00)[];
	 FROM_EQ_ENVFROM(0.00)[];
	 RCVD_TLS_LAST(0.00)[];
	 ASN(0.00)[asn:24679, ipnet:81.3.0.0/18, country:DE];
	 FORGED_RECIPIENTS(0.00)[peter.mueller@ipfire.org,peter.mueller@link38.eu];
	 MID_RHS_MATCH_FROM(0.00)[];
	 TO_DOM_EQ_FROM_DOM(0.00)[]
X-Spam-Status: No, score=-5.96
X-Rspamd-Server: mx-nbg.link38.eu

Rejecting some mails falsely is now very unlikely.

Closing this ticket, but we need to make sure our infrastructure always sends mails with equal envelope and MIME sender, using some domain (e.g., @ipfire.org) with at least valid MX records.