Spam Detection

• SPAM Detection Architecture

• Spam Detection Methods on the Central Mail Server

1. SPAM Detection Architecture _[menu]

The following diagram shows how the individual parts of the anti-spam filter work in the school network.

Notes:

• • The central mail server is two mutually representing servers – vse.vse.cz and mx2.vse.cz.
  • E-mails sent from InSIS or Office 365 do not pass through anti-spam filter, e-mails sent from other computers in the school (e.g. from web servers) already go through the anti-spam filter.
  • Only a limited number of e-mails per day can be sent via smtp.vse.cz (authenticated sending).
  • Messages sent to Office 365 are still undergoing additional anti-virus and anti-spam checks at Microsoft.
  • If a message is marked as spam on the central mail server, InSIS will place it in the “Spam” folder. Office 365 puts it in the “Nevyžádaná pošta” spam folder. If Microsoft algorithms detect an incoming message as spam, it also places it in the “Spam” folder.
  • Users can influence spam detection:
    • In Office 365, they can create their own rules, including blocked and allowed users.
    • When processing mail in Office 365 via the web interface – marking an e-mail as spam or unmarking it can be sent to Microsoft for lectures of heuristic filtering.
    • In InSIS – if the user marks the email as SPAM, it is used to ‘teach’ heuristic filtering on the central mail server.

2. Spam Detection Methods on the Central Mail Server _[menu]

The following spam detection methods are used on the central mail server:

• • Blocking clients (computers sending mail) according to manually maintained lists of IP addresses or domain names. Blocking senders (FROM email address) based on manually maintained lists. E-mails sent with a non-existent domain in the sender’s address or e-mails with a recipient outside the school are also blocked (only applies to e-mails received from the Internet).
  • Blocking clients based on external lists of IP addresses of spammers (DNS Blacklist). Two lists are used: SpamCop (bl.spamcop.net) and Spamhaus (zen.spamhaus.org).
  • Greylisting – if the trio of sender, recipient and client from the message is not known, then the e-mail will be temporarily rejected. Mail from some clients is not checked (e.g. from seznam.cz). Many spammers do not try to send spam for the second time.
  • Limiting the Number of E-mails Sent per Hour/Day. There will be a delay in sending spam or legitimate business offers, thus creating a time reserve for a suitable response. Most of these e-mails are delivered later.
    • When sending e-mails to smtp.vse.cz, the logged-in user has two limits – a maximum of 500 messages and a maximum of 2,000 recipients per day (there can be more than one recipient in one message).
    • A specific client (mail server) on the Internet can send a maximum of 500 messages per hour to a maximum of 1,000 recipients at school.
    • A specific sender (email address From) can send a maximum of 500 messages per hour to a maximum of 1,000 recipients.

• • E-mails to a non-existent final e-mail address are rejected.
  • E-mails with infected or executable attachments are discarded.
  • Spam Detection in SpamAssassin. If the program determines that it is spam, then the message is not discarded, but only marks it – it adds the string “***** SPAM! *****” to the title and the header “X-Spam-Flag: YES” to the message.
    • Rules based on the detection of patterns in the e-mail, e.g. contains an image with a large number of words, different freemails in From and in Reply-To or the text contains the names of drugs (e.g. viagra). The creators of SpamAssassin update and test the rules. Even several local rules are used on the central mail server.
    • Bayesian filtering – the administrator or users will teach SpamAssassin what spam is. The program then marks other e-mails as spam based on the similarity.
    • Network filtering methods – determines whether the client is not on DNS Blacklists, whether the domain does not use a domain on the URI DNS blacklists, whether the same content is not sent to other mail servers (e.g. Vipul Razor), Sender Policy Framework and DKIM checks.