Detection Tuning

This article guides you through different ways on how to tune the system to have more accurate decisions. You can also jump to specific chapters if you would like to tune only specific verdicts.

​

Authentication

Goal: Make sure all emails sent by your organization (“guarded sender”) are marked as authentic. Symptoms of a wrong authentication configuration: Legitimate emails sent by your organization are marked as IMPERSONATION, POLICY, or PHISHING. Required know-how: Trusted MTAs Actions:

1. List all domains that belong to your organization using the search query in xorlab Security Platform: messages AND NOT(status:pending) from:matches:*@*example* #inc, where example is the root domain name used by your organization, e.g., xorlab. This way, you can find new domains that should be added to the list of Guarded Tenant Domains. Follow step #4 in Basic settings to make these domains guarded.
2. List all unauthenticated emails sent by your organization using another search query in XSP: rule:TAG_sender_is_guarded #noauth.
3. For every unique sender address in the results, open up a sample in a new tab using Ctrl+click.
4. Have a look at every sample in the detail view.
5. Open the HEADERS tab and read the “Received” headers to understand where the email came from.
6. If the email came from an external source that you trust, we recommend that you set up SPF or DKIM for this location. If this is not possible, continue with the next step.
7. Extend the trusted MTA configuration as described in Trusted MTAs with another entry that specifies the sender domain of this email in whitelistedDomains. This instructs XSP to mark every email that passes through this server specified in trusted as authentic. Note: make sure that this server is trusted and no malicious emails can go through it.

Shortcuts: If you have an upstream email gateway that already does full spoofing protection, you can simplify the configuration and only add this email gateway to the trustedMTA and specify the domains of your guarded tenants as whitelistedDomains. If you have external web forms that send emails to you with your domain, add them to one of the contact form lists. This will prevent XSP from marking those emails as POLICY or IMPERSONATION.

​

False positive tuning

​

General tuning

Goal: Reduce the number of false positives in the quarantine (i.e., legitimate emails that were put into quarantine). Symptoms: Many false positives especially in spam and phishing quarantine. Required know-how:

• Inbound Email Security
• Actions, Rules and Profiles

1. Create a deliver campaign—a tailored whitelist for specific emails, e.g., a whitelist based on sender’s address and recipient’s address.
2. If certain senders send recurring legitimate emails detected as spam, phishing, or extortion, you can add them to the Low trusted sender addresses or Low trusted sender domains list. This will prevent aggressive spam or phishing rules to quarantine those emails, but basic malware filtering will still be active. This is usually used for newsletters or similar emails where no trust has been established due to missing outgoing emails.
3. If you have public email addresses where you receive many important emails (e.g., a billing address for invoices), consider adding those to the Exposed public contacts list. They will be subject to less aggressive spam and phishing filtering.
4. Use whitelists:
   • Sender domain whitelist, Sender address whitelist (see Safe senders).
   • Registered domains whitelist: you can add domains to this list if
     • you own domains that are not guarded by xorlab Security Platform
     • or you are using external, third-party services sending messages from other domains than the guarded ones (e.g., yourDomain.attlassian.net).
     Registered domains are seen as something external, so checks are still applied, but this communication is slightly less suspicious (see Safe senders).
   • Envelope FROM domain whitelist, Envelope FROM sender whitelist (see Safe SMTP Senders).
   • Link domain whitelist: this ensures that a specific domain (as part of a URL) will not be seen as suspicious. It is not a guarantee that emails with this URL will be let through (see Accepted links).
5. Activate ADAPT rules (if some passive adapt rules are found under the MATCHED RULES tab in the Message Detail view)—see Phishing below.

Actions: The false positive tuning is done per verdict. In the following, you will find the query for every verdict that lists the emails that are (or would be, in case of the toothless mode) put into the corresponding quarantine. We recommend creating a saved query for at least phishing and spam to facilitate tuning.

​

Phishing

Query: verdict:phishing #quarantine

• Evaluate the following passive rules to see which one of them reduces the number of false positives:
  ADAPT_phishing_confidence_low_based_on_spamscore_and_mta_reputation
  ADAPT_phishing_confidence_low_quite_aggressive
  ADAPT_phishing_confidence_low_very_aggressive
  ADAPT_phishing_confidence_low_based_on_link_count
  ADAPT_phishing_confidence_low_based_on_link_with_from_domain_no_auth
  ADAPT_phishing_verdict_based_on_known_sender
  ADAPT_phishing_verdict_based_on_link_count
  ADAPT_phishing_verdict_based_on_link_with_from_domain
  ADAPT_phishing_confidence_low_based_on_link_with_from_domain_auth_only
  
  You can do this by using the following query for each rule—just replace the rule name after passiveRule with the rule you would like to evaluate:
  #quarantine AND passiveRule:ADAPT_phishing_confidence_low_based_on_spamscore_and_mta_reputation
  
  If you find a rule that matches mostly false positives, you can activate it in the local.properties file like this:
  ## NOTE: add "_state=ACTIVE" to the end of the rule name to activate it
  ADAPT_phishing_confidence_low_based_on_spamscore_and_mta_reputation_state=ACTIVE
  
• If you would like to generally reduce the number of false positives in this result, configure the fp_sensitive.properties profile as described in Actions, Rules and Profiles. We generally recommend using this profile for deployments bigger than a few hundred seats.
• If you spot many document-based false positives (indicated with the paperclip icon), run this query:
  verdict:phishing #quarantine passiveRule:ADAPT_phishing_confidence_low_based_on_document_no_negligible_mismatching_link
  
  If this matches mostly legitimate emails, add the following line to the local.properties file:
  ADAPT_phishing_confidence_low_based_on_document_no_negligible_mismatching_link_state=ACTIVE
  
• Use the low trusted sender list as described in General tuning for recurring senders.
• Report remaining important messages as false positives using the Report misclassification feature.

​

Spam

Query: verdict:spam #quarantine

• Use the low-trusted sender list as described in General tuning for recurring senders.
• Report important false positives using the Report misclassification feature.

​

Malware

Query: verdict:malware #quarantine

• If you see frequent false positives with encrypted Office or archive files, you can add the following lines to the local.properties file:
  MALWARE_encrypted_archive_state=PASSIVE
  MALWARE_office_encrypted_state=PASSIVE
  
  Note: these properties are also set through the fp_sensitive.properties profile.
• If you see frequent false positives with Office files that contain ActiveX (indicator: “Office document contains ActiveX object”), you can add the following line to the local.properties file:
  MALWARE_office_activex_state=PASSIVE
  
  Note: this property is also set through the fp_sensitive.properties profile.
• If you see recurring Office false positives due to a specific macro, you can add the file hash of the macro embedding to the Macro file whitelist.
• If you see recurring false positives due to a network connection attempt to a specific domain during dynamic analysis (sandbox), you can add this domain to the Dynamic DNS request whitelist. If you see many false positives with connection attempts to a *.local domain, you can whitelist all those requests by adding (?i)\.local$ to the Dynamic DNS request whitelist (regex). Also, adding (?i)\.loc$ and (?i)\.intra$ would work.
• Report important false positives using the Report misclassification feature.

​

Extortion

Query: verdict:extortion #quarantine

• Add the following two lines to the local.properties file:
  EXTORTION_crypto_currency_ransom_image_state=PASSIVE
  EXTORTION_crypto_currency_ransom_keyword_based_state=PASSIVE
  
  Note: these properties are also set through the fp_sensitive.properties profile.
• Use the low-trusted sender list as described in General tuning for recurring senders.
• Report important false positives using the Report misclassification feature.

​

Impersonation

Query: (verdict:impersonation or verdict:vipfraud) #quarantine

• If you see VIP Fraud false positives where the email is legitimate and coming from a private address of a VIP, you can add the sender address to the VIP sender email whitelist.
• If you see other Impersonation false positives, they are often due to a wrong authentication configuration from the previous section.

​

Filter (Policy)

Query: verdict:policy #quarantine

• Filters are policies enforced by xorlab Security Platform. For every email in this result, you can find the corresponding Filter rule under MATCHED RULE starting with FILTER_, like, for example, FILTER_sender_forged_guarded. If you would like to disable one of the default filters, find its name in the MATCHED RULE tab, append _state=PASSIVE, and add it to local.properties like this: FILTER_sender_forged_guarded_state=PASSIVE.

​

BEC

Query: verdict:bec #quarantine

• Report important false positives using the Report misclassification feature.

​

False negative tuning

Goal: Reduce the number of malicious emails that were let through and not put into quarantine. Symptoms:

• Malicious email let through and is shown as benign in xorlab Security Platform (green verdict).
• Malicious email let through even though it is shown as malicious in xorlab Security Platform (red verdict).

Required know-how:

• Inbound Email Security
• Actions, Rules and Profiles

In case of a symptom 1: Report the message as false negative using the Report misclassification feature. In case of a symptom 2: Open the email in xorlab Security Platform and go to the MATCHED RULE tab. Scroll down and check if a rule starting with ADAPT_ is listed. If yes, check if its decision UNDECIDED on the right is shown in blue (and not in grey). If both are true, this email has been let through due to an ADAPT rule that lowered the confidence from high to low to prevent false positives. It is now possible to disable this adapt rule to prevent such false negatives:

1. Copy the name of the ADAPT rule and search for it with rule:rule-name (see Search).
2. Go through the results to assess the impact of this ADAPT rule.
3. Disable it by adding its name appended with _state=PASSIVE to local.properties, for example: ADAPT_phishing_confidence_high_state=PASSIVE Besides this, report the message as false negative using the Report misclassification feature.

​

General tuning

• If you have public email addresses where you receive a lot of commodity spam and phishing emails (e.g., a jobs@ or info@ email address), consider adding those to the Exposed accounts list. They will be subject to more aggressive spam and phishing filtering.
• If VIP fraud emails are let through, make sure that you add the name of the impersonated VIP to the VIP names list.