Format Converters

The format converter is part of the appender element and enables you to customize how log messages are structured. This flexibility enhances log clarity for security analysts and ensures compatibility with various SIEM systems.

Converters

JSON Message

This is the most common format converter used on XSP. It captures all event-specific attributes. It can be used for all loggers except those recording Message Tracing Events.

<appender>
  <encoder>
    <pattern>%jsonMsg%n</pattern>
  </encoder>
</appender>

Message Tracing

This format converter can only be used for Message Tracing Events (numbered from 1000 to 1999). This converter records all the results of the email analysis as a single log entry. Structure:

<appender>
  <encoder>
    <pattern>%coreJsonMsg%n</pattern>
  </encoder>
</appender>

Log Entry Structure

Below you find an example of a single log entry containing the complete results of the email analysis performed by xorlab Security Platform.

Expand to view

  {
      "source": "activeguard",
      "version": "release-6.7.0",
      "ggrid": "1b35699e-5ca4-4ee6-84b6-3c4b54c3d5e2",
      "decision": "UNDECIDED",
      "type": "Emails",
      "created": "2020-03-30T16:51:48.433+0200",
      "decided": "",
      "completed": "",
      "risk": 0,
      "trust": 0,
      "threat": 20,
      "reputation": 0,
      "tags": [
          "#suspicious"
      ],
      "mail": {
          "headers": [
              {
                  "name": "x-spam-status",
                  "value": "-----spam-status-----"
              }
          ],
          "subject", "Good morning"
          "sent": "",
          "direction": "INCOMING",
          "from": [
              {
                  "email": "han.solo@lucasartsthe-rebels.com",
                  "display_name": ""
              }
          ],
          "to": [
              {
                  "email": "chewy@lucasartschewbacca@the-rebels.com",
                  "display_name": ""
              },
              {
                  "email": "luke@lucasartsthe-rebels.com",
                  "display_name": "Lightsaber Guy"
              }
          ],
          "reply_to": [
              {
                  "email": "theguy@lucasartsadmin@the-rebels.com",
                  "display_name": ""
              }
          ],
          "attachments": [
              {
                  "name": "archive.zip",
                  "sha256": "beef123",
                  "size": 1027
              },
              {
                  "name": "logo.png",
                  "sha256": "beef123",
                  "size": 127
              }
          ],
          "threat_intel_emails": [
              "admin@death-star.com"
          ],
          "threat_intel_domains": [
              "suspicious1.dangerous.com",
              "suspicious2.dangerous.com"
          ],
          "threat_intel_urls": [
              "http://www.ww3.com/foo.sh"
          ]
      }
  }

Parameters

The email result converter allows you to customize the logged email results through additional parameters:

Include or exclude the subject
Add headers of the email to the JSON message.

The parameters are specified in curly brackets: %coreJsonMsg{param1,param2,param3,...}

Parameter	Description
`param1`	Include or exclude the subject in the JSON. Supported values: `ALWAYS` (default) `NEVER` `ON_DROP_BOUNCE_AND_QUARANTINE`
`param2` to `paramX`	A list of headers of the email that will be added to the JSON message

As an example, the following appender pattern will not include the subject but always add the message-id and x-custom-header header to the log message:

<encoder>
<pattern>%coreJsonMsg{NEVER,message-id,x-custom-header}%n</pattern>
</encoder>

Specific Attributes

The %logArg{} converter allows you to log specific key-value pairs from a larger log event. This can be used for any event, in any appender, and across all xorlab components. To identify which keys to log, initially configure the appender to log the entire JSON message and inspect the structure. Important: Always end the %logArg{} sequence with a line break (%n) to ensure proper log entry formatting.

<appender name="customListAppender" class="ch.qos.logback.core.FileAppender">
  <file>${log_dir}/Custom_list.log</file>
  <append>true</append>
  <encoder>
    <pattern>%logArg{item} %logArg{listName}%n</pattern>
  </encoder>
</appender>

<logger name="audit.user.lists.item_added">
  <appender-ref ref="customListAppender"/>
</logger>

Common Event Format (CEF)

The CEF format with its converter is used by specifying %cef in the pattern section of logback-audit.xml:

<appender name="syslogCEFAppender" class="com.xorlab.sgappliance.shared.logback.SyslogAppender">
  <syslogHost>syslog.xorlab.com</syslogHost>
  <port>514</port>
  <protocol>UDP</protocol>
  <pattern>%cef%n</pattern>
</appender>

A CEF event first contains some static properties like the event ID and event name, and afterward the event specific payload:

CEF:0|xorlab|ActiveGuard|release-6.7.0-253-gcce39f1b9|1300|trace.msg_verdict|4|ggrid=b950ffd2-00a1-498f-a248-d8376e47f9b9 decision=QUARANTINE direction=INCOMING from=hansolo <han.solo@lucasarts.com> to="Chewbacca" 'chewy' <chewy@lucasarts.com>; "Lightsaber Guy" <luke@lucasarts.com>

The static properties added for every event are:

Name	Type	Example
`format`	String	`CEF:0`
`vendor`	String	`xorlab`
`product`	String	`ActiveGuard`
`version`	String	`release-<MAJOR>.<MINOR>.<PATCH>`
`event_id`	Number (1-10000)	`1300`
`event_name`	String	`trace.msg_verdict`
`severity`	Number (1-10)	`7`

Additional converters

So far, we only used the different JSON and CEF converters in the pattern section of logback-audit.xml. However, the patterns can be enhanced with all standard Logback converters to add additional information. Please have a look at the table at Pattern Layout to find a complete list of all standard Logback converters. For example, the following configuration adds a date string to the logged message payload:

<appender name="syslogCEFAppender" class="com.xorlab.sgappliance.shared.logback.SyslogAppender">
  <syslogHost>syslog.xorlab.com</syslogHost>
  <port>514</port>
  <protocol>UDP</protocol>
  <!-- Add a date string in front of the CEF payload -->
  <encoder>
    <pattern>%date %cef%n</pattern>
  </encoder>
</appender>

​Converters

​JSON Message

​Message Tracing

​Log Entry Structure

​Parameters

​Specific Attributes

​Common Event Format (CEF)

​Additional converters