Format Converters

The format converter is part of the appender element.

Converters

JSON

This is the most common format converter used on xorlab. It captures all event-specific attributes. It can be used for all loggers except those recording Message Tracing Events.

<appender>
  <encoder>
    <pattern>%jsonMsg%n</pattern>
  </encoder>
</appender>

JSON Extended

This format converter can only be used for Message Tracing Events (numbered from 1000 to 1999). This converter records all the results of the email analysis as a single log entry. Structure:

<appender>
  <encoder>
    <pattern>%coreJsonMsg%n</pattern>
  </encoder>
</appender>

Example JSON

  {
      "source": "activeguard",
      "version": "release-6.7.0",
      "ggrid": "1b35699e-5ca4-4ee6-84b6-3c4b54c3d5e2",
      "decision": "UNDECIDED",
      "type": "Emails",
      "created": "2020-03-30T16:51:48.433+0200",
      "decided": "",
      "completed": "",
      "risk": 0,
      "trust": 0,
      "threat": 20,
      "reputation": 0,
      "tags": [
          "#suspicious"
      ],
      "mail": {
          "headers": [
              {
                  "name": "x-spam-status",
                  "value": "-----spam-status-----"
              }
          ],
          "subject": "Good morning",
          "sent": "",
          "direction": "INCOMING",
          "from": [
              {
                  "email": "han.solo@lucasartsthe-rebels.com",
                  "display_name": ""
              }
          ],
          "to": [
              {
                  "email": "chewy@lucasartschewbacca@the-rebels.com",
                  "display_name": ""
              },
              {
                  "email": "luke@lucasartsthe-rebels.com",
                  "display_name": "Lightsaber Guy"
              }
          ],
          "reply_to": [
              {
                  "email": "theguy@lucasartsadmin@the-rebels.com",
                  "display_name": ""
              }
          ],
          "attachments": [
              {
                  "name": "archive.zip",
                  "sha256": "beef123",
                  "size": 1027
              },
              {
                  "name": "logo.png",
                  "sha256": "beef123",
                  "size": 127
              }
          ],
          "threat_intel_emails": [
              "admin@death-star.com"
          ],
          "threat_intel_domains": [
              "suspicious1.dangerous.com",
              "suspicious2.dangerous.com"
          ],
          "threat_intel_urls": [
              "http://www.ww3.com/foo.sh"
          ]
      }
  }

Parameters

The extended JSON can be customized through additional parameters:

Include or exclude the subject
Add headers of the email to the JSON message.

The parameters are specified in curly brackets: %coreJsonMsg{param1,param2,param3,...}

Parameter	Description
`param1`	Include or exclude the subject in the JSON. Supported values: `ALWAYS` (default) `NEVER` `ON_DROP_BOUNCE_AND_QUARANTINE`
`param2` to `paramX`	A list of headers of the email that will be added to the JSON message

As an example, the following appender pattern will not include the subject but always add the message-id and x-custom-header header to the log message:

<encoder>
<pattern>%coreJsonMsg{NEVER,message-id,x-custom-header}%n</pattern>
</encoder>

Specific Attributes

The %logArg{} converter allows you to log specific key-value pairs from a larger log event. This can be used for any event, in any appender, and across all xorlab components. To identify which keys to log, initially configure the appender to log the entire JSON message and inspect the structure. Important: Always end the %logArg{} sequence with a line break (%n) to ensure proper log entry formatting.

<appender name="customListAppender" class="ch.qos.logback.core.FileAppender">
  <file>${log_dir}/Custom_list.log</file>
  <append>true</append>
  <encoder>
    <pattern>%logArg{item} %logArg{listName}%n</pattern>
  </encoder>
</appender>

<logger name="audit.user.lists.item_added">
  <appender-ref ref="customListAppender"/>
</logger>

Common Event Format (CEF)

The CEF format with its converter is used by specifying %cef in the pattern section of logback-audit.xml:

<appender name="syslogCEFAppender" class="com.xorlab.sgappliance.shared.logback.SyslogAppender">
  <syslogHost>syslog.xorlab.com</syslogHost>
  <port>514</port>
  <protocol>UDP</protocol>
  <pattern>%cef%n</pattern>
</appender>

A CEF event first contains some static properties like the event ID and event name, and afterward the event specific payload:

CEF:0|xorlab|ActiveGuard|release-6.7.0-253-gcce39f1b9|1300|trace.msg_verdict|4|ggrid=b950ffd2-00a1-498f-a248-d8376e47f9b9 decision=QUARANTINE direction=INCOMING from=hansolo <han.solo@lucasarts.com> to="Chewbacca" 'chewy' <chewy@lucasarts.com>; "Lightsaber Guy" <luke@lucasarts.com>

The static properties added for every event are:

Name	Type	Example
`format`	String	`CEF:0`
`vendor`	String	`xorlab`
`product`	String	`ActiveGuard`
`version`	String	`release-<MAJOR>.<MINOR>.<PATCH>`
`event_id`	Number (1-10000)	`1300`
`event_name`	String	`trace.msg_verdict`
`severity`	Number (1-10)	`7`

Additional converters

So far, we only used the different JSON and CEF converters in the pattern section of logback-audit.xml. However, the patterns can be enhanced with all standard Logback converters to add additional information. Please have a look at the table at Pattern Layout to find a complete list of all standard Logback converters. For example, the following configuration adds a date string to the logged message payload:

<appender name="syslogCEFAppender" class="com.xorlab.sgappliance.shared.logback.SyslogAppender">
  <syslogHost>syslog.xorlab.com</syslogHost>
  <port>514</port>
  <protocol>UDP</protocol>
  <!-- Add a date string in front of the CEF payload -->
  <encoder>
    <pattern>%date %cef%n</pattern>
  </encoder>
</appender>

​Converters

​JSON

​JSON Extended

​Parameters

​Specific Attributes

​Common Event Format (CEF)

​Additional converters