Format Converters

The shape of a record is set by the pattern inside the encoder of an appender, so the same choice applies to Syslog, email, and file appenders alike:

<appender>
  <encoder>
    <pattern>%jsonMsg%n</pattern>
  </encoder>
</appender>

Converter	Pattern	Scope
JSON	`%jsonMsg%n`	Every event
JSON extended	`%coreJsonMsg%n`	Message-tracing (`trace.*`) events
CEF	`%cef%n`	Every event
Specific attributes	`%logArg{key}%n`	Every event

Always end a pattern with %n, the line break. Without it, records run into each other.

Converters

JSON

The most common choice on xorlab, and the default in every guide. It records the attributes of the event itself, as one JSON object per record.

<appender>
  <encoder>
    <pattern>%jsonMsg%n</pattern>
  </encoder>
</appender>

JSON extended

Only for Message-tracing events, numbered 1000 to 1999. Instead of the attributes of that one event, it records the whole result of the email analysis as a single entry: verdict, scores, tags, participants, attachments with hashes, and extracted indicators.

<appender>
  <encoder>
    <pattern>%coreJsonMsg%n</pattern>
  </encoder>
</appender>

Example JSON

  {
      "source": "activeguard",
      "version": "release-6.7.0",
      "ggrid": "1b35699e-5ca4-4ee6-84b6-3c4b54c3d5e2",
      "decision": "UNDECIDED",
      "type": "Emails",
      "created": "2020-03-30T16:51:48.433+0200",
      "decided": "",
      "completed": "",
      "risk": 0,
      "trust": 0,
      "threat": 20,
      "reputation": 0,
      "tags": [
          "#suspicious"
      ],
      "mail": {
          "headers": [
              {
                  "name": "x-spam-status",
                  "value": "-----spam-status-----"
              }
          ],
          "subject": "Good morning",
          "sent": "",
          "direction": "INCOMING",
          "from": [
              {
                  "email": "han.solo@lucasartsthe-rebels.com",
                  "display_name": ""
              }
          ],
          "to": [
              {
                  "email": "chewy@lucasartschewbacca@the-rebels.com",
                  "display_name": ""
              },
              {
                  "email": "luke@lucasartsthe-rebels.com",
                  "display_name": "Lightsaber Guy"
              }
          ],
          "reply_to": [
              {
                  "email": "theguy@lucasartsadmin@the-rebels.com",
                  "display_name": ""
              }
          ],
          "attachments": [
              {
                  "name": "archive.zip",
                  "sha256": "beef123",
                  "size": 1027
              },
              {
                  "name": "logo.png",
                  "sha256": "beef123",
                  "size": 127
              }
          ],
          "threat_intel_emails": [
              "admin@death-star.com"
          ],
          "threat_intel_domains": [
              "suspicious1.dangerous.com",
              "suspicious2.dangerous.com"
          ],
          "threat_intel_urls": [
              "http://www.ww3.com/foo.sh"
          ]
      }
  }

A record like the one above is well over the 1500-byte limit that UDP Syslog truncates at. Use TCP or TLS with this converter.

One exception is worth knowing: trace.msg_released does not support the email result extension, so a release from quarantine is recorded without the analysis result.

Parameters

Extended JSON takes parameters in curly brackets, %coreJsonMsg{param1,param2,param3,...}, to control the subject and to add email headers:

Parameter	Description
`param1`	Include or exclude the subject in the JSON. Supported values: `ALWAYS` (default) `NEVER` `ON_DROP_BOUNCE_AND_QUARANTINE`
`param2` to `paramX`	A list of headers of the email that will be added to the JSON message

The following pattern leaves the subject out and always adds the message-id and x-custom-header headers, which is a common requirement where the subject line is treated as personal data:

<encoder>
<pattern>%coreJsonMsg{NEVER,message-id,x-custom-header}%n</pattern>
</encoder>

ON_DROP_BOUNCE_AND_QUARANTINE is the middle ground: the subject is recorded only for mail that was dropped, bounced, or quarantined. See also Change Storing of Subject.

Specific attributes

%logArg{} records named key-value pairs instead of the whole event. It works for any event, in any appender, on every component. To find the key names, configure the appender with %jsonMsg%n first, look at one record, then narrow it down.

<appender name="customListAppender" class="ch.qos.logback.core.FileAppender">
  <file>${log_dir}/Custom_list.log</file>
  <append>true</append>
  <encoder>
    <pattern>%logArg{item} %logArg{listName}%n</pattern>
  </encoder>
</appender>

<logger name="audit.user.lists.item_added">
  <appender-ref ref="customListAppender"/>
</logger>

This is also how you build a readable email body out of a single event. See Send an Email When an Event Occurs.

Common Event Format (CEF)

CEF is what ArcSight-style platforms and CEF data connectors expect. Select it with %cef:

<appender name="syslogCEFAppender" class="com.xorlab.sgappliance.shared.logback.SyslogAppender">
  <syslogHost>syslog.xorlab.com</syslogHost>
  <port>514</port>
  <protocol>UDP</protocol>
  <encoder>
    <pattern>%cef%n</pattern>
  </encoder>
</appender>

A CEF record starts with static properties, the event ID and name among them, followed by the event-specific payload:

CEF:0|xorlab|ActiveGuard|release-6.7.0-253-gcce39f1b9|1300|trace.msg_verdict|4|ggrid=b950ffd2-00a1-498f-a248-d8376e47f9b9 decision=QUARANTINE direction=INCOMING from=hansolo <han.solo@lucasarts.com> to="Chewbacca" 'chewy' <chewy@lucasarts.com>; "Lightsaber Guy" <luke@lucasarts.com>

The static properties, present on every event:

Name	Type	Example
`format`	String	`CEF:0`
`vendor`	String	`xorlab`
`product`	String	`ActiveGuard`
`version`	String	`release-<MAJOR>.<MINOR>.<PATCH>`
`event_id`	Number (1-10000)	`1300`
`event_name`	String	`trace.msg_verdict`
`severity`	Number (1-10)	`7`

Additional converters

A pattern is not limited to one converter. Every standard Logback converter can be combined with the ones above — see the table under Pattern Layout for the full list. For example, this prefixes the CEF payload with a date:

<appender name="syslogCEFAppender" class="com.xorlab.sgappliance.shared.logback.SyslogAppender">
  <syslogHost>syslog.xorlab.com</syslogHost>
  <port>514</port>
  <protocol>UDP</protocol>
  <!-- Add a date string in front of the CEF payload -->
  <encoder>
    <pattern>%date %cef%n</pattern>
  </encoder>
</appender>

​Converters

​JSON

​JSON extended

​Parameters

​Specific attributes

​Common Event Format (CEF)

​Additional converters

Converters

JSON

JSON extended

Parameters

Specific attributes

Common Event Format (CEF)

Additional converters