> ## Documentation Index
> Fetch the complete documentation index at: https://docs.xorlab.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Pages under /latest/ document the current release. Paths that begin with a version number, for example /10.0/, are frozen snapshots of superseded releases kept for reference only — never present their instructions as current. If the reader's version is unknown, answer from /latest/ and say which version the answer describes.
> xorlab is deployed on-premises, hybrid, or as a cloud service, and integrates with either Microsoft 365 or on-premises mail infrastructure. Configuration steps often differ between these. State which deployment and integration mode an instruction applies to instead of presenting one as universal.
> Distinguish inline mode from monitoring mode when describing anything that acts on email. Monitoring mode observes a copy and cannot block or quarantine; inline mode routes mail through xorlab and can.
> Write the product name as lowercase 'xorlab'. Use the documented component names: xorlab Control Center (XCC), xorlab MTA, xorlab Sandbox (DANA), xorlab Natural Language Understanding (NLU). After the first mention, use the short forms XCC, MTA, Sandbox, and NLU. Do not use DANA as a standalone name for the Sandbox, but keep it where it is a literal string in configuration keys, container names, and hostnames.
> Do not invent configuration keys, rule parameters, list names, log properties, or API fields. If a value is not present in this documentation, say that it is not documented rather than guessing.

# Format Converters

> Customize the structure of log messages with format converters for compatibility with your SIEM.

The format converter is part of the `appender` element.

## Converters

### JSON

This is the most common format converter used on xorlab. It captures all event-specific attributes. It can be used for all loggers **except** those recording [Message Tracing Events](/latest/logging-events#message-tracing-events).

```xml theme={null}
<appender>
  <encoder>
    <pattern>%jsonMsg%n</pattern>
  </encoder>
</appender>
```

### JSON Extended

This format converter can only be used for [Message Tracing Events](/latest/logging-events#message-tracing-events) (numbered from 1000 to 1999). This converter records all the results of the email analysis as a single log entry.

Structure:

```xml theme={null}
<appender>
  <encoder>
    <pattern>%coreJsonMsg%n</pattern>
  </encoder>
</appender>
```

<Accordion title="Example JSON">
  ```yaml theme={null}
    {
        "source": "activeguard",
        "version": "release-6.7.0",
        "ggrid": "1b35699e-5ca4-4ee6-84b6-3c4b54c3d5e2",
        "decision": "UNDECIDED",
        "type": "Emails",
        "created": "2020-03-30T16:51:48.433+0200",
        "decided": "",
        "completed": "",
        "risk": 0,
        "trust": 0,
        "threat": 20,
        "reputation": 0,
        "tags": [
            "#suspicious"
        ],
        "mail": {
            "headers": [
                {
                    "name": "x-spam-status",
                    "value": "-----spam-status-----"
                }
            ],
            "subject": "Good morning",
            "sent": "",
            "direction": "INCOMING",
            "from": [
                {
                    "email": "han.solo@lucasartsthe-rebels.com",
                    "display_name": ""
                }
            ],
            "to": [
                {
                    "email": "chewy@lucasartschewbacca@the-rebels.com",
                    "display_name": ""
                },
                {
                    "email": "luke@lucasartsthe-rebels.com",
                    "display_name": "Lightsaber Guy"
                }
            ],
            "reply_to": [
                {
                    "email": "theguy@lucasartsadmin@the-rebels.com",
                    "display_name": ""
                }
            ],
            "attachments": [
                {
                    "name": "archive.zip",
                    "sha256": "beef123",
                    "size": 1027
                },
                {
                    "name": "logo.png",
                    "sha256": "beef123",
                    "size": 127
                }
            ],
            "threat_intel_emails": [
                "admin@death-star.com"
            ],
            "threat_intel_domains": [
                "suspicious1.dangerous.com",
                "suspicious2.dangerous.com"
            ],
            "threat_intel_urls": [
                "http://www.ww3.com/foo.sh"
            ]
        }
    }
  ```
</Accordion>

#### Parameters

The extended JSON can be customized through additional parameters:

* Include or exclude the subject
* Add headers of the email to the JSON message.

The parameters are specified in curly brackets: `%coreJsonMsg{param1,param2,param3,...}`

| Parameter            | Description                                                                                                                             |
| :------------------- | :-------------------------------------------------------------------------------------------------------------------------------------- |
| `param1`             | Include or exclude the subject in the JSON. Supported values:<br />`ALWAYS` (default)<br />`NEVER`<br />`ON_DROP_BOUNCE_AND_QUARANTINE` |
| `param2` to `paramX` | A list of headers of the email that will be added to the JSON message                                                                   |

As an example, the following appender pattern will not include the subject but always add the `message-id` and `x-custom-header` header to the log message:

```xml theme={null}
<encoder>
<pattern>%coreJsonMsg{NEVER,message-id,x-custom-header}%n</pattern>
</encoder>
```

### Specific Attributes

The `%logArg{}` converter allows you to log specific key-value pairs from a larger log event. This can be used for any event, in any appender, and across all xorlab components. To identify which keys to log, initially configure the appender to log the entire JSON message and inspect the structure.

Important: Always end the `%logArg{}` sequence with a line break (%n) to ensure proper log entry formatting.

```xml theme={null}
<appender name="customListAppender" class="ch.qos.logback.core.FileAppender">
  <file>${log_dir}/Custom_list.log</file>
  <append>true</append>
  <encoder>
    <pattern>%logArg{item} %logArg{listName}%n</pattern>
  </encoder>
</appender>

<logger name="audit.user.lists.item_added">
  <appender-ref ref="customListAppender"/>
</logger>
```

### Common Event Format (CEF)

The CEF format with its converter is used by specifying `%cef` in the `pattern` section of `logback-audit.xml`:

```xml theme={null}
<appender name="syslogCEFAppender" class="com.xorlab.sgappliance.shared.logback.SyslogAppender">
  <syslogHost>syslog.xorlab.com</syslogHost>
  <port>514</port>
  <protocol>UDP</protocol>
  <pattern>%cef%n</pattern>
</appender>
```

A CEF event first contains some static properties like the event ID and event name, and afterward the event specific payload:

```text theme={null}
CEF:0|xorlab|ActiveGuard|release-6.7.0-253-gcce39f1b9|1300|trace.msg_verdict|4|ggrid=b950ffd2-00a1-498f-a248-d8376e47f9b9 decision=QUARANTINE direction=INCOMING from=hansolo <han.solo@lucasarts.com> to="Chewbacca" 'chewy' <chewy@lucasarts.com>; "Lightsaber Guy" <luke@lucasarts.com>
```

The static properties added for every event are:

| Name         | Type             | Example                           |
| :----------- | :--------------- | :-------------------------------- |
| `format`     | String           | `CEF:0`                           |
| `vendor`     | String           | `xorlab`                          |
| `product`    | String           | `ActiveGuard`                     |
| `version`    | String           | `release-<MAJOR>.<MINOR>.<PATCH>` |
| `event_id`   | Number (1-10000) | `1300`                            |
| `event_name` | String           | `trace.msg_verdict`               |
| `severity`   | Number (1-10)    | `7`                               |

### Additional converters

So far, we only used the different JSON and CEF converters in the `pattern` section of `logback-audit.xml`. However, the patterns can be enhanced with all standard Logback converters to add additional information.

Please have a look at the table at [Pattern Layout](http://logback.qos.ch/manual/layouts.html#ClassicPatternLayout) to find a complete list of all standard Logback converters.

For example, the following configuration adds a date string to the logged message payload:

```xml theme={null}
<appender name="syslogCEFAppender" class="com.xorlab.sgappliance.shared.logback.SyslogAppender">
  <syslogHost>syslog.xorlab.com</syslogHost>
  <port>514</port>
  <protocol>UDP</protocol>
  <!-- Add a date string in front of the CEF payload -->
  <encoder>
    <pattern>%date %cef%n</pattern>
  </encoder>
</appender>
```