Skip to main content

Reporting And Data Management

The Reporting & Data page contains the following options for the reporting functionality in Zenarmor (Sensei):

  • Reporting Database Backend

  • Temporary Memory Disk

  • DNS Enrichment for Reports

  • Reports Data Management

  • Stream Reporting Data to External Elastic Search Database

  • Stream Reporting Data to Remote Syslog Server

  • Scheduled Reports

You may configure the Zenarmor Reporting & Data Management options by navigating to ZenarmorConfigurationReporting & Data on OPNsense web GUI.

Reporting Database Backend: Elasticsearch & MongoDB (Internal)#

You can install and configure to use a Elasticsearch or MongoDB database as an internal database server on your OPNsense firewall.

Directory storing Database Files:(Path for Elasticsearch or MongoDB)#

This option allows you to change the directory path of the backend reporting data and indexes.

You may modify your reporting database directory path with a new one.

To change the path:

  1. Enter the new path.

  2. Click Change DB Path.

Changing the Database Directory and Prefix

Figure 1: Changing the Database Directory and Prefix

Database Prefix#

The database prefix uniquely defines the source firewall. If you are using more than one firewall, you need to use a different database prefix to avoid a database file name conflict in Elasticsearch. Otherwise, the system will overwrite data on the same prefixed files.

To change the database prefix:

  1. Enter the new database prefix .

  2. Click Change Prefix.

  3. After the change is made, you will see a message box displaying Done as in the message below.

  4. Close the window.

A dialog box indicating that new indexes are created.

Figure 2. A dialog box indicating that new indexes are created.

Size of the Fast Temporary Memory Disk (in Megabytes / Capacity):#

Changing the Size of the Temporary Disk#

Zenarmor temporarily stores IPDR files in the directory /usr/local/sensei/output/temp until the information is ingested into the database or the files are rotated. If you see warnings in the system log that this partition is full, you will need to increase the disk space size. The minimum and maximum size of the temporary disk is 10 MB and 500 MB, respectively.

To change the size of the temporary disk:

  1. Enter a value between 10 and 500.

  2. Click Change Temp Size.

The Zenarmor packet engine will restart after this change.

Changing the Size of the Fast Temporary Memory Disk

Figure 3. Changing the Size of the Fast Temporary Memory Disk

DNS Enrichment for Reports#

Configuring the DNS Enrichment for Reports

Figure 4. Configuring the DNS Enrichment for Reports

DNS server IP addresses to do reverse IP lookups#

You may define local or external DNS servers for reverse IP lookups to resolve hostnames for the reports.

Enter the DNS server IP address(es) in the DNS server IP addresses to do a reverse IP lookups field. You should use a comma between the IP addresses to enter more than one DNS server IP address.

Perform real-time DNS reverse queries for local IP addresses#

To obtain the hostname of each IP address on your local network, you should enable the real-time DNS reverse queries option. Zenarmor uses MDNS and LLMNR packets to retrieve hostnames.

Click on the related toggle button to enable/disable the Perform real-time DNS reverse queries for local IP addresses feature in the DNS Enrichment for Reports pane.

Use OPNsense Host aliases for DNS enrichment#

You may use OPNsense Host Aliases for DNS enrichment. When this feature is enabled, the alias name will be used in reports when a hostname cannot be obtained by a reverse DNS lookup.

Click on the related toggle button to enable/disable the Use OPNsense Host aliases for DNS enrichment feature in the DNS Enrichment for Reports pane.

To apply the configuration changes made in the DNS Enrichment for Reports pane, click Save Changes at the end of the page.

Reports Data Management#

Maximum number of days to store reporting data:#

The maximum number of days to store the reporting data on the system may be configured with this option. When you increase the number of days to store reporting data, the amount of disk space consumed by the data increases. Please refer to the Reporting & Disk Space section to calculate the amount of disk space required for reporting.

Perform health check for indices:#

To apply a health check for the database indexes, click Perform Index Check. If all indexes are healthy, you will see the following message:

Performing Index Check

Figure 5. Performing Index Check

If there is a problem with the indexes, they will be fixed or re-created automatically.

Anonymize local IP address:#

You may anonymize your local IP addresses for security and privacy purposes. To anonymize the local IP addresses, click the toggle button.

Anonymize remote IP address:#

You may anonymize remote IP addresses for security and privacy purposes. To anonymize the remote IP addresses, click the toggle button.

Do not perform DNS Enrichment:#

For GDPR or other privacy requirements, you may not be allowed to use any DNS Enrichment functionality. Enabling this option will disable all DNS Enrichment settings.

Do not perform User Enrichment:#

In your policy configuration, you may associate IP addresses with users. However, you may not be allowed to include usernames in your reports due to regulation requirements. Therefore, you may disable the user enrichment feature by clicking on the toggle button for this option.

Erase reporting data:#

If you wish to clear out old reporting data to start fresh, you may delete all reporting data. In certain cases, erasing the report data can eliminate issues with reports that are crashing. To delete all reporting data, click Erase Reporting Data.

Reports Data Management Pane

Figure 6. Reports Data Management Pane

To apply the configuration changes you made in the Reports Data Management pane, click Save Changes at the end of the page.

Stream Reporting Data to External Elasticsearch#

If you have an existing Elasticsearch database deployed on your network or in the cloud, you may stream your data to that database server in addition to the local Elasticsearch database if you selected the local database option during installation.

Streaming Reporting Data to An External ECS Database

Figure 7. Streaming Reporting Data to An External ECS Database

Enabled:#

First, enable the system to stream your data by clicking the Enabled toggle button. When reporting data is streamed externally, your data will be stored on both the local and remote Elasticsearch databases.

External Elasticsearch URI:#

Enter the External Elasticsearch URI, username, and password.

The system will check:

  • the availability of the remote Elasticsearch database.

  • if the database is available and the credentials are correct, new index files will be generated for the remote system.

info

The Stream Reporting Data to External Elasticsearch feature allows you to keep a local copy of the reporting database while streaming to an external database if you installed a local Elasticsearch database during the installation of Zenarmor. If you decide to only use an external Elasticsearch database without keeping a local copy after the installation of Zenarmor, see How to Change to an External Only Elasticsearch Database after Installation.

Stream Reporting Data#

Streaming Reporting Data to an External Syslog Server

Figure 8. Streaming Reporting Data to an External Syslog Server

Your log data may be streamed to an external Syslog server for centralized reporting.

To configure the external Syslog server:

  1. Enable the system to stream your data by clicking the Enabled toggle button.

  2. Enter the Syslog Server IP address.

  3. Enter the Syslog Server Port (the default port is 514).

  4. Select the protocol to use for streaming data.

  5. Finally, select the data which will be streamed to Syslog.

To apply the configuration changes made in the Stream Reporting Data pane, click Save Changes at the end of the page.

IMPORTANT NOTE

The Stream Reporting Data feature is not available for the Free & Home Editions. To gain the benefit of the Stream Reporting Data feature, you must have either the SOHO or Business Zenarmor subscriptions. For more information, see the plans & pricing.

IPDR files are located under /usr/local/sensei/output/active/temp folder. They should be examined if you wish to integrate the information into a SIEM server.

Scheduled Reports#

Reports may be sent via email on a regular basis.

Configuring a Scheduled Report

Figure 9. Configuring a Scheduled Report

Enabled:#

To create scheduled reports, click the Enabled toggle button.

Mail Server:#

Enter the IP address or the hostname of the mail server (ex: 192.168.2.10).

Mail Server Port:#

Enter the port number of the mail server. When you select a Connection Security type, the port will fill automatically (ex: 25, 465, or 587).

Connection Security:#

Select the connection security type. There are two security options for connections.

The first option is SMTPS (SMTP SSL) {Preferred}. SMTPS (Simple Mail Transfer Protocol Secure) is a method for securing the SMTP using transport layer security. It is intended to provide authentication of the communication partners, as well as data integrity and confidentiality.

The second option is STARTTLS. It is an extension of plain text communication protocols, which offers a way to upgrade a plain text connection to an encrypted TLS or SSL connection instead of using a separate port for encrypted communication.

Generally, SSL/TLS is only used between end-clients and servers. STARTTLS is more commonly used between MTAs to secure inter-server transport.

Username:#

Enter the username of your email account.

Password:#

Enter the password of the user.

Send Mail(s) From:#

You may choose another sender’s email address as the “from” address.

Send Mail(s) To:#

Enter the recipient’s email address. You may add more than one email address by separating each address by a semicolon (;).

Reporting Criteria:#

You may choose between 3 different types of reports:

  • Packet

  • Session

  • Volume

Reporting Schedule:#

Scheduling may be set for a single day of the week or for all days of the week.

Do not require TLS server certificate verification:#

If your email server has a self-signed certificate, you must enable this feature. This will bypass the certificate verification step. Enabling TLS server certificate verification is advised to guarantee proper security of the traffic sent between sender and the receiver based on a valid certificate.

Send reports as PDF attachments:#

The scheduled reports may be generated as PDF files.

As shown in the screenshot below, PDF files are generated by APIs hosted in the Sunny Valley Networks datacenter. Sunny Valley Networks does not store any data processed by this API.

 Dialog box indicating that SVN does not store any data used for creating PDF reports

Figure 10. Dialog box indicating that SVN does not store any data used for creating PDF reports.

note

The PDF report feature is not available for the Free Edition. To gain the benefit of the PDF report feature, you must have one of the paid Zenarmor Subscriptions. For more information, see the plans & pricing.

To apply the configuration changes you made in the Scheduled Reports pane, click Save Changes at the end of the page.