MongoDB Enterprise Server Onboarding Steps

Onboarding a data source requires preparation, such as gathering permissions and collecting relevant information for your deployment. Assistance may be necessary from a database administrator, network administrator, and an IT administrator to successfully begin monitoring your data source.

Please ensure all of the following items are properly configured or available for use. The information and permissions gathered in this step are required for the remaining onboarding steps.

Network Prerequisites

Below is an overview of the necessary requirements to enable the Agentless Gateway to communicate with data sources and other components across the network. Please ensure these are completed by a Network Administrator.

• Ensure the MongoDB Enterprise Server host’s firewall allows outbound TCP traffic to the Agentless Gateway host IP on port 10501.

  Note:

  On Linux, to verify that the port is open, run the command below and ensure that it is listed in the results.

  sudo semanage port -l | grep syslogd_port_t

IT Prerequisites

Below is an overview of the necessary steps which will need to be completed on the Agentless Gateway host or the data source. Please ensure these steps are completed by an IT Administrator.

On Windows

• You have "administrator" privileges on the MongoDB Enterprise Server host to install NXLog (a universal log collector and forwarder tool) if not already installed and update the nxlog.conf file. 
• If NXLog is not already installed on the system, install and set it up. For more information and steps for downloading and installing NXLog, please visit https://docs.nxlog.co/userguide/deploy/windows.html.

On Linux

• You have sudo privileges on the MongoDB Enterprise Server host to create the rsyslog configuration file and update the MongoDB configuration file.

• If the MongoDB host has SElinux enabled, ensure that the rsyslog service is able to access the audit log files. 

ls -dZ /path/to/my/audit.log

Database Permissions (Optional)

To enable audit log rotation on either Windows or Linux, a database user with the "root" role is required. This user can be used to connect to the admin database and run the "logRotate" command.

Once the required permissions and information have been obtained, please complete the following steps to enable audit on the data source. 

This documentation provides instructions for enabling MongoDB Enterprise Server audit on both Linux and Windows platforms. Follow the instructions provided in the below sections depending on the platform on which your data source is hosted.

Enable Audit on Linux

Enabling audit on Linux involves the following two steps. Please complete the instructions under each step.

• Step One: Updating the audit configuration parameters in the mongo configuration file.
  
• Step Two: Configuring rsyslog to forward the audit logs to Agentless Gateway host.

Step One: Updating the audit configuration parameters in the mongo configuration file

1. As "root" user, edit the mongod.conf file using the below command.

   sudo vi /etc/mongod.conf

2. In the mongod.conf file, the values for parameters should be set as explained below. 

   • security: Set authorization to enabled. This is required for role-based access control.

   • setParameter: Set auditAuthorizationSuccessEnable to true. This will log authorization successes.

   • auditLog: Set the destination to "file" with JSON format and path (full path) of the audit log file. Set a value for filter (the "filter" parameter is optional).
     

     Note:

     Configuring Audit filter to filter out logs from background operations (OPTIONAL)
     

     MongoDB generates many logs due to background operations occurring in regular intervals. To filter these out, a custom audit filter can be used.
     For example, to filter out logs with local port being 0, use this:

     filter: '{ "local.port": { $ne: 0 } }'

     To filter out logs from background operation databases (e.g. "config", "admin"), use this:

     filter: '{ "param.args.$db": { $nin: ["config"] } }'

     
     

     To filter the background actions in a sharded cluster this filter can be used. For more details and examples, please see the MongoDB documentation. 
     

     An example of a custom audit filter is as follows. This filter operates on $or operator and executes when it matches on any one of the following 2 blocks:

     • 1st block executes when both $and conditions are met - Audits logs where user is not "__system" and "$client.mongos" does not exist
     • 2nd block executes when both $and conditions are met - Audits logs where user is "__system" on all databases except "local"

       filter:
       '{
         $or: [
           { $and : [
               { "param.args.$client.mongos": { $exists:false } },
               { "users.user" : { $exists: true, $ne: "__system" } }
             ]  },
       
           { $and : [
               { "users.user" : { $exists: true, $eq: "__system" } },
               { "users.db": { $ne: "local" } }
             ]  }
         ]
       }'

     
     

     Uncomment or add the parameters in the mongod.conf file as shown below.

     To apply the filter, copy and paste it as the "filter" value. Ensure the indentation is correct.
     

     security:
       authorization: enabled
      
     auditLog:
       destination: file
       format: JSON
       path: <audit log path>/auditLog.json   # usually /var/lib/mongo/auditLog.json
       filter: '<custom filter>'
     setParameter: {auditAuthorizationSuccess: true}

3. Restart the mongod service.
   

   sudo systemctl restart mongod

   
   

   Note:

   For a sharded cluster setup, audit should be enabled on all the nodes of the cluster. 

   1. Repeat the instructions in Step 1 on all the config servers and shard servers.  Update the mongod.conf file on all your config servers and shard replica sets with audit parameters as shown above and restart the "mongod" service. 

   2. On the mongos server, repeat the same instructions but the audit parameters should be added to mongos.conf (not mongod) and the "mongos" service should be restarted. 

    

Step Two: Configuring Rsyslog to forward the audit logs to Agentless Gateway host

On the MongoDB Enterprise Server host, set the Rsyslog config file to forward the messages to the Agentless Gateway host:

sudo vi /etc/rsyslog.d/mongodb_audit_forward.conf

Add the following lines, and replace the necessary parameter values:

• target - the Agentless Gateway host IP 
• File - the full path of audit log file
  
• If you are not using the default Server Port, replace "27017" in the code below with your port value. 

$MaxMessageSize 18000000
 
module(load="imfile")
input(type="imfile" Tag="jsonaraudit:" File="<audit log path>/auditLog.json" ruleset="pRuleset")
  
template(name="mongo_rawmsg" type="list") {
    constant(value="{ ")
    constant(value="\"Server Port\":\"27017\"")
    constant(value=" }")
    constant(value="PR3N0RM")
    property(name="rawmsg")
}
  
ruleset(name="pRuleset") {
action(type="omfwd"
     keepalive="on"
     protocol="tcp"
     target="<Agentless-Gateway-IP>"
     port="10501"
     template="mongo_rawmsg")
stop
}

Run this command to verify the Rsyslog configuration file is correct:

rsyslogd -N1

Restart the Rsyslog service:

sudo systemctl restart rsyslog

Enable Audit on Windows

Enabling audit on Windows involves the following two steps. Please complete the instructions listed under each step below.

• Step One: Updating the audit configuration parameters in the mongo configuration file.
  
• Step Two: Configuring NXLog to forward the audit logs to Agentless Gateway host

Step One: Updating the audit configuration parameters in the mongo configuration file

1. As "administrator", open the mongod.cfg file with a compatible application. The file can be found in the MongoDB installation path, e.g. C:\Program Files\MongoDB\Server\<Version>\bin\mongod.cfg
2. Uncomment or add the below parameters with the values suggested below. 

• security - Set authorization: enabled. This is done to use role-based access control 

• auditLog: set the following parameters:

  • destination - Set it to "file"

  • format: Should be set to "JSON"

  • path: Specify filename path (full or relative path). This depends on 

  • filter: This is an optional parameter. MongoDB Enterprise Server generates many logs from its background operations that occur at regular intervals. To filter these out, a custom audit filter can be used.
    For example:

    - To filter out logs with local port being 0, replace the <custom filter> with the code below:

    '{ "local.port": { $ne: 0 } }'

    - To filter out logs from background operations dbs (e.g "config","admin"), replace the <custom filter> with the section below

    '{ "param.args.$db": { $nin: ["config"] } }

• setParameter: enable the audit system to log authorization successes using the auditAuthorizationSuccess parameter, to capture read and write operations in the audit.

The conf file should have the below parameters and values. Make sure the indentation is correct:

security:
  authorization: enabled
auditLog:
  destination: file
  format: JSON
  path: <audit log path>\auditLog.json   # usually C:\Program Files\MongoDB\Server\4.2\log\auditLog.json
  filter: '<custom filter>'
setParameter: {auditAuthorizationSuccess: true}

As "administrator", open a command prompt and run the following commands to restart the mongod service:

net stop MongoDB
net start MongoDB

Step Two: Configuring NXLog to forward the audit logs to Agentless Gateway host

Edit the nxlog.conf file and replace the parameters below with the following values:

• Host: IP of the Agentless Gateway host
• File: full path of the audit log file.
• Server Port : If you are not using the default Server Port, replace "27017" in the code below with your port value. 

<Input in_auditlog>
    Module im_file
    File '<audit log path>\auditLog.json'
    SavePos TRUE
    ReadFromLast TRUE
    PollInterval 1
    Exec $Message = '{ "Server Port":"27017" }PR3N0RM' + $raw_event; $SyslogFacilityValue = 22;
</Input>
  
<Output out_jsonaraudit>
    Module om_tcp
    Host <Agentless Gateway IP>
    Port 10501
    Exec to_syslog_bsd();
</Output>
  
<Route route_jsonaraudit>
    Path in_auditlog => out_jsonaraudit
</Route>

As "administrator", open command prompt and run the following commands to restart the NXLog service:

net stop nxlog
net start nxlog

Configuring Audit Log Rotation (Optional)

On a Windows Server:

1. To create the "AuditLog_cleanup.bat" file in the desired location, enter the command below:

   REM @echo off

2. Edit the following lines to include the full path to the file, replacing the following values:

   <batch file path> with the full path to the "AuditLog_cleanup.bat" file 

   <audit log path> with the full path of the "auditLog.*.json" file

   @echo %date% %time% >> <batch file path>\AuditLog_cleanup.log

   forfiles -p "<audit log path>\auditLog.*.json" -m * /D -<# of days to keep logs> /C "cmd /c del /s/q @path >> <batch file path>\AuditLog_cleanup.log"

3. Next set up the Windows task scheduler to run the cleanup.bat script that was created in the existing schedule or create a new one, if needed.

On Windows and Linux:

MongoDB has the option of rotating the log files specified in /etc/mongos.conf. This will be the same path specified in the "auditLog" section while enabling audit on the data source.

1. Connect to your admin database with a user that has "root" role, and run the below command. For more details please see the MongoDB documentation.
   

   db.adminCommand({logRotate: 1})

2. Check the log path directory, and you will see that the log file was rotated, for example:

   auditLog.json
   auditLog.json.2023-06-27T17-26-08

After completing the prerequisites and enabling audit, the data source is ready to be onboarded onto DSF. This can be accomplished using ANY ONE of the methods listed below:

• Importing Assets via Unified Settings Console (USC)
• Importing Assets via Assets Dashboard
• Importing Assets via DSF Open APIs

Please use the Asset Specifications below as a guide to fill in the field values for this data source. 

MongoDB Asset Specifications

Importing Assets via Unified Settings Console (USC)

The USC feature in the DSF Portal allows users to configure a full audit flow, including importing new data assets. To access the USC, visit the DSF Portal and under Apps, click the Unified Settings Console link. To add a new data source asset, please complete these steps:

1. From the DSF Portal, under Apps, click Unified Settings Console.
2. In the Appliances pane, select DSF Hub.
3. Click the Data Sources tab to open the Data Sources page.
4. Click "Add" to open the Add Data Source form.
5. In the Data Source Type section, select a data source via the dropdown menu.
6. Specific data source configuration sections will display: Details, Connections, and Monitoring. Configure the mandatory configuration fields under Details and any optional configuration fields displayed under Advanced.
7. Under Connection, select an authentication method (Auth Mechanism) from the drop-down menu. The mandatory fields for the selected Auth Mechanism are displayed; to see optional configuration fields available, click Advanced.
8. Click "Save". The Add Data Source form closes and the Data Sources page opens.
9. Locate the asset you want to connect. Click on "Enable Audit Collection" to start collecting audit data. 

For additional instructions on adding, viewing and editing Data Source assets, see  Adding Assets via Unified Settings Console (USC)  documentation. 

Importing Assets using an Asset Spreadsheet

Complete these steps to import data source assets using Asset Spreadsheets: 

1. Log into the DSF Hub with your username and password.
2. On the DSF Hub homepage, under Apps, click Sync Spreadsheet. 
3. A new window will open, click Import Assets.
4. On the Import Assets page, go to the Assets Templates dropdown menu.
5. Select the data source template and click Download.
6. Use the Asset Specification documentation as a guide, and fill in the necessary field values. 
7. On the Import Assets page, find Upload 'Assets and Connections to Import' spreadsheet. 
8. Navigate to your completed asset spreadsheet and click Open. Click Upload.
9. Click Validate All to validate the current configuration of the spreadsheet.
10. Click Run 'Import Assets' to complete the process.
11. On the Asset Dashboard page, locate the asset that was imported and click “Connect Gateway” on the database asset to start collecting Audit data.

For more details, please visit Adding Assets via the Import Assets Page.

Importing Assets via DSF Open APIs

Data Security Fabric (DSF) Open APIs provide functions for onboarding and managing assets (log aggregators, cloud accounts, data sources, secret managers and other assets) via a RESTful API. For more information on the supported assets and how to onboard them, please see Using DSF Open APIs.