This document provides a detailed guide on configuring a custom Splunk app,
PhoneHomeInterval, designed to set the phoneHomeIntervalInSecs for
Deployment Clients (DPL2, DPL3, DPL4) in a production environment. The
deployment architecture consists of DPL1 as the Deployment Server (Load
Balancer), with DPL2, DPL3, and DPL4 as its clients. Together, these three servers
manage around 8000 forwarders. First, the custom setting will apply to these DPLs
(DPL2–4), and only after that, the setting will propagate from them to their
respective connected forwarders.
The primary objective is to reduce the load on the Deployment Server by increasing
the phoneHomeIntervalInSecs to 1800 seconds (30 minutes) using a custom app
deployed through server class configurations.
Step 1: Create the Custom App on DPL1 (Deployment
Server)
Location: /opt/splunk/etc/deployment-apps
Navigate to the deployment apps directory:
Step 2: Backup and Modify serverclass.conf on DPL1
Location: /opt/splunk/etc/system/local/serverclass.conf
Backup the existing serverclass.conf file:
Once the configuration is added, reload the Deployment Server:
Some time later after reloading, you may observe that the app
PhoneHomeInterval gets pushed to DPL2, DPL3, and DPL4, but instead of
being installed in the usual /etc/apps/ directory, it appears under /etc/deploymentapps/.
To investigate this behavior, check the configuration of the following app on the
DPL2–4 nodes:
Normally, Deployment Clients (forwarders) receive apps under /etc/apps.
However, this stanza is used in setups where a server acts both as:
a Deployment Server (for its forwarders), and
a Deployment Client (of an upstream DS — in this case, DPL1).
This configuration:
Forces the instance to install apps only into /etc/deployment-apps.
Explicitly rejects any apps being installed into /etc/apps.
🔁 Application to Your Setup:
DPL2, DPL3, and DPL4 act as Deployment Clients of DPL1.
Due to the stanza, apps pushed from DPL1 (like PhoneHomeInterval)
install into /etc/deployment-apps.
Attempts to push apps into /etc/apps are rejected due to
serverRepositoryLocationPolicy = rejectAlways.
Note: Since Splunk only applies app configurations present under /etc/apps/, we had
to manually copy the app from /opt/splunk/etc/deployment-apps/ to
/opt/splunk/etc/apps/ on DPL2, DPL3, and DPL4 to ensure the configuration was
applied.
Step 4: Modify serverclass.conf on DPL2, DPL3, and
DPL4
Location: /opt/splunk/etc/system/local/serverclass.conf
Backup the serverclass.conf file on each server:
DPL3, or DPL4 as they will receive it from DPL1.
Step 5: Reload Deployment Server Configuration on
DPL2, DPL3, and DPL4
Run the following command on all four DPLs to reload the deployment server
configuration:
You may be prompted to enter credentials.
What Happens Next?
DPL2, DPL3, and DPL4 will receive the custom app PhoneHomeInterval
from DPL1.
This app contains the configuration setting their phoneHomeIntervalInSecs to
1800 seconds.
Similarly, all forwarders (~8,000) connected to DPL2, DPL3, and DPL4 will receive
this configuration via the server class settings.
Summary: Key Technical Points
phoneHomeIntervalInSecs = 1800 means clients will check in every 30 minutes,
reducing the load on the Deployment Server.
The app is created only on DPL1; DPL2–4 receive it automatically.
Server class settings must exist on all DPLs.
splunk reload deploy-server is required on all for changes to take effect.
Due to quanta_child_ds_deploymentclient, apps pushed from DPL1 land in
deployment-apps (not apps) on DPL2–4.
Appendix: Useful Paths and Commands
Deployment Apps Directory: /opt/splunk/etc/deployment-apps/
System Config Directory: /opt/splunk/etc/system/local/
Reload Command: /opt/splunk/bin/splunk reload deploy-server
If you are still facing an issue, feel free to Ask Doubts in the Comment Section Below and Don’t Forget to Follow us on 👍 Social Networks.
| Happy Splunking 😉
