This article will demonstrate how to configure the Syncplicity Storage Connector to send its health metrics data feed to Grafana, and how to set up some basic graphs which can be used to monitor the performance and health of the Storage Connector.
Grafana is an open source application for visualizing large-scale measurement data. It supports several back ends, including Graphite. Grafana is user-friendly allowing you to use its WYSIWYG editor to create efficient dashboards.
This article details the method to visualize health metrics data using Grafana dashboards.
Set up
Since Grafana is only a visualization tool you need to select, install and configure a compatible backend for it. For this article please reference the How to configure Storage Connector health metrics using Graphite guide to first set-up Graphite so that Grafana can leverage the metrics collector (Carbon) and database (Whisper) for storing and querying time-series data.
This article assumes that you have already installed and configured your Storage Connector(s) and they are up and running. Carbon is set-up on the host graphite and is listening to metrics on TCP port 2003 (default). Refer to the Graphite installation guide for detailed steps on how to do this.
We also assume that Grafana is installed and the instance is already available on the host graphana on port 80. Refer to the Grafana installation guide for detailed steps on how set-up and configure Grafana.
Storage Connector configuration
The first action is to add several new configuration settings in the Storage Connector config file. The default location of this config file is /etc/syncp-storage/syncp-storage.conf. The file is written in HOCON format; please see Using HOCON for reference.
You will need to add the following 5 lines to your config file:
syncplicity.health.enable syncplicity.health.external.enable syncplicity.health.external.host syncplicity.health.external.port syncplicity.health.external.prefix
It is recommended that you add these to the very end of the file.
The proceeding steps will go through how to input the values for each setting.
Step 1
On the Storage Connector node, open the config file for editing.
Step 2
Ensure that health monitoring is enabled.
syncplicity.health.enable = true
Step 3
Enable health metrics to be exported to an external port.
syncplicity.health.external.enable = true
Step 4
Specify the Carbon daemon host URL.
syncplicity.health.external.host = graphite.internaldomain.com
Step 5
Confirm that the Carbon daemon TCP port is specified (the default is port 2300).
syncplicity.health.external.port = 2300
Step 6
Set a prefix for the the health metrics. When an additional prefix is specified, such as "myCluster.myNode", all metrics roll up under the myCluster.myNode.syncp.compute.v1.* path.
syncplicity.health.external.prefix = myCluster.myNode
Alternatively you can use UNIX environment variables to set the prefix. For example, if you have these environment variables set:
HOST=$(hostname -s) CLUSTER="myCluster"
Then you can configure this setting in the config file as:
syncplicity.health.external.prefix = $CLUSTER.$HOST
Step 7
Save your changes to the config file and restart Storage Connector.
Grafana configuration
Now that you have changed the settings on the Storage Connector to enable health metrics and send them to Grafana, you need to make some changes within Grafana to start visualizing those metrics.
Step 8
Add the Graphite data source to the list of data sources on the Grafana dashboard and save your changes.
Step 9
Confirm that Grafana is correctly connected to Graphite and has started to receive metrics data from the Storage Connector.
Navigate to the Grafana interface and create a new dashboard. Then click Edit and start adding a new query. The system provides an auto-complete option for various metrics. The prefix you specified in the "syncplicity.health.external.prefix" will be displayed or a hostname of the storage connector will be displayed if the prefix is not specified. It should be followed by *.syncp.compute.v1 after which you should be able to choose from the metrics to display.
At this point Grafana should be configured correctly and ready to visualize Storage Connector health metrics. Next we will demonstrate how to create Grafana dashboard panels that graph a few of the interesting Storage Connector metrics.
Using Grafana to visualize performance data
The following examples will demonstrate how to visualize Storage Connector health metrics as graphs below for 3 scenarios:
- The total number of requests (uploads + downloads)
- The total amount of traffic (number of bytes uploaded and downloaded)
- The JVM heap status
Example 1 - Visualize the total number of requests (uploads + downloads)
- Navigate to the http://graphite:1080/grafana web interface
- Create a new dashboard or select an existing one
-
Add a new panel, name it "Bytes In/Out" and add the following two queries in the Metrics view:
"*.*.syncp.compute.v1.transfer.in.bytes" "scale(*.*.syncp.compute.v1.transfer.out.bytes, -1)"
-
Return to the dashboard. The result should be similar to the screen shown below.
Example 2 - Visualize the total amount of traffic (number of bytes uploaded and downloaded)
- Navigate to the http://graphite:1080/grafanaweb interface
- Create a new dashboard or select an existing one
-
Add a new panel, name it "Concurrent Requests" and add the following two queries in the Metrics view:
"*.*.syncp.compute.v1.syncRequests.currentDownloadRequestsCounter" "*.*.syncp.compute.v1.syncRequests.currentUploadRequestsCounter”
-
Return to the dashboard. The result should be similar to the screen shown below.
Example 3 - Visualize the JVM heap status
- Navigate to the http://graphite:1080/grafana web interface
- Create a new dashboard or select an existing one
-
Add a new panel, name it "JVM Heap" and add the following two queries in the Metrics view:
"alias(scale(*.*.syncp.compute.v1.jvm.heapUsed, 0.000001), 'heap used (MB) ')" "alias(*.*.syncp.compute.v1.jvm.threadCount, 'thread count')"
-
Return to the dashboard. The result should be similar to the screen shown below.