Workspaces Metrics Documentation
================================
Welcome to Workspaces Metrics!
------------------------------
- `Introduction <#WorkspacesMetricsDocumentation-Introduc>`__
- `View Specifications <#WorkspacesMetricsDocumentation-ViewSpec>`__
- `General Capability
Metrics <#WorkspacesMetricsDocumentation-GeneralC>`__
- `Download Metrics <#WorkspacesMetricsDocumentation-Download>`__
- `Direct Database
Queries <#WorkspacesMetricsDocumentation-DirectDa>`__
- `Suggested Usage <#WorkspacesMetricsDocumentation-Suggeste>`__
- `General Capability
Metrics <#WorkspacesMetricsDocumentation-GeneralC>`__
- `Download Metrics <#WorkspacesMetricsDocumentation-Download>`__
- `Warnings and Notes <#WorkspacesMetricsDocumentation-Warnings>`__
- `Workspaces Metrics CLI <#WorkspacesMetricsDocumentation-Workspac>`__
- `Structure <#WorkspacesMetricsDocumentation-Structur>`__
- `Usage <#WorkspacesMetricsDocumentation-Usage>`__
- `General Capability
Metrics <#WorkspacesMetricsDocumentation-GeneralC>`__
- `Download Metrics <#WorkspacesMetricsDocumentation-Download>`__
Introduction
~~~~~~~~~~~~
There are two ways of obtaining metrics data from the workspaces system:
1) Via direct database queries on the view tables created specifically
for workspaces metrics analytics
2) Via the new Workspaces Metrics CLI
View Specifications
~~~~~~~~~~~~~~~~~~~
`WS-164 <https://open-jira.nrao.edu/browse/WS-164>`__ - Getting issue
details... STATUS
`WS-161 <https://open-jira.nrao.edu/browse/WS-161>`__ - Getting issue
details... STATUS
`WS-162 <https://open-jira.nrao.edu/browse/WS-162>`__ - Getting issue
details... STATUS
Currently there are two view tables, *capability_execution_metrics*
and *download_metrics*, available as this system is still under active
development.
General Capability Metrics
^^^^^^^^^^^^^^^^^^^^^^^^^^
The purpose of the *capability_execution_metrics* table is to provide
information regarding the number of times any specified Capability is
executed by the workspaces system.
The *capability_execution_metrics* view holds the following fields:
+-----------------------+---------------------------------------------+
| request_id | The id of the execution's parent |
| | CapabilityRequest. |
| | |
| | (Anticipated need, not currently used) |
+=======================+=============================================+
| cap_name | The name of the Capability being executed |
+-----------------------+---------------------------------------------+
| request_created | The timestamp at which the |
| | CapabilityRequest was created in format |
| | year-month-day hour:minute:seconds (UTC). |
| | |
| | (Anticipated need, not currently used) |
+-----------------------+---------------------------------------------+
| execution_id | The id of this execution |
+-----------------------+---------------------------------------------+
| execution_created | The timestamp at which this execution was |
| | created in format year-month-day |
| | hour:minute:seconds (UTC) |
+-----------------------+---------------------------------------------+
| execution_state | The current state of this execution. |
| | |
| | (Anticipated need, not currently used) |
+-----------------------+---------------------------------------------+
| execution_workflow_id | The id of this execution's WorkflowRequest. |
| | |
| | (Anticipated need, not currently used) |
+-----------------------+---------------------------------------------+
Download Metrics
^^^^^^^^^^^^^^^^
The purpose of the *download_metrics* table is to provide information
regarding the amount of data downloaded for a specific capability
request or over a specified length of time.
The *download_metrics* view holds the following fields:
+-------------------+-------------------------------------------------+
| request_id | The id of the CapabilityRequest |
+===================+=================================================+
| cap_name | The name of the Capability that was run |
+-------------------+-------------------------------------------------+
| execution_created | The timestamp at which the request's execution |
| | was created in format year-month-day |
| | hour:minute:seconds (UTC) |
+-------------------+-------------------------------------------------+
| state | The state of the CapabilityExecution |
+-------------------+-------------------------------------------------+
| product_locator | The product locator that was downloaded |
+-------------------+-------------------------------------------------+
| filegroup_id | The filegroup of the product locator |
+-------------------+-------------------------------------------------+
| datasize | The size of the data that was downloaded |
+-------------------+-------------------------------------------------+
Direct Database Queries
~~~~~~~~~~~~~~~~~~~~~~~~
The first method of metrics collection is the currently used method of
direct database queries against view tables.
For this method, the metrics analyst will be provided with a set of DB
credentials, the names of the metrics views, and allowed to formulate
their own queries.
Suggested Usage
^^^^^^^^^^^^^^^
.. _general-capability-metrics-1:
General Capability Metrics
''''''''''''''''''''''''''
Based on the current system expectations, there are two possible ways to
run metrics data:
1) By Capability name only
2) By a given date range.
Query for returning total count of all capability executions of a
specific name:
**All Capabilities Executed by Name**
SELECT count(*)
FROM capability_execution_metrics
WHERE cap_name = '<name>';
Query for returning total count of all capability executions of a
specific name in a specific date range:
**All Capabilities by Name and Date Range**
SELECT count(*)
FROM capability_execution_metrics
WHERE cap_name = '<name>' AND
execution_created between '<beginning_date>' and '<end_date>';
.. _download-metrics-1:
Download Metrics
''''''''''''''''
There are currently two available download metrics:
1) Size of data downloaded by request
2) Size of data downloaded total in a given date range
Query for returning the size of the data downloaded in one request:
**Data Size by Request**
SELECT sum(datasize)
FROM download_metrics
WHERE request_id = '<id>'
AND state = 'Complete'
GROUP BY request_id
Query for returning the total amount of data downloaded in a date range:
**Data Downloaded by Date Range**
SELECT sum(datasize)
FROM download_metrics
WHERE state = 'Complete'
AND execution_created BETWEEN <beginning_date> AND <end_date>
Warnings and Notes
^^^^^^^^^^^^^^^^^^
Timestamps should be in the following format: '2021-03-30T12:00:00'.
Warning: Don't forget the single quotes! Especially since we have a
Capability named *null*!
Warning: Please note that all timestamps are stored in UTC time, not
local time.
Workspaces Metrics CLI
~~~~~~~~~~~~~~~~~~~~~~
The second method of metrics collection is the Workspaces Metrics CLI.
This is a command line tool that will be available on servers were the
workspaces system is deployed via the command **ws_metrics**.
Structure
^^^^^^^^^
The CLI currently has two available options:
+------------+---------------------------+---------------------------+
| **option** | **parameters needed** | **purpose** |
+============+===========================+===========================+
| -c | capability_name | Returns the total count |
| | | of all <capability_name> |
| | | executions since the |
| | | database was initialized |
+------------+---------------------------+---------------------------+
| -b | capability_name, | Returns the total count |
| | beginning_timestamp, | of all <capability_name> |
| | end_timestamp | executions in the range |
| | | beginning_timestamp to |
| | | end_timestamp |
+------------+---------------------------+---------------------------+
| -rd | capability_request_id | Returns the data size |
| | | that was downloaded by |
| | | this request in kilobytes |
+------------+---------------------------+---------------------------+
| -d | beginning_timestamp, | Returns the data size |
| | end_timestamp | that was downloaded in |
| | | the range |
| | | beginning_timestamp to |
| | | end_timestamp |
+------------+---------------------------+---------------------------+
Usage
^^^^^
To access ws_metrics do the following:
1) On a server running workspaces, perform the following command to
access the capability docker container:
**Docker Container Bash Access**
docker exec -it workspaces_capability_1 bash
2) Now that you are in the proper container, you can access ws_metrics!
**ws_metrics usage**
ws_metrics <option> <parameters>
.. _general-capability-metrics-2:
General Capability Metrics
^^^^^^^^^^^^^^^^^^^^^^^^^^
To acquire the total number of *all* capability executions of a specific
name run since the database was created, use:
**ws_metrics capability usage**
ws_metrics -c <capability_name>
To acquire the total number of capability executions of a specific
capability name run within a specified date range, use:
**ws_metrics capability range usage**
ws_metrics -b <capability_name> <beginning_timestamp> <end_timestamp>
.. _download-metrics-2:
Download Metrics
^^^^^^^^^^^^^^^^
To acquire the data size for a capability request of interest, use:
**ws_metrics request datasize usage**
ws_metrics -rd <request_id>
To acquire the data size of everything downloaded in a specified date
range, use:
**ws_metrics total download size usage**
ws_metrics -d <beginning_timestamp> <end_timestamp>