Events APIs
New in version 3.9
NOTE : glusterfs-selinux package would have to be installed for events feature to function properly when the selinux is in enforced mode. In addition to that, the default port to be used for eventsd has now been changed to 55555 and it has to lie between the ephemeral port ranges.
Set PYTHONPATH(Only in case of Source installation)
If Gluster is installed using source install, cliutils
will get
installed under /usr/local/lib/python.2.7/site-packages
Set
PYTHONPATH by adding in ~/.bashrc
# export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH
Enable and Start Events APIs
Enable and Start glustereventsd in all peer nodes
In Systems using Systemd,
# systemctl enable glustereventsd
# systemctl start glustereventsd
FreeBSD or others, add the following in /etc/rc.conf
glustereventsd_enable="YES"
And start the glustereventsd using,
# service glustereventsd start
SysVInit(CentOS 6),
# chkconfig glustereventsd on
# service glustereventsd start
Status
Status Can be checked using,
# gluster-eventsapi status
Example output:
Webhooks:
None
+-----------+-------------+-----------------------+
| NODE | NODE STATUS | GLUSTEREVENTSD STATUS |
+-----------+-------------+-----------------------+
| localhost | UP | UP |
| node2 | UP | UP |
+-----------+-------------+-----------------------+
Webhooks
Webhooks are similar to callbacks(over HTTP), on event Gluster will call the Webhook URL(via POST) which is configured. Webhook is a web server which listens on a URL, this can be deployed outside of the Cluster. Gluster nodes should be able to access this Webhook server on the configured port.
Example Webhook written in python,
from flask import Flask, request
app = Flask(__name__)
@app.route("/listen", methods=["POST"])
def events_listener():
gluster_event = request.json
if gluster_event is None:
# No event to process, may be test call
return "OK"
# Process gluster_event
# {
# "nodeid": NODEID,
# "ts": EVENT_TIMESTAMP,
# "event": EVENT_TYPE,
# "message": EVENT_DATA
# }
print (gluster_event)
return "OK"
app.run(host="0.0.0.0", port=9000)
Test and Register webhook using following commands,
usage: gluster-eventsapi webhook-test [-h] [--bearer_token BEARER_TOKEN] url
positional arguments:
url URL of Webhook
optional arguments:
-h, --help show this help message and exit
--bearer_token BEARER_TOKEN, -t BEARER_TOKEN
Bearer Token
Example(Webhook server is running in 192.168.122.188:9000
),
# gluster-eventsapi webhook-test http://192.168.122.188:9000/listen
+-----------+-------------+----------------+
| NODE | NODE STATUS | WEBHOOK STATUS |
+-----------+-------------+----------------+
| localhost | UP | OK |
| node2 | UP | OK |
+-----------+-------------+----------------+
If Webhook status is OK from all peer nodes then register the Webhook using,
usage: gluster-eventsapi webhook-add [-h] [--bearer_token BEARER_TOKEN] url
positional arguments:
url URL of Webhook
optional arguments:
-h, --help show this help message and exit
--bearer_token BEARER_TOKEN, -t BEARER_TOKEN
Bearer Token
Example,
# gluster-eventsapi webhook-add http://192.168.122.188:9000/listen
+-----------+-------------+-------------+
| NODE | NODE STATUS | SYNC STATUS |
+-----------+-------------+-------------+
| localhost | UP | OK |
| node2 | UP | OK |
+-----------+-------------+-------------+
Note: If Sync status is Not OK for any node, then make sure to run following command from a peer node when that node comes up.
# gluster-eventsapi sync
To unsubscribe from events, delete the webhook using following command
usage: gluster-eventsapi webhook-del [-h] url
positional arguments:
url URL of Webhook
optional arguments:
-h, --help show this help message and exit
Example,
# gluster-eventsapi webhook-del http://192.168.122.188:9000/listen
Configuration
View all configurations using,
usage: gluster-eventsapi config-get [-h] [--name NAME]
optional arguments:
-h, --help show this help message and exit
--name NAME Config Name
Example output:
+--------------------+-------+
| NAME | VALUE |
+--------------------+-------+
| log-level | INFO |
| port | 55555 |
| disable-events-log | False |
+--------------------+-------+
To change any configuration,
usage: gluster-eventsapi config-set [-h] name value
positional arguments:
name Config Name
value Config Value
optional arguments:
-h, --help show this help message and exit
Example output,
+-----------+-------------+-------------+
| NODE | NODE STATUS | SYNC STATUS |
+-----------+-------------+-------------+
| localhost | UP | OK |
| node2 | UP | OK |
+-----------+-------------+-------------+
To Reset any configuration,
usage: gluster-eventsapi config-reset [-h] name
positional arguments:
name Config Name or all
optional arguments:
-h, --help show this help message and exit
Example output,
+-----------+-------------+-------------+
| NODE | NODE STATUS | SYNC STATUS |
+-----------+-------------+-------------+
| localhost | UP | OK |
| node2 | UP | OK |
+-----------+-------------+-------------+
Note: If any node status is not UP or sync status is not OK, make
sure to run gluster-eventsapi sync
from a peer node.
Add node to the Cluster
When a new node added to the cluster,
- Enable and Start Eventsd in the new node using the steps mentioned above
- Run
gluster-eventsapi sync
command from a peer node other than the new node.
APIs documentation
Glustereventsd pushes the Events in JSON format to configured Webhooks. All Events will have following attributes.
Attribute | Description |
---|---|
nodeid | Node UUID |
ts | Event Timestamp |
event | Event Type |
message | Event Specific Data |
Example:
{
"nodeid": "95cd599c-5d87-43c1-8fba-b12821fd41b6",
"ts": 1468303352,
"event": "VOLUME_CREATE",
"message": {
"name": "gv1"
}
}
"message" can have following attributes based on the type of event.
Peer Events
Event Type | Attribute | Description |
---|---|---|
PEER_ATTACH | host | Hostname or IP of added node |
PEER_DETACH | host | Hostname or IP of detached node |
Volume Events
Event Type | Attribute | Description |
---|---|---|
VOLUME_CREATE | name | Volume Name |
VOLUME_START | force | Force option used or not during Start |
name | Volume Name | |
VOLUME_STOP | force | Force option used or not during Stop |
name | Volume Name | |
VOLUME_DELETE | name | Volume Name |
VOLUME_SET | name | Volume Name |
options | List of Options[(key1, val1), (key2, val2),..] | |
VOLUME_RESET | name | Volume Name |
option | Option Name |
Brick Events
Event Type | Attribute | Description |
---|---|---|
BRICK_RESET_START | volume | Volume Name |
source-brick | Source Brick details | |
BRICK_RESET_COMMIT | volume | Volume Name |
destination-brick | Destination Brick | |
source-brick | Source Brick details | |
BRICK_REPLACE | volume | Volume Name |
destination-brick | Destination Brick | |
source-brick | Source Brick details |
Georep Events
Event Type | Attribute | Description |
---|---|---|
GEOREP_CREATE | force | Force option used during session Create |
secondary | Secondary Details(Secondaryhost::SecondaryVolume) | |
no_verify | No verify option is used or not | |
push_pem | Push pem option is used or Not | |
ssh_port | If SSH port is configured during Session Create | |
primary | Primary Volume Name | |
GEOREP_START | force | Force option used during session Start |
Primary | Primary Volume Name | |
secondary | Secondary Details(Secondaryhost::SecondaryVolume) | |
GEOREP_STOP | force | Force option used during session Stop |
primary | Primary Volume Name | |
secondary | Secondary Details(Secondaryhost::SecondaryVolume) | |
GEOREP_PAUSE | force | Force option used during session Pause |
primary | Primary Volume Name | |
secondary | Secondary Details(Secondaryhost::SecondaryVolume) | |
GEOREP_RESUME | force | Force option used during session Resume |
primary | Primary Volume Name | |
secondary | Secondary Details(Secondaryhost::SecondaryVolume) | |
GEOREP_DELETE | force | Force option used during session Delete |
primary | Primary Volume Name | |
secondary | Secondary Details(Secondaryhost::SecondaryVolume) | |
GEOREP_CONFIG_SET | primary | Primary Volume Name |
secondary | Secondary Details(Secondaryhost::SecondaryVolume) | |
option | Name of Geo-rep config | |
value | Changed Value | |
GEOREP_CONFIG_RESET | primary | Primary Volume Name |
secondary | Secondary Details(Secondaryhost::SecondaryVolume) | |
option | Name of Geo-rep config |
Bitrot Events
Event Type | Attribute | Description |
---|---|---|
BITROT_ENABLE | name | Volume Name |
BITROT_DISABLE | name | Volume Name |
BITROT_SCRUB_THROTTLE | name | Volume Name |
value | Changed Value | |
BITROT_SCRUB_FREQ | name | Volume Name |
value | Changed Value | |
BITROT_SCRUB_OPTION | name | Volume Name |
value | Changed Value |
Quota Events
Event Type | Attribute | Description |
---|---|---|
QUOTA_ENABLE | volume | Volume Name |
QUOTA_DISABLE | volume | Volume Name |
QUOTA_SET_USAGE_LIMIT | volume | Volume Name |
path | Path in Volume on which Quota option is set | |
limit | Changed Value | |
QUOTA_SET_OBJECTS_LIMIT | volume | Volume Name |
path | Path in Volume on which Quota option is set | |
limit | Changed Value | |
QUOTA_REMOVE_USAGE_LIMIT | volume | Volume Name |
path | Path in Volume on which Quota option is Reset | |
QUOTA_REMOVE_OBJECTS_LIMIT | volume | Volume Name |
path | Path in Volume on which Quota option is Reset | |
QUOTA_ALERT_TIME | volume | Volume Name |
time | Changed Alert Time | |
QUOTA_SOFT_TIMEOUT | volume | Volume Name |
soft-timeout | Changed Value | |
QUOTA_HARD_TIMEOUT | volume | Volume Name |
hard-timeout | Changed Value | |
QUOTA_DEFAULT_SOFT_LIMIT | volume | Volume Name |
default-soft-limit | Changed Value |
Snapshot Events
Event Type | Attribute | Description |
---|---|---|
SNAPSHOT_CREATED | snapshot_name | Snapshot Name |
volume_name | Volume Name | |
snapshot_uuid | Snapshot UUID | |
SNAPSHOT_CREATE_FAILED | snapshot_name | Snapshot Name |
volume_name | Volume Name | |
error | Failure details | |
SNAPSHOT_ACTIVATED | snapshot_name | Snapshot Name |
snapshot_uuid | Snapshot UUID | |
SNAPSHOT_ACTIVATE_FAILED | snapshot_name | Snapshot Name |
error | Failure details | |
SNAPSHOT_DEACTIVATED | snapshot_name | Snapshot Name |
snapshot_uuid | Snapshot UUID | |
SNAPSHOT_DEACTIVATE_FAILED | snapshot_name | Snapshot Name |
error | Failure details | |
SNAPSHOT_SOFT_LIMIT_REACHED | volume_name | Volume Name |
volume_id | Volume ID | |
SNAPSHOT_HARD_LIMIT_REACHED | volume_name | Volume Name |
volume_id | Volume ID | |
SNAPSHOT_RESTORED | snapshot_name | Snapshot Name |
volume_name | Volume Name | |
snapshot_uuid | Snapshot UUID | |
SNAPSHOT_RESTORE_FAILED | snapshot_name | Snapshot Name |
error | Failure details | |
SNAPSHOT_DELETED | snapshot_name | Snapshot Name |
snapshot_uuid | Snapshot UUID | |
SNAPSHOT_DELETE_FAILED | snapshot_name | Snapshot Name |
error | Failure details | |
SNAPSHOT_CLONED | clone_uuid | Snapshot Clone UUID |
snapshot_name | Snapshot Name | |
clone_name | Snapshot Clone Name | |
SNAPSHOT_CLONE_FAILED | snapshot_name | Snapshot Name |
clone_name | Snapshot Clone Name | |
error | Failure details | |
SNAPSHOT_CONFIG_UPDATED | auto-delete | Auto delete Value if available |
config_type | Volume Config or System Config | |
hard_limit | Hard Limit Value if available | |
soft_limit | Soft Limit Value if available | |
snap-activate | Snap activate Value if available | |
SNAPSHOT_CONFIG_UPDATE_FAILED | error | Error details |
SNAPSHOT_SCHEDULER_INITIALISED | status | Succss Status |
SNAPSHOT_SCHEDULER_INIT_FAILED | error | Error details |
SNAPSHOT_SCHEDULER_ENABLED | status | Succss Status |
SNAPSHOT_SCHEDULER_ENABLE_FAILED | error | Error details |
SNAPSHOT_SCHEDULER_DISABLED | status | Succss Status |
SNAPSHOT_SCHEDULER_DISABLE_FAILED | error | Error details |
SNAPSHOT_SCHEDULER_SCHEDULE_ADDED | status | Succss Status |
SNAPSHOT_SCHEDULER_SCHEDULE_ADD_FAILED | error | Error details |
SNAPSHOT_SCHEDULER_SCHEDULE_EDITED | status | Succss Status |
SNAPSHOT_SCHEDULER_SCHEDULE_EDIT_FAILED | error | Error details |
SNAPSHOT_SCHEDULER_SCHEDULE_DELETED | status | Succss Status |
SNAPSHOT_SCHEDULER_SCHEDULE_DELETE_FAILED | error | Error details |
Svc Events
Event Type | Attribute | Description |
---|---|---|
SVC_MANAGER_FAILED | volume | Volume Name if available |
svc_name | Service Name | |
SVC_CONNECTED | volume | Volume Name if available |
svc_name | Service Name | |
SVC_DISCONNECTED | svc_name | Service Name |
Peer Events
Event Type | Attribute | Description |
---|---|---|
PEER_STORE_FAILURE | peer | Hostname or IP |
PEER_RPC_CREATE_FAILED | peer | Hostname or IP |
PEER_REJECT | peer | Hostname or IP |
PEER_CONNECT | host | Hostname or IP |
uuid | Host UUID | |
PEER_DISCONNECT | host | Hostname or IP |
uuid | Host UUID | |
state | Disconnect State | |
PEER_NOT_FOUND | peer | Hostname or IP |
uuid | Host UUID |
Unknown Events
Event Type | Attribute | Description |
---|---|---|
UNKNOWN_PEER | peer | Hostname or IP |
Brick Events
Event Type | Attribute | Description |
---|---|---|
BRICK_START_FAILED | peer | Hostname or IP |
volume | Volume Name | |
brick | Brick | |
BRICK_STOP_FAILED | peer | Hostname or IP |
volume | Volume Name | |
brick | Brick | |
BRICK_DISCONNECTED | peer | Hostname or IP |
volume | Volume Name | |
brick | Brick | |
BRICK_CONNECTED | peer | Hostname or IP |
volume | Volume Name | |
brick | Brick |
Bricks Events
Event Type | Attribute | Description |
---|---|---|
BRICKS_START_FAILED | volume | Volume Name |
Brickpath Events
Event Type | Attribute | Description |
---|---|---|
BRICKPATH_RESOLVE_FAILED | peer | Hostname or IP |
volume | Volume Name | |
brick | Brick |
Notify Events
Event Type | Attribute | Description |
---|---|---|
NOTIFY_UNKNOWN_OP | op | Operation Name |
Quorum Events
Event Type | Attribute | Description |
---|---|---|
QUORUM_LOST | volume | Volume Name |
QUORUM_REGAINED | volume | Volume Name |
Rebalance Events
Event Type | Attribute | Description |
---|---|---|
REBALANCE_START_FAILED | volume | Volume Name |
REBALANCE_STATUS_UPDATE_FAILED | volume | Volume Name |
Import Events
Event Type | Attribute | Description |
---|---|---|
IMPORT_QUOTA_CONF_FAILED | volume | Volume Name |
IMPORT_VOLUME_FAILED | volume | Volume Name |
IMPORT_BRICK_FAILED | peer | Hostname or IP |
brick | Brick details |
Compare Events
Event Type | Attribute | Description |
---|---|---|
COMPARE_FRIEND_VOLUME_FAILED | volume | Volume Name |
Ec Events
Event Type | Attribute | Description |
---|---|---|
EC_MIN_BRICKS_NOT_UP | subvol | Subvolume |
EC_MIN_BRICKS_UP | subvol | Subvolume |
Georep Events
Event Type | Attribute | Description |
---|---|---|
GEOREP_FAULTY | primary_node | Hostname or IP of Primary Volume |
brick_path | Brick Path | |
secondary_host | Secondary Hostname or IP | |
primary_volume | Primary Volume Name | |
current_secondary_host | Current Secondary Host to which Geo-rep worker was trying to connect to | |
secondary_volume | Secondary Volume Name |
Quota Events
Event Type | Attribute | Description |
---|---|---|
QUOTA_CROSSED_SOFT_LIMIT | usage | Usage |
volume | Volume Name | |
path | Path |
Bitrot Events
Event Type | Attribute | Description |
---|---|---|
BITROT_BAD_FILE | gfid | GFID of File |
path | Path if Available | |
brick | Brick details |
Client Events
Event Type | Attribute | Description |
---|---|---|
CLIENT_CONNECT | client_identifier | Client Identifier |
client_uid | Client UID | |
server_identifier | Server Identifier | |
brick_path | Path of Brick | |
CLIENT_AUTH_REJECT | client_identifier | Client Identifier |
client_uid | Client UID | |
server_identifier | Server Identifier | |
brick_path | Path of Brick | |
CLIENT_DISCONNECT | client_identifier | Client Identifier |
client_uid | Client UID | |
server_identifier | Server Identifier | |
brick_path | Path of Brick |
Posix Events
Event Type | Attribute | Description |
---|---|---|
POSIX_SAME_GFID | gfid | GFID of File |
path | Path of File | |
newpath | New Path of File | |
brick | Brick details | |
POSIX_ALREADY_PART_OF_VOLUME | volume-id | Volume ID |
brick | Brick details | |
POSIX_BRICK_NOT_IN_VOLUME | brick | Brick details |
POSIX_BRICK_VERIFICATION_FAILED | brick | Brick details |
POSIX_ACL_NOT_SUPPORTED | brick | Brick details |
POSIX_HEALTH_CHECK_FAILED | path | Path |
brick | Brick details | |
op | Error Number | |
error | Error |
Afr Events
Event Type | Attribute | Description |
---|---|---|
AFR_QUORUM_MET | subvol | Sub Volume Name |
AFR_QUORUM_FAIL | subvol | Sub Volume Name |
AFR_SUBVOL_UP | subvol | Sub Volume Name |
AFR_SUBVOLS_DOWN | subvol | Sub Volume Name |
AFR_SPLIT_BRAIN | subvol | Sub Volume Name |
Tier Events
Event Type | Attribute | Description |
---|---|---|
TIER_ATTACH | vol | Volume Name |
TIER_ATTACH_FORCE | vol | Volume Name |
TIER_DETACH_START | vol | Volume Name |
TIER_DETACH_STOP | vol | Volume Name |
TIER_DETACH_COMMIT | vol | Volume Name |
TIER_DETACH_FORCE | vol | Volume Name |
TIER_PAUSE | vol | Volume Name |
TIER_RESUME | vol | Volume Name |
TIER_WATERMARK_HI | vol | Volume Name |
TIER_WATERMARK_DROPPED_TO_MID | vol | Volume Name |
TIER_WATERMARK_RAISED_TO_MID | vol | Volume Name |
TIER_WATERMARK_DROPPED_TO_LOW | vol | Volume Name |
Volume Events
Event Type | Attribute | Description |
---|---|---|
VOLUME_ADD_BRICK | volume | Volume Name |
bricks | Bricks details separated by Space | |
VOLUME_REMOVE_BRICK_START | volume | Volume Name |
bricks | Bricks details separated by Space | |
VOLUME_REMOVE_BRICK_COMMIT | volume | Volume Name |
bricks | Bricks details separated by Space | |
VOLUME_REMOVE_BRICK_STOP | volume | Volume Name |
bricks | Bricks details separated by Space | |
VOLUME_REMOVE_BRICK_FORCE | volume | Volume Name |
bricks | Bricks details separated by Space | |
VOLUME_REBALANCE_START | volume | Volume Name |
VOLUME_REBALANCE_STOP | volume | Volume Name |
VOLUME_REBALANCE_FAILED | volume | Volume Name |
VOLUME_REBALANCE_COMPLETE | volume | Volume Name |