Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option for suspend before starting. Add support multiple riemann … #23

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,13 @@ tags: ['flapper', 'notify']


# If omitted, defaults to system's hostname
# Can be list like this
# host:
# - "host1.example.com"
# - "host2.example.com"
# or this:
# host: ["host1.example.com", "host2.example.com"]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'host' option here is for setting the host field of the event (ie, source), not the destination Riemann server.

# In this case message will be sent to all hosts.
host: "myhost.example.com"

# Set arbitrary attributes, optional
Expand Down
38 changes: 25 additions & 13 deletions bin/sumd
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
# Add cwd-relative lib to path
import os
import sys
sys.path.append("lib")

dirname = os.path.dirname(os.path.dirname(__file__))
sys.path.append(os.path.join(dirname, 'lib'))

# Misc standard library imports
import time
Expand Down Expand Up @@ -51,6 +53,7 @@ parser.add_option("--log-dir", dest="log_directory", default="/var/log", help="D
parser.add_option("--config-dir", dest="config_directory", default="/etc/sumd", help="Root path where tasks.d and tags.d can be found")
parser.add_option("--run-dir", dest="run_directory", default="/var/run", help="Directory for where pidfiles should be placed")
parser.add_option("--foreground", dest="foreground", action='store_true', default=False, help="Don't daemonize.")
parser.add_option("--suspend", dest="suspend", default=0, help="Suspend time before task starting.")
parser.add_option("--debug", dest="debug", action='store_true', default=False, help="Increase logger verbosity")
parser.add_option("--client-key-file", dest="client_key_file", help="Path to client SSL key file")
parser.add_option("--client-cert-file", dest="client_cert_file", help="Path to client SSL cert file")
Expand Down Expand Up @@ -128,18 +131,22 @@ def load_configs(schedule):
schedule.add(task)

def riemann_client():
if options.client_key_file != None and options.client_cert_file != None:
riemann = bernhard.SSLClient(
host=options.riemann_host,
port=options.riemann_port,
keyfile=options.client_key_file,
certfile=options.client_cert_file,
ca_certs=options.ca_certs_file)
else:
riemann = bernhard.Client(
host=options.riemann_host,
port=options.riemann_port,
transport=bernhard.UDPTransport)
riemann = []
if type(options.riemann_host) not in (list, tuple):
options.riemann_host = [options.riemann_host]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting over 'options.riemann_host' feels a bit weird here, because it comes from the argparser. Does it make sense to assign to a new variable?

for h in options.riemann_host:
if options.client_key_file != None and options.client_cert_file != None:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This option is generally host-specific, and would require the Riemann servers to be configured very similarly (from a client ssl perspective) to work, because this is still a single option in the configuration.

Perhaps I could help suggest some improvements here, but I don't think I fully understand the use case yet. Can you expand on your architecture and goal here?

riemann.append(bernhard.SSLClient(
host=h,
port=options.riemann_port,
keyfile=options.client_key_file,
certfile=options.client_cert_file,
ca_certs=options.ca_certs_file))
else:
riemann.append(bernhard.Client(
host=h,
port=options.riemann_port,
transport=bernhard.UDPTransport))

return riemann

Expand All @@ -163,6 +170,11 @@ def main():
if not options.foreground:
log.info("Creating pidfile with PID: %s" % (pidfile(create=True)))

if options.suspend:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you share some context about the problem this is trying to solve? I don't necessarily object to this change, but I don't yet understand the use case.

For example, in the default configuration of sumd, and our Riemann config, the TTLs are set such that a delay here could cause you to fall 'outside' that TTLs and the event to expire. I also believe (but would have to doublecheck) that events are scheduled for now+interval on startup, so that does provide some natural delay already.

log.info("Suspend tasks for {}s".format(options.suspend))
time.sleep(float(options.suspend))
log.info("Continue to start.")

# Create scheduler
schedule = scheduler.TaskSchedule()

Expand Down
11 changes: 6 additions & 5 deletions lib/sender.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,12 @@ def send(self, events):
log.debug("Sending %s events..." % (len(events)))
while len(events) > 0:
event = events.pop(0)
try:
log.debug("Event: %s" % (event.dict()))
self.riemann.send(event.dict())
except socket.error:
log.error("Unable to send event '%s' to %s:%s" % (event.service, self.riemann.host, self.riemann.port))
log.debug("Event: %s" % (event.dict()))
for r in self.riemann:
try:
r.send(event.dict())
except socket.error:
log.error("Unable to send event '%s' to %s:%s" % (event.service, r.host, r.port))
else:
log.warning("Send called with no events to send.")

Expand Down