Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: Ensure socket is closed when error is raised while opening socket #328

Merged
merged 2 commits into from
Dec 5, 2024

Conversation

tgxworld
Copy link
Contributor

@tgxworld tgxworld commented Dec 4, 2024

/139110

@tgxworld tgxworld force-pushed the ensure_socket_close branch 2 times, most recently from 8141df9 to 6f79322 Compare December 5, 2024 01:33
Comment on lines 56 to 69
def test_close_socket_on_error
logs = StringIO.new
logger = Logger.new(logs)
logger.level = :error

client = PrometheusExporter::Client.new(logger: logger, port: 321, once: true)
client.send("put a message in the queue")

assert_includes(
logs.string,
"Prometheus Exporter, failed to send message Connection refused - connect(2) for \"localhost\" port 321",
)
end
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test does not directly test that the socket is closed but it does cause the internal close_socket! method to be called.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why aren't we explicitly checking for socket closure though?

Copy link
Contributor Author

@tgxworld tgxworld Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it is an internal implementation detail and there is basically no way for us to assert for that in the test since the reference to the socket is thrown away after it is closed.

@tgxworld tgxworld force-pushed the ensure_socket_close branch from 6f79322 to dbd8288 Compare December 5, 2024 01:39
@@ -234,8 +241,7 @@ def ensure_socket!

nil
rescue StandardError
@socket = nil
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, we were just assigning the @socket to nil which causes a TCPSocket to remain open.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need an explicit ensure close_socket! in worker_loop?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No the worker loop is meant to ensure that a socket is opened and reuse the socket across loops. Is there a reason you think we should close the socket each time we process a message?

Copy link

@nattsw nattsw Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not so much for each time we process a message, but I'm tracing the flow right now and I do see other places e.g. process_queue, where there's @socket = nil as well

logger.warn "Prometheus Exporter is dropping a message: #{e}"
@socket = nil

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch and I've made the change to call close_socket!. However, I don't think we can easily test for this and I think it is OK since the risk of dropping a message is low.

@@ -59,7 +59,8 @@ def initialize(
json_serializer: nil,
custom_labels: nil,
logger: Logger.new(STDERR),
log_level: Logger::WARN
log_level: Logger::WARN,
once: false
Copy link

@nattsw nattsw Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry can you elaborate here what is "once: false"?

Export it once only and kill after? Perhaps some verbosity in variable naming can help

Copy link
Contributor Author

@tgxworld tgxworld Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated to process_queue_once_and_stop which should be much clearer.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

@tgxworld tgxworld force-pushed the ensure_socket_close branch from dbd8288 to 4f308f2 Compare December 5, 2024 02:38
@tgxworld tgxworld force-pushed the ensure_socket_close branch from 4f308f2 to a5f8af5 Compare December 5, 2024 02:53
@tgxworld
Copy link
Contributor Author

tgxworld commented Dec 5, 2024

Thank you for reviewing @nattsw 👍

@tgxworld tgxworld merged commit 46e88af into main Dec 5, 2024
11 checks passed
@tgxworld tgxworld deleted the ensure_socket_close branch December 5, 2024 02:57
2.2.0 - 2024-12-05

- FIX: Ensure socket is closed when error is raised while opening socket
- Feature: Add Dalli::Client memcache metrics for web_collector
Copy link
Member

@nbianca nbianca Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick:

- FEATURE: Add Dalli::Client memcache metrics for web_collector

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants