Skip to content

Commit

Permalink
Switch execution order of scrub_items() and prune_news()
Browse files Browse the repository at this point in the history
Rationale was that cache scrub would reduce number of items to iterate
through later. However, if expiration hasn't been updated in "long"
time eg. due to error or low traffic(HTTP Staus Code 304), then the
whole cache gets invalidated which leads to excess flood.

One solution is to increase cache expiration(in other words one day is
really low).
Another solution seems to be to switch order of execution, because
prune_news() updates time of expiration. Therefore, it should prevent
scrubbing of news which are still present in RSS feed.
And it's also possible there is a bug somewhere in the code.
  • Loading branch information
zstyblik committed Jun 9, 2024
1 parent 7823e29 commit eadf83f
Show file tree
Hide file tree
Showing 3 changed files with 3 additions and 3 deletions.
2 changes: 1 addition & 1 deletion phpbb2slack.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,8 @@ def main():
sys.exit(0)

source.extract_caching_headers(rsp.headers)
scrub_items(logger, cache)
prune_news(logger, cache, news, args.cache_expiration)
scrub_items(logger, cache)

slack_client = rss2slack.get_slack_web_client(
slack_token, args.slack_base_url, args.slack_timeout
Expand Down
2 changes: 1 addition & 1 deletion rss2irc.py
Original file line number Diff line number Diff line change
Expand Up @@ -174,8 +174,8 @@ def main():
sys.exit(0)

source.extract_caching_headers(rsp.headers)
scrub_items(logger, cache)
prune_news(logger, cache, news, args.cache_expiration)
scrub_items(logger, cache)

if not args.cache_init:
write_data(logger, news, args.output, args.handle, args.sleep)
Expand Down
2 changes: 1 addition & 1 deletion rss2slack.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,8 @@ def main():
sys.exit(0)

source.extract_caching_headers(rsp.headers)
rss2irc.scrub_items(logger, cache)
rss2irc.prune_news(logger, cache, news, args.cache_expiration)
rss2irc.scrub_items(logger, cache)

slack_client = get_slack_web_client(
slack_token,
Expand Down

0 comments on commit eadf83f

Please sign in to comment.