feat: add full_drain_mode option to ensure complete data flush over network before shutdown #1395
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
messixukejia
requested changes
Mar 16, 2024
core/sender/Sender.cpp
Outdated
| @@ -2073,7 +2073,7 @@ SendResult Sender::SendToNetSync(sdk::Client* sendClient, | |||
| void Sender::SendToNetAsync(LoggroupTimeValue* dataPtr) { | |||
| auto& exactlyOnceCpt = dataPtr->mLogGroupContext.mExactlyOnceCheckpoint; | |||
|
|
|||
| if (IsFlush()) // write local file avoid binary update fail | |||
| if (!BOOL_FLAG(sidecar_mode) && Application::GetInstance()->IsExiting()) // write local file avoid binary update fail | |||
There was a problem hiding this comment.
如果网络不稳定，发送极慢，会有什么影响？影响最终退出时间？
There was a problem hiding this comment.
常规模式下，一旦开始退出流程，所有缓存队列的数据全部写到本地磁盘，不再通过网络发送，所以网络对退出时间无影响；
新模式下，是强制发完所有数据后退出，如果网络问题导致发不出去，就会一直等着，直到K8s强制kill。这个是迅飞提的要求，发不出去就死等，而不是超时自动退出。
yyuuttaaoo
reviewed
Mar 18, 2024
| if (BOOL_FLAG(sidecar_mode) && Application::GetInstance()->IsExiting() | ||
| && EventDispatcher::GetInstance()->IsAllFileRead()) { | ||
| break; | ||
| } |
There was a problem hiding this comment.
非sidecar_mode在mInteruptFlag+Application::GetInstance()->IsExiting()的时候也可以break出来？
There was a problem hiding this comment.
之前的设计里是input的循环是不会主动退出的，只会暂停
yyuuttaaoo
approved these changes
Mar 19, 2024
yyuuttaaoo
changed the title
messixukejia
approved these changes
Mar 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introducing the full_drain_mode configuration, which when enabled, triggers a comprehensive data flush across the network during the shutdown sequence. This mode is particularly beneficial for environments with sidecar and fat containers where data persistence is not guaranteed post-application exit. By ensuring all data is transmitted prior to shutdown, we prevent potential data loss that could occur with local dumps, making this feature a critical addition for systems that require high data integrity and resilience.