replace binlogIO with io.BinlogIO in datanode #14
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wayblink
force-pushed
the
binlogio
branch
5 times, most recently
from
f1977d9 to
b868dc1
Compare
Signed-off-by: yangxuan <[email protected]>
pr: milvus-io#29675 Signed-off-by: aoiasd <[email protected]>
…o#29858) See also milvus-io#29803 This PR: - Add trace span for `LoadIndex` & `LoadFieldData` in segment loader - Add `TraceCtx` parameter for `Index.Load` in segcore - Add span for ReadFiles & Engine Load for Memory/Disk Vector index --------- Signed-off-by: Congqi Xia <[email protected]>
this supports mmap for marisa trie index related milvus-io#21866 Signed-off-by: yah01 <[email protected]>
…9860) milvus-io#29759 Signed-off-by: luzhang <[email protected]> Co-authored-by: luzhang <[email protected]>
issue: milvus-io#29793 Use `DocSetCollector` instead of `TopDocsCollector`, which will avoid scoring and sorting. --------- Signed-off-by: longjiquan <[email protected]>
this protect the loading index from failure, and speed up the loading progress Signed-off-by: yah01 <[email protected]>
If segment has more than 128 log fils, drop segment will exceed etcd txn ops limit, which will failed the drop segment request This PR drop segment meta info with prefix, to avoid drop segment meta failed --------- Signed-off-by: Wei Liu <[email protected]>
issue: milvus-io#29709 #291712 to avoid concurrent recursive RLock and Lock cause deadlock, This PR remove the unnecessary lock in config manager --------- Signed-off-by: Wei Liu <[email protected]>
…9873) This pull request simplifies the integration test for cross-cluster routing by reusing `integration.MiniClusterSuite`, instead of defining custom Milvus clients, servers, and etcd client. issue: milvus-io#29874 Signed-off-by: bigsheeper <[email protected]>
…#29870) this leads to the EntitiesNum metric would be never reduced fix: milvus-io#29766 Signed-off-by: yah01 <[email protected]>
issue：milvus-io#22837 Add bfloat16 vector, add the index part of float16 vector. Signed-off-by: Writer-X <[email protected]>
milvus-io#29759 Signed-off-by: luzhang <[email protected]> Co-authored-by: luzhang <[email protected]>
This PR change some frequency log to rated level Signed-off-by: Wei Liu <[email protected]>
Signed-off-by: Sammy Huang <[email protected]>
Signed-off-by: Sammy Huang <[email protected]>
…9641) milvus-io#22882 /kind feature Signed-off-by: wayblink <[email protected]>
Signed-off-by: nico <[email protected]>
pr: milvus-io#29634 After verifying that the problem is fixed, it is necessary to run this case every night for regression. Signed-off-by: elstic <[email protected]>
Update Pytest image changes See changes: milvus-io@930735c Signed-off-by: sre-ci-robot [email protected] Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
issue: milvus-io#29846 Signed-off-by: jaime <[email protected]>
…e up (milvus-io#29890) milvus-io#29889 Signed-off-by: Sammy Huang <[email protected]>
Update Knowhere Commit Signed-off-by: sre-ci-robot [email protected] Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Add tracing span for search/query operation waiting tsafe duration Signed-off-by: Congqi Xia <[email protected]>
milvus-io#29912) related milvus-io#29843 Signed-off-by: yah01 <[email protected]>
…o#30500) See also milvus-io#27675 milvus-io#30469 For a sync task, the segment could be compacted during sync task. In previous implementation, this sync task will hold only the old segment id as KeyLock, in which case compaction on compacted to segment may run in parallel with delta sync of this sync task. This PR introduces sync target segment verification logic. It shall check target segment lock it's holding beforing actually syncing logic. If this check failed, sync task shall return`errTargetSegementNotMatch` error and make manager re-fetch the current target segment id. Signed-off-by: Congqi Xia <[email protected]>
Fixes: milvus-io#30517 Signed-off-by: yangxuan <[email protected]>
This PR mainly improve two items: 1. Target observer should refresh loading status during init time. An uninitialized loading status blocks search/query. Currently, the target observer refreshes every 10 seconds, i.e. we'd need to wait for 10s for no reason. That's also the reason why we constantly see false log "collection unloaded" upon mixcoord restarts. 2. Delete session when service is stopped. So that the new service doesn't need to wait for the previous session to expire (~10s). Item 1 is the major improvement of this PR, which should speed up init time by 10s. Item 2 is not a big concern in most cases as coordinators usually shut down after stop(). In those cases, coordinator restart triggers serverID change which further triggers an existing logic that deletes expired session. This PR only fixes rare cases where serverID doesn't change. integration test: `go test -tags dynamic -v -coverprofile=profile.out -covermode=atomic tests/integration/coordrecovery/coord_recovery_test.go -timeout=20m` Performance after the change: Average init time of coordinators: 10s Hardware: M2 Pro Test setup: 1000 collections with 1000 rows (dim=128) per collection. issue: milvus-io#29409 Signed-off-by: yiwangdr <[email protected]>
Fixes: milvus-io#30507 Signed-off-by: yangxuan <[email protected]>
/kind enhancement Signed-off-by: wayblink <[email protected]>
…ilvus-io#30504) issue: milvus-io#30358 and milvus-io#30491 Signed-off-by: cqy123456 <[email protected]>
Signed-off-by: Sammy Huang <[email protected]>
Data write through rawkv API may pollute tikv data. It should be disallowed. We will add this check to all repos that involves metadata access. In the longer term, we should have a metadata service that implements access control. relate: milvus-io#30029 Signed-off-by: yiwangdr <[email protected]>
issue: milvus-io#29507 Removing deprecated code from integration test lib Signed-off-by: yiwangdr <[email protected]>
…io#30555) See also milvus-io#28817 Signed-off-by: Congqi Xia <[email protected]>
relate: milvus-io#30404 --------- Signed-off-by: aoiasd <[email protected]>
) related: milvus-io#29883 Signed-off-by: MrPresent-Han <[email protected]>
Related to milvus-io#27675 Signed-off-by: Congqi Xia <[email protected]>
…lvus-io#30188) issue: milvus-io#30039 --------- Signed-off-by: sunby <[email protected]>
…vus-io#30549) issue: milvus-io#29880 Signed-off-by: nico <[email protected]>
…0432) milvus-io#30350 Signed-off-by: luzhang <[email protected]> Co-authored-by: luzhang <[email protected]>
feat: restful support new features 1. search with groupingField milvus-io#25324 2. hybrid search milvus-io#25639 Signed-off-by: PowderLi <[email protected]>
…io#30532) issue : milvus-io#30369 Signed-off-by: zhenshan.cao <[email protected]>
…ilvus-io#30479) issue: milvus-io#30295 Signed-off-by: jaime <[email protected]>
Signed-off-by: Alexandr Guzhva <[email protected]>
issue has been fixed then remove xfail for testcase Signed-off-by: zhuwenxing <[email protected]>
milvus-io#30523) See also milvus-io#27606 --------- Signed-off-by: Congqi Xia <[email protected]>
…0603) See also milvus-io#30571 When `compactionExecutor` stops one compaction task, the `stop` method will case `injectDone` called. However in `executeTask` when `compact` method returns error, it shall also invoke `injectDone` as well. That the reason `Unlock of unlocked RWMutex` panicking happened. This PR add sync.Once to make sure that `injectDone` is called only once. We did not remove any of the `injectDone` since removal any of those invocation may cause logic problem. --------- Signed-off-by: Congqi Xia <[email protected]>
milvus-io#28410 Signed-off-by: wayblink <[email protected]>
wayblink
force-pushed
the
binlogio
branch
4 times, most recently
from
d26adbc to
d0d2468
Compare
Signed-off-by: wayblink <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.