When does juicefs delete the file on s3?

[201341]

I created 100m file, then delete it. But juicefs didn't delete it on s3. So I want to know when does juicefs delete file on s3? By the way, when the file is small, such as 10byte, jucefs deletes it right away.

[davies]

JuiceFS delete the objects in background under limited concurrency, it will retry the unfinished or failed deletion later.

Also, you can use juicefs gc to cleanup any leak or delayed deletion.

[201341]

Hi, I have know the reason, It's a timezone bug. minio uses UTC, but the juicefs uses time.Now(), which return the current local time.

[davies]

Cool, can you create an issue or send a PR?

[Xuanwo]

I created an issue about this problem: #1002

However, I don't think it's timezone-related problem: both time.Now() and time.After() are already taken timezone into consideration. So the different timezone between juciefs and minio should not affect the gc.

For reference:

juicefs/cmd/gc.go

Lines 220 to 231 in ccb1330

	if obj == nil {
	break // failed listing
	}
	if obj.IsDir() {
	continue
	}
	if obj.Mtime().After(maxMtime) || obj.Mtime().Unix() == 0 {
	logger.Debugf("ignore new block: %s %s", obj.Key(), obj.Mtime())
	bar.Increment()
	skipped.add(obj.Size())
	continue
	}

[Xuanwo]

BTW, here is a hardcoded logic to skip all objects that created in 1-hour:

juicefs/cmd/gc.go

Line 196 in ccb1330

maxMtime := time.Now().Add(time.Hour * -1)

Maybe this is the root cause?

[Xuanwo]

Already fixed in #728