When juicefs uses CEPH (S3) to store data, errors are reported because the capacity is not increased enough

[1397436824]

What happened:
How to add the used capacity continuously, resulting in insufficient capacity and error reporting
被使用的容量不断怎加，导致容量不足报错
What you expected to happen:
No error is reported, and the used capacity is reduced in time
不报错，被使用的容量及时减少
How to reproduce it (as minimally and precisely as possible):
#exec this fio script:
dir=("/usr/local/redis/etc/juicefs")
rm -rf ${dir[0]}/*
fio --name=big-file-multi-write --directory=${dir[0]} --rw=write --refill_buffers --bs=4M --size=4G --numjobs=4 --end_fsync=1
rm -rf ${dir[0]}/*
fio --name=big-file-multi-write --directory=${dir[0]} --rw=write --refill_buffers --bs=4M --size=4G --numjobs=4 --end_fsync=1
rm -rf ${dir[0]}/*
fio --name=big-file-multi-write --directory=${dir[0]} --rw=write --refill_buffers --bs=4M --size=4G --numjobs=4 --end_fsync=1
rm -rf ${dir[0]}/*
fio --name=big-file-multi-write --directory=${dir[0]} --rw=write --refill_buffers --bs=4M --size=4G --numjobs=4 --end_fsync=1
rm -rf ${dir[0]}/*
fio --name=big-file-multi-write --directory=${dir[0]} --rw=write --refill_buffers --bs=4M --size=4G --numjobs=4 --end_fsync=1
rm -rf ${dir[0]}/*

Anything else we need to know?
The RM command in the middle will not increase the available capacity of CEPH, but will increase slowly after the last RM is executed. If CEPH is 3 copies, an error will be reported if the CEPH capacity is less than the minimum capacity of 4 * 4 * 4 * 3 = 192 + CEPH under normal conditions after 4 FIOs!
中间的rm命令不会导致ceph可用容量的增加，而是执行最后的rm后，才会慢慢增加，如果ceph为3副本，此时当执行4次FIO后，ceph容量如果小于4x4x4x3(=192) + ceph正常情况的最小容量时就会报错！

Environment:
juicefs + redis( metadata) + ceph radosgw (s3) (data)

• JuiceFS version (use juicefs --version) or Hadoop Java SDK version: 0.17.5

[SandyXSD]

The objects in Ceph are deleted asynchronously after file removed. It takes a while to actually delete all object of a 4G file.
With the command juicefs stats <mount_point> --verbosity 1, you can find del_c, which shows the number of recently completed deletions.