Problem syncing large s3 buckets in the background

[wasphin]

Hi, I am trying to sync a large bucket. Since the bucket is very large, it will take days to sync, I cannot open the terminal for so long, so I tried to sync in the background like bellow:

nohup ./sync.sh &> x.log &

content of sync.sh like:

juicefs sync s3:/from s3:/to

The problem is that, when not opening a tty, it cannot log any progress info.
And, after a night of syncing background, the to bucket doesn't have enough data as expected(as syncing in the foreground), and there is no way to know whether it's working or not.

Any tips or suggestions on this?

Thanks.

[karlkashofer]

apt install screen
screen
juicefs sync s3:/from s3:/to
ctrl-a+d
screen -r

[wasphin]

Thanks. It may be better to implement some solutions within juicefs.

It seems that using nohup to execute a script wrapper for juicefs in the background is not working. When directly executing juicefs, it begins syncing. I haven't delved into the details, but perhaps some inputs or outputs are getting blocked.

With two workers, the syncing tasks were completed in two days.