Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to properly detect if storage is full? #11

Open
cannium opened this issue Sep 10, 2019 · 5 comments
Open

How to properly detect if storage is full? #11

cannium opened this issue Sep 10, 2019 · 5 comments

Comments

@cannium
Copy link
Contributor

cannium commented Sep 10, 2019

As I can see, there're several limitations:

  • journal area is full
  • data area is full
  • data area is not full, but cannot allocate new objects larger than "certain" size
  • file system is full, which is avoided by pre-allocation

Upper layer application needs this information to determine if a volume is still writable, maybe this could be reported by Usage.

@thesues
Copy link
Owner

thesues commented Oct 10, 2019

I updated the Usage function in storage.go.

the storage.Usage function now should be thread-safe. you can call it anytime you want.

the output is like this:


{
  "journalcapacity": 10737664,
  "datacapacity": 1063003648,
  "filecounts": 272,
  "datafreebytes": 652605440,
  "journalusagebytes": 7920
}

@cannium
Copy link
Contributor Author

cannium commented Oct 11, 2019

Thread-safe is nice to have, since it's already handled in upper layer: https://github.com/journeymidnight/seaweedfs/blob/yig2/weed/storage/volume_read_write.go#L92

And what about case 3: data area is not full, but cannot allocate new objects larger than "certain" size?

@thesues
Copy link
Owner

thesues commented Oct 11, 2019

You are brilliant to notice this corner case. Since the the data is aligned to block size,
I think certain size = min(lump.LUMP_MAX_SIZE, datacapacity - datafreebytes -2)

@cannium
Copy link
Contributor Author

cannium commented Oct 12, 2019

I think free space could be fragmented.

@thesues
Copy link
Owner

thesues commented Oct 13, 2019

Done in commit#806dfbba
each allocation or release will update maxSegmentSize to size of freelist's max portion

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants