some QA in dev-session 5

[httaotao]

hello, when I saw dev-session, I had some questions.

1. What is the function of nlookup in inode_t. I know lookup operation,but I don't understand this.
2. in dev-session 5 ppt ,there was a setence
   if the number of inodes in LRU is huge and kernel doesn't call forgets quick enough ...
   what is forget call ? Is it a system call ?
3. there was also a setence
   when refcount of inode is 0 but the nlookup count is non-zero , what happen ? I have understood this for a long time, but I don't quite understand it.
   hope for some answers , thanks

[pranithk]

hello, when I saw dev-session, I had some questions.

1. What is the function of nlookup in inode_t.

nlookup maintains the number of times kernel looked up the inode. It can either happen through LOOKUP fop or through READDIRP of the inode's parent directory. When fuse-kernel thinks it doesn't need the inode anymore, it will call either FUSE_FORGET/FUSE_BATCH_FORGET on the inodes.

2. in dev-session 5 ppt ,there was a setence
   if the number of inodes in LRU is huge and kernel doesn't call forgets quick enough ...
   what is forget call ? Is it a system call ?

You can check FUSE_FORGET/FUSE_BATCH_FORGET fuse_handler_t operations defined in fuse-bridge.c, these are part of interface between kernel fuse and glusterfs.

3. there was also a setence
   when refcount of inode is 0 but the nlookup count is non-zero , what happen ? I have understood this for a long time, but I don't quite understand it.
   hope for some answers , thanks

They will be put in LRU list of the inode table. These inodes will be evicted from the table when

• fuse-kernel calls forget on it
• Glusterfs' LRU limit is reached and gluster can force kernel to forget the inode.

[httaotao]

hello, when I saw dev-session, I had some questions.

1. What is the function of nlookup in inode_t.

nlookup maintains the number of times kernel looked up the inode. It can either happen through LOOKUP fop or through READDIRP of the inode's parent directory. When fuse-kernel thinks it doesn't need the inode anymore, it will call either FUSE_FORGET/FUSE_BATCH_FORGET on the inodes.

1. in dev-session 5 ppt ,there was a setence
   if the number of inodes in LRU is huge and kernel doesn't call forgets quick enough ...
   what is forget call ? Is it a system call ?

You can check FUSE_FORGET/FUSE_BATCH_FORGET fuse_handler_t operations defined in fuse-bridge.c, these are part of interface between kernel fuse and glusterfs.

1. there was also a setence
   when refcount of inode is 0 but the nlookup count is non-zero , what happen ? I have understood this for a long time, but I don't quite understand it.
   hope for some answers , thanks

They will be put in LRU list of the inode table. These inodes will be evicted from the table when

• fuse-kernel calls forget on it
• Glusterfs' LRU limit is reached and gluster can force kernel to forget the inode.

thanks for answer, it meas that FUSE_FORGET/FUSE_BATCH_FORGET like the unref operation in glusterfs fuse , is it right ? thanks

[pranithk]

From kernel fuse perspective, you can think about it that way.

[httaotao]

From kernel fuse perspective, you can think about it that way.

I get it , thanks.

[httaotao]

From kernel fuse perspective, you can think about it that way.

Sorry, I don't quite understand the difference between invalidation and forget. In my opinion, forget means that the kernel no longer needs the inode, while invalidation means that the inode is invalid. then can we use prune to destroy the invalid inode? or invalidation means that move the invalid inode to the inactive list and save the LRU size . thanks

[pranithk]

From kernel fuse perspective, you can think about it that way.

Sorry, I don't quite understand the difference between invalidation and forget. In my opinion, forget means that the kernel no longer needs the inode, while invalidation means that the inode is invalid. then can we use prune to destroy the invalid inode? or invalidation means that move the invalid inode to the inactive list and save the lrc size . thanks

@amarts Why is forcing forgetting of the inode LRU called invalidation instead of forget? Like @httaotao mentions, cache is not invalid at that point right?

[amarts]

Sorry, I don't quite understand the difference between invalidation and forget. In my opinion, forget means that the kernel no longer needs the inode, while invalidation means that the inode is invalid. then can we use prune to destroy the invalid inode? or invalidation means that move the invalid inode to the inactive list and save the lrc size . thanks

This is not so straight forward in fuse layer.

'Invalidation' of inode in fuse (or client) is done to reduce the memory pressure. Ie, in this scenario, we want to forget the valid inodes, just that they are not accessed for some time now. The inodes which are getting invalidated are 'valid' w.r.to kernel. Hence, we first need to kernel from inside the process to send a forget. We can't delete/purge the inode till we get 'forget()' in fuse, as it may result in crash.

[httaotao]

Sorry, I don't quite understand the difference between invalidation and forget. In my opinion, forget means that the kernel no longer needs the inode, while invalidation means that the inode is invalid. then can we use prune to destroy the invalid inode? or invalidation means that move the invalid inode to the inactive list and save the lrc size . thanks

This is not so straight forward in fuse layer.

'Invalidation' of inode in fuse (or client) is done to reduce the memory pressure. Ie, in this scenario, we want to forget the valid inodes, just that they are not accessed for some time now. The inodes which are getting invalidated are 'valid' w.r.to kernel. Hence, we first need to kernel from inside the process to send a forget. We can't delete/purge the inode till we get 'forget()' in fuse, as it may result in crash.

thanks for answer, there are some note in inode.c I found in gfs9.2 code.

Details as per Xavi:

 I think we should have 3 lists: active, lru and invalidate.

We'll need 3 things: refs, nlookups and invalidate_sent flag. Any change of
refs, invalidate_sent flag and moving from one list to another must be done
atomically.

With this information, these are the states that cause a transition:

    refs nlookups inv_sent    op
      1      0        0      unref  -> refs = 0, active--->destroy
      1      1        0      unref  -> refs = 0, active--->lru
      1      1        0     forget  -> nlookups = 0, active--->active
     *0      1        0     forget  -> nlookups = 0, lru--->destroy
     *0      1        1     forget  -> nlookups = 0, invalidate--->destroy
      0      1        0       ref   -> refs = 1, lru--->active
      0      1        1       ref   -> refs = 1, inv_sent = 0, invalidate--->active
      0      1        0    overflow -> refs = 1, inv_sent = 1, lru--->invalidate
      1      1        1      unref  -> refs = 0, invalidate--->invalidate
      1      1        1     forget  -> nlookups = 0, inv_sent = 0, invalidate--->active

(*) technically these combinations cannot happen because a forget sent by the
kernel first calls ref() and then unref(). However it's equivalent.

overflow means that lru list has grown beyond the limit and the inode needs to
be invalidated. All other combinations do not cause a change in state or are not
possible.

So in conclusion, in my opinion

1. in this node ,invalidate_sent flag meas that some xlator call inode_invalidate and then the inv_sent flag is 1 , right ?
2. other xlators or fops call inode_invalidate and fuse kernel calls forge, right ?
3. valid inode meas that we also can write or read, but it can reduce the memory pressure.
4. what does inode_invalidate work in inode.c ? there are no STACK_WIND or any call fops ,but maybe I don't know .

thanks.

[pranithk]

Sorry, I don't quite understand the difference between invalidation and forget. In my opinion, forget means that the kernel no longer needs the inode, while invalidation means that the inode is invalid. then can we use prune to destroy the invalid inode? or invalidation means that move the invalid inode to the inactive list and save the lrc size . thanks

This is not so straight forward in fuse layer.

'Invalidation' of inode in fuse (or client) is done to reduce the memory pressure. Ie, in this scenario, we want to forget the valid inodes, just that they are not accessed for some time now. The inodes which are getting invalidated are 'valid' w.r.to kernel. Hence, we first need to kernel from inside the process to send a forget. We can't delete/purge the inode till we get 'forget()' in fuse, as it may result in crash.

Should we call it force_forget instead of invalidate to avoid confusion? Glusterfs already has invalidate when perf xlators decide to invalidate an inode when they detect cache is not valid anymore.

[amarts]

Good to call it force_forget 👍