Dynamic training of datasets

[kobox11]

Hi Team,

I'm not a real expert in machine learning, but I had an idea I wanted to share with you.

During training it is easy to overtrain/overfit/overbake a Lora/model.

Is it possible to calculate a maximum/average necessary training value and compare it during the training with the training status? If one of the training images files approximates to this value, then it flies out of training and maybe such a validation is also possible for the text encoder too?
So training will continued, but the material to train on will get less and less during training time.

Maybe this calculation can be done before the training starts. As some kind of detector for possible training potential in these combinations.
As example: the network seems not to have connections between humanlike cucumbers and walking, or standing. So there would be maybe more potential than fine-tuning against 100 more Emma Watson images.

Thank you for reading! I hope it is something useful and not just junk :-/