Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError when using strategy="random" and optimization_order="sequential" without validation set when using Trainer #259

Open
GeoKafkias opened this issue Nov 10, 2024 · 0 comments
Assignees
Labels
bug Something isn't working, either in /adalflow, /tutorials, or /use cases...

Comments

@GeoKafkias
Copy link

GeoKafkias commented Nov 10, 2024

Bug description

A TypeError: object of type 'NoneType' has no len() occurs when using the trainer with the following configuration:

  • strategy="random"
  • optimization_order="sequential"
  • Training and testing sets provided
  • No validation set provided

The error originates from adalflow/optim/trainer/adal.py line 372 in the pred_step function, where it attempts to get the length of the batch which is None in this case

What version are you seeing the problem on?

The version of the package "adalfow" is 0.2.3, and it has been installed using pip.

How to reproduce the bug

  • Configure the trainer with strategy="random" and optimization_order="sequential".
  • Provide training and testing sets to the trainer.
  • Do not provide a validation set.
  • Run
trainer.fit(
  train_dataset=data["train"],
  val_dataset=data["val"],
  debug=False,
  resume_from_ckpt=resume_from_ckpt,
)

Error messages and logs

completed_samples: 25, len: 25
Optimizer step: 0.6 > 0.56
None
Training Step: 3:  17%|██▊              | 2/12 [00:33<02:47, 16.79s/it]
Epoch:   0%|                                     | 0/1 [00:33<?, ?it/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 87, in train
  File "/home/geo/miniconda3/envs/paper_2_v2/lib/python3.11/site-packages/adalflow/optim/trainer/trainer.py", line 470, in fit
    trainer_results = self._fit_text_grad_random(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/geo/miniconda3/envs/paper_2_v2/lib/python3.11/site-packages/adalflow/optim/trainer/trainer.py", line 1452, in _fit_text_grad_random
    test_output = self.adaltask.validation_step(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/geo/miniconda3/envs/paper_2_v2/lib/python3.11/site-packages/adalflow/optim/trainer/adal.py", line 473, in validation_step
    completed_y_preds, completed_samples, index_to_score = self.pred_step(
                                                           ^^^^^^^^^^^^^^^
  File "/home/geo/miniconda3/envs/paper_2_v2/lib/python3.11/site-packages/adalflow/optim/trainer/adal.py", line 372, in pred_step
    y_preds = [None] * len(batch)
                       ^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

Environment

Current environment
  • OS: Linux
  • Python version: 3.11.5

More info

This bug seems to be related to how the trainer handles the absence of a validation/test set when using the "random" strategy and "sequential" optimization order. It might be necessary to add a check for the existence of the validation set or handle the None value for batch appropriately in the pred_step function.

@GeoKafkias GeoKafkias added the bug Something isn't working, either in /adalflow, /tutorials, or /use cases... label Nov 10, 2024
@liyin2015 liyin2015 mentioned this issue Nov 18, 2024
9 tasks
@liyin2015 liyin2015 self-assigned this Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working, either in /adalflow, /tutorials, or /use cases...
Projects
None yet
Development

No branches or pull requests

2 participants