How to reproduce the results on Tab.2 Prompt learning methods

[mmmeya]

Dear authors,

I followed your given command "python ./scripts/train_test_ood.py", but it cannot reproduce the results.

Compared with the paper, I found some differences between the code and the paper:

1. code "open_set_method = 'MSP' open_score = 'OE'". but in paper "open_set_method = 'OE'" = open_score = 'MSP'"
2. in the paper, you said that there are two stages for training. what is the stage 3. what's the difference between stage 2 and 3.
3. what's the number of epochs for stage 2 and 3. In the code, you set 100 for stage 3. But in the paper, it's 10 for stage 2.

Could you give us an updated command for reproducing the result?

I look forward to your reply. Thanks.

[arbiterleo]

I have met the same problem as him.
Besides, I also disocver that the code require more then 48G Vram to excute, but in the paper it says the code can be run with only one V100(32G).
I wonder does any one meet the same problem with me.
Any update on this will be good, thanks!

[myhakureimu]

May I ask if you met the following issue when running the script train_test_ood.py?

However, POMP and POMP_k are not defined in the code.

[myhakureimu]

Dear authors,

I followed your given command "python ./scripts/train_test_ood.py", but it cannot reproduce the results.

Compared with the paper, I found some differences between the code and the paper:

1. code "open_set_method = 'MSP' open_score = 'OE'". but in paper "open_set_method = 'OE'" = open_score = 'MSP'"
2. in the paper, you said that there are two stages for training. what is the stage 3. what's the difference between stage 2 and 3.
3. what's the number of epochs for stage 2 and 3. In the code, you set 100 for stage 3. But in the paper, it's 10 for stage 2.

Could you give us an updated command for reproducing the result?

I look forward to your reply. Thanks.

May I ask whether you met the issue of "KeyError: 'CSC'" when running their code with "python ./scripts/train_test_ood.py"?

[BRONYA-818]

I have met the same problem as him. Besides, I also disocver that the code require more then 48G Vram to excute, but in the paper it says the code can be run with only one V100(32G). I wonder does any one meet the same problem with me. Any update on this will be good, thanks!

In my practice, it seems that 60GB Vram is required for training. I hope that after the reproduction is completed, the effect shown in the paper can be achieved and I will continue to update under this issue.

[slcheng97]

Did you successfully reproduce the experimental results? @BRONYA-818