Add support for AdapterPlus

[julian-fong]

This PR aims to add support for AdapterPlus

Github: https://github.com/visinf/adapter_plus
Paper: https://arxiv.org/pdf/2406.06820

Integration of AdapterPlus into the adapters library will involve adding new parameters/options to the BnConfig

Checklist of things that are added/to be added

1. New type of scaling called channel, in which we add learnable parameters for the channel/input_size dimension
2. New type of init_weights called houlsby, where the projection matrices $W_{down}$ and $W_{up}$ will be initialized with zero-centered Gaussian with a standard deviation of $10^{-2}$ truncated at 2 standard deviations, and zeros for bias
3. Support for drop_path, also known as stochastic depth ONLY applicable for vision based tasks using residual networks - located under a new file called /methods/vision.py

[calpt]

Did a first review pass, looks pretty solid overall! thanks for working on this!

Could you additionally add the new config to relevant locations in the docs. e.g. methods.md and classes/adapter_config?

[calpt]

imports are added twice in this file, once here and once in the type checking block below

[julian-fong]

@calpt It looks like there is a missing torchvision import error happening in the CI. Is the CI configured to install the package? I don't see it inside as a core dependency in setup.py but not exactly sure if that is the correct place to look.

[calpt]

@calpt It looks like there is a missing torchvision import error happening in the CI. Is the CI configured to install the package? I don't see it inside as a core dependency in setup.py but not exactly sure if that is the correct place to look.

you're right, it's not currently installed in CI. we could add it as optional dependency in setup.py, similar to what HF does here: https://github.com/huggingface/transformers/blob/405b56269812056d9593869e22b7b264d806cb1e/setup.py#L312 and then install that additional dep package here

adapters/.github/workflows/tests_torch.yml

Line 66 in bcace97

pip install .[sklearn,testing,sentencepiece]

[julian-fong]

@calpt It looks like there is a missing torchvision import error happening in the CI. Is the CI configured to install the package? I don't see it inside as a core dependency in setup.py but not exactly sure if that is the correct place to look.

you're right, it's not currently installed in CI. we could add it as optional dependency in setup.py, similar to what HF does here: https://github.com/huggingface/transformers/blob/405b56269812056d9593869e22b7b264d806cb1e/setup.py#L312 and then install that additional dep package here

adapters/.github/workflows/tests_torch.yml

Line 66 in bcace97

pip install .[sklearn,testing,sentencepiece]

Would you like that in a separate pr? I know other open-source libraries prefer another pr to differentiate scopes of pull requests, so just wondering what your preference is

[julian-fong]

@calpt It looks like there is a missing torchvision import error happening in the CI. Is the CI configured to install the package? I don't see it inside as a core dependency in setup.py but not exactly sure if that is the correct place to look.

you're right, it's not currently installed in CI. we could add it as optional dependency in setup.py, similar to what HF does here: https://github.com/huggingface/transformers/blob/405b56269812056d9593869e22b7b264d806cb1e/setup.py#L312 and then install that additional dep package here

adapters/.github/workflows/tests_torch.yml

Line 66 in bcace97

pip install .[sklearn,testing,sentencepiece]

Please correct me if I'm wrong, but wouldn't torchvision need to be included as a core dependency since (and maybe there is an alternative that avoids this) we're importing the torchvision stochactic_depth function in methods/modeling.py? Users will be missing the library for that import whenever the file is being used if they install adapters straight up.

Thats sort of why I originally wrote the native implementation in the library so that we don't need to include any new core dependencies for timm or torchvision.

[calpt]

@calpt It looks like there is a missing torchvision import error happening in the CI. Is the CI configured to install the package? I don't see it inside as a core dependency in setup.py but not exactly sure if that is the correct place to look.

you're right, it's not currently installed in CI. we could add it as optional dependency in setup.py, similar to what HF does here: https://github.com/huggingface/transformers/blob/405b56269812056d9593869e22b7b264d806cb1e/setup.py#L312 and then install that additional dep package here

adapters/.github/workflows/tests_torch.yml

Line 66 in bcace97

pip install .[sklearn,testing,sentencepiece]

Would you like that in a separate pr? I know other open-source libraries prefer another pr to differentiate scopes of pull requests, so just wondering what your preference is

fine to include here since this is directly related to the new functionality implemented in this pr

[calpt]

@calpt It looks like there is a missing torchvision import error happening in the CI. Is the CI configured to install the package? I don't see it inside as a core dependency in setup.py but not exactly sure if that is the correct place to look.

you're right, it's not currently installed in CI. we could add it as optional dependency in setup.py, similar to what HF does here: https://github.com/huggingface/transformers/blob/405b56269812056d9593869e22b7b264d806cb1e/setup.py#L312 and then install that additional dep package here

adapters/.github/workflows/tests_torch.yml

Line 66 in bcace97

pip install .[sklearn,testing,sentencepiece]

Please correct me if I'm wrong, but wouldn't torchvision need to be included as a core dependency since (and maybe there is an alternative that avoids this) we're importing the torchvision stochactic_depth function in methods/modeling.py? Users will be missing the library for that import whenever the file is being used if they install adapters straight up.

Thats sort of why I originally wrote the native implementation in the library so that we don't need to include any new core dependencies for timm or torchvision.

that's a good point. I think what we can do is to make the import of torchvision conditional on the package being available. this is pretty standard practice across Transformers and they have helper methods for checking all sorts of packages, including torchvision: https://github.com/huggingface/transformers/blob/33868a057c02f0368ba63bd1edb746be38fe3d90/src/transformers/utils/import_utils.py#L361

If torchvision is not installed & stochastic_depth is used, we should thrown an exception, otherwise we don't care whether it's installed. This keeps it optional for all non-vision use cases.