Add linting capability

[hickeyma]

Added the following linter:

• pylint

Note to reviewers:

• Added tox target which can kick off linting.
• Separated packages that require CUDA into cuda_requirements.txt as they are not required for linting and also are not required for non CUDA tuning.
• The linting is not automated yet as need to add CI/CD
• Any changes recommended by the linter are not committed until we automate as part of the build
• The linting can be run manually:
  • Installing tox and dependencies: pip install -r setup_requirements.txt
  • Running it: tox -e lint

[anhuong]

A few questions Martin

• Could you add how to run the linter and formatter in the README? Especially if this is something that is expected to be run by all contributors else PRs will fail (in near term future when automation added)
• If we separate torch_requirements, this should also be updated in the README since it will require users to run pip install -r torch_requirements.txt and pip install -r requirements.txt. Which wil affect any current automation that is installing this repo.

[hickeyma]

@anhuong Thanks for the review and feedback.

Could you add how to run the linter and formatter in the README? Especially if this is something that is expected to be run by all contributors else PRs will fail (in near term future when automation added)

I intend to do that in the contributors doc as part of the enablement of linting in CI/CD. Next PR essentially.

If we separate torch_requirements, this should also be updated in the README since it will require users to run pip install -r torch_requirements.txt and pip install -r requirements.txt. Which wil affect any current automation that is installing this repo.

Added to the README now.

[hickeyma]

@anhuong Updated and ready for review again.

[anhuong]

I tried uninstalling packages, wheel, and flash-attn and running tuning to make sure it can run without these deps.

When trying to run PT, I get error:

Traceback (most recent call last):
  File "fms-hf-tuning/tuning/sft_trainer.py", line 8, in <module>
    from peft.utils.other import fsdp_auto_wrap_policy
  File "fms-hf-tuning/venv/lib/python3.9/site-packages/peft/__init__.py", line 22, in <module>
    from .auto import (
  File "fms-hf-tuning/venv/lib/python3.9/site-packages/peft/auto.py", line 23, in <module>
    from transformers import (
  File "fms-hf-tuning/venv/lib/python3.9/site-packages/transformers/__init__.py", line 26, in <module>
    from . import dependency_versions_check
  File "fms-hf-tuning/venv/lib/python3.9/site-packages/transformers/dependency_versions_check.py", line 16, in <module>
    from .utils.versions import require_version, require_version_core
  File "fms-hf-tuning/venv/lib/python3.9/site-packages/transformers/utils/__init__.py", line 19, in <module>
    from packaging import version
ModuleNotFoundError: No module named 'packaging'

This is a package that is usually installed with flash-attn I believe, so if flash-attn is not installed this may need to be added to core requirements.

These are the other packages that rely on packaging:

$ pip show packaging
Name: packaging
Version: 23.2
Summary: Core utilities for Python packages
Home-page: 
Author: 
Author-email: Donald Stufft <[email protected]>
License: 
Requires: 
Required-by: accelerate, aim, datasets, huggingface-hub, peft, transformers

Pip installing packaging resolves the above error and PT runs successfully.

[anhuong]

Can verify that with set of optional deps uninstalled and with packaging installed, Lora tuning also runs successfully.

[hickeyma]

As discussed @anhuong, I have moved packaging to the core requirements file.

[anhuong]

LGTM, thanks Martin!

[tedhtchang]

We should enable fmt and lint checks for each PR #59