Merge remote-tracking branch 'upstream/main'

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,34 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ""
+labels: ""
+assignees: ""
+---
+
+## Describe the bug
+
+A clear and concise description of what the bug is.
+
+## Platform
+
+Please provide details about the environment you are using, including the following:
+
+- Interpreter version:
+- Library version:
+
+## Sample Code
+
+Please include a minimal sample of the code that will (if possible) reproduce the bug in isolation
+
+## Expected behavior
+
+A clear and concise description of what you expected to happen.
+
+## Observed behavior
+
+What you see happening (error messages, stack traces, etc...)
+
+## Additional context
+
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,23 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ""
+labels: ""
+assignees: ""
+---
+
+## Is your feature request related to a problem? Please describe.
+
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+## Describe the solution you'd like
+
+A clear and concise description of what you want to happen.
+
+## Describe alternatives you've considered
+
+A clear and concise description of any alternative solutions or features you've considered.
+
+## Additional context
+
+Add any other context about the feature request here.

.github/ISSUE_TEMPLATE/user_story.md

@@ -0,0 +1,23 @@
+---
+name: User story
+about: A user-oriented story describing a piece of work to do
+title: ""
+labels: ""
+assignees: ""
+---
+
+## Description
+
+As a <user type>, I want to <do something>, so that I can <accomplish something>
+
+## Discussion
+
+Provide detailed discussion here
+
+## Acceptance Criteria
+
+<!-- Remove any that don't apply -->
+
+- [ ] Unit tests cover new/changed code
+- [ ] Examples build against new/changed code
+- [ ] READMEs are updated

.github/pull_request_template.md

@@ -0,0 +1,19 @@
+<!-- Thank you for the contribution! -->
+
+### Description of the change
+
+<!-- Please summarize the changes -->
+
+### Related issue number
+
+<!-- For example: "Closes #1234" -->
+
+### How to verify the PR
+
+<!-- Please provide instruction or screenshots on how to verify the PR.-->
+
+### Was the PR tested
+
+<!-- Describe how PR was tested -->
+- [ ] I have added >=1 unit test(s) for every new method I have added.
+- [ ] I have ensured all unit tests pass

.github/workflows/format.yml

@@ -35,4 +35,6 @@ jobs:
           python -m pip install -r setup_requirements.txt
       - name: Check Formatting
         run: tox -e fmt
+      - name: Run pylint
+        run: tox -e lint
 

.github/workflows/test.yaml

@@ -0,0 +1,22 @@
+name: Test
+
+on:
+  push:
+    branches: [ "main" ]
+  pull_request:
+    branches: [ "main" ]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Set up Python 3.9
+        uses: actions/setup-python@v4
+        with:
+          python-version: 3.9
+      - name: Install dependencies
+        run: |
+          python -m pip install -r setup_requirements.txt
+      - name: Run unit tests
+        run: tox -e py

.gitignore

@@ -7,7 +7,6 @@ durations/*
 coverage*.xml
 dist
 htmlcov
-build
 test
 
 # IDEs

.pylintrc

@@ -443,7 +443,9 @@ disable=raw-checker-failed,
         attribute-defined-outside-init,
         abstract-method,
         pointless-statement,
-        wrong-import-order
+        wrong-import-order,
+        duplicate-code,
+        unbalanced-tuple-unpacking
 
 # Enable the message, report, category or checker with the given id(s). You can
 # either give multiple identifier separated by comma (,) or put this option

CODEOWNERS

@@ -0,0 +1,11 @@
+#####################################################
+#
+# List of approvers for fms-hf-tuning repository
+#
+#####################################################
+#
+# Learn about CODEOWNERS file format:
+#  https://help.github.com/en/articles/about-code-owners
+#
+
+*  @anhuong @Ssukriti @alex-jw-brooks

CONTRIBUTING.md

@@ -0,0 +1,141 @@
+# Contributing
+
+👍🎉 First off, thank you for taking the time to contribute! 🎉👍
+
+The following is a set of guidelines for contributing. These are just guidelines, not rules. Use your best judgment, and feel free to propose changes to this document in a pull request.
+
+## What Should I Know Before I Get Started?
+
+### Code of Conduct
+
+This project adheres to the [Contributor Covenant](./code-of-conduct.md). By participating, you are expected to uphold this code.
+
+Please report unacceptable behavior to one of the [Code Owners](./CODEOWNERS).
+
+### How Do I Start Contributing?
+
+The below workflow is designed to help you begin your first contribution journey. It will guide you through creating and picking up issues, working through them, having your work reviewed, and then merging.
+
+Help on open source projects is always welcome and there is always something that can be improved. For example, documentation (like the text you are reading now) can always use improvement, code can always be clarified, variables or functions can always be renamed or commented on, and there is always a need for more test coverage. If you see something that you think should be fixed, take ownership! Here is how you get started:
+
+## How Can I Contribute?
+TODO: Add link to ADR  and add template to this repository
+
+For any contributions that need design changes/API changes, first contribute an ADR. Reason for ADR: teams agree on the design, to avoid back and forth after writing code. An ADR gives context on the code being written.
+
+When contributing, it's useful to start by looking at [issues](https://github.com/foundation-model-stack/fms-hf-tuning/issues). After picking up an issue, writing code, or updating a document, make a pull request and your work will be reviewed and merged. If you're adding a new feature or find a bug, it's best to [write an issue](https://github.com/foundation-model-stack/fms-hf-tuning/issues/new) first to discuss it with maintainers. 
+
+To contribute to this repo, you'll use the Fork and Pull model common in many open source repositories. For details on this process, check out [The GitHub Workflow
+Guide](https://github.com/kubernetes/community/blob/master/contributors/guide/github-workflow.md)
+from Kubernetes.
+
+When your contribution is ready, you can create a pull request. Pull requests are often referred to as "PR". In general, we follow the standard [GitHub pull request](https://help.github.com/en/articles/about-pull-requests) process. Follow the template to provide details about your pull request to the maintainers. It's best to break your contribution into smaller PRs with incremental changes, and include a good description of the changes.
+
+Before sending pull requests, make sure your changes pass formatting, linting and unit tests.
+
+#### Code Review
+
+Once you've [created a pull request](#how-can-i-contribute), maintainers will review your code and may make suggestions to fix before merging. It will be easier for your pull request to receive reviews if you consider the criteria the reviewers follow while working. Remember to:
+
+- Run tests locally and ensure they pass
+- Follow the project coding conventions
+- Write detailed commit messages
+- Break large changes into a logical series of smaller patches, which are easy to understand individually and combine to solve a broader issue
+
+Maintainers will perform "squash and merge" actions on PRs in this repo, so it doesn't matter how many commits your PR has, as they will end up being a single commit after merging.
+
+### Reporting Bugs
+
+This section guides you through submitting a bug report. Following these guidelines helps maintainers and the community understand your report ✏️, reproduce the behavior 💻, and find related reports 🔎.
+
+#### How Do I Submit A (Good) Bug Report?
+
+Bugs are tracked as [GitHub issues using the Bug Report template]((https://github.com/foundation-model-stack/fms-hf-tuning/issues/new?template=bug_report.md). Create an issue on that and provide the information suggested in the bug report issue template. 
+
+### Suggesting Enhancements
+
+This section guides you through submitting an enhancement suggestion, including completely new features, tools, and minor improvements to existing functionality. Following these guidelines helps maintainers and the community understand your suggestion ✏️ and find related suggestions 🔎
+
+#### How Do I Submit A (Good) Enhancement Suggestion?
+
+Enhancement suggestions are tracked as [GitHub issues using the Feature Request template](https://github.com/foundation-model-stack/fms-hf-tuning/issues/new?template=feature_request.md). Create an issue and provide the information suggested in the feature requests or user story issue template.
+
+#### How Do I Submit A (Good) Improvement Item?
+
+Improvements to existing functionality are tracked as [GitHub issues using the User Story template](https://github.com/foundation-model-stack/fms-hf-tuning/issues/new?template=user_story.md). Create an issue and provide the information suggested in the feature requests or user story issue template.
+
+## Development
+
+### Set up your dev environment
+
+The following tools are required:
+
+- [git](https://git-scm.com)
+- [python](https://www.python.org) (v3.8+)
+- [pip](https://pypi.org/project/pip/) (v23.0+)
+
+Installation:
+``` 
+pip install -r requirements.txt
+pip install -U datasets
+pip install -e .
+```
+<details>
+<summary>Linting</summary>
+
+To lint your code:
+```shell
+tox -e lint
+```
+
+We use Pylint to checks your Python code for errors, coding standards, code convention and refactoring suggestions.
+
+Pylint emits [messages](https://pylint.pycqa.org/en/latest/user_guide/messages/index.html) that provides explanations of the failed checks.
+
+You should fix all message in the following order:
+1. Fix each message provided. Select a message [description](https://pylint.pycqa.org/en/latest/user_guide/messages/messages_overview.html#messages-overview) to fix a message.
+2. Disable a message (i.e: unbalanced-tuple-unpacking) caused by a particular line of code:
+    ```python
+    a, b = ... # pylint: disable=unbalanced-tuple-unpacking
+    ```
+    Please see [here](https://pylint.pycqa.org/en/latest/user_guide/messages/message_control.html#block-disables) for the progma syntax.
+
+3. Disable a checker globally. Please extend the `disable=` list in the [pylintrc](.pylintrc) file.
+    > Note: Disable checkers only if there is good reason.
+</details>
+
+<details>
+<summary>Formatting</summary>
+
+To format your code:
+```shell
+tox -e fmt
+```
+We use [black](https://github.com/psf/black) formatter to format the code.
+
+You could optionally install the git pre-commit hooks if you would like to format the code automatically for each commit:
+```
+brew install pre-commit
+pre-commit install
+```
+</details>
+
+<details>
+<summary>Unit tests</summary>
+
+To run unit tests:
+```shell
+tox -e py
+```
+Running unit tests ensures your contributions do not break exiting code.
+We use [pytest](https://docs.pytest.org/) framework to run unit tests. The framework is setup to run all run all test_*.py or *_test.py in the [tests](./tests) directory.
+
+> Optionally, run `make test` command to do formatting, linting, and testing at once.
+</details>
+
+## Your First Code Contribution
+
+Unsure where to begin contributing? You can start by looking through these issues:
+
+- Issues with the [`good first issue` label](https://github.com/foundation-model-stack/fms-hf-tuning/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) - these should only require a few lines of code and are good targets if you're just starting contributing.
+- Issues with the [`help wanted` label](https://github.com/foundation-model-stack/fms-hf-tuning/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) - these range from simple to more complex, but are generally things we want but can't get to in a short time frame.

Makefile

@@ -0,0 +1,14 @@
+# Run unit tests
+.PHONY: test
+test: fmt lint
+	tox -e py
+
+# Format python code
+.PHONY: fmt
+fmt:
+	tox -e fmt
+
+# Run pylint to check code
+..PHONY: lint
+lint:
+	tox -e lint