WIP: [python-package] support Python 3.13

[jameslamb]

Add support for Python 3.13, now that it's available: https://docs.python.org/3/whatsnew/3.13.html

This also proposes a new standard process for Python version upgrades in this project. Proposing that when a new version is generally available, we do the following:

• bump every Python version in all CI up by 1 major version (e.g., all 3.10 -> 3.11)
• add any end-of-life versions to the test-old-versions CI job (as I've done here with Python 3.8)
  • based on https://devguide.python.org/versions/

This makes the updates easy to make and review, and reduces the risk of issues like #5969 and #6680 leading to maintenance work.

Notes for Reviewers

Just opening this to test. It's not ready for review... I expect more changes will be necessary, or at least that we'll need to wait for dependencies to update.

[jameslamb]

We'll need to wait for the rest of LightGBM's dependencies to get Python 3.13 packages on conda-forge.

LibMambaUnsatisfiableError: Encountered problems while solving:
  - package mypy-1.11.1-py310h5b4e0ec_0 requires python >=3.10,<3.11.0a0, but none of the providers can be installed

Could not solve for environment specs
The following packages are incompatible
├─ mypy >=1.11.1  is installable with the potential options
│  ├─ mypy [1.11.1|1.11.2|1.12.0] would require
│  │  └─ python >=3.10,<3.11.0a0 , which can be installed;
│  ├─ mypy [1.11.1|1.11.2|1.12.0] would require
│  │  └─ python >=3.11,<3.12.0a0 , which can be installed;
│  ├─ mypy [1.11.1|1.11.2|1.12.0] would require
│  │  └─ python_abi 3.12.* *_cp312, which requires
│  │     └─ python 3.12.* *_cpython, which can be installed;
│  ├─ mypy [1.11.1|1.11.2] would require
│  │  └─ python >=3.8,<3.9.0a0 , which can be installed;
│  ├─ mypy [1.11.1|1.11.2] would require
│  │  └─ pypy3.9 >=7.3.15 , which requires
│  │     └─ python 3.9.* *_73_pypy, which can be installed;
│  ├─ mypy [1.11.1|1.11.2|1.12.0] would require
│  │  └─ python >=3.9,<3.10.0a0 , which can be installed;
│  └─ mypy 1.12.0 would require
│     └─ python_abi 3.13.* *_cp313 with the potential options
│        ├─ python_abi 3.13 would require
│        │  └─ python 3.13.* *_cpython, which can be installed;
│        └─ python_abi 3.13 would require
│           └─ python 3.13.* *_cp313, which can be installed;
├─ pyarrow-core >=17.0  is installable with the potential options
│  ├─ pyarrow-core 17.0.0 would require
│  │  └─ __cuda >=11.8 , which is missing on the system;
│  ├─ pyarrow-core 17.0.0 would require
│  │  └─ python >=3.10,<3.11.0a0 , which can be installed;
│  ├─ pyarrow-core 17.0.0 would require
│  │  └─ python >=3.11,<3.12.0a0 , which can be installed;
│  ├─ pyarrow-core 17.0.0 would require
│  │  └─ python_abi 3.12.* *_cp312, which can be installed (as previously explained);
│  ├─ pyarrow-core 17.0.0 would require
│  │  └─ python >=3.8,<3.9.0a0 , which can be installed;
│  └─ pyarrow-core 17.0.0 would require
│     └─ python >=3.9,<3.10.0a0 , which can be installed;
└─ python 3.13 *_cp* is not installable because it conflicts with any installable versions previously reported.

(build link)

But hey, at least lightgbm itself already has Python 3.13 packages on conda-forge! (conda-forge/lightgbm-feedstock#63). That's thanks to lightgbm having such a small set of required dependencies... a good reminder that it's worthwhile to keep it that way 😁

We can check the state of LightGBM's dependencies here: https://conda-forge.org/status/migration/?name=python313