-
Notifications
You must be signed in to change notification settings - Fork 409
Comparing changes
Open a pull request
base repository: skorch-dev/skorch
base: v1.1.0
head repository: skorch-dev/skorch
compare: v1.2.0
- 15 commits
- 40 files changed
- 6 contributors
Commits on Jan 10, 2025
-
Configuration menu - View commit details
-
Copy full SHA for 547d3ff - Browse repository at this point
Copy the full SHA 547d3ffView commit details
Commits on Jan 18, 2025
-
Fix docstring parsing issue in Python 3.13 (#1082)
Python 3.13 auto dedents docstrings, breaking the parsing. With this fix, parsing works the same as before.
Configuration menu - View commit details
-
Copy full SHA for bb1bac4 - Browse repository at this point
Copy the full SHA bb1bac4View commit details
Commits on Jan 27, 2025
-
FIX Unpickling without using torch.load (#1092)
Resolves #1090. PyTorch plans to make the switch to weights_only=True for torch.load. We already partly dealt with that in #1064 when it comes to save_params/load_params. However, we still had a gap. Namely, when using pickle directly, i.e. when going through __getstate__ and __setstate__, we are still using torch.load and torch.save without handling weights_only. This will cause trouble in the future when the default is switched. But it's also annoying right now, because users will get the FutureWarning about weights_only, even if they correctly pass torch_load_kwargs (see #1090). The reason why we use torch.save/torch.load for pickle is that those functions are basically _extended_ pickle functions that have the benefit of supporting the map_location argument to handle the device of torch tensors, which we don't have for pickle. The map_location argument is important, e.g. when saving a net that uses CUDA and loading it on a machine without CUDA, we would otherwise run into an error. However, with the move to weights_only=True, these torch.save/torch.load will become _reduced_ pickle functions, as they will only support a small subset of objects by default. Therefore, we wouldn't be able to rely on torch.save/torch.load for pickling the whole skorch object. In this PR, we thus move to using plain pickle for this. However, now we run into the issue of how to handle the map_location. The solution I ended up with is now to intercept torch's _load_from_bytes using a custom Unpickler, and to specifically use torch.load there. That way, we can pass the map_location and other torch_load_kwargs. The remaining unpickling process just works as normal. Yes, this is a private function, so we cannot be sure if it'll work indefinitely, If there is a better suggestion, I'm open to it. However, the function has existed for 7 years, so it's not very likely that it will change anytime soon: https://github.com/pytorch/pytorch/blame/0674ab7e33c3f627ca6781ce98468ec1dd4743a5/torch/storage.py#L525 A drawback of the solution is that we cannot just load old skorch nets that were saved with torch.save using pickle.load. This is because torch uses custom persistent_load functions. When trying to load with pickle, we thus get: _pickle.UnpicklingError: A load persistent id instruction was encountered, but no persistent_load function was specified. Therefore, I had to keep torch.load as a fallback to avoid backwards incompatibility. The bad news is that the initial problem persists, namely that even when passing torch_load_kwargs, users get the FutureWarning about weights_only. The good news is that users can just re-save their net with the new skorch version and from then on they won't see the warning again. Note that I didn't add a specific test for this problem of loading backwards nets from before the change, because test_pickle_load, which uses a checked in pickle file, already covers this. Other considered solutions: 1. Why not continue using torch.save/torch.load and just pass the torch_load_kwargs argument to it? This is unforunately not that easy. When switching to weights_only=True, torch will refuse to load any custom objects, e.g. class MyModule. There is a way to prevent that, namely via torch.serialization.add_safe_globals, but it is a ton of work to add all required objects there, as even builtin Python types are mostly not supported. 2. We cannot use with torch.device, as this is not honored during unpickling. 3. During __getstate__, we could recursively go through the state, pop all torch tensors, and replace them with, say, numpy arrays and additional meta data like the device, then use this info to restore those objects during __setstate__. Even though this looks like a cleaner solution, it is much more complex and therefore, I'd argue more error prone. 4. Don't do anything and just live with the warning: This will work -- until PyTorch switches the default. Therefore, we had to tackle this sooner or later. Notes While working on this, I thought that we could most likely remove the cuda_dependent_attributes_ (which contains the net.module_, net.optimizer_, etc.). Their purpose was to call torch.load on these attributes specifically, but with the new Unpickler, it should also work without this. However, I kept the attribute for now, mainly for these reasons: 1. I didn't want to change more than necessary, as these changes are delicate and I don't to break any existing skorch code or pickle files. 2. The attribute itself is public, so in theory, users may rely on its existence (not sure if in practice). We would thus have to keep most of the code related to this attribute. But LMK if you think we should deprecate and eventually remove this attribute.
Configuration menu - View commit details
-
Copy full SHA for be93b77 - Browse repository at this point
Copy the full SHA be93b77View commit details
Commits on Feb 4, 2025
-
Update: PyTorch v2.6.0 (#1093)
- Add torch 2.6.0 to CI - Remove torch 2.2.2 - Update torch install instructions, as they no longer provide conda packages - Add test for new default of weights_only - Update pickle file test artifact (explained in #1092) - Update some comments - Conditionally install triton 3.1 for torch < 2.6
Configuration menu - View commit details
-
Copy full SHA for 2337b5b - Browse repository at this point
Copy the full SHA 2337b5bView commit details
Commits on Mar 12, 2025
-
Configuration menu - View commit details
-
Copy full SHA for 393c48a - Browse repository at this point
Copy the full SHA 393c48aView commit details
Commits on Mar 21, 2025
-
Configuration menu - View commit details
-
Copy full SHA for 21ae449 - Browse repository at this point
Copy the full SHA 21ae449View commit details
Commits on Mar 24, 2025
-
Configuration menu - View commit details
-
Copy full SHA for 0b4885f - Browse repository at this point
Copy the full SHA 0b4885fView commit details
Commits on Mar 26, 2025
-
Add Contributing Guidelines (#1097)
* Add Contributing guidelines. * Update pr number in changes.md
Configuration menu - View commit details
-
Copy full SHA for fd86afb - Browse repository at this point
Copy the full SHA fd86afbView commit details
Commits on Apr 24, 2025
-
Update: PyTorch v2.7.0 (#1104)
* Add PyTorch 2.7.0 * Dropped PyTorch 2.3.1 * Install triton 3.2 for torch 2.6 * Remove numpy<2.0 install, see #1059
Configuration menu - View commit details
-
Copy full SHA for 052ab09 - Browse repository at this point
Copy the full SHA 052ab09View commit details
Commits on Jun 13, 2025
-
Configuration menu - View commit details
-
Copy full SHA for b40d905 - Browse repository at this point
Copy the full SHA b40d905View commit details
Commits on Jun 26, 2025
-
Add pyproject.toml to Improve CI/CD and Tooling (#1108)
As described in PEP 639, PEP 518 and PEP 621, the pyproject.toml file is a modern python tooling file which can manage multiple tasks of testing, linting, building of package, codecoverage ,etc. It has key-value configuration format. Building the source and wheels for the library This functionality does not have any breaking changes, I've tested it out with the python -m build command. Linting with pylint and flake8 Linting with flake8 has no breaking changes. pylint has some changes. version 2.6 has deprecated no-space-check option with no_self_use being made optional. pylint options locally-enabled, super-on-old-class were deprecated in version 2.14. I've updated it with the relevant rules to make the lint efficient. I've tested out both tools, they are working correctly. Using classifiers for PyPI When we publish packages on PyPI, classifers help categorize Python packages. These classifiers provide metadata about the package, such as its development status, intended audience, license, supported Python versions, and more. I've current added a standard set of packages for the library. Changes can be made to them as required. Testing and Code Coverage We can run the tests by running the pytest command with no breaking changes. Testing Code coverage has no breaking changes as well. You can check it with the pytest -v --cov=./skorch --cov-report=term-missing command. Deprecating Manifest.in, requirements.txt and requirements-dev.txt The functions of Manifest.in can be done by the .toml file as described in the setuptools docs here. The optional dependencies (prev. requirements-dev.txt) are listed in [project.optional-dependencies] and dependencies (prev requirements.txt) are listed in the project section. Lastly, I have updated the gh-actions file to reflect these changes as well.
Configuration menu - View commit details
-
Copy full SHA for d562403 - Browse repository at this point
Copy the full SHA d562403View commit details
Commits on Jul 22, 2025
-
FIX HF integration test workflow file (#1113)
Due to #1108, pip install -r requirements.txt is no longer working. Also, I took the liberty to bump the Python version used to 3.12.
Configuration menu - View commit details
-
Copy full SHA for 2d4c721 - Browse repository at this point
Copy the full SHA 2d4c721View commit details
Commits on Aug 7, 2025
-
TST FIX Issue with mocking Accelerator
The latest accelerate version has attributes on Accelerator that raise an error when being accessed. This is causing an issue when trying to Mock the Accelerator, resulting in a skorch test failing. This PR fixes this by monkeypatching those attributes.
Configuration menu - View commit details
-
Copy full SHA for dc4a113 - Browse repository at this point
Copy the full SHA dc4a113View commit details -
Update: PyTorch v2.8.0 (#1116)
- drop 2.4.1 - update 2.7.0 -> 2.7.1
Configuration menu - View commit details
-
Copy full SHA for 4378f31 - Browse repository at this point
Copy the full SHA 4378f31View commit details
Commits on Aug 8, 2025
-
# Version 1.2.0 This is a smaller release, most changes concern examples and development and thus don't affect users of skorch. ## Changed - Loading of skorch nets using pickle: When unpickling a skorch net, you may come across a PyTorch warning that goes: "FutureWarning: You are using torch.load with weights_only=False [...]"; to avoid this warning, pickle the net again and use the new pickle file (#1092) ## Added - Add Contributing Guidelines for skorch. (#1097) - Add an example of hyper-parameter optimization using [Optuna](https://optuna.org/) [here](https://github.com/skorch-dev/skorch/tree/master/examples/optuna) (#1098) - Add Example for Streaming Dataset(#1105) - Add pyproject.toml to Improve CI/CD and Tooling (#1108) Thanks @raphaelrubrice, @omahs, and @ParagEkbote for their contributions. **Full Changelog**: v1.1.0...v1.2.0 Release commit specific: * Bump verison to 1.2.0 * Update CHANGES.md * Remove workarounds that have been fixed in sklearn Only affects tests
Configuration menu - View commit details
-
Copy full SHA for db77adb - Browse repository at this point
Copy the full SHA db77adbView commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff v1.1.0...v1.2.0