Notebook for Learning Rate Scheduler by ParagEkbote · Pull Request #1074 · skorch-dev/skorch

ParagEkbote · 2024-11-28T17:37:40Z

As described in #275, I've created a notebook that shows the visualization of the learning rate schedulers using the custom scheduler callback method in skorch.

Please let me know if any improvements are needed and I will make the necessary changes.

Fixes #275

cc: @BenjaminBossan

review-notebook-app · 2024-11-28T17:37:45Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

BenjaminBossan

Thanks a lot for the PR. I haven't done an in-depth review yet but the first impression is already quite good.

Before proceeding, I have a few points, could you please take a look?

Right now, the notebook starts with pip install .... We typically have a check if the notebook runs in a colab environment and only run the installs there (also, with a link to open the notebook in colab). E.g. check out this notebook. Could you please copy the approach?
Right now, you apply the StandardScaler directly on the data. How about creating an sklearn Pipeline with the StandardScaler as the first step and the neural net as the 2nd? That would be a bit cleaner IMO.
You can remove the if __name__ == "__main__": part.
Regarding the visualization, we have a simulate method in LRScheduler. Maybe that can be used instead of having to implement LRHistoryCallback.
Could you please add an entry to the notebook/README.md and to the CHANGES.md?

Modify LR notebook.

ParagEkbote · 2024-12-02T19:22:30Z

I have modified the notebook, and markdown files as per the suggested changes. Here's a summary:

Completed.
Completed.
Completed.
Completed.
I have added placeholder links in the notebooks. When this PR gets merged, I will submit a follow-up PR since the notebook needs to be present in the repo before I can add the links to the markdown files.

Could you please review the changes?

@BenjaminBossan

BenjaminBossan

Thanks a lot for the updates. I have added some comments, please take a look.

ParagEkbote · 2024-12-05T15:40:08Z

Could you please review the updated changes?

@BenjaminBossan

ParagEkbote · 2024-12-10T13:57:07Z

Could you please review the updated changes?

@BenjaminBossan

BenjaminBossan

Sorry for the delay. I reviewed the PR again and added a few comments. One change is blocked by a new PR, so I'd say let's wait for that PR first. Please take a look.

…duler

add new training method for rate scheduler.

ParagEkbote · 2024-12-20T17:42:23Z

I have addressed the mentioned comments.

Could you please review the updated changes?

cc: @BenjaminBossan

ParagEkbote · 2024-12-27T08:34:41Z

Could you please review the updated changes?

cc: @BenjaminBossan

BenjaminBossan

Thanks so much for the updates. I went through the notebook again and found only a few small things I'd like to be adjusted. Please check my comments.

subheading text.

ParagEkbote · 2024-12-28T14:38:31Z

I have addressed the mentioned comments.

Could you please review the updated changes?

cc: @BenjaminBossan

BenjaminBossan

Thanks for making the final changes, the notebook looks great, well done.

ParagEkbote · 2024-12-30T15:53:10Z

Thank you for assigning me the issue and being a patient guide.

Could you please see that #1025 gets closed to declutter?

cc: @BenjaminBossan

BenjaminBossan · 2024-12-30T16:01:53Z

Thanks for the reminder!

Please welcome skorch 1.1.0 - a smaller release with a few fixes, a new notebook showcasing learning rate schedulers and mainly support for scikit-learn 1.6.0. Full list of changes: ### Added - Added a [notebook](https://github.com/skorch-dev/skorch/blob/master/notebooks/Learning_Rate_Scheduler.ipynb) that shows how to use Learning Rate Scheduler in skorch.(#1074) ### Changed - All neural net classes now inherit from sklearn's [`BaseEstimator`](https://scikit-learn.org/stable/modules/generated/sklearn.base.BaseEstimator.html). This is to support compatibility with sklearn 1.6.0 and above. Classification models additionally inherit from [`ClassifierMixin`](https://scikit-learn.org/stable/modules/generated/sklearn.base.ClassifierMixin.html) and regressors from [`RegressorMixin`](https://scikit-learn.org/stable/modules/generated/sklearn.base.RegressorMixin.html). (#1078) - When using the `ReduceLROnPlateau` learning rate scheduler, we now record the learning rate in the net history (`net.history[:, 'event_lr']` by default). It is now also possible to to step per batch, not only by epoch (#1075) - The learning rate scheduler `.simulate()` method now supports adding step args which is useful when simulation policies such as `ReduceLROnPlateau` which expect metrics to base their schedule on. (#1077) - Removed deprecated `skorch.callbacks.scoring.cache_net_infer` (#1088) ### Fixed - Fix an issue with using `NeuralNetBinaryClassifier` with `torch.compile` (#1058)

Add Notebook for Learning Rate Scheduler

817b4ac

ParagEkbote changed the title ~~Add Notebook for Learning Rate Scheduler~~ Nov 28, 2024

BenjaminBossan requested changes Dec 2, 2024

View reviewed changes

ParagEkbote added 4 commits December 2, 2024 17:36

Update Changes.md and Readme.md

fc09cac

Modify LR notebook.

Update notebook.

0b312d8

Update LR Scheduler Notebook.

04a98f2

Update markdown headings of notebook.

d6ee05c

ParagEkbote requested a review from BenjaminBossan December 2, 2024 19:22

BenjaminBossan requested changes Dec 3, 2024

View reviewed changes

Comment thread notebooks/Learning_Rate_Scheduler.ipynb Outdated

Comment thread CHANGES.md Outdated

Comment thread notebooks/Learning_Rate_Scheduler.ipynb Outdated

Comment thread notebooks/Learning_Rate_Scheduler.ipynb Outdated

Comment thread notebooks/Learning_Rate_Scheduler.ipynb Outdated

ParagEkbote added 4 commits December 5, 2024 10:26

Update Notebook Subheadings and Release ReadMe.

6f89a95

Update skorch and plotting code.

fd47ba6

Organize notebook and plot simulated results.

33325d5

Add comment

0c7fa25

ParagEkbote requested a review from BenjaminBossan December 5, 2024 15:40

BenjaminBossan requested changes Dec 11, 2024

View reviewed changes

Comment thread notebooks/Learning_Rate_Scheduler.ipynb Outdated

Comment thread notebooks/Learning_Rate_Scheduler.ipynb Outdated

Comment thread notebooks/Learning_Rate_Scheduler.ipynb Outdated

Comment thread notebooks/README.md Outdated

ParagEkbote added 4 commits December 12, 2024 15:44

Update missing link.

49ea76b

Merge branch 'skorch-dev:master' into notebook-for-learning-rate-sche…

b5e2800

…duler

Revert data processing to Standard Scaler and

381bf12

add new training method for rate scheduler.

Update pytorch tensors to numpy arrays.

ff126a8

ParagEkbote requested a review from BenjaminBossan December 20, 2024 17:43

Cite the contributor.

229c4b2

BenjaminBossan requested changes Dec 28, 2024

View reviewed changes

Comment thread notebooks/Learning_Rate_Scheduler.ipynb Outdated

Comment thread notebooks/Learning_Rate_Scheduler.ipynb

Comment thread notebooks/Learning_Rate_Scheduler.ipynb Outdated

Comment thread notebooks/Learning_Rate_Scheduler.ipynb

ParagEkbote added 2 commits December 28, 2024 14:17

Update Broken Links in Colab and GitHub, Update

6eae1f0

subheading text.

Add verbose argument.

dca83e1

ParagEkbote requested a review from BenjaminBossan December 28, 2024 14:38

BenjaminBossan approved these changes Dec 30, 2024

View reviewed changes

BenjaminBossan merged commit 79ad021 into skorch-dev:master Dec 30, 2024

ParagEkbote deleted the notebook-for-learning-rate-scheduler branch December 30, 2024 15:53

BenjaminBossan mentioned this pull request Dec 30, 2024

Fixes #275 #1025

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Notebook for Learning Rate Scheduler#1074

Notebook for Learning Rate Scheduler#1074
BenjaminBossan merged 16 commits intoskorch-dev:masterfrom
ParagEkbote:notebook-for-learning-rate-scheduler

ParagEkbote commented Nov 28, 2024 •

edited

Loading

review-notebook-app Bot commented Nov 28, 2024

BenjaminBossan left a comment

ParagEkbote commented Dec 2, 2024 •

edited

Loading

BenjaminBossan left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ParagEkbote commented Dec 5, 2024 •

edited

Loading

ParagEkbote commented Dec 10, 2024

BenjaminBossan left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ParagEkbote commented Dec 20, 2024

ParagEkbote commented Dec 27, 2024

BenjaminBossan left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ParagEkbote commented Dec 28, 2024

BenjaminBossan left a comment

ParagEkbote commented Dec 30, 2024

BenjaminBossan commented Dec 30, 2024

Labels

2 participants

Conversation

ParagEkbote commented Nov 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

review-notebook-app Bot commented Nov 28, 2024

BenjaminBossan left a comment

Choose a reason for hiding this comment

ParagEkbote commented Dec 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

BenjaminBossan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ParagEkbote commented Dec 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ParagEkbote commented Dec 10, 2024

BenjaminBossan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ParagEkbote commented Dec 20, 2024

ParagEkbote commented Dec 27, 2024

BenjaminBossan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ParagEkbote commented Dec 28, 2024

BenjaminBossan left a comment

Choose a reason for hiding this comment

ParagEkbote commented Dec 30, 2024

BenjaminBossan commented Dec 30, 2024

Labels

2 participants

ParagEkbote commented Nov 28, 2024 •

edited

Loading

ParagEkbote commented Dec 2, 2024 •

edited

Loading

ParagEkbote commented Dec 5, 2024 •

edited

Loading