Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
| ## | |
| <pre> | |
| from accelerate import Accelerator | |
| -accelerator = Accelerator() | |
| +accelerator = Accelerator(log_with="wandb") | |
| train_dataloader, model, optimizer scheduler = accelerator.prepare( | |
| dataloader, model, optimizer, scheduler | |
| ) | |
| +accelerator.init_trackers() | |
| model.train() | |
| for batch in train_dataloader: | |
| optimizer.zero_grad() | |
| inputs, targets = batch | |
| outputs = model(inputs) | |
| loss = loss_function(outputs, targets) | |
| + accelerator.log({"loss":loss}) | |
| accelerator.backward(loss) | |
| optimizer.step() | |
| scheduler.step() | |
| +accelerator.end_training() | |
| </pre> | |
| ## | |
| To use experiment trackers with `accelerate`, simply pass the desired tracker to the `log_with` parameter | |
| when building the `Accelerator` object. Then initialize the tracker(s) by running `Accelerator.init_trackers()` | |
| passing in any configurations they may need. Afterwards call `Accelerator.log` to log a particular value to your tracker. | |
| At the end of training call `accelerator.end_training()` to call any finalization functions a tracking library | |
| may need automatically. | |
| ## | |
| To learn more checkout the related documentation: | |
| - <a href="https://huggingface.co/docs/accelerate/usage_guides/tracking" target="_blank">Using experiment trackers</a> | |
| - <a href="https://huggingface.co/docs/accelerate/package_reference/accelerator#accelerate.Accelerator.log" target="_blank">Accelerator API Reference</a> | |
| - <a href="https://huggingface.co/docs/accelerate/package_reference/tracking" target="_blank">Tracking API Reference</a> | |