Search Ctrl K
Home / Tools
THE FEUTEX TOOLKIT

Gradient Accumulation Effective Batch

Calculate effective global batch and optimizer steps from microbatches and data parallel workers.

AI Infrastructure workspace● Runs in your browser
01 Your input
02 Your result
View or copy raw result data
FormulaEffective global batch = micro*acc*workers; Full optimizer steps = Math.floor(examples/(micro*acc*workers)); Remaining examples = examples%(micro*acc*workers)

Made for real tasks

  • Calculate effective global batch and optimizer steps from microbatches and data parallel workers.
Explore step-by-step workflows →

Worked example

Inputs

Examples per microbatch per worker
4 examples
Accumulation microbatches
8 microbatches
Data parallel workers
8 workers
Training examples
1000000 examples

Expected results

Effective global batch
256 examples
Full optimizer steps
3906 steps
Remaining examples
64 examples

Load the example in the workspace to reproduce these results. Display rounding does not change the underlying calculation.

How to use this tool

  1. Enter examples per microbatch per worker, accumulation microbatches, data parallel workers, training examples in the displayed units.
  2. Calculate the scenario, then compare it with measurements before changing a system.

Try this example

{
  "micro": 4,
  "acc": 8,
  "workers": 8,
  "examples": 1000000
}

Accuracy and scope

Equal workers with no dropped examples before final incomplete batch; framework handling of final accumulation varies.

Is my data uploaded?

This tool runs locally in your browser. Input and output are not sent to a processing server. Copy or download your result before leaving the page.