Search Ctrl K
Home / Tools
THE FEUTEX TOOLKIT

Adam Training State Memory

Account for parameter, gradient, master-weight and two optimizer-moment arrays.

AI Infrastructure workspace● Runs in your browser
01 Your input
02 Your result
View or copy raw result data
FormulaStatic training state = params*1e6*(weight+gradient+master+2*moment)/1073741824

Made for real tasks

  • Account for parameter, gradient, master-weight and two optimizer-moment arrays.
Explore step-by-step workflows →

Worked example

Inputs

Trainable parameters
100 millions
Weight bytes
2 bytes
Gradient bytes
2 bytes
Master-weight bytes or zero
4 bytes
Bytes per moment
4 bytes

Expected results

Static training state
1.490116119 GiB

Load the example in the workspace to reproduce these results. Display rounding does not change the underlying calculation.

How to use this tool

  1. Enter trainable parameters, weight bytes, gradient bytes, master-weight bytes or zero, bytes per moment in the displayed units.
  2. Calculate the scenario, then compare it with measurements before changing a system.

Try this example

{
  "params": 100,
  "weight": 2,
  "gradient": 2,
  "master": 4,
  "moment": 4
}

Accuracy and scope

Unsharded dense Adam accounting; excludes activations, temporary buffers, embeddings sparsity and allocator overhead.

Is my data uploaded?

This tool runs locally in your browser. Input and output are not sent to a processing server. Copy or download your result before leaving the page.