Search Ctrl K
Home / Tools
THE FEUTEX TOOLKIT

Attention Score Tensor Memory

Estimate explicit dense attention score storage before memory-efficient attention.

AI Infrastructure workspace● Runs in your browser
01 Your input
02 Your result
View or copy raw result data
FormulaOne layer score tensor = batch*heads*tokens*tokens*bytes/1073741824

Made for real tasks

  • Estimate explicit dense attention score storage before memory-efficient attention.
Explore step-by-step workflows →

Worked example

Inputs

Batch sequences
2 sequences
Query heads
32 heads
Sequence length
4096 tokens
Bytes per score
2 bytes

Expected results

One layer score tensor
2 GiB

Load the example in the workspace to reproduce these results. Display rounding does not change the underlying calculation.

How to use this tool

  1. Enter batch sequences, query heads, sequence length, bytes per score in the displayed units.
  2. Calculate the scenario, then compare it with measurements before changing a system.

Try this example

{
  "batch": 2,
  "heads": 32,
  "tokens": 4096,
  "bytes": 2
}

Accuracy and scope

Materialized square attention tensor only; FlashAttention and other fused implementations do not allocate this full matrix.

Is my data uploaded?

This tool runs locally in your browser. Input and output are not sent to a processing server. Copy or download your result before leaving the page.