Search Ctrl K
Home / Tools
THE FEUTEX TOOLKIT

Dictionary Encoding Size

Estimate a fixed-width dictionary index and dictionary payload for repeated strings.

Developer workspace● Runs in your browser
01 Your input
02 Your result
View or copy raw result data
FormulaMinimum fixed index width = Math.ceil(Math.log2(distinct)); Dictionary plus packed indices = (distinct*length+Math.ceil(rows*Math.ceil(Math.log2(distinct))/8))/1e6; Unencoded string payload = rows*length/1e6

Made for real tasks

  • Estimate a fixed-width dictionary index and dictionary payload for repeated strings.
Explore step-by-step workflows →

Worked example

Inputs

String values
1000000 values
Distinct strings
1000 strings
Mean encoded string length
20 bytes

Expected results

Minimum fixed index width
10 bits
Dictionary plus packed indices
1.27 MB
Unencoded string payload
20 MB

Load the example in the workspace to reproduce these results. Display rounding does not change the underlying calculation.

How to use this tool

  1. Enter string values, distinct strings, mean encoded string length in the displayed units.
  2. Calculate the scenario, then compare it with measurements before changing a system.

Try this example

{
  "rows": 1000000,
  "distinct": 1000,
  "length": 20
}

Accuracy and scope

Ideal bit-packed indices; dictionary offsets, nulls and metadata excluded.

Is my data uploaded?

This tool runs locally in your browser. Input and output are not sent to a processing server. Copy or download your result before leaving the page.