update explanation of parity

api.py

@@ -74,7 +74,7 @@ COLUMN_DESCRIPTIONS = {
     "model": "The tokenizer model name with a link to its Hugging Face page.",
     "fertility": "Fertility Score (Tokens/Word): Average number of tokens generated per word. Lower is better — fewer tokens means more efficient representation.",
     "compression_bytes": "Compression (Bytes/Token): Average number of UTF-8 bytes per token. Higher is better — more bytes per token means more compact representation.",
-    "parity_en": "Parity vs English: Ratio of tokens generated for this language vs English on the same text. Lower is better (1.0 = equal efficiency). Not applicable for English or Programming Languages (English is the reference).",
+    "parity_en": "Parity vs English: Ratio of tokens for this language vs English on the same text. Closer to 1.0 is better. Above 1.0 means the tokenizer uses more tokens for this language than for English (less efficient, higher inference cost for non-English speakers). Below 1.0 means it is more efficient for this language than for English. Not applicable for English or Programming Languages (English is the reference).",
     "reversible_ratio": "Reversible (% docs): Percentage of documents of the benchmark perfectly reconstructed after tokenize→decode. Higher is better — 100% indicates lossless encoding of all examples.",
     "unk_ratio": "Unknown (% tokens): Percentage of tokens of the benchmark identified as unknown. Lower is better — indicates better vocabulary coverage.",
     "total_tokens": "Total Number of Tokens: Total tokens generated across all selected datasets of the benchmark.",

display.py

@@ -12,7 +12,7 @@ We evaluate tokenizers using the following core metrics:
 
 *   **🪺 Fertility Score (Tokens/Word)**: The average number of tokens generated per word. **Lower scores are better**, indicating the tokenizer uses fewer tokens to represent words. The number of words is calculated by the libaries [NLTK (Punkt/Treebank)](https://www.nltk.org/) or [spaCy](https://spacy.io/) with the respective models for each natural language, for programming languages we use their Lexer from the [pygments](https://pygments.org/) libary.
 *   **🗃 Compression (Bytes/Token)**: The average number of UTF-8 bytes represented by a single token. **Higher scores are better**, indicating more compact representation.
-*   **⚖️ Parity (vs. Reference Language)**: The ratio of tokens generated for a given language compared to a reference language (typically English) for the same text. **Lower scores are better**, a score of 1.0 means the tokenizer uses a similar number of tokens for the same content in both languages. A score greater than 1.0 indicates the tokenizer uses more tokens (is less efficient) for the target language than for the reference language. This metric helps assess cross-lingual fairness. (Introduced by Petrov et al., 2023, [https://arxiv.org/abs/2305.15425](https://arxiv.org/abs/2305.15425)). To calculate this metric we use the parallel corpus [FLORES+](https://huggingface.co/datasets/openlanguagedata/flores_plus).
+*   **⚖️ Parity (vs. Reference Language)**: The ratio of tokens generated for a given language compared to a reference language (typically English) for the same text. **Closer to 1.0 is better.** A score of 1.0 means the tokenizer is equally efficient for both languages. A score **above 1.0** means the tokenizer uses more tokens for the target language than for English — penalising non-English speakers with higher inference costs. A score **below 1.0** means the tokenizer is actually more efficient for that language than for English. In the multilingual view this is averaged across all selected languages, so a value close to 1.0 indicates a well-balanced tokenizer with no strong language bias. (Introduced by Petrov et al., 2023, [https://arxiv.org/abs/2305.15425](https://arxiv.org/abs/2305.15425)). To calculate this metric we use the parallel corpus [FLORES+](https://huggingface.co/datasets/openlanguagedata/flores_plus).
 *   **🔄 Reversible (% docs)**: The percentage of documents that can be perfectly reconstructed (losslessly) after being tokenized and then decoded back to text (including spaces, tabs and newlines). **Higher scores are better**, indicating the tokenizer preserves the original information more accurately.
 *   **❓ Unknown (% tokens)**: The percentage of tokens that are identified as unknown by the tokenizer. **Lower scores are better**, as they indicate the tokenizer has a better vocabulary coverage for the given text.