Best- and worst-predicted answer-side SAE features — issue #1482

Short character sequences (typically 2-4 characters) that form partial fragments or syllables within longer words, particularly in non-English languages including Romance languages, Asian languages, and other scripts, often appearing at morpheme or syllable boundaries.

Foreign names

Tokens that mark the start of list items, bullet points, enumerated steps, or structured elements in formatted text, including dashes, numbers, brackets, and keywords like 'def' that introduce organized content.

Email/code sections

Short tokens (typically 1-4 characters) that function as abbreviations, suffixes, prefixes, or technical notation within formal, academic, or programming contexts across multiple languages.

technical language snippets

Tokens representing health, healthcare, wellness, or medical-related concepts across multiple languages, typically appearing in contexts discussing healthcare systems, health monitoring, medical conditions, wellbeing, or health-related services.

health

Whitespace or formatting tokens immediately following list markers (bullets, dashes, numbers) and preceding the actual content text in structured lists or enumerated items across multiple languages.

>

Tokens that are abbreviations, acronyms, technical identifiers (like variable names or API terms), proper name fragments, or word stems in non-English languages, typically appearing in technical, programming, or multilingual contexts.

Kir

Word stems and roots that appear immediately before common suffixes or at morphological boundaries where suffixes attach, including both complete words that function as stems and partial words representing the root portion before derivational/inflectional endings.

the ending -ly/-less/-ful/-ess

Suffixes that form abstract nouns or technical terms, particularly noun-forming morphemes like -tion, -ment, -ization, -ness in formal or technical writing contexts across multiple languages.

ending in "ation"

Short character sequences (2-4 characters) that form parts of words in non-English languages, particularly capturing mid-word fragments in multilingual text including Romance languages, Germanic languages, Slavic languages, Asian languages, and Arabic.

names and companies

Non-English tokens (particularly in Chinese, Russian, Japanese, Vietnamese, Portuguese, Italian) that serve connective, explanatory, or functional grammatical roles, often appearing at transitional points in sentences or as parts of technical/formal vocabulary.

Pen

Subword units, morphemes, or short complete words that function as meaningful components within technical documentation, programming code, or multilingual text, including variable name suffixes, linguistic particles, domain-specific terminology fragments, and structural elements.

code

Tokens that are the word 'red' or its variants in multiple languages, or tokens that are part of words/phrases associated with red, pink, orange, or warm colors, including descriptions of objects, emotions, or visual scenes involving these hues.

0

Internal word fragments, syllables, or morpheme components that appear as meaningful subword segments within longer words across multiple languages, often representing common letter sequences, roots, or morphological parts (like 'str-' in English compound words or syllabic fragments in non-Latin scripts).

Str

Tokens that express or introduce commonality, shared properties, unifying concepts, or systematic connections between multiple elements, including words like 'common', '共同' (shared), and structural markers that signal explanatory synthesis.

thread

Morphemes and word fragments expressing concepts of 'higher', 'superior', 'elevated', 'increase', or vertical position (above/below) across multiple languages, particularly in Chinese, Russian, and European languages.

higher

Word endings, suffixes, or final portions of multi-token identifiers, particularly in programming variable/function names (snake_case, camelCase) and technical terminology across multiple languages.

code

Numeric digits that function as list item numbers, section indicators, or hierarchical labels in structured or enumerated content, typically appearing at the beginning of list items or subsections.

1

Short tokens (single characters, numbers, or brief words) that serve as intermediate or connective elements within longer sequences, appearing in technical, procedural, or structured contexts rather than at prominent sentence boundaries.

non-English words

Words and morphemes meaning 'hold', 'maintain', 'keep', 'grasp', 'contain', or related concepts of physical or abstract holding/maintaining across multiple languages.

hold

Tokens that are neutral or positive in connotation and are not associated with downward movement, decline, or negative directionality (the feature suppresses 'down' and related terms).

statistical relationships

Tokens or morphemes that express the concept of 'right', 'correct', 'proper', 'truth', or 'righteousness' across multiple languages (English, Spanish, Chinese, Russian, German, Indonesian), including directional uses of 'right', legal/moral rights, and correctness.

no description

Punctuation marks, symbols, and formatting delimiters (hyphens, parentheses, brackets, equals signs, colons, periods, commas, asterisks, backticks) that serve as syntactic or structural elements in code, mathematical notation, technical documentation, or structured text formats.

code and equations

Words or morphemes meaning 'true', 'real', 'genuine', 'truly', or expressing concepts of truthfulness, authenticity, and reality across multiple languages.

True

Short, common tokens that serve structural, grammatical, or formatting functions rather than carrying primary semantic content, including single characters, punctuation marks, syllables, brief function words, and programming syntax elements across multiple languages.

Code snippets and formatting

Greek or Latin-derived prefixes and affixes that indicate scale (mini-, micro-, macro-, multi-), opposition (anti-), position (mono-), or other technical modifiers (eco-, neo-, meta-, ultra-) in formal, technical, or scientific terminology across multiple languages.

multi

Punctuation marks and single characters that function as delimiters, separators, or structural boundaries between clauses, list items, or text segments across multiple languages.

commas

Words and morphemes meaning 'middle', 'medium', 'moderate', 'central', 'average', 'intermediate', or 'balance' across multiple languages, indicating concepts of middling position, moderate degree, or balanced state rather than extremes.

medium

Tokens appearing in multilingual text, code snippets, technical documentation, or at language/script boundaries, particularly where non-English characters or formatting syntax are present in the surrounding context.

random snippets of text

Tokens appearing at the end of sentences, paragraphs, or sections immediately before structural breaks, formatting transitions, or new sections begin, serving as textual boundary markers across multiple languages.

no description

Suffixes and word-endings that denote professional roles, specialists, experts, artistic/philosophical movements, or ideological positions, particularly '-ist', '-ists', '-ism', and their equivalents in other languages.

ist

Word stems, roots, or fragments that form or are part of descriptive words expressing qualities, emotions, states, characteristics, or properties, particularly those that can take adjectival or noun-forming suffixes.

describing textures

Non-English tokens or grammatical morphemes in multilingual text, including function words, verb conjugations, and common vocabulary items across Romance, Germanic, Slavic, and Asian languages.

legal disclaimers

Partial word fragments or syllables, particularly within proper nouns, technical terms, or foreign language words embedded in multilingual text contexts, where tokenization breaks words into non-standard subword units.

Nor

Grammatical function tokens, word fragments, and morphological continuations that serve structural rather than primary semantic roles, including articles, particles, suffixes, and mid-word segments across multiple languages.

the/a

Subword tokens or morphemes that form meaningful components of larger words across multiple languages, particularly including fragments related to legal/legislative terms, words for 'legend/legacy', body parts (legs), and significant word stems in Chinese, Romance languages, English, Russian, and Arabic.

Leg

Tokens that represent continuations, alternatives, or additional options in their context, including: parts of words being completed, translation equivalents in multilingual text, alternative phrasings, code syntax completions, or technical term extensions.

foreign words

Short token fragments (1-3 characters) that are subword pieces within longer words in non-English languages, proper nouns, or whitespace tokens in code, often representing incomplete morphological units within multilingual text.

code and articles

Content words and meaningful morphemes in non-English text, particularly Chinese characters carrying semantic meaning (nouns, verbs, technical terms), as well as substantive vocabulary in Spanish, Portuguese, Russian, and other languages.

technical/academic text

Tokens that are word-internal or directly attached to the preceding text without whitespace, including word continuations in various languages, code syntax elements, and non-space-separated segments in multilingual contexts.

code snippets

Words and morphemes meaning 'general', 'universal', 'common', 'comprehensive', or 'typical' across multiple languages (English, Chinese, Portuguese, Italian, Russian, Japanese, French, etc.).

General

Tokens that are parts of words containing negative prefixes (un-, non-, im-) or negation-related morphemes across multiple languages, typically appearing at the boundary where the negative prefix joins the root word.

Negative prefixes

Function words, grammatical morphemes, abstract concepts, and semantically light tokens including pronouns, modal verbs, conjunctions, suffixes, punctuation, digits, and word fragments rather than concrete content-bearing nouns or verbs.

3

Pronouns (personal, possessive, reflexive) across multiple languages that refer to specific entities mentioned in the context.

you

Tokens representing ordinal numbers, list indices, sequential identifiers, or discrete labels within structured contexts such as enumerated lists, code arrays, step-by-step instructions, or categorical references.

digits"

Words and morphemes meaning 'select', 'choose', or 'selection' across multiple languages (English, Chinese, Russian, Vietnamese, Korean, Turkish, Japanese, Spanish), including SQL SELECT commands and technical contexts involving selection operations.

select

Tokens that are word fragments, syllables, or short words appearing in multilingual text (especially Cyrillic, Chinese, Arabic transliterations) or technical/code contexts, often representing morphemes or syntax elements in non-English languages or programming code.

sorting numbers

Tokens that serve as suffixes, file extensions, technical abbreviations, closing delimiters, or components completing technical identifiers in structured text like code, mathematical expressions, filenames, and domain-specific terminology.

code

The token marks the numeral '1' or equivalent formatting at the beginning of the first item in a numbered list or enumerated sequence across various languages and document formats.

1

Tokens that are being referenced as themselves, used as identifiers or variables, repeated literally, or appearing in self-referential or technical/notational contexts where the token's form rather than meaning is emphasized.

code

Mid-sequence tokens that occur within words, identifiers, or code constructs across multiple programming languages and natural languages, excluding tokens at clear sentence or statement boundaries.

research papers