Disentangling lexical and grammatical information in word embeddings

Li Liu (li (point) liu (point) 2 <at> umontreal (point) ca)

OLST, Département de linguistique et de traduction

Le 17 septembre 2025 à 11 h 30

Salle C-9019, Pavillon Lionel-Groulx — En présentiel, avec diffusion simultanée sur Zoom


To enable finer-grained linguistic analysis, we propose a method for the separation of lexical and grammatical information within contextualized word embeddings. Using CamemBERT embeddings for French, we apply our method to 14,472 inflected word forms extracted from the French Lexical Network (LN-fr), covering 1,468 nouns, 202 adjectives, and 299 verbs inflected via 14 distinct grammatical feature values. Our iterative distillation process alternates two steps until convergence: (i) estimating lexical or grammatical vectors by averaging the embeddings of words that share the same lexeme or grammatical feature value, and (ii) isolating the complementary component of each word embedding by subtracting the estimated vector. To assess the quality of the decomposition, we measure whether the resulting lexical and grammatical vectors form more compact clusters within their respective groups and whether their sum better reconstructs the original word embeddings. All evaluations rely on L2 distance. The observed improvements in both clustering and reconstruction accuracy demonstrate the effectiveness of our approach.

La présentation sera en anglais; ces résultats seront présentés à IWCS 2025.


Li Liu est étudiante au doctorat en linguistique à l'Université de Montréal.


Pour recevoir les annonces hebdomadaires par courriel, visitez http://rali.iro.umontreal.ca/rali/?q=fr/node/1631

Liste des autres séminaires