Research | Verena Blaschke

Publications

What do dialect speakers want?
A survey of attitudes towards language technology for German dialects

Verena Blaschke, Christoph Purschke, Hinrich Schütze & Barbara Plank
Preprint | Abstract Cite PDF

Natural language processing (NLP) has largely focused on modelling standardized languages. More recently, attention has increasingly shifted to local, non-standardized languages and dialects. However, the relevant speaker populations’ needs and wishes with respect to NLP tools are largely unknown. In this paper, we focus on dialects and regional languages related to German – a group of varieties that is heterogeneous in terms of prestige and standardization. We survey speakers of these varieties (N=327) and present their opinions on hypothetical language technologies for their dialects. Although attitudes vary among subgroups of our respondents, we find that respondents are especially in favour of potential NLP tools that work with dialectal input (especially audio input) such as virtual assistants, and less so for applications that produce dialectal output such as machine translation or spellcheckers.
Verena Blaschke, Christoph Purschke, Hinrich Schütze, and Barbara Plank (2024). “What do dialect speakers want? A survey of attitudes towards language technology for German dialects.” Computing Research Repository.
@article{blaschke2024what, title = {What do dialect speakers want? {A} survey of attitudes towards language technology for {German} dialects}, author = {Blaschke, Verena and Purschke, Christoph and Schütze, Hinrich and Plank, Barbara}, year = {2024}, journal = {Computing Research Repository}, volume = {arXiv:2402.11968} }
MaiBaam: A multi-dialectal Bavarian Universal Dependency treebank

Verena Blaschke, Barbara Kovačić, Siyao Peng, Hinrich Schütze & Barbara Plank
LREC–COLING 2024 | Abstract PDF Data Code Annotation guidelines

Despite the success of the Universal Dependencies (UD) project exemplified by its impressive language breadth, there is still a lack in within-language breadth’: most treebanks focus on standard languages. Even for German, the language with the most annotations in UD, so far no treebank exists for one of its language varieties spoken by over 10M people: Bavarian. To contribute to closing this gap, we present the first multi-dialect Bavarian treebank (MaiBaam) manually annotated with part-of-speech and syntactic dependency information in UD, covering multiple text genres (wiki, fiction, grammar examples, social, non-fiction). We highlight the morphosyntactic differences between the closely-related Bavarian and German and showcase the rich variability of speakers’ orthographies. Our corpus includes 15k tokens, covering dialects from all Bavarian-speaking areas spanning three countries. We provide baseline parsing and POS tagging results, which are lower than results obtained on German and vary substantially between different graph-based parsers. To support further research on Bavarian syntax, we make our dataset, language-specific guidelines and code publicly available.
Sebastian, Basti, Wastl?!
Recognizing named entities in Bavarian dialectal data

Siyao Peng, Zihang Sun, Huangyan Shan, Marie Kolm, Verena Blaschke, Ekaterina Artemova & Barbara Plank
LREC–COLING 2024 | Abstract PDF Data

Named Entity Recognition (NER) is a fundamental task to extract key information from texts, but annotated resources are scarce for dialects. This paper introduces the first dialectal NER dataset for German, BarNER, with 161K tokens annotated on Bavarian Wikipedia articles (bar-wiki) and tweets (bar-tweet), using a schema adapted from German CoNLL 2006 and GermEval. The Bavarian dialect differs from standard German in lexical distribution, syntactic construction, and entity information. We conduct in-domain, cross-domain, sequential, and joint experiments on two Bavarian and three German corpora and present the first comprehensive NER results on Bavarian. Incorporating knowledge from the larger German NER (sub-)datasets notably improves on bar-wiki and moderately on bar-tweet. Inversely, training first on Bavarian contributes slightly to the seminal German CoNLL 2006 corpus. Moreover, with gold dialect labels on Bavarian tweets, we assess multi-task learning between five NER and two Bavarian-German dialect identification tasks and achieve NER SOTA on bar-wiki. We substantiate the necessity of our low-resource BarNER corpus and the importance of diversity in dialects, genres, and topics in enhancing model performance.
Exploring the robustness of task-oriented dialogue systems for colloquial German varieties

Ekaterina Artemova, Verena Blaschke & Barbara Plank
EACL 2024 | Abstract Cite PDF Slides Poster Code

Mainstream cross-lingual task-oriented dialogue (ToD) systems leverage the transfer learning paradigm by training a joint model for intent recognition and slot-filling in English and applying it, zero-shot, to other languages.We address a gap in prior research, which often overlooked the transfer to lower-resource colloquial varieties due to limited test data.Inspired by prior work on English varieties, we craft and manually evaluate perturbation rules that transform German sentences into colloquial forms and use them to synthesize test sets in four ToD datasets.Our perturbation rules cover 18 distinct language phenomena, enabling us to explore the impact of each perturbation on slot and intent performance.Using these new datasets, we conduct an experimental evaluation across six different transformers.Here, we demonstrate that when applied to colloquial varieties, ToD systems maintain their intent recognition performance, losing 6% (4.62 percentage points) in accuracy on average. However, they exhibit a significant drop in slot detection, with a decrease of 31% (21 percentage points) in slot F_1 score.Our findings are further supported by a transfer experiment from Standard American English to synthetic Urban African American Vernacular English.
Ekaterina Artemova, Verena Blaschke, and Barbara Plank (2024). “Exploring the robustness of task-oriented dialogue systems for colloquial German varieties.” In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 445–468. St. Julian’s, Malta. Association for Computational Linguistics.
@inproceedings{artemova2024exploring, title = {Exploring the robustness of task-oriented dialogue systems for colloquial {G}erman varieties}, author = {Artemova, Ekaterina and Blaschke, Verena and Plank, Barbara}, editor = {Graham, Yvette and Purver, Matthew}, booktitle = {Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)}, year = {2024}, month = mar, address = {St. Julian{'}s, Malta}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2024.eacl-long.28}, pages = {445--468}, }
A survey of corpora for Germanic low-resource languages and dialects

Verena Blaschke, Hinrich Schütze & Barbara Plank
NoDaLiDa 2023 | Abstract Cite Website PDF Slides

Despite much progress in recent years, the vast majority of work in natural language processing (NLP) is on standard languages with many speakers. In this work, we instead focus on low-resource languages and in particular non-standardized low-resource languages. Even within branches of major language families, often considered well-researched, little is known about the extent and type of available resources and what the major NLP challenges are for these language varieties. The first step to address this situation is a systematic survey of available corpora (most importantly, annotated corpora, which are particularly valuable for NLP research). Focusing on Germanic low-resource language varieties, we provide such a survey in this paper. Except for geolocation (origin of speaker or document), we find that manually annotated linguistic resources are sparse and, if they exist, mostly cover morphosyntax. Despite this lack of resources, we observe that interest in this area is increasing: there is active development and a growing research community. To facilitate research, we make our overview of over 80 corpora publicly available.
Verena Blaschke, Hinrich Schütze, and Barbara Plank (2023). “A survey of corpora for Germanic low-resource languages and dialects.” In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pp. 392–414. Tórshavn, Faroe Islands. University of Tartu Library.
@inproceedings{blaschke2023survey, title = {A survey of corpora for {G}ermanic low-resource languages and dialects}, author = {Blaschke, Verena and Sch{\"u}tze, Hinrich and Plank, Barbara}, year = {2023}, month = may, booktitle = {Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)}, address = {T{\'o}rshavn, Faroe Islands}, publisher = {University of Tartu Library}, url = {https://aclanthology.org/2023.nodalida-1.41}, pages = {392--414}, }
Does manipulating tokenization aid cross-lingual transfer?
A study on POS tagging for non-standardized languages

Verena Blaschke, Hinrich Schütze & Barbara Plank
VarDial @ EACL 2023 | Abstract Cite PDF Slides Video Poster Code

One of the challenges with finetuning pretrained language models (PLMs) is that their tokenizer is optimized for the language(s) it was pretrained on, but brittle when it comes to previously unseen variations in the data. This can for instance be observed when finetuning PLMs on one language and evaluating them on data in a closely related language variety with no standardized orthography. Despite the high linguistic similarity, tokenization no longer corresponds to meaningful representations of the target data, leading to low performance in, e.g., part-of-speech tagging.
In this work, we finetune PLMs on seven languages from three different families and analyze their zero-shot performance on closely related, non-standardized varieties. We consider different measures for the divergence in the tokenization of the source and target data, and the way they can be adjusted by manipulating the tokenization during the finetuning step. Overall, we find that the similarity between the percentage of words that get split into subwords in the source and target data (the split word ratio difference) is the strongest predictor for model performance on target data.
Verena Blaschke, Hinrich Schütze, and Barbara Plank (2023). “Does manipulating tokenization aid cross-lingual transfer? A study on POS tagging for non-standardized languages.” In Proceedings of the Tenth Workshop on NLP for Similar Languages, Varieties and Dialects, pp. 40–54. Dubrovnik, Croatia. Association for Computational Linguistics.
@inproceedings{blaschke2023manipulating, title = {Does manipulating tokenization aid cross-lingual transfer? {A} study on {POS} tagging for non-standardized languages}, author = {Blaschke, Verena and Sch{\"u}tze, Hinrich and Plank, Barbara}, booktitle = {Proceedings of the Tenth Workshop on NLP for Similar Languages, Varieties and Dialects}, year = {2023}, month = may, address = {Dubrovnik, Croatia}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/2023.vardial-1.5}, pages = {40--54}, }
Navigable atom-rule interactions in PSL models enhanced by rule verbalizations, with an application to etymological inference

Verena Blaschke, Thora Daneyko, Jekaterina Kaparina, Zhuge Gao & Johannes Dellert
ILP 2022 | Abstract Cite PDF Slides Poster Code (PSL-Infrastructure) Code (PSL-RAGviewer)

Adding to the budding landscape of advanced analysis tools for Probabilistic Soft Logic (PSL), we present a graphical explorer for grounded PSL models. It exposes the structure of the model from the perspective of any single atom, listing the ground rules in which it occurs. The other atoms in these rules serve as links for navigation through the resulting rule-atom graph (RAG). As additional diagnostic criteria, each associated rule is further classified as exerting upward or downward pressure on the atom’s value, and as active or inactive depending on its importance for the MAP estimate.
Our RAG viewer further includes a general infrastructure for making PSL results explainable by stating the reasoning patterns in terms of domain language. For this purpose, we provide a Java interface for “talking” predicates and rules which can generate verbalized explanations of the atom interactions effected by each rule. If the model’s rules are structured similarly to the way the domain is conceptualized by users, they will receive an intuitive explanation of the result in natural language.
As an example application, we present the current state of the loanword detection component of EtInEn, our upcoming software for machine-assisted etymological theory development.
Verena Blaschke, Thora Daneyko, Jekaterina Kaparina, Zhuge Gao, and Johannes Dellert (2024). “Navigable atom-rule interactions in PSL models enhanced by rule verbalizations, with an application to etymological inference.” In Inductive Logic Programming, pp. 15–24. Cham. Springer Nature Switzerland. DOI: 10.1007/978-3-031-55630-2_2
@inproceedings{blaschke2024navigable, title = {Navigable atom-rule interactions in {PSL} models enhanced by rule verbalizations, with an application to etymological inference}, author = {Blaschke, Verena and Daneyko, Thora and Kaparina, Jekaterina and Gao, Zhuge and Dellert, Johannes}, year = {2024}, url = {https://link.springer.com/chapter/10.1007/978-3-031-55630-2_2}, doi = {10.1007/978-3-031-55630-2_2}, booktitle = {Inductive Logic Programming}, publisher = {Springer Nature Switzerland}, address = {Cham}, pages = {15--24}, editor = {Muggleton, Stephen H. and Tamaddoni-Nezhad, Alireza}, }
CyberWallE at SemEval-2020 task 11: An analysis of feature engineering for ensemble models for propaganda detection

Verena Blaschke, Maxim Korniyenko & Sam Tureski
SemEval @ COLING 2020 | Abstract Cite PDF Poster Code

This paper describes our participation in the SemEval-2020 task Detection of Propaganda Techniques in News Articles. We participate in both subtasks: Span Identification (SI) and Technique Classification (TC). We use a bi-LSTM architecture in the SI subtask and train a complex ensemble model for the TC subtask. Our architectures are built using embeddings from BERT in combination with additional lexical features and extensive label post-processing. Our systems achieve a rank of 8 out of 35 teams in the SI subtask (F1-score: 43.86%) and 8 out of 31 teams in the TC subtask (F1-score: 57.37%).
Verena Blaschke, Maxim Korniyenko, and Sam Tureski (2020). “CyberWallE at SemEval-2020 task 11: An analysis of feature engineering for ensemble models for propaganda detection.” In Proceedings of the Fourteenth Workshop on Semantic Evaluation (SemEval 2020), pp. 1469–1480. Barcelona (online). International Committee for Computational Linguistics. DOI: 10.18653/v1/2020.semeval-1.192
@inproceedings{blaschke2020cyberwalle, title = {{C}yber{W}all{E} at {S}em{E}val-2020 task 11: An analysis of feature engineering for ensemble models for propaganda detection}, author = {Blaschke, Verena and Korniyenko, Maxim and Tureski, Sam}, booktitle = {Proceedings of the Fourteenth Workshop on Semantic Evaluation (SemEval 2020)}, year = {2020}, address = {Barcelona (online)}, publisher = {International Committee for Computational Linguistics}, url = {https://aclanthology.org/2020.semeval-1.192}, doi = {10.18653/v1/2020.semeval-1.192}, pages = {1469--1480}, }
Tübingen-Oslo Team at the VarDial 2018 evaluation campaign: An analysis of n-gram features in language variety identification

Çağrı Çöltekin, Taraka Rama & Verena Blaschke
VarDial @ COLING 2018 | Abstract Cite PDF

This paper describes our systems for the VarDial 2018 evaluation campaign. We participated in all language identification tasks, namely, Arabic dialect identification (ADI), German dialect identification (GDI), discriminating between Dutch and Flemish in Subtitles (DFS), and Indo-Aryan Language Identification (ILI). In all of the tasks, we only used textual transcripts (not using audio features for ADI). We submitted system runs based on support vector machine classifiers (SVMs) with bag of character and word n-grams as features, and gated bidirectional recurrent neural networks (RNNs) using units of characters and words. Our SVM models outperformed our RNN models in all tasks, obtaining the first place on the DFS task, third place on the ADI task, and second place on others according to the official rankings. As well as describing the models we used in the shared task participation, we present an analysis of the n-gram features used by the SVM models in each task, and also report additional results (that were run after the official competition deadline) on the GDI surprise dialect track.
Çağrı Çöltekin, Taraka Rama, and Verena Blaschke (2018). “Tübingen-Oslo Team at the VarDial 2018 evaluation campaign: An analysis of n-gram features in language variety identification.” In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pp. 55–65. Santa Fe, New Mexico, USA. Association for Computational Linguistics.
@inproceedings{coltekin2018tubingen, title = {{T}{\"u}bingen-{O}slo Team at the {V}ar{D}ial 2018 evaluation campaign: An analysis of n-gram features in language variety identification}, author = {{\c{C}}{\"o}ltekin, {\c{C}}a{\u{g}}r{\i} and Rama, Taraka and Blaschke, Verena}, booktitle = {Proceedings of the Fifth Workshop on {NLP} for Similar Languages, Varieties and Dialects ({V}ar{D}ial 2018)}, year = {2018}, address = {Santa Fe, New Mexico, USA}, publisher = {Association for Computational Linguistics}, url = {https://aclanthology.org/W18-3906}, pages = {55--65}, }

Talks

(Excluding paper presentations, which are linked in the Publications section where applicable.)

Large language models and small language varieties:
Challenges and current methods

Verena Blaschke & Barbara Plank
Embracing Variability in Natural Language Processing @ ICLaVE 2024 |
Natural dialect processing:
NLP for non-standardized language varieties

Verena Blaschke & Barbara Plank
Quantitative approaches in dialectology and variationist sociolinguistics (12/2023) | Slides
Configurable language-specific tokenization for CLDF databases

Johannes Dellert & Verena Blaschke
Exploiting standardized cross-linguistic data in historical linguistics @ ICHL 2023 | Abstract

In any workflow for computational historical linguistics, tokenization of IPA sequences is a crucial preprocessing step, as it shapes the alignments which provide the input of algorithms for cognate detection and proto-form reconstruction. This is also true for EtInEn (Dellert 2019), our forthcoming integrated development environment for etymological theories. An EtInEn project can be created from any CLDF database such as the ones that have been aggregated and unified by the Lexibank initiative (List ea. 2022). Whereas the tools for preparing CLDF databases (Forkel & List 2020) encourage the application of a uniform tokenization across all languages in a dataset, our view is that in many contexts, it is more natural to tokenize phonetic sequences in ways that differ between languages. To provide a simple example, many geminates in Italian need to be aligned to consonant clusters in other Romance languages (e.g. notte vs. Romanian noapte “night”), which is much easier if they are tokenized into two instances of the same consonant, whereas geminates in Swedish are best treated as cognate to their shortened counterparts in other Germanic languages.
To provide comprehensive support for such cases, EtInEn includes configurable language-specific tokenizers as an additional abstraction layer that allows to reshape forms after the import, and also serves as a generic way to bridge phonetic surface forms and the underlying forms that historical linguists are primarily interested in. Each tokenizer is defined by a token alphabet which is used for greedy tokenization, a list of allophone sets which can be used to abstract over irrelevant subphonemic distinctions, and a list of non-IPA symbols that are defined in terms of phonetic features. The initial state of each tokenizer is based on an analysis of the tokens used by the imported CLDF database. Tokenizer definitions are stored in a human-editable plain-text format which we would like to propose as a new standard.
In EtInEn, tokenizer definitions are manipulated through a graphical editor in which the potential tokens for each language are arranged in the familiar layout of consonant and vowel charts, enhanced by additional panels for diphthongs and tones. Currently defined tokens are highlighted, and allophone sets are summarized under their canonical symbols. Basic edit operations serve to group several sounds into an allophone set, and to join or split a multi-symbol sequence, such as a diphthong or a sound with a coarticulation. More complex operations support workflows for parallel configuration of multiple tokenizers.
Additional non-IPA symbols can be given semantics in terms of a combination of phonetic features, and declared to be part of the token set for any language. On the representational level, this provides the option to use non-IPA symbols for form display, whereas underlyingly, the system will interpret the symbols in terms of their features. On the conceptual level, underspecified definitions provide support for metasymbols. In addition to some predefined metasymbols (such as V for vowels and C for consonants), the user can assign additional symbols to arbitrary classes of sounds. These are then available throughout EtInEn for various purposes, such as concisely representing the conditioning environments for a soundlaw, or summarizing the probabilistic output of an automated reconstruction module.
In addition to configurable tokenizers, EtInEn provides the option to define form-specific tokenization overrides, allowing to substitute the result of automated tokenization with any sequence over the current token alphabet for the relevant language. This is currently our strategy for handling otherwise challenging phenomena such as metathesis or root-pattern morphology, which we normalize into alignable and concatenative representations. This forms a bridge to existing standards for representing morphology in the CLDF framework (e.g. Schweikhard & List 2020), which currently only support the annotation of morpheme boundaries in terms of simple splits in phonetic IPA sequences.

References:
Dellert, Johannes (2019): “Interactive Etymological Inference via Statistical Relational Learning.” Workshop on Computer-Assisted Language Comparison at SLE-2019.
Forkel, Robert and Johann-Mattis List (2020): “CLDFBench. Give your Cross-Linguistic data a lift.” Proceedings of LREC 2020, 6997-7004.
List, Johann-Mattis, Robert Forkel, S. J. Greenhill, Christoph Rzymski, Johannes Englisch & Russell Gray (2022): “Lexibank, A public repository of standardized wordlists with computed phonological and lexical features.” Scientific Data 9.316, 1-31.
Schweikhard, Nathanael E. and Johann-Mattis List (2020): “Developing an annotation framework for word formation processes in comparative linguistics.“ SKASE Journal of Theoretical Linguistics 17(1), 2-26.
Correlating borrowing events across concepts to derive a data-driven source of evidence for loanword etymologies

Verena Blaschke & Johannes Dellert
Model and Evidence in Quantitative Comparative Linguistics @ DGfS 2021 | Abstract Slides Code

Computational methods for approximating various aspects of the reasoning of a historical linguist have great potential as components of a future generation of systems for more rapid machine-aided theory development (List 2019). One of the main challenges for such methods is that some of the heuristics and reasoning patterns commonly used in historical linguistics are difficult to formalize completely. Etymological arguments frequently appeal more to the shared experience of experts than to a fully developed theoretical framework. Computationally emulating this process will require experience in the shape of data with annotations that represent the heuristics and preferences employed within human expert communities.
Our first application of this general paradigm focuses on informal evidence used for establishing loanword etymologies. Classical arguments for assigning a loanword etymology to a word rely on deviations from the sound laws which would have applied if the word had been inherited, or borrowed at a different point in time. For instance, it is clear that the German word Person is a borrowing and not strictly cognate with Latin persona, because otherwise the initial p would have had to undergo a sound shift to f. Such a criterion would be rather straightforward to formalize based on a formal description of the expected sound laws. However, this criterion is only helpful if some known sound law would have applied to a part of the phonetic material of the word in question. In many cases, we are not in this comfortable position, and the etymological discussion will be based on more elusive evidence.
In some cases, historical, geographical or archaeological knowledge will help to make the decision, but the most systematically exploitable type of evidence builds on the tendency for loanwords to appear in batches. For instance, if some language has already been established as a donor language for some words, it becomes more likely as a candidate donor for other words as well, even if the evidence from the individual words alone would not warrant such a conclusion. Even more crucially, arguments often rely on the observation that words from the same semantic field tend to get borrowed together. This applies to obvious cases like numbers and month names as well as to less obviously connected sets of concepts such as tools belonging to a certain craft (Tadmor 2009, Carling et al. 2009).
A helpful automated method for inferring possible loanword relations will have to emulate at least some of these types of informal reasoning. As a first step in this direction, we develop data-driven measures of how much evidence establishing one borrowing event provides for assuming others. We also explore the extent to which such a correlation structure of borrowing events can be extracted from the limited amounts of existing cross-linguistic loanword data.
Given a set of parallel wordlists annotated with loanword status and semantic concept information, we extract how often each concept was loaned and by which pairs of donor and target languages. To quantify the non-independence of borrowing events for each pair of concepts, we average the normalized pointwise mutual information across 1,000 bootstrap samples. In order to additionally retrieve some directional signal that can be interpreted as an approximation to implicational universals of borrowing, the same procedure is applied to the conditional probabilities of concept pairs given one of the concepts.
We execute our methods on WOLD (Haspelmath and Tadmor 2009), and find that even from this limited sample of 41 languages, it is possible to extract quite a few of the expected within-domain correlations (such as the ones between numbers or between kinship terms), which validated our approach. In addition, we also receive some more surprising cross-domain correlations (such as between NARROW and HOLE and between KNEEL and DEFEAT, but also between BEESWAX and KIDNEY) which require further investigation.

References:
Carling, Gerd, Sandra Cronhamn, Robert Farren, Elnur Aliyev, and Johan Frid. 2019. “The causality of borrowing: Lexical loans in Eurasian languages.” PloS one 14(10): e0223588.
Haspelmath, Martin and Uri Tadmor, eds. 2009. World Loanword Database. Leipzig: Max Planck Institute for Evolutionary Anthropology. Available at https://wold.clld.org/.
List, Johann-Mattis. 2019. “Automated methods for the investigation of language contact, with a focus on lexical borrowing.” Language and Linguistics Compass 13(10): e12355.
Tadmor, Uri. 2009. “Loanwords in the world’s languages: Findings and results.” In Martin Haspelmath, and Uri Tadmor, eds. Loanwords in the world’s languages: A comparative handbook. Berlin: De Gruyter Mouton. 55-75.
Clustering dialect varieties based on historical sound correspondences

Verena Blaschke
GSCL Student Award nominee presentations @ KONVENS 2019 | Abstract Summary Slides Code

While information on historical sound shifts plays an important role for examining the relationships between related language varieties, it has rarely been used for computational dialectology. This thesis explores the performance of two algorithms for clustering language varieties based on sound correspondences between Proto-Germanic and modern continental West Germanic dialects. Our experiments suggest that the results of agglomerative clustering match common dialect groupings more closely than the results of (divisive) bipartite spectral graph co-clustering. We also observe that adding phonetic context information to the sound correspondences yields clusters that are more frequently associated with representative and distinctive sound correspondences).

Theses

Explainable machine learning in linguistics and applied NLP:
Two case studies of Norwegian dialectometry and sexism detection in French tweets

Master’s thesis, 2021, supervised by Çağrı Çöltekin & John Nerbonne | Abstract PDF Code

This thesis presents an exploration of explainable machine learning in the context of a traditional linguistic area (dialect classification) and an applied task (sexism detection). In both tasks, the input features deemed especially relevant for the classification form meaningful groups that fit in with previous research on the topic, although not all such features are easy to understand or provide plausible explanations. In the case of dialect classification, some important features show that the model also learned patterns that are not typically presented by dialectologists. For both case studies, I use LIME [1] to rank features by their importance for the classification. For the sexism detection task, I additionally examine attention weights, which produce feature rankings that are in many cases similar to the LIME results but that are overall worse at showcasing tokens that are especially characteristic of sexist tweets.

[1] M. T. Ribeiro, S. Singh, and C. Guestrin (2016). “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, New York, NY, USA, pp. 1135–1144. Association for Computing Machinery.
Clustering dialect varieties based on historical sound correspondences

Bachelor’s thesis, 2018, supervised by Çağrı Çöltekin. Finalist for the GSCL Award for the best Bachelor’s thesis in computational linguistics 2017–2019 | Abstract PDF Summary Slides Code

While information on historical sound shifts plays an important role for examining the relationships between related language varieties, it has rarely been used for computational dialectology. This thesis explores the performance of two algorithms for clustering language varieties based on sound correspondences between Proto-Germanic and modern continental West Germanic dialects. Our experiments suggest that the results of agglomerative clustering match common dialect groupings more closely than the results of (divisive) bipartite spectral graph co-clustering. We also observe that adding phonetic context information to the sound correspondences yields clusters that are more frequently associated with representative and distinctive sound correspondences).