A Small, Auditable Dataset for Detecting Misogyny in Brazilian Portuguese

Online misogyny can be blunt, but it can also rely on irony, stereotypes or attacks that question a woman’s credibility. Those subtler cases are difficult for automated systems to identify, especially when the training data are limited or poorly documented. The researchers behind MISO-BR set out to create an open resource focused specifically on misogyny in Brazilian Portuguese, rather than treating it as just one category within general offensive-language detection.

Their dataset contains 896 social-media texts labeled as misogynistic or not. The team gathered posts from X using keyword lists and searches focused on public figures, then used a large language model to sort those posts before human review. They also brought in examples from OLID-BR, an existing Portuguese-language dataset, and relabeled them using MISO-BR’s own guidelines. Three annotators made the judgments; disagreements were resolved by majority vote. The release includes documentation such as the annotation guide, model prompt, data sources and experimental scripts.

The annotators agreed on the misogyny label to a substantial degree: Fleiss’ kappa was 0.788 across 816 items. That is a measure of agreement beyond what might occur by chance, not proof that the labels are objectively settled. The paper notes that disagreement clustered around quotation, irony and cases where criticism of a woman might or might not be rooted in gender stereotypes. Those difficult examples are part of the problem the dataset is meant to represent.

The researchers compared four conventional classifiers—including Naive Bayes, a linear support-vector machine and a random forest—with fine-tuned versions of BERTimbau, a model for Brazilian Portuguese. On the fixed test set, the conventional models scored between 0.651 and 0.676 on macro-F1, a metric that averages performance across the two classes. The BERTimbau models each scored 0.833. They also made the same number of errors on the test examples. The larger model did not improve that score over the smaller Base model, despite taking about 2.6 times as long to train.

These findings suggest that, in this experiment, the transformer models handled the classification task better than the simpler approaches. But a test set of 180 texts provides limited precision: the authors caution against ranking the conventional models based on small score differences. Nor does a strong score on this test establish how a system would perform on a different platform, topic or stream of posts.

That distinction matters because the dataset was deliberately balanced: half its examples are labeled misogynistic. Real online content may have a much lower share. The authors estimate that if the model’s measured true- and false-positive rates held steady, precision would fall to about 0.22 at a 5% misogyny prevalence. In other words, many flagged posts could be false alarms. This is a calculation based on assumed prevalence and test-set rates, not a measurement of performance on a real-world stream.

The collection process also creates a possible shortcut for models. Scraped posts and examples imported from OLID-BR are not evenly distributed across the labels. A classifier might learn differences in source or writing style rather than misogyny itself. The dataset records each item’s provenance, making that possibility testable, but the paper does not establish how much it affects the reported score. The authors also did not manually review a random sample of posts rejected by the automated triage stage, so they cannot estimate how much relevant content that stage missed. Keyword-guided collection may likewise favor explicit language over subtler cases.

MISO-BR is therefore best understood as a small, inspectable starting point. Its value lies not only in the benchmark numbers but in making the data and decisions open to scrutiny. The authors recommend human oversight rather than autonomous moderation and call for broader sampling, additional annotation and checks across data sources. The work is a preprint, so its claims have not necessarily been evaluated through peer review. Its reported results support further study, not conclusions about the prevalence of misogyny or the reliability of moderation systems in general.

Source: Passos, Miranda and Cavalcanti, “MISO-BR: An Open Corpus for Misogyny Detection in Brazilian Portuguese,” arXiv:2611.14036v1. The paper and corpus are publicly available; findings described here are from the preprint.