Auditing Machine Learning Models

Current Machine Learning model evaluation methods, e.g., the use of test sets, will only detect whether a model’s predictions match the data. They cannot exclude the possibility that both predictions and data are biased. More targeted efforts to reduce or eliminate training data biases require either manual adjustments of the models, domain knowledge, or assume that data or model issues can easily be identified. This is seldom the case. Model auditing generally relies on manual analysis of the models, on existing data, and knowledgeable auditors. Automatically identifying deficiencies in both data and training, and how they impact application of models, is still an unsolved question. Bias mitigation approaches require either the ground-truth distribution or concrete information on the bias — or unbiased / differently biased data from other sources. In this project, we aim to develop a model-agnostic framework that will not be limited by any of these requirements.

2024

Lyu, Jiachen; Dost, Katharina; Koh, Yun Sing; Wicker, Jörg

Regional Bias in Monolingual English Language Models Journal Article

In: Machine Learning, 2024, ISSN: 1573-0565.

Abstract | Links | BibTeX | Altmetric | PlumX

Chang, Xinglong; Brydon, Liam; Wicker, Jörg

Memento: v1.1.1 Miscellaneous

Zenedo, 2024.

Links | BibTeX | Altmetric | PlumX

2023

Dost, Katharina; Tam, Jason; Lorsbach, Tim; Schmidt, Sebastian; Wicker, Jörg

Defining Applicability Domain in Biodegradation Pathway Prediction Unpublished Forthcoming

Forthcoming.

Abstract | Links | BibTeX | Altmetric | PlumX

Chang, Xinglong; Dost, Katharina; Dobbie, Gillian; Wicker, Jörg

Poison is Not Traceless: Fully-Agnostic Detection of Poisoning Attacks Unpublished Forthcoming

Forthcoming.

Abstract | Links | BibTeX | Altmetric | PlumX

Chang, Xinglong; Dobbie, Gillian; Wicker, Jörg

Fast Adversarial Label-Flipping Attack on Tabular Data Unpublished Forthcoming

Forthcoming.

Abstract | Links | BibTeX | Altmetric | PlumX

Pullar-Strecker, Zac; Chang, Xinglong; Brydon, Liam; Ziogas, Ioannis; Dost, Katharina; Wicker, Jörg

Memento: Facilitating Effortless, Efficient, and Reliable ML Experiments Proceedings Article

In: Morales, Gianmarco De Francisci; Perlich, Claudia; Ruchansky, Natali; Kourtellis, Nicolas; Baralis, Elena; Bonchi, Francesco (Ed.): Machine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo Track, pp. 310-314, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-43430-3.

Abstract | Links | BibTeX | Altmetric | PlumX

Dost, Katharina; Pullar-Strecker, Zac; Brydon, Liam; Zhang, Kunyang; Hafner, Jasmin; Riddle, Pat; Wicker, Jörg

Combatting over-specialization bias in growing chemical databases Journal Article

In: Journal of Cheminformatics, vol. 15, iss. 1, pp. 53, 2023, ISSN: 1758-2946.

Abstract | Links | BibTeX | Altmetric | PlumX