Explainable AI

• 
  Generative Inpainting for Shapley-Value-Based Anomaly Explanation. Tritscher, Julian; Lissmann, Philip; Wolf, Maximilian; Krause, Anna; Hotho, Andreas; Schlör, Daniel. In The World Conference on eXplainable Artificial Intelligence (xAI 2024) - to appear. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Feature relevance explanations currently constitute the most used type of explanation in anomaly detection related tasks such as cyber security and fraud detection. Recent works have underscored the importance of optimizing hyperparameters of post-hoc explainers which show a large impact on the resulting explanation quality. In this work, we propose a new method to set the hyperparameter of replacement values within Shapley-value-based post-hoc explainers. Our method leverages ideas from the domain of generative image inpainting, where generative machine learning models are used to replace parts of a given input image. We show that these generative models can also be applied to tabular replacement value generation for Shapley-value-based feature relevance explainers. Experimentally, we train a denoising diffusion probabilistic model for generative inpainting on two tabular anomaly detection datasets from the domains of network intrusion detection and occupational fraud detection, and integrate the generative inpainting model into the SHAP explanation framework. We empirically show that generative inpainting may be used to achieve consistently strong explanation quality when explaining different anomaly detectors on tabular data.

  @article{tritscher2024generative, abstract = {Feature relevance explanations currently constitute the most used type of explanation in anomaly detection related tasks such as cyber security and fraud detection. Recent works have underscored the importance of optimizing hyperparameters of post-hoc explainers which show a large impact on the resulting explanation quality. In this work, we propose a new method to set the hyperparameter of replacement values within Shapley-value-based post-hoc explainers. Our method leverages ideas from the domain of generative image inpainting, where generative machine learning models are used to replace parts of a given input image. We show that these generative models can also be applied to tabular replacement value generation for Shapley-value-based feature relevance explainers. Experimentally, we train a denoising diffusion probabilistic model for generative inpainting on two tabular anomaly detection datasets from the domains of network intrusion detection and occupational fraud detection, and integrate the generative inpainting model into the SHAP explanation framework. We empirically show that generative inpainting may be used to achieve consistently strong explanation quality when explaining different anomaly detectors on tabular data.}, author = {Tritscher, Julian and Lissmann, Philip and Wolf, Maximilian and Krause, Anna and Hotho, Andreas and Schlör, Daniel}, journal = {The World Conference on eXplainable Artificial Intelligence (xAI 2024) - to appear}, keywords = {xai}, month = {04}, title = {Generative Inpainting for Shapley-Value-Based Anomaly Explanation}, year = 2024 }

• 
  Feature relevance XAI in anomaly detection: Reviewing approaches and challenges. Tritscher, Julian; Krause, Anna; Hotho, Andreas. In Frontiers in Artificial Intelligence, 6. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  With complexity of artificial intelligence systems increasing continuously in past years, studies to explain these complex systems have grown in popularity. While much work has focused on explaining artificial intelligence systems in popular domains such as classification and regression, explanations in the area of anomaly detection have only recently received increasing attention from researchers. In particular, explaining singular model decisions of a complex anomaly detector by highlighting which inputs were responsible for a decision, commonly referred to as local post-hoc feature relevance, has lately been studied by several authors. In this paper, we systematically structure these works based on their access to training data and the anomaly detection model, and provide a detailed overview of their operation in the anomaly detection domain. We demonstrate their performance and highlight their limitations in multiple experimental showcases, discussing current challenges and opportunities for future work in feature relevance XAI for anomaly detection.

  @article{tritscher2023feature, abstract = {With complexity of artificial intelligence systems increasing continuously in past years, studies to explain these complex systems have grown in popularity. While much work has focused on explaining artificial intelligence systems in popular domains such as classification and regression, explanations in the area of anomaly detection have only recently received increasing attention from researchers. In particular, explaining singular model decisions of a complex anomaly detector by highlighting which inputs were responsible for a decision, commonly referred to as local post-hoc feature relevance, has lately been studied by several authors. In this paper, we systematically structure these works based on their access to training data and the anomaly detection model, and provide a detailed overview of their operation in the anomaly detection domain. We demonstrate their performance and highlight their limitations in multiple experimental showcases, discussing current challenges and opportunities for future work in feature relevance XAI for anomaly detection.}, author = {Tritscher, Julian and Krause, Anna and Hotho, Andreas}, journal = {Frontiers in Artificial Intelligence}, keywords = {xai}, month = {02}, title = {Feature relevance XAI in anomaly detection: Reviewing approaches and challenges}, volume = 6, year = 2023 }
• 
  Evaluating feature relevance XAI in network intrusion detection. Tritscher, Julian; Wolf, Maximilian; Hotho, Andreas; Schlör, Daniel. In The World Conference on eXplainable Artificial Intelligence (xAI 2023). 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  As machine learning models become increasingly complex, there is a growing need for explainability to understand and trust the decision-making processes. In the domain of network intrusion detection, post-hoc feature relevance explanations have been widely used to provide insight into the factors driving model decisions. However, recent research has highlighted challenges with these methods when applied to anomaly detection, which can vary in importance and impact depending on the application domain. In this paper, we investigate the challenges of post-hoc feature relevance explanations for network intrusion detection, a critical area for ensuring the security and integrity of computer networks. To gain a deeper understanding of these challenges for the application domain, we quantitatively and qualitatively investigate the popular feature relevance approach SHAP when explaining different network intrusion detection approaches. We conduct experiments to jointly evaluate detection quality and explainability, and explore the impact of replacement data, a commonly overlooked hyperparameter of post-hoc feature relevance approaches. We find that post-hoc XAI can provide high quality explanations, but requires a careful choice of its replacement data as default settings and common choices do not transfer across different detection models. Our study showcases the viability of post-hoc XAI for network intrusion detection systems, but highlights the need for rigorous evaluations of produced explanations.

  @article{tritscher2023evaluating, abstract = {As machine learning models become increasingly complex, there is a growing need for explainability to understand and trust the decision-making processes. In the domain of network intrusion detection, post-hoc feature relevance explanations have been widely used to provide insight into the factors driving model decisions. However, recent research has highlighted challenges with these methods when applied to anomaly detection, which can vary in importance and impact depending on the application domain. In this paper, we investigate the challenges of post-hoc feature relevance explanations for network intrusion detection, a critical area for ensuring the security and integrity of computer networks. To gain a deeper understanding of these challenges for the application domain, we quantitatively and qualitatively investigate the popular feature relevance approach SHAP when explaining different network intrusion detection approaches. We conduct experiments to jointly evaluate detection quality and explainability, and explore the impact of replacement data, a commonly overlooked hyperparameter of post-hoc feature relevance approaches. We find that post-hoc XAI can provide high quality explanations, but requires a careful choice of its replacement data as default settings and common choices do not transfer across different detection models. Our study showcases the viability of post-hoc XAI for network intrusion detection systems, but highlights the need for rigorous evaluations of produced explanations.}, author = {Tritscher, Julian and Wolf, Maximilian and Hotho, Andreas and Schlör, Daniel}, journal = {The World Conference on eXplainable Artificial Intelligence (xAI 2023)}, keywords = {xai}, title = {Evaluating feature relevance XAI in network intrusion detection}, year = 2023 }
• 
  Occupational Fraud Detection through Agent-based Data Generation. Tritscher, Julian; Roos, Alexander; Schlör, Daniel; Hotho, Andreas; Krause, Anna. In The 8th Workshop on MIning DAta for financial applicationS MIDAS 2023. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Occupational fraud is an increasing concern for enterprises that is estimated to cause losses of around 5% of company revenue each year. With the increasing data tracked by companies through enterprise resource planning systems, recent research has taken interest in the automated detection of occupational fraud. Automated detection is however hindered by the unavailability of labeled fraud cases which require known occupational frauds within company data and costly expert annotation. Even despite the existence of anomaly detection methods that can be trained on unsupervised data, selecting the ideal preprocessing techniques, the most suitable model, and the optimal hyperparameters necessitates the availability of labeled data for evaluation purposes. To alleviate this issue, we propose to use simulation through multi-agent systems for generating business processes according to best practices from economics and creating labeled synthetic data that closely matches a given unlabeled real-world dataset. We extend an existing simulation by incorporating functionality for including, tracking and automatic labeling of occupational fraud cases. Using this simulation, we propose a framework that decides on important design choices for fraud detection models in enterprise resource planning data and does not require labeled real-world data. We demonstrate in multiple experiments that the framework can aid automated occupational fraud detection through data generation.

  @article{tritscher2023occupational, abstract = {Occupational fraud is an increasing concern for enterprises that is estimated to cause losses of around 5% of company revenue each year. With the increasing data tracked by companies through enterprise resource planning systems, recent research has taken interest in the automated detection of occupational fraud. Automated detection is however hindered by the unavailability of labeled fraud cases which require known occupational frauds within company data and costly expert annotation. Even despite the existence of anomaly detection methods that can be trained on unsupervised data, selecting the ideal preprocessing techniques, the most suitable model, and the optimal hyperparameters necessitates the availability of labeled data for evaluation purposes. To alleviate this issue, we propose to use simulation through multi-agent systems for generating business processes according to best practices from economics and creating labeled synthetic data that closely matches a given unlabeled real-world dataset. We extend an existing simulation by incorporating functionality for including, tracking and automatic labeling of occupational fraud cases. Using this simulation, we propose a framework that decides on important design choices for fraud detection models in enterprise resource planning data and does not require labeled real-world data. We demonstrate in multiple experiments that the framework can aid automated occupational fraud detection through data generation.}, author = {Tritscher, Julian and Roos, Alexander and Schlör, Daniel and Hotho, Andreas and Krause, Anna}, journal = {The 8th Workshop on MIning DAta for financial applicationS MIDAS 2023}, keywords = {mas}, title = {Occupational Fraud Detection through Agent-based Data Generation}, year = 2023 }

• 
  Open ERP System Data For Occupational Fraud Detection. Tritscher, Julian; Gwinner, Fabian; Schlör, Daniel; Krause, Anna; Hotho, Andreas. In arxiv. 2022.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @article{tritscher2022erpData, author = {Tritscher, Julian and Gwinner, Fabian and Schlör, Daniel and Krause, Anna and Hotho, Andreas}, journal = {arxiv}, keywords = {author:hotho}, title = {Open ERP System Data For Occupational Fraud Detection}, year = 2022 }
• 
  Towards Explainable Occupational Fraud Detection. Tritscher, Julian; Schlör, Daniel; Gwinner, Fabian; Krause, Anna; Hotho, Andreas. In Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2022, Communications in Computer and Information Science(1753), pp. 79–96. 2022.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @article{tritscher2022towards, author = {Tritscher, Julian and Schlör, Daniel and Gwinner, Fabian and Krause, Anna and Hotho, Andreas}, journal = {Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2022}, keywords = {xai}, number = 1753, pages = {79-96}, title = {Towards Explainable Occupational Fraud Detection}, volume = {Communications in Computer and Information Science}, year = 2022 }

• 
  A financial game with opportunities for fraud. Tritscher, Julian; Krause, Anna; Schl"o}r, Daniel; Gwinner, Fabian; von Mammen, Sebastian; Hotho, Andreas. In IEE COG 2021, 2021. 2021.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Even though companies store large amounts of business data in enterprise resource planning (ERP) systems, obtaining data for financial fraud detection is prohibitively difficult due to privacy concerns and companies protecting trade secrets. One possible solution is game-based generation of synthetic ERP data, which poses the challenge of designing an environment that generates realistic ERP data and allows players to commit many different types of fraud. In this work, we design a multiplayer game that allows players to cooperatively run a fictional company, while simultaneously challenging them to maximize their personal gain. We introduce an approach for letting players explore fraud scenarios through emergent gameplay and present a prototype that may be primed with information from real world ERP systems to generate realistic data.

  @article{tritscherfinancial, abstract = {Even though companies store large amounts of business data in enterprise resource planning (ERP) systems, obtaining data for financial fraud detection is prohibitively difficult due to privacy concerns and companies protecting trade secrets. One possible solution is game-based generation of synthetic ERP data, which poses the challenge of designing an environment that generates realistic ERP data and allows players to commit many different types of fraud. In this work, we design a multiplayer game that allows players to cooperatively run a fictional company, while simultaneously challenging them to maximize their personal gain. We introduce an approach for letting players explore fraud scenarios through emergent gameplay and present a prototype that may be primed with information from real world ERP systems to generate realistic data.}, author = {Tritscher, Julian and Krause, Anna and Schl{\"o}r, Daniel and Gwinner, Fabian and von Mammen, Sebastian and Hotho, Andreas}, journal = {IEE COG 2021}, keywords = {research_imbalanced_data}, title = {A financial game with opportunities for fraud}, volume = 2021, year = 2021 }

• 
  Evaluation of post-hoc XAI approaches through synthetic tabular data. Tritscher, Julian; Ring, Markus; Schlör, Daniel; Hettinger, Lena; Hotho, Andreas. In International Symposium on Methodologies for Intelligent Systems. Springer, 2020.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Evaluating the explanations given by post-hoc XAI approaches on tabular data is a challenging prospect, since the subjective judgement of explanations of tabular relations is non trivial in contrast to e.g. the judgement of image heatmap explanations. In order to quantify XAI performance on categorical tabular data, where feature relationships can often be described by Boolean functions, we propose an evaluation setting through generation of synthetic datasets. To create gold standard explanations, we present a definition of feature relevance in Boolean functions. In the proposed setting we evaluate eight state-of-the-art XAI approaches and gain novel insights into XAI performance on categorical tabular data. We find that the investigated approaches often fail to faithfully explain even basic relationships within categorical data.

  @article{tritscher2020evaluation, abstract = {Evaluating the explanations given by post-hoc XAI approaches on tabular data is a challenging prospect, since the subjective judgement of explanations of tabular relations is non trivial in contrast to e.g. the judgement of image heatmap explanations. In order to quantify XAI performance on categorical tabular data, where feature relationships can often be described by Boolean functions, we propose an evaluation setting through generation of synthetic datasets. To create gold standard explanations, we present a definition of feature relevance in Boolean functions. In the proposed setting we evaluate eight state-of-the-art XAI approaches and gain novel insights into XAI performance on categorical tabular data. We find that the investigated approaches often fail to faithfully explain even basic relationships within categorical data.}, author = {Tritscher, Julian and Ring, Markus and Schlör, Daniel and Hettinger, Lena and Hotho, Andreas}, booktitle = {International Symposium on Methodologies for Intelligent Systems}, keywords = {xai}, note = {Accepted but not published}, organization = {Springer}, title = {Evaluation of post-hoc XAI approaches through synthetic tabular data}, year = 2020 }
• 
  Evaluation of Post-hoc XAI Approaches Through Synthetic Tabular Data. Tritscher, Julian; Ring, Markus; Schlör, Daniel; Hettinger, Lena; Hotho, Andreas. In Foundations of Intelligent Systems, D. Helic, G. Leitner, M. Stettinger, A. Felfernig, Z. W. Raś (eds.), pp. 422–430. Springer International Publishing, Cham, 2020.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Evaluating the explanations given by post-hoc XAI approaches on tabular data is a challenging prospect, since the subjective judgement of explanations of tabular relations is non trivial in contrast to e.g. the judgement of image heatmap explanations. In order to quantify XAI performance on categorical tabular data, where feature relationships can often be described by Boolean functions, we propose an evaluation setting through generation of synthetic datasets. To create gold standard explanations, we present a definition of feature relevance in Boolean functions. In the proposed setting we evaluate eight state-of-the-art XAI approaches and gain novel insights into XAI performance on categorical tabular data. We find that the investigated approaches often fail to faithfully explain even basic relationships within categorical data.

  @inproceedings{tritscher2020evaluation, abstract = {Evaluating the explanations given by post-hoc XAI approaches on tabular data is a challenging prospect, since the subjective judgement of explanations of tabular relations is non trivial in contrast to e.g. the judgement of image heatmap explanations. In order to quantify XAI performance on categorical tabular data, where feature relationships can often be described by Boolean functions, we propose an evaluation setting through generation of synthetic datasets. To create gold standard explanations, we present a definition of feature relevance in Boolean functions. In the proposed setting we evaluate eight state-of-the-art XAI approaches and gain novel insights into XAI performance on categorical tabular data. We find that the investigated approaches often fail to faithfully explain even basic relationships within categorical data.}, address = {Cham}, author = {Tritscher, Julian and Ring, Markus and Schlör, Daniel and Hettinger, Lena and Hotho, Andreas}, booktitle = {Foundations of Intelligent Systems}, editor = {Helic, Denis and Leitner, Gerhard and Stettinger, Martin and Felfernig, Alexander and Raś, Zbigniew W.}, keywords = {xai}, pages = {422–430}, publisher = {Springer International Publishing}, title = {Evaluation of Post-hoc XAI Approaches Through Synthetic Tabular Data}, year = 2020 }