research

2025

1. RLC

   

   Reinforcement Learning under State and Outcome Uncertainty: A Foundational Distributional Perspective

   Larry Preuett, Qiuyi Zhang, and Muhammad Aurangzeb Ahmad

   In Reinforcement Learning Conference (RLC), 2025

   Abs arXiv Bib HTML PDF Code

   In many real-world planning tasks, agents must tackle uncertainty about the environment’s state and variability in the outcomes of any chosen policy. We address both forms of uncertainty as a first step toward safer algorithms in partially observable settings. Specifically, we extend Distributional Reinforcement Learning (DistRL) — which models the entire return distribution for fully observable domains — to Partially Observable Markov Decision Processes (POMDPs), allowing an agent to learn the distribution of returns for each conditional plan. Concretely, we introduce new distributional Bellman operators for partial observability and prove their convergence under the supremum p-Wasserstein metric. We also propose a finite representation of these return distributions via psi-vectors, generalizing the classical alpha-vectors in POMDP solvers. Building on this, we develop Distributional Point-Based Value Iteration (DPBVI), which integrates psi-vectors into a standard point-based backup procedure — bridging DistRL and POMDP planning. By tracking return distributions, DPBVI naturally enables risk-sensitive control in domains where rare, high-impact events must be carefully managed.

   @inproceedings{preuett2025distributional,
     category = {selected_research},
     title = {Reinforcement Learning under State and Outcome Uncertainty: A Foundational Distributional Perspective},
     author = {Preuett, Larry and Zhang, Qiuyi and Ahmad, Muhammad Aurangzeb},
     booktitle = {Reinforcement Learning Conference (RLC)},
     year = {2025},
   }

2. arXiv

   

   Algorithmic Fairness in AI Surrogates for End-of-Life Decision-Making

   Muhammad Aurangzeb Ahmad

   2025

   Abs arXiv Bib HTML PDF

   Artificial intelligence surrogates are systems designed to infer preferences when individuals lose decision-making capacity. Fairness in such systems is a domain that has been insufficiently explored. Traditional algorithmic fairness frameworks are insufficient for contexts where decisions are relational, existential, and culturally diverse. This paper explores an ethical framework for algorithmic fairness in AI surrogates by mapping major fairness notions onto potential real-world end-of-life scenarios. It then examines fairness across moral traditions. The authors argue that fairness in this domain extends beyond parity of outcomes to encompass moral representation, fidelity to the patient’s values, relationships, and worldview.

   @misc{ahmad2025algorithmic,
     category = {selected_research},
     title = {Algorithmic Fairness in AI Surrogates for End-of-Life Decision-Making},
     author = {Ahmad, Muhammad Aurangzeb},
     year = {2025},
   }

2023

1. Surgical Clinics

   

   Show Your Work: Responsible Model Reporting in Health Care Artificial Intelligence.

   Muhammad Aurangzeb Ahmad, and Carly Marie Eckert

   The Surgical Clinics of North America, 2023

   Abs Bib HTML

   Standardized and thorough model reporting is an integral component in the development and deployment of machine learning models in health care. Model reporting includes sharing multiple model performance metrics and incorporating metadata to provide the necessary context for model evaluation. Thorough model reporting addresses common concerns about artificial intelligence in health care including model explainability, transparency, fairness, and generalizability. All stages in the model development lifecycle, from initial design to data capture to model deployment, can be communicated openly to stakeholders with responsible model reporting. Physician involvement throughout these processes can ensure clinical concerns and potential consequences are considered.

   @article{ahmad2023show,
     category = {selected_research},
     title = {Show Your Work: Responsible Model Reporting in Health Care Artificial Intelligence.},
     author = {Ahmad, Muhammad Aurangzeb and Eckert, Carly Marie},
     journal = {The Surgical Clinics of North America},
     volume = {103},
     number = {2S},
     pages = {e1--e11},
     year = {2023},
     url = {https://www.sciencedirect.com/science/article/abs/pii/S0039610923000348?via%3Dihub},
   }

2. ICHI

   

   Validation of a Hospital Digital Twin with Machine Learning

   Muhammad Aurangzeb Ahmad, Vijay Chickarmane, Farinaz Sabz Ali Pour, and 2 more authors

   2023

   Abs Bib PDF

   Recently there has been a surge of interest in developing Digital Twins of process flows in healthcare to better understand bottlenecks and areas of improvement. A key challenge is in the validation process. We describe a work in progress for a digital twin using an agent based simulation model for determining bed turnaround time for patients in hospitals. We employ a strategy using machine learning for validating the model and implementing sensitivity analysis.

   @article{aurangzeb2023validation,
     category = {selected_research},
     title = {Validation of a Hospital Digital Twin with Machine Learning},
     author = {Ahmad, Muhammad Aurangzeb and Chickarmane, Vijay and Pour, Farinaz Sabz Ali and Shariari, Nima and Dutta Roy, Taposh},
     booktitle = {2023 IEEE 11th International Conference on Healthcare Informatics (ICHI)},
     pages = {465--469},
     year = {2023},
     organization = {IEEE},
     url = {https://arxiv.org/pdf/2303.04117},
   }

2021

1. IEEE ICHI

   

   Interpretable phenotyping for electronic health records

   Christine Allen, Juhua Hu, Vikas Kumar, and 2 more authors

   In 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI), 2021

   Abs Bib PDF

   Datasets from Electronic Health Records (EHRs) are increasingly large and complex, creating challenges in their use for predictive modeling. The two major challenges are large-scale and high-dimensionality. One of the common way to address the large-scale challenge is through use of data phenotypes: clinically relevant characteristic groupings that can be expressed as logical queries (e.g., “senior patients with diabetes”). With the increasing use of machine learning across the continuum of care, phenotypes play an important role in modeling for population management, clinical trials, observational and interventional research, and quality measures. Yet, phenotype interpretation can often be difficult and require post-hoc clarifications from experienced clinicians. For example, detailed analysis may be needed to find that all patients in a a phenotype are diabetic seniors with complications from previous surgery. Moreover, the high-dimensionality problem is often addressed either separately or simultaneously with phenotyping by dimension reduction methods that may further hinder interpretability. In this paper, we introduce the notion of interpretable data phenotypes generated by an unsupervised learning technique. Methods are designed to disambiguate relative feature memberships, thus facilitating general clinical validation, and alleviating the problem of high-dimensionality. The empirical study applies the proposed unsupervised interpretable phenotyping method to a real world healthcare dataset (MIMIC), then uses hospital length of stay as a reference prediction task. The results demonstrate that the proposed method produces phenotypes with improved interpretability and without diminishing the quality of prediction results.

   @inproceedings{allen2021interpretable,
     category = {selected_research},
     title = {Interpretable phenotyping for electronic health records},
     author = {Allen, Christine and Hu, Juhua and Kumar, Vikas and Ahmad, Muhammad Aurangzeb and Teredesai, Ankur},
     booktitle = {2021 IEEE 9th International Conference on Healthcare Informatics (ICHI)},
     pages = {161--170},
     year = {2021},
     organization = {IEEE},
     url = {https://ieeexplore.ieee.org/abstract/document/9565719},
   }

2014

1. Book

   

   Predicting real world behaviors from virtual world data

   Muhammad Aurangzeb Ahmad, Cuihua Shen, Jaideep Srivastava, and 1 more author

   2014

   Abs Bib HTML

   There is a growing body of literature that focuses on the similarities and differences between how people behave in the offline world vs. how they behave in these virtual environments. Data mining has aided in discovering interesting insights with respect to how people behave in these virtual environments. The book addresses prediction, mining and analysis of offline characteristics and behaviors from online data and vice versa. Each chapter will focus on a different aspect of virtual worlds to real world prediction e.g., demographics, personality, location, etc.

   @book{ahmad2014predicting,
     category = {selected_book},
     title = {Predicting real world behaviors from virtual world data},
     author = {Ahmad, Muhammad Aurangzeb and Shen, Cuihua and Srivastava, Jaideep and Contractor, Noshir},
     year = {2014},
     publisher = {Springer},
   }