Combining Sources and Leveraging Contexts Citation Mazaheri, Bijan Henrik Socrates (2024) Combining Sources and Leveraging Contexts. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/7t4d-xg91. https://resolver.caltech.edu/CaltechTHESIS:06042024-004345614 Abstract In this thesis we discuss two levels of knowledge beyond regression and classification. The first involves the identification of exchangeable scenarios or individuals from which causal relationships can be ascertained. We discuss one key difficulty of this task, the "Multi-Source Conundrum," which emerges whenever data is merged from multiple sources. This motivates the "Principle of Limited Latent Classes," an assumption which allows us to introduce new algorithms for deconfounding and causal structure learning. The second level of knowledge involves the expansion from contextual exchangeability to contextual synthesis. We will study a paradox of nontransitivity that occurs when combining multiple contexts, as well a demonstrating robustness gains from using context-dependent counterfactuals as training features. Through these points, we present contextual synthesis as a new frontier with promise for advances in out-of-distribution robustness, fairness, and privacy. Item Type: Thesis (Dissertation (Ph.D.)) Subject Keywords: Causal Inference, Expert Graphs, Causal Discovery, Mixture Models, Distribution Shift Degree Grantor: California Institute of Technology Division: Engineering and Applied Science Major Option: Computing and Mathematical Sciences Thesis Availability: Public (worldwide access) Research Advisor(s): Bruck, Jehoshua (co-advisor) Schulman, Leonard J. (co-advisor) Thesis Committee: Eberhardt, Frederick D. (chair) Janzing, Dominik Rabani, Yuval Bruck, Jehoshua Schulman, Leonard J. Defense Date: 15 August 2023 Non-Caltech Author Email: bijanmazaheri (AT) gmail.com Funders: Funding Agency Grant Number NSF Graduate Research Fellowship DGE‐1745301 Amazon AI4Science Fellowship UNSPECIFIED Record Number: CaltechTHESIS:06042024-004345614 Persistent URL: https://resolver.caltech.edu/CaltechTHESIS:06042024-004345614 DOI: 10.7907/7t4d-xg91 Related URLs: URL URL Type Description https://proceedings.mlr.press/v213/gordon23a.html Publisher Article adapted for Chapter 4. https://arxiv.org/abs/2311.07454 arXiv Article adapted for Chapter 5. https://arxiv.org/abs/2311.06840 arXiv Article adapted for Chapters 6 and 7. https://arxiv.org/abs/2107.07054 arXiv Article adapted for Chapter 7. https://proceedings.mlr.press/v216/mazaheri23a.html Publisher Article adapted for Chapter 8. https://proceedings.mlr.press/v134/gordon21a/gordon21a.pdf Related Document Article mentioned in Chapter 9. https://papers.nips.cc/paper/2020/file/24368c745de15b3d2d6279667debcba3-Paper.pdf Related Document Related Document. https://arxiv.org/abs/2007.08101 Related Document Related Document. https://www.biorxiv.org/content/10.1101/518878v1 Related Document Article material adapted for Chapter 3 motivation. ORCID: Author ORCID Mazaheri, Bijan Henrik Socrates 0000-0001-9690-8686 Default Usage Policy: No commercial reproduction, distribution, display or performance rights in this work are provided. ID Code: 16496 Collection: CaltechTHESIS Deposited By: Bijan Mazaheri Deposited On: 06 Jun 2024 21:49 Last Modified: 17 Jun 2024 20:32 Thesis Files PDF - Final Version See Usage Policy. 2MB Repository Staff Only: item control page

Combining Sources and Leveraging Contexts Thesis by Bijan Mazaheri In Partial Fulfillment of the Requirements for the Degree of Doctorate of Philosophy CALIFORNIA INSTITUTE OF TECHNOLOGY Pasadena, California 2024 Defended August 15, 2023 ii © 2024 Bijan Mazaheri ORCID: 0000-0001-9690-8686 Some rights reserved. This thesis is distributed under a Creative Commons Attribution-NonCommercial-ShareAlike License iii ACKNOWLEDGEMENTS This work was generously funded by the Caltech Computing and Mathematical Sciences Department, a National Science Foundation Graduate Research Fellowship, an Amazon AI4Science Fellowship, and funding from both Prof. Leonard Schulman and Prof. Jehoshua Bruck’s labs. I would like to thank my undergraduate professors at Williams, especially Prof. Duane Bailey, Prof. Keith McPartland, and Prof. Daniel Aalberts, who laid the foundation and frameworks from which I have built my intuition. I would like to thank my collaborators, Dr. Spencer Gordon, Prof. Yuval Rabani, Dr. Siddharth Jain, Prof. Matthew Cook, Dr. Michaela Hardt, Dr. Atalanti Mastakouri, and Dr. Dominik Janzing, who have helped direct research problems and fill out my understanding of related fields. I would like to thank my advisors, Prof. Jehoshua Bruck and Prof. Leonard Schulman, for their support throughout my time at Caltech. I thank my running friends, Brandon Wolfe, Jonathan Koch, Adolfo Carvalho, Kevin Horchler, Nick Spector, Chris Myers, and Tony Tomsich, for sharing many miles across California (and Germany) and occasionally enduring small mathematical lectures based on whatever I was fixated on at the time. I would like to thank my late father, Dr. Mohsen Mazaheri, for inspiring me to pursue a Ph.D. and his continued belief in me. He is dearly missed. In addition, I would like to thank my family, Anna Mazaheri, Nasser Mazaheri, and Said Mazaheri, for their support throughout this process. Finally, I would like to thank my wife, Rebecca, who I met during my first year at Caltech and has been a sounding board for my ideas and shared in my frustrations, sorrows, and triumphs. iv ABSTRACT In this thesis we discuss two levels of knowledge beyond regression and classification. The first involves the identification of exchangeable scenarios or individuals from which causal relationships can be ascertained. We discuss one key difficulty of this task, the “Multi-Source Conundrum,” which emerges whenever data is merged from multiple sources. This motivates the “Principle of Limited Latent Classes,” an assumption which allows us to introduce new algorithms for deconfounding and causal structure learning. The second level of knowledge involves the expansion from contextual exchangeability to contextual synthesis. We will study a paradox of nontransitivity that occurs when combining multiple contexts, as well a demonstrating robustness gains from using context-dependent counterfactuals as training features. Through these points, we present contextual synthesis as a new frontier with promise for advances in out-ofdistribution robustness, fairness, and privacy. v PUBLISHED CONTENT AND CONTRIBUTIONS Gordon, Spencer et al. (2023). “Causal inference despite limited global confounding via mixture models”. In: 2nd Conference on Causal Learning and Reasoning. Bijan Mazaheri participated in the conception of the project, formation of theory, implementation of the experiments, and writing of the manuscript. Mazaheri, Bijan, Spencer Gordon, et al. (2023). Causal Discovery under Latent Class Confounding. arXiv: 2311.07454 [cs.LG]. Bijan Mazaheri participated in the conception of the project, formation of theory, and writing of the manuscript. Mazaheri, Bijan, Siddharth Jain, Matthew Cook, et al. (2023). Omitted Labels in Causality: A Study of Paradoxes. arXiv: 2311.06840 [cs.LG]. Bijan Mazaheri participated in the conception of the project, formation of theory, implementation of the experiments, and writing of the manuscript. Mazaheri, Bijan, Atalanti Mastakouri, et al. (July 2023). “Causal information splitting: Engineering proxy features for robustness to distribution shifts”. In: Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence. Ed. by Robin J. Evans and Ilya Shpitser. Vol. 216. Proceedings of Machine Learning Research. PMLR, pp. 1401–1411. url: https://proceedings.mlr.press/ v216/mazaheri23a.html. Bijan Mazaheri participated in the conception of the project, formation of theory, implementation of the experiments, and writing of the manuscript. Mazaheri, Bijan, Siddharth Jain, and Jehoshua Bruck (2021). “Synthesizing New Expertise via Collaboration”. In: 2021 IEEE International Symposium on Information Theory (ISIT), pp. 2447–2452. doi: 10.1109/ISIT45174.2021.9517822. Bijan Mazaheri participated in the conception of the project, formation of theory, implementation of the experiments, and writing of the manuscript. Jain, Siddharth et al. (2019). “Short tandem repeats information in tcga is statistically biased by amplification”. In: BioRxiv, p. 518878. Bijan Mazaheri participated in the writing of the manuscript. vi TABLE OF CONTENTS Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Published Content and Contributions . . . . . . . . . . . . . . . . . . . . . . v Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v List of Illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi I Introduction 1 Chapter I: The Hierarchy of Knowledge . . . . . . . . . . . . . . . . . . . . 2 1.1 The Hierarchy of Knowledge . . . . . . . . . . . . . . . . . . . . . 3 1.2 Summary and Structure . . . . . . . . . . . . . . . . . . . . . . . . 5 Chapter II: Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Causal Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3 Structural Causal Models . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 Information Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 14 II Level 3: Causality 16 Chapter III: Limited Latent Classes . . . . . . . . . . . . . . . . . . . . . . . 3.1 The Many-Source Conundrum . . . . . . . . . . . . . . . . . . . . . 3.2 The Principle of Limited Latent Classes . . . . . . . . . . . . . . . . 3.3 Mixture Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter IV: Confounder Identification . . . . . . . . . . . . . . . . . . . . . 4.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Applying a k-MixProd run . . . . . . . . . . . . . . . . . . . . . . . 4.4 Combining Runs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Collections of runs . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter V: Structure Learning under Global Confounding . . . . . . . . . . . 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Additional Background . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Rank Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Algorithm Phase I . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Phase II: Handle FP Edges . . . . . . . . . . . . . . . . . . . . . . . 5.6 Utilizing the Phase I Graph for k-MixProd . . . . . . . . . . . . . . 5.7 Empirical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 17 18 20 23 23 28 29 30 34 39 41 41 46 48 50 53 54 58 vii 5.8 Deferred Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 IIILevel 4: Wisdom 64 Chapter VI: Graphically Modeled Contexts . . . . . . . . . . . . . . . . . . 65 6.1 The Domain Expertise Paradox . . . . . . . . . . . . . . . . . . . . 65 6.2 Simpson’s Paradox . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 6.3 Omitted Label Contexts . . . . . . . . . . . . . . . . . . . . . . . . 68 6.4 Networks of Contexts . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.5 Context-Based Features . . . . . . . . . . . . . . . . . . . . . . . . 71 Chapter VII: Expert Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.2 Aggregations of Rankings and Soft Rankings . . . . . . . . . . . . . 77 7.3 Curl and the curl condition . . . . . . . . . . . . . . . . . . . . . . 83 7.4 When the curl condition is sufficient . . . . . . . . . . . . . . . . . 84 7.5 Synthetic experts . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Chapter VIII: Causal Information Splitting . . . . . . . . . . . . . . . . . . . 93 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 8.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 8.3 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 8.4 Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 8.5 Context Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8.6 Causal Information Splitting . . . . . . . . . . . . . . . . . . . . . . 105 8.7 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 8.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 8.9 Deferred Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 8.10 Experimental Details . . . . . . . . . . . . . . . . . . . . . . . . . . 115 IV Discussion 117 Chapter IX: Rethinking Dimensionality and Errors . . . . . . . . . . . . . . 118 9.1 Diverse Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 9.2 The Necessity (and Blessing?) of Dimensionality . . . . . . . . . . . 118 9.3 The Information Theoretical Value of Errors . . . . . . . . . . . . . 119 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 viii LIST OF ILLUSTRATIONS Number Page 1.1 A dependency chart of this thesis. Technical chapters, which are adapted from papers, are shaded in blue. . . . . . . . . . . . . . . . . 6 2.1 Examples of active and inactive paths between 𝑆 and 𝑇. Conditioned vertices are filled in. . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.2 (a) 𝐶 exerts influence on 𝑌 , but not on the covariate 𝑋. (b) shows both a direct and indirect causal path from 𝑋 to 𝑌 . (c) shows a DAG of an instrumental variable setup. (d) shows an example of a DAG on which the front-door criterion can be applied. . . . . . . . . . . . . . 13 3.1 (a) A DAG representing the many-source conundrum, with 𝐷 representing the dataset source. (b) A DAG representing an attempted solution to the many-source conundrum, involving an unobserved latent class that is simpler than 𝐷. (c) A DAG representing 𝑘-MixProd. 19 4.1 The reduction process of conditioning on COND to create an instance of 𝑘-MixProd. A Bayesian network with four vertices 𝑉1 , 𝑉6 , 𝑉9 , 𝑉13 and their corresponding disjoint Markov boundaries are indicated. . 25 4.2 We can decompose Pr𝑢 (𝑣 1 , 𝑣 2 , 𝑣 3 , 𝑦, 𝑣 4 , 𝑣 5 ) = Pr𝑢 (𝑣 1 , 𝑣 2 , 𝑣 3 ) Pr𝑢 (𝑦 | 𝑣 1 , 𝑣 2 ) Pr𝑢 (𝑣 4 | 𝑦, 𝑣 3 ) Pr𝑢 (𝑣 5 | 𝑦, 𝑣 4 ). 𝑈 and any other variables in the graph are omitted for clarity. . . . . . . . . . . . . . . . . . . . 32 4.3 An alignment spanning tree of the default assignment 𝑎 0 (COND𝑎0 arbitrarily assigns all Markov boundaries to 0) and six other central runs. The runs on the left cover all possible assignments to MB(𝑋2 ) ∈ {(0, 0), (0, 1), (1, 0)}, while maintaining the default assignment to MB(𝑋1 ) to allow alignment with 𝑎 0 . The right runs similarly cover all possible assignments to MB(𝑋1 ), aligned at 𝑋2 . . . . . . . . . . . 36 5.1 The goal is to learn the graph structure 𝔊 without observing 𝑈. . . . 41 5.2 An illustration of an FP edge after Phase I due to a large set of immoral descendants. The population variable 𝑈 is omitted to avoid clutter. While 𝑉𝑖 and 𝑉 𝑗 are d-separated by C = ∅ no IPA can be made because all of the leftover vertices are immoral descendants. . . . . . . . . . . 52 ix 5.3 The given graph has an FP edge between 𝑉3 and 𝑉4 , indicated by a dashed line, caused by a large set of immoral descendants (shown in red). Conditioning on 𝑉7 , 𝑉11 , 𝑉12 creates an instance of 𝑘-MixProd on T𝑖 𝑗 , X1 , X2 . Notice that 𝑉7 , 𝑉11 , 𝑉12 are all in FB(𝑉3 , 𝑉4 ), which means that the Pr(T𝑖 𝑗 | 𝑉7 , 𝑉11 , 𝑉12 , 𝑢′) recovered by 𝑘-MixProd will not be sufficient for detecting the FP edge. This obstacle will be solved in Subsection 5.6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 5.4 The results of Test 1. . . . . . . . . . . . . . . . . . . . . . . . . . . 59 5.5 Results from Test 2. In blue, we show correctly returned edges. In red, we show edges that were returned which are not in the true model. The opacity of the lines show the percentage of the time that the edge was returned (ideally, we would want faint red lines and strong blue lines). To the right of the graph, we show a table of the frequency of returning the edge colored according to the same scheme. . . . . . . 61 5.6 The results of Test 3. Horizontal ticks represent the median accuracy for recovering edges (blue) or lack of edges (orange). A violin plot is also shown, representing the density of results over 20 iterations at each 𝑝 (probability of adding an edge). . . . . . . . . . . . . . . . . 61 6.1 (a) A causal DAG depicting confounding from a common cause 𝑋. (b) The causal DAG that “severs” 𝑋 → 𝑇 by reweighting for exchangeability. (c) The causal DAG depicting the effect of a omitted label context 𝐶 which has been conditioned on. . . . . . . . . . . . 69 6.2 The Condorcet paradox as an aggregation of rankings. . . . . . . . . 70 6.3 (a) 𝑈 exerts unobserved influence on 𝑌 , but not on the covariate 𝑋, meaning an auxiliary training task predicting 𝑌 using 𝑋 can remove the effect of 𝑈. (b) shows a DAG of an instrumental variable setup. (c) shows an unobserved active path for which an auxiliary training task predicting 𝑌 using 𝑋 can capture and isolate information about 𝑈. 71 6.4 A visual proof for Lemma 24. . . . . . . . . . . . . . . . . . . . . . 73 7.1 Examples of decision boundaries from classifiers trained on pairs of differently colored Gaussians. Regions of nontransitivity are shaded in grey. In the case on the right, this includes both the center and the outer region of all outliers. The classifiers on the left are trained with sklearn’s linear SVM, and on the right they are trained with sklearn’s nonlinear kernel SVM. . . . . . . . . . . . . . . . . . . . . . . . . . 76 x 7.2 A demonstration of the inductive step in the proof for Lemma 29. The weights on the LHS are the aggregate probabilities (𝑦 (𝑖) , 𝑦 ( 𝑗) ) that we wish to generate, while the numbers within each vertex 𝑦 (𝑖) specify 𝑝𝑖 . The weights of the graphs on the RHS are given by Equation 7.11, with adjusted (re-normalized) probabilities 𝑝 [−𝑘] specified within the vertices. Three subgraphs are highlighted in red, which represent the smaller sets of labels which can be decomposed according to the inductive hypothesis. . . . . . . . . . . . . . . . . . . . . . . . . . . 82 7.3 An example of a triangulated cycle. . . . . . . . . . . . . . . . . . . 85 7.4 The shortest path from 𝑠 → 𝑡, given by 𝑠 → 𝑦 (1) → 𝑡 gives 𝑓𝑑 (𝑠, 𝑡) < 0.7. The shortest path from 𝑡 to 𝑠, given by 𝑡 → 𝑦 (2) → 𝑆 gives 𝑓𝑑 (𝑠, 𝑡) > 1 − .9 = .1. Together, we get 𝐹 (𝑠, 𝑡) ∈ (0.1, 0.7). . . . . . 89 7.5 An example of how to create 𝜁-accurate synthetic experts that make up any curl consistent graph by adding additional paths. All edge weights given in color have a + 4𝜁 added to their weight that has been omitted to reduce clutter. Here, the cycle on 𝑦 (1) , 𝑦 (2) , 𝑦 (3) is created by: (1) Adding shortest paths (shown in red on the other part of the cycle) with total weight equal to the desired edge weight 𝑓 (𝑒) + 2𝜁 . (2) Adding shortest reverse paths (shown in blue on the inner part of the cycle) with total weight equal to 1 − 𝑓 (𝑒) + 2𝜁 . . . . . . . . . . . . . 91 7.6 An example of how we cannot always achieve any combination of synthetic expert bounds. Here, a choice of just under .7 for all the synthetic experts (given in dashed lines) would violate the curl condition on cycle 𝑦 (1) → 𝑦 (2) → 𝑦 (3) → 𝑦 (1) . . . . . . . . . . . . . 91 8.1 Examples of the 𝔊+ considered for the paper. (a) shows a generic setup where 𝑈1 is a hidden cause of 𝑌 , and 𝑈2 , 𝑈3 are hidden effects. (b) shows a plausible model explaining the success of our real-data experiment in Section 8.7. . . . . . . . . . . . . . . . . . . . . . . . 98 8.2 A diagram showing separability. . . . . . . . . . . . . . . . . . . . . 100 8.3 𝑉𝐺 ∈ VGOOD , 𝑉𝐵 ∈ VBAD . 𝑉𝐴 ∈ VAMBIG is a linear transformation of two components, 𝑉𝐴(𝐺) , 𝑉𝐴(𝐵) , which are good and bad respectively. . . 106 8.4 Results from our experiments on synthetic data. Single standard deviation confidence intervals are shaded in the corresponding colors. 109 xi LIST OF TABLES Number Page 1.1 The hierarchy of knowledge . . . . . . . . . . . . . . . . . . . . . . 3 6.1 Three tables discussed in this paper. . . . . . . . . . . . . . . . . . . 67 8.1 Comparison of out-of-domain (2021) performance via mean of accuracy.110 8.2 Comparison of out-of-domain (2021) performance on predicting high income via F1 scores. . . . . . . . . . . . . . . . . . . . . . . . . . 116 8.3 Comparison of in-domain (2019) performance on predicting high income via Accuracies. . . . . . . . . . . . . . . . . . . . . . . . . . 116 Part I Introduction 1 2 Chapter 1 THE HIERARCHY OF KNOWLEDGE For the past 20 years, researchers have repeatedly demonstrated the predictive capabilities of machine learning (ML) and the generative capabilities of artificial intelligence (AI). While the foundations for these technologies date back to the 1960s, their recent success stems in part from access to large comprehensive datasets (Schmidhuber, 2022). Knowledge is often limited to the context of the data that lead to it, so larger datasets that cover more contexts are generally more powerful than smaller and more specific ones. Unfortunately, broadening the scope of data is not the panacea that this simplistic view promises. For example, consider a broad-reaching survey of alcohol usage, followed by a longitudinal study on the health outcomes of those individuals. For many years, researchers observed what they called a “J-shaped curve,” in which optimal mortality rates emerged with one to two alcoholic drinks per day. We now know that this relationship is driven by the influence of patient condition on recommendations for alcohol consumption. Specifically, many people with pre-existing health conditions were advised to avoid alcohol, leading to a significantly less healthy population of teetotalers. When these conditions are adjusted for, the “J” shaped curve disappears, showing a monotonically negative impact of alcohol (T. S. Naimi et al., 2005; Chikritzhs, Fillmore, and Stockwell, 2009). Evidently, comparisons between health recommendations must happen between similarly healthy groups of populations. Such “case-controlled studies” (G. W. Imbens and Rubin, 2015) involve a constriction in data-set size in order to isolate a more correct causal relationship. It is not always possible to extend data in a case-controlled fashion. Within science, data is often collected in sets called “batches,” giving rise to “batch effects” that can induce spurious correlations in the aggregate dataset. Extending beyond the laboratory, human-subject studies extend their scope by relaxing entry criteria, giving rise to studies on diverse populations. Historical studies similarly expand by gathering data over extended time-frames. In all of these processes, big data comes hand in hand with heterogeneity, which helps some tasks and hurts others. In order to understand the role of data heterogeneity, we must carefully consider the 3 context that our data comes from and the target contexts on which we hope to apply our conclusions. Such generalization highly nuanced, so we will begin by breaking generalization goals into a “hierarchy of knowledge.” 1.1 The Hierarchy of Knowledge We will describe four “levels” of knowledge generalization which are separated by the key challenges involved and the mathematical techniques we use to overcome them. This hierarchy will bear a close resemblance to “Pearl’s Causal Ladder,” which partitions causality into three “rungs”: (1) Associations, (2) Interventions, and (3) Counterfactuals (Judea Pearl, 2009). The hierarchy of knowledge will attempt a similar partitioning, instead separated by the scope of generalization errors. Level Name 0 Association Tools Distance, Feature Importance Curve-fitting, Regularization Capabilities Anecdotes, Association between examples Generalized prediction, Pattern-detection Transfer learning, Domain adaptation How does it fail? 1 Synthesis 2 Transferal Data re-weighting 3 Causality Causal models Counterfactuals, Universal relationships, Interventions Incorrect causal modeling, incomplete measurement of the system 4 Wisdom Combining contextbased conclusions Imagination, Understanding beyond the data Contradiction Bad data Over-fitting, Under-fitting, Insufficient data Insufficient data overlap Table 1.1: The hierarchy of knowledge Level 0: Association All knowledge stems from experiences and events. The simplest form of thought is anecdotal, involving matching new experiences to previous ones. An example of such reasoning would be: “My grandmother is 100 years old and she drinks a glass of wine every night.” Level 0 hinges on the determination of a distance-metric between samples. Principal component analysis (PCA) (KPFRS, 1901) and auto-encoders (Kingma and Welling, 2013) make up crucial tools for this level by helping determine latent features that are necessary to capture the variation between instances. While most inference techniques reach beyond this level into pattern-detection, some elementary techniques like 𝑘-nearest neighbors rely only on Level 0 anecdotal evidence. Level 1: Synthesis With enough data, the first level of knowledge involves pattern-detection for generalization out of sample. In Level 1, we seek to separate example-specific information from general trends. These trends can be used for prediction, but their guarantees are limited to the scope of the distribution from which they are drawn. 4 While Level 0 only requires correct data, Level 1’s attainability hinges on data quantity: More data allows for the generalization of more complex relationships. Out of sample generalization has been well studied, with the most common tool being regularization (Sugiyama, 2015). Level 2: Transferal While Level 1 generalizes patterns from samples to the distribution from which those samples were generated, transferal involves making conclusions about a different, but related target distribution. This process is known as domain adaptation (DA). More generally, transfer learning extends this notion to the transfer of tasks (Weiss, Khoshgoftaar, and D. Wang, 2016). The most common approach to domain adaptation involves re-weighting source-data to “look” like a target distribution. DA is made possible through assumptions on the shared information between 𝑝 and 𝑞. Two such assumptions are covariate shift (Shimodaira, 2000) and label shift (Schweikert et al., 2008). Both settings involve assuming a constant “label function” Pr(𝑌 | X) when prediction label 𝑌 using covariates X. For example, a covariate shift task may involve adapting skin cancer predictions from Europe to Africa — while the function we seek to learn remains constant while the focus shifts to darker-skinned populations. Level 3: Causality Causal inference can be thought of as a special case of DA that involves a shift from “observational” data to an “interventional” target context. Such a setting differs from Level 2 because interventions have a directional impact on a system. For example, going to the beach will not warm the weather. Interventional settings are difficult to predict without further modeling because they often depart from the domain of the data we have at hand. For example, we have likely not seen the climate-impact of forcing many people to go to the beach in the winter. Instead, causality often falls within the realm of extrapolation. At the core of determining the impact of interventions is the comparison between “exchangeable” settings. In order to determine what constitutes exchangeability we must make use of causal assumptions which usually come in the form of a structural causal model (SCM) (Judea Pearl, 2009; G. W. Imbens and Rubin, 2015; Peters, Janzing, and Schölkopf, 2017). “Causal discovery” automates the process of learning these structures (Squires and Uhler, 2022), but is still subject to data-limitations and restrictive assumptions (such as the “faithfullness” assumption, see Uhler et al., 2013). 5 A primary argument posed by this thesis will be that an additional assumption of inherently low dimensional data, which we will call “the principle of limited latent classes,” can extend the power of SCMs beyond canonical examples where exchangeability is easy to define. The essence of this assumption is that the complexity of unmeasured forces does not continue to grow as we expand the scope of our study. When this assumption holds, tools within mixture models can be added to the toolbox for causal inference. Level 4: Wisdom The fourth level of knowledge involves understanding the process that lead to our observed data and how those processes affect our conclusions. The most common challenge at this level is that of sampling/survivorship bias, which selectively removes certain data points from our purview. This thesis will demonstrate that wisdom falls within it’s own category of generalization by showing how incorrect conclusions can be drawn from standard causal adjustments in settings with sampling bias. While we will not propose a complete solution to Level 4 generalization, we will present results that suggest approaching the problem using multiple contexts with different sampling biases. 1.2 Summary and Structure This thesis will begin with an in-depth study of Level 3 knowledge — particularly in the case of multiple batches or populations, which we call “latent global confounding.” Within this setting, traditional causal inference is insufficient to identify causal relationships. In Chapter 3, we justify the principal of Limited Latent Classes (LLC) and explain how it can be used to address this insufficiency. Chapter 4 and Chapter 5 will present the first known algorithms for harnessing the LLC to deconfound and learn causal structures this setting. We will then progress to the study of Level 4 knowledge. Chapter 6 will begin by explaining a paradox that arises among settings with different types of label bias, demonstrating the breakdown of standard causality adjustments and principles. In Chapter 7, we continue to study this paradox and its implications for networks of experts and high-level “decision fusion.” Chapter 6 also introduces techniques for obtaining Level 4 knowledge, namely that auxiliary training tasks can be used as filters of information. In Chapter 8, we see that carefully engineered counterfactual questions and their contradictions with observed 6 data carry important information for contextually-robust models. Finally, Chapter 9 will discuss the implications of the presented work and the future directions they imply. This thesis consists of high-level conceptual chapters (Chapters 1, 2, 3, 6, 9) which introduce and discuss the ideas in the low-level technical chapters (Chapters 4, 5, 7, 8). A dependency diagram for these chapters is given in Figure 1.1. Chapter 1: Introduction The Knowledge Hierarchy Chapter 2: Background General Notation Causal Inference Information Theory Chapter 6: Graphically Modeled Contexts Omitted Label Paradoxes Context-Based Features Chapter 3: Limited Latent Classes The Many-Source Conundrum Mixture Models Chapter 4: Confounder Identification Chapter 5: Structure Learning Chapter 7: Expert Graphs Chapter 8: Causal Information Splitting Chapter 9: Future Work Figure 1.1: A dependency chart of this thesis. Technical chapters, which are adapted from papers, are shaded in blue. 7 Chapter 2 BACKGROUND 2.1 Notation The capital Latin alphabet will be used to denote random variables. When referring to sets of these variables, we will use bolded font, e.g. X = {𝑋1 , 𝑋2 , . . .}. The variables discussed in this thesis will generally be discrete. The supports, or “alphabets” of these variables will be denoted using caligraphic script, i.e. 𝑋𝑖 ∈ X𝑖 . The cardinality of these supports will be denoted |X| and sometimes just referred to as the cardinality of the variable. Probability distributions on random variables will be denoted Pr(·). Conditional probabilities will be used frequently, for which we will use a lowercase shorthand to denote assignment. For example, 𝑥 denotes 𝑋 = 𝑥 and x denotes X = x. Pr(𝑦 | 𝑥) ∈ [0, 1] is a single probability, whereas Pr(𝑌 | 𝑥) ∈ △ |Y| and Pr(𝑦 | 𝑋) ∈ [0, 1] |X| can both be thought of as vectors indexed by the unspecified assignment to 𝑌 or 𝑋. When these probabilities are parameters of a model, we will use the lowercase Greek alphabet. The majority of graphs in this thesis will be “structural causal models,” which are discussed later in this chapter. Such models are graphs whose vertices are random variables (i.e. capital Latin alphabet). In general, we will try to use U to denote the set of unobserved variables in the system, and V to denote the visible or observed variables. Sometimes, when a specific prediction task is clear, we will also use X, 𝑌 for visible variables, where 𝑌 is a label and X the covariates used to predict that label. In Chapter 7, a different type of directed graph will be discussed in which the vertices are specific classes of a label variable. Vertices in these graphs are given by the lowercase Latin alphabet because they are assignments to a single variable. Sets of these assignments are denoted using calligraphic script, i.e. Y = {𝑦 1 , 𝑦 2 , . . .} or lowercase c = (𝑐 1 , 𝑐 2 , . . .) ⊂ Y for paths and cycles. There will be a few exceptions to these general rules, which will be specified in their relevant chapters. 8 Graph Notation To refer to components of a graph, we will use the following operators: • PA(𝑉), CH(𝑉) will refer to the parents and children of a vertex 𝑉. • AN(𝑉), DE(𝑉) will refer to the ancestors and descendants of 𝑉. AN(𝑉) ∪ {𝑉 } and DE(𝑉) ∪ {𝑉 } are denoted using AN+ (𝑉), DE+ (𝑉) respectively. • MB(𝑉) = PA(𝑉) ∪ CH(𝑉) ∪ PA(CH(𝑉)) will refer to the Markov boundary of 𝑉.1 • NBℓ (𝑉) refers to the undirected distance ℓ neighborhood of 𝑉. As these operators act on graphs, they can specify the graph structure being used in the superscript, e.g. PA𝔊 (𝑉). We will also occasionally write tuples to indicate the intersection of the sets for two vetices, e.g. CH(𝑉, 𝑊) = CH(𝑉) ∩ CH(𝑊). Finally, these operators can also act on sets to indicate the union of the operation, e.g. Ø PA(X) = PA(𝑋) \ X. (2.1) 𝑋∈X Note that our lowercase/uppercase convention can also be applied to operators, e.g. mb(𝑉) denotes an assignment to MB(𝑉). Assignments for these operators can also be obtained from a larger set of assignments using a subscript, e.g. mbc (𝑉) obtains assignments for MB(𝑉) ⊆ C from the assignments of c to C. 2.2 Causal Inference The fundamental problem of causal inference Causality encompasses both counterfactual (what could have been) and hypothetical (what could be) statements. If a patient is given treatment and cured, saying that treatment caused recovery is equivalent to saying that the patient would not have recovered without treatment (a counterfactual). Similarly, recommending an intervention of treatment carries with it the implication that the patient is better off with treatment than without it. One framework for understanding causal inference developed by G. W. Imbens and Rubin (2015) involves defining “potential outcomes” of an intervention. The “fundamental problem” of causal statements is that counterfactual and hypothetical outcomes are inherently unobserved. We only have one world and we therefore only 1The Markov boundary is the minimal set that d-separates 𝑉 from all other vertices (Judea Pearl, 2009). 9 see one potential outcome (e.g. the patient recovers with treatment). We do not see the counterfactual — for all we know, the treatment may have had no effect and the patient would have recovered either way. We will develop the rest of this section using random variables that represent the potential outcomes of whether the patient receives (𝑌 (1) ) or does not receive 𝑌 (0) treatment. The treatment effect is therefore defined to be the difference between these potential outcomes: 𝑌 (1) − 𝑌 (0) . Average Treatment Effects One way to access a treatment effect is with a twin. For example, if two identical individuals are given different treatments (e.g. one receives treatment and one receives a control), then we can interpret the outcome of one twin as the counterfactual for the other. In the absence of identical individuals, we can relax our goal to the identification of an average treatment effect (ATE) between two “exchangeable” groups: ATE B E[𝑌 (1) ] − E[𝑌 (0) ]. (2.2) The simplest way to ensure exchangeability is a randomized controlled trial (RCT), first introduced in Fisher and Wishart (1930). RCTs assign treatments randomly to a population, ensuring that there are no common-causes for the treatment and the outcome. RCTs are not always possible. It is not ethical to withhold potentially life-saving medication from a sick patient and voters do not generally embrace experimental public policy. In such settings, we must determine how to ascertain causality without active interventions. Instrumental Variables Natural causes of a treatment can mimic the randomization of RCTs. Such causes are known as instrumental variables and are often used in economics (J. Angrist and G. Imbens, 1995). A famous real-world example involved the use of birthdays to estimate the economic impact of a military draft (J. D. Angrist and A. B. Krueger, 2001). To remove confounding between socioeconomic status and draft eligibility (or perhaps avoidance), economists instead studied the relationship between the birthdays of draft-age individuals and their eventual economic well-being. Birthdays are unrelated to economic status, but functioned as the primary mechanism under which the draft took place. Any relationship between birthday and economic well-being could therefore be attributed to the draft. 10 Inverse Propensity Weighting Exchangeability gives us insight into how we can mathematically “simulate” the outcome of an RCT. Suppose we have access to observational data on patients, some of whom are prescribed a treatment by their doctors. It is common for prescriptions to depend on the condition of the patient — sicker patients are more likely to receive treatment while milder cases are monitored but not necessarily treated. Selective prescription therefore drives unexchangeable treated and untreated patient groups — the treated group is on average significantly worse off than the untreated group. To enforce exchangeability, the health and epidemiological sciences often employ “inverse propensity weighting” (IPW) (G. W. Imbens and Rubin, 2015). Weighting each datapoint by the inverse of the patient’s likelihood of receiving their corresponding treatment enforces matching (weighted) distributions of severity between the treated and untreated groups. IPW techniques rely on assumptions that are typical within epidemiological settings, but do not hold in generality. For example, common effects of both treatment and outcome do not induce confounding and therefore need not be a part of the definition of exchangeability. Furthermore, (Judea Pearl, 2009) explains that adjusting for these “collider” variables can actually lead to incorrect calculations of causal effect. The do-calculus has therefore been developed to study causal identifiability with respect to more general graphical models, known as Structural Causal Models, or SCMS (Judea Pearl, 2009). 2.3 Structural Causal Models Structural Causal Models (Judea Pearl, 2009; Peters, Janzing, and Schölkopf, 2017) graphically model causal systems with arrows representing causation: e.g. 𝐴 → 𝐵 indicates “𝐴 causes 𝐵.” While many dynamical systems may contain cyclic dependencies, restricting theory to acyclic graphs (known as directed acyclic graphs or DAGs) is popular.2 SCMs represent a data-generating process, in which each variable is a function of its causes and an independent source of noise. This process implies a factorization of the joint probability distribution, Ö Pr(v) = Pr(𝑣 | pa𝔊 (2.3) v (𝑣)). 𝑣∈v 2 Cyclic systems can usually be reduced to acyclic graphs by time-indexing the variables (i.e. replacing vertices V with V1 , . . . , V𝑇 ) and enforcing the directionality of time by forbidding any “future to past” or “present to present” arrows. 11 We will call this the “Markov factorization,” and the satisfaction of this factorization is referred to causal Markov condition. We sometimes say that V is “Markovian” in graph 𝔊. Response Functions When we have more than one cause of a variable 𝑌 , the function determining 𝑌 relative to a subset of the causes is indexed by the excluded causes. For example, consider the SCM in Figure 2.2 (a). We can write Pr(𝑌 = 1 | 𝑐) = Γ𝑐 (𝑋). (2.4) We refer to these indexed functions as response functions, for which we reserve the capital Greek alphabet. “Causal sufficiency” represents the assumption that all of the variables needed to determine a casual relationship are known and measured. Figure 2.2 (b), (c), and (d) show violations of causal sufficiency due to unobserved causes of 𝑌 . In these cases, the response functions that govern the values of 𝑌 are not known when they are indexed by 𝑈. When the distribution of these response functions depends on the covariates, such as in Figure 2.2 (b), (c) and (d), the relationships between the label and the covariates become confounded. The independence between covariates and their response functions in unconfounded systems is referred to as the “principle of independent mechanisms” in Peters, Mooij, et al., 2014. Independence Properties For any DAG 𝔊 = (V, E), we call P ⊆ E a path if it connects 𝐴 and 𝐵 with no repeated vertices. The path is directed if it obeys the directions of the edges and undirected if it does not. Both directed and undirected paths in a causal DAG can result in dependencies between variables. To understand the conditions for dependence/independence, we will use the concepts of active and inactive paths, which are defined relative to a conditioning set (Judea Pearl, 2009; Peters, Janzing, and Schölkopf, 2017). Intuitively, whether a path is active or not indicates whether it “carries dependence” between the variables. In the absence of any conditioning, a path between 𝐴 to 𝐵 is active if it is directed or if it is made up of two directed paths from a common cause along that path. In the same unconditioned setting, inactive paths are paths that contain a collider, i.e. a vertex for which the path has two inward pointing arrows such as 𝐴 → 𝐶 ← 𝐵. 12 Active Paths Inactive Paths 𝑆 𝑉1 𝑇 𝑆 𝑉1 𝑇 𝑆 𝑉1 𝑇 𝑆 𝑉1 𝑇 𝑆 𝑉1 𝑇 𝑆 𝑉1 𝑇 𝑆 𝑉1 𝑇 𝑆 𝑉1 𝑇 𝑉2 𝑉2 Figure 2.1: Examples of active and inactive paths between 𝑆 and 𝑇. Conditioned vertices are filled in. When we are given a conditioning set Z, conditional dependencies differ from unconditional ones. Active paths can be blocked (thus becoming inactive paths) if some vertex 𝑍 along the path between 𝐴 to 𝐵 is included in C. Similarly, inactive paths with a collider 𝐶 can become unblocked by including 𝐶 or some descendant of the collider variable in the conditioning set Z. If two variables 𝐴, 𝐵 contain no active paths (they may contain inactive paths), then we say they are d-separated (𝐴 ⊥⊥𝑑 𝐵 | Z). If two variables contain at least one active path for a conditioning set Z, we say that they are d-connected. See Figure 2.1 for some examples of active and inactive paths. The data-generating process of structural causal models allows them to be factorized according to the causal Markov condition, from which we conclude that d-separation always implies independence. In the opposite direction, it is possible that two d-connected variables by chance exhibit some unexpected statistical independence via cancellation. Under tje assumption of faithfulness (Peter Spirtes, Clark Glymour, Scheines, and David Heckerman, 2000), d-connectedness always implies statistical dependence. Interventions Intervening on a “treatment” 𝑋 has an effect on “outcome” 𝑌 that can be calculated by removing all backdoor paths, or paths between 𝑋 and 𝑌 which are active but not directed. An example of a backdoor path is given in Figure 2.2 (b). The idea 13 𝐶 (a) 𝑋 𝑈 𝐶 𝑌 (b) 𝑋 𝑌 (c) 𝐼 𝑋 𝑈 𝑌 (d) 𝑋 𝑍 𝑌 Figure 2.2: (a) 𝐶 exerts influence on 𝑌 , but not on the covariate 𝑋. (b) shows both a direct and indirect causal path from 𝑋 to 𝑌 . (c) shows a DAG of an instrumental variable setup. (d) shows an example of a DAG on which the front-door criterion can be applied. of the backdoor adjustment is to not allow 𝐶 to vary by changing 𝑥, since 𝑋 = 𝑥 is driven by an outside intervention rather than the natural variation of 𝐶. This fixes the distribution on 𝐶 to the observed marginal probability distribution despite conditioning on 𝑋. Do interventions can therefore be interpreted graphically as shifting from the DAG in Figure 2.2 (b) 𝔊, to the DAG in (a) 𝔊 𝑋 (i.e. the graph with all incoming edges to 𝑋 removed). One way to derive the backdoor adjustment is to modify the observed probability distribution Pr(·), which is Markovian in 𝔊𝑏 , to the intervened distribution Pr (𝑎) (·), which is Markovian 𝔊𝑎 . The “do intervention” in this setting is given by Pr(𝑦 | do(𝑥)) := Pr (𝑎) (𝑦 | 𝑥) = Pr (𝑎) (𝑦, 𝑥) Pr (𝑎) (𝑥) (2.5) By marginalizing over 𝐶 and applying the Markov factorization for 𝔊𝑎 , we get Í Pr (𝑎) (𝑐) Pr (𝑎) (𝑥) Pr (𝑎) (𝑦 | 𝑥, 𝑐) ∑︁ (𝑎) Pr(𝑦 | do(𝑥)) = 𝑐∈C = Pr (𝑐) Pr (𝑎) (𝑦 | 𝑐, 𝑥). (𝑎) Pr (𝑥) 𝑐∈C (2.6) Finally, we replace all of the probabilities Pr (𝑎) (·) with the observed probabilities Pr(·) to get the commonly accepted backdoor adjustment for this setting. ∑︁ Pr(𝑦 | do(𝑥)) = Pr(𝑐) Pr(𝑦 | 𝑐, 𝑥). (2.7) 𝑐∈C The Frontdoor Criterion In the absence of the ability to break a backdoor path, we can instead infer the affect of interventions through the front-door criterion (Judea Pearl, 2009), provided that we have access to all directed paths from 𝑋 to 𝑌 and provided that those paths themselves are unconfounded. An example of this setting is given in Figure 2.2(d). Here, we can first infer Pr(𝑧 | do(𝑥)) = Pr(𝑧 | 𝑥) (for all 𝑥) by noting that no backdoor paths 14 exist between 𝑋 and 𝑍. We can then compose the effect of this intervention with Pr(𝑦 | do(𝑧)) to calculate Pr(𝑦 | do(𝑥)): ∑︁ Pr(𝑦 | do(𝑥)) = Pr(𝑧 | 𝑥) Pr(𝑦 | do(𝑧)). (2.8) 𝑧∈Z Pr(𝑦 | do(𝑧)) does contain a backdoor path via 𝑍 ← 𝑋 ← 𝑈 → 𝑌 , but has an observable adjustment set (𝑋) which blocks this path. Hence ∑︁ ∑︁ Pr(𝑦 | do(𝑥)) = Pr(𝑧 | 𝑥) Pr(𝑥) Pr(𝑦 | 𝑧, 𝑥). (2.9) 𝑧∈Z 2.4 𝑥∈X Information Theory This thesis will also use concepts from information theory, with H( 𝐴) indicating the entropy of 𝐴, I( 𝐴 : 𝐵) = H( 𝐴) − H( 𝐴 | 𝐵) indicating the mutual information between 𝐴, 𝐵, and I( 𝐴 : 𝐵 : 𝐶) = I( 𝐴 : 𝐵) − I( 𝐴 : 𝐵 | 𝐶) indicating the interaction information between 𝐴, 𝐵, 𝐶. Lemma 1 (Chain Rule, (Cover, 1999)). For sets of variables A, B, and subset B′ ⊂ B I(A : B) = I(A : B′) + I(A : B \ B′ | B′) (2.10) Definition 1 (Cover, 1999). For sets of variables A, B, C, the interaction information is defined, I(A : B : C) := I(A : B) − I(A : B | C). (2.11) A key property of interaction information is that it is symmetric to permutations in its three inputs, I(A : B : C) = H(A, B, C)+H(A)+H(B)+H(C)−H(A, B)−H(B, C)−H(C, A). (2.12) Another key property is that interaction information can be either positive or negative, differing from mutual information which is non-negative. The following lemmas will describe two common situations in which we can expect positive and negative interaction information. Lemma 2. Given three sets of random variables A, B, C if A ⊥⊥ C | B then I(A : B : C) ≥ 0. 15 Graphically, Lemma 2 represents a situation where conditioning on B d-separates A and C (i.e. B is a separating set of A and B). The symmetry of interaction information means that it is not important which set of variables is the separating set. The data processing inequality uses each “step” of an active path to upper bound the mutual information. Lemma 3 (Data Processing Inequality (modified from Cover, 1999)). If A ⊥⊥ C | B, D then I(A : C | D) ≤ min(I(A : B | D), I(B : C | D)) ≤ H(B | D). (2.13) Part II Level 3: Causality 16 17 Chapter 3 LIMITED LATENT CLASSES In this chapter, we introduce the “many-source conundrum,” a difficulty that arises when combining data from multiple sources. Though this problem cannot be addressed by classical methods in causal inference, the addition of a new assumption, “the principle of limited latent classes,” allows us to employ mixture models. This chapter will provide some musings as to why this assumption is both reasonable and likely and show how it allows mixture models to become a central tool for solving the many-source conundrum and, more generally, handling unobserved, but large-scale confounding. 3.1 The Many-Source Conundrum The motivation for this problem emerged from using a combined TCGA cancer database (Jain et al., 2021). Our study used the accuracy of ML classifiers as evidence for the presence of signals; if a feature was predictive of cancer-type, we reasoned that it must be related to the development of that cancer. Midway through some exciting results, we discovered that we could distinguish between two types of equipment (“D” and “W” amplification) with correct identification (recalls) of 71% and 86% respectively (Jain et al., 2019). Consequently, our models could leverage this signal to make predictions about any attribute which correlated with this equipment use. Unfortunately, cancer-type was among these equipment-correlated attributes, raising concern about the true origin of our cancer-type prediction accuracy. We had stumbled apon confounding due to heterogeneity, which emerges when equipment or environmental details vary between “batches” of data. This issue extends outside batched experiments, coming into play whenever data spans multiple populations or environments. Counfounding effects sometimes lay dormant, only to awaken during the search for new signals. The equipment-specific signal in TCGA had evaded detection because it was limited to non-coding repeat regions, which are seldom studied in genome-wide association studies (GWAS). This emergence of novel confounding mechanisms is exacerbated by the unprecedented power of modern ML. The novelty of these phenomena conceal them from the intuitions of domain knowledge, making them silent killers of scientific rigor. 18 Known methods for correcting confounding from heterogeneity involve restricting analysis to homogeneous sub-populations using either case-controlled studies (Schlesselman, 1982) or covariate adjustments (Judea Pearl, 2009). Restriction-based approaches reduce dataset size — limiting the power of many data-demanding machine-learning techniques. This limitation is even more severe in scientific disciplines that rely on batch-based data, upper bounding dataset-size by batch-size. In such settings, the promises of big data become elusive as we fail to grow the size of homogeneous sub-components. We refer to this problem as the “many-source conundrum.” 3.2 The Principle of Limited Latent Classes To handle the many-source conundrum, we will introduce an assumption that allows us to reduce heterogeneity to a limited number of latent classes. This assumption will allow homogeneous components to grow with combined data so long as they are identifiable. Beyond Graphical Assumptions Techniques for handing confounding make up the primary task of causal inference. The backdoor and front-door adjustments and the criteria for their application have been synthesized into a general theory of causal identifiability, known as the “docalculus” (Judea Pearl, 2009). Instrumental variables do not fully identify causal effects, but can be added to the list of techniques for partially determining causal influence. Traditional approaches to confounding are built on one of two components of a causal model: 1. Observable confounders, for which we can apply the backdoor adjustment. 2. Observable variables which are not affected by the confounder, either along causal paths from the treatment to the effect (for the front door adjustment) or as parents of the treatment (for instrumental variables). With respect to these classical approaches, the situation presented by the many-source conundrum is grim. If each lab chooses both their equipment (which affects the DNA sequencing output) and the types of cancer they study, then both sequencing output (𝑆) and cancer type (𝐶) are affected by a common cause via the data source (𝐷), graphically modeled as 𝑆 ← 𝐷 → 𝐶. In more complicated systems, all of the 19 𝑈 𝐷 𝑉1 𝑉2 𝑉3 (a) 𝑉4 𝑉1 (b) 𝑉2 𝑈 𝑉3 𝑉4 𝑉1 𝑉2 𝑉3 𝑉4 (b) Figure 3.1: (a) A DAG representing the many-source conundrum, with 𝐷 representing the dataset source. (b) A DAG representing an attempted solution to the many-source conundrum, involving an unobserved latent class that is simpler than 𝐷. (c) A DAG representing 𝑘-MixProd. variables in the system could potentially be affected by 𝐷, as shown in Figure 3.1 (a). In the absence of any variables unaffected by 𝐷, we must apply the backdoor adjustment, giving rise to the many-source conundrum. The key problem in the many-source conundrum is that the observed class (the dataset source) is too rich. Hence, a reasonable approach is to assume that some simpler unobserved 𝑈 (often called a latent class) is sufficient to control for the confounding induced by heterogeneity. Figure 3.1 (b) shows the causal DAG for this scenario. Examples of such a 𝑈 could include private attributes like health status or artificially constructed classes such as individual price sensitivity. Such a 𝑈 would allow us, in principal, to merge and re-partition data from many sources into homogeneous “unconfounded components.” Unfortunately, the existence of a latent class is perhaps even worse than 𝐷, because we have now also eliminated the possibility of the backdoor adjustment. Evidently, we must depart from graphical notions of causal identifiability. Limited Latent Classes The weakness of the graphical perspective is that it poses no restrictions on the relationships between the variables in the graph. In the context of the many-source conundrum, this worst case analysis ignores the fact that limiting the cardinality of 𝑈 would also limit 𝑈’s ability to completely hide the causal dynamics. As it turns out, reality appears to be far from the worst case. In principal, our backdoor adjustment could apply an even finer partitioning than 𝐷, such as assigning each datapoint to its own partition. While a skeptical critic can always assert the inherent uniqueness of every individual (perhaps aided by the cautionary tale from earlier in this chapter), merged datasets have many success stories and science has made measurable progress. 20 Embedded in our ability to progress to even Level 0 knowledge is the assumption that individual data points share some common information that can be synthesized. A consequence of this fact is limited confounding induced by combining datapoints and datasets. For a discrete confounder 𝑈, one way of limiting this confounding is by limiting |U|, thereby assuming that the datapoints can be limited to a small number of latent classes. We call this assumption the “principle of Limited Latent Classes” (LLC). We can hypothesize the reasons why LLC appears to be an inherent property of our world. One observation is that humans and animals are the product of an evolutionary trajectory, so data will inherently have latent classes governed by common ancestors. Clusters also form from large branching effects driven by outside influences, such as migration or a change in environmental conditions. Such events are often driven by low dimensional variables, such as geography and time. Finally, the existence of these latent classes may be a meta-driver for our desire to gather and study the data in the first place. Regardless the source of this phenomenon, it appears to exist and it arms us with the statistical power to solve the many-source conundrum. 3.3 Mixture Models Under the LLC assumption, we now turn our attention to deconfounding by identifying the probability distribution of 𝑈 and Pr(X, 𝑌 | 𝑢). In the case of discrete 𝑈, such a problem can be viewed as identifying a mixture of distributions, where each mixture “source” corresponds to a specific assignment 𝑢. There are two problems within mixture models: (1) Learning the model, namely, producing any model consistent with (or close to) the observations; (2) Identifying the model, namely, producing the true model (or one close to it) up to permutations in the source label. The feasibility of the identification problem hinges on a one-to-one mapping between the observed statistics and the model’s parameters. When using the resulting model to deconfound causal relationships, it is imperative that the joint probability distribution with the confounder be identified. Parametric vs. Graphical Assumptions Say we observe that the probability density function of 𝑌 | 𝑋 forms a bimodal distribution. If we knew the noise in our SCM was additive and Gaussian, it would be possible to infer the response functions Γ𝑢 (𝑋) by recovering a mixture of Gaussians. Hence, parametric assumptions can enable the recovery of response functions. 21 In the absence of parametric assumptions, graphical assumptions can also be used to recover the mixture by harnessing independence properties implied by the causal Markov condition. The simplest case is identifying discrete 𝑘-mixtures of product distributions, which we call 𝑘-MixProd. Such a setting is described by V = 𝑉1 , . . . , 𝑉𝑛 and a latent global confounder or “source” 𝑈 such that 𝑉𝑖 ⊥⊥ 𝑉 𝑗 | 𝑈 for all 𝑖, 𝑗, for which Figure 3.1 (c) shows a causal DAG. The key complexity parameter for identifiability is 𝑘 = |U|. This setting well studied (Allman, Matias, and Rhodes, 2009; Jon Feldman, Ryan O’Donnell, and Rocco A Servedio, 2008; Chen and A. Moitra, 2019; S. Gordon, B. H. Mazaheri, et al., 2021). At a high level, the identification algorithms for 𝑘-MixProd are primarily method-of-moments approaches that harness information from dependencies in the marginal distribution on 𝑋 (over 𝑈). To achieve varying levels of stability, different approaches require different numbers of independent variables to be affected by 𝑈. Scope Requirements for Identification Of course, it is possible for 𝑈 with sufficiently large 𝑘 to completely control the distribution on X. For example, a cardinality of 𝑘 = 2𝑛 would be sufficient for binary 𝑋𝑖 ∈ {0, 1} to assign each sequence in X to a latent class in 𝑈. Such a powerful 𝑈 could generate any desired probability distribution on X by simply controlling the probability distribution on 𝑈. Limiting 𝑘, however, limits the space of marginal probability distributions on X, eventually giving rise to identifiability. Under a cardinality bound 𝑘 on the support of 𝑈, Allman, Matias, and Rhodes, 2009 showed that 𝑛 ≥ Ω(log(𝑘)) is sufficient for the generic identification of 𝑘-MixProd. In other words, other than a Lebesgue measue 0 set of exceptions, most instances of 𝑘-MixProd have a one-to-one correspondence with their observed statistics (the probability distribution on X marginalized over 𝑈) and generating model (up to a set of 𝑘! models with permuted labels of 𝑈). For guaranteed identifiability, a lower bound in the case of independently distributed variables was shown to be 2𝑘 − 1 in Li et al., 2015. Tahmasebi, Motahari, and Maddah-Ali, 2018 demonstrated that 𝑛 ≥ 2𝑘 − 1 in conjunction with a separation condition in the distributions of 𝑋𝑖 | 𝑈, is a sufficient upper bound. S. Gordon, B. H. Mazaheri, et al., 2021 recently showed that 𝑛 ≥ 3𝑘 − 1 is able to give further stability guarantees for the problem. The chapters of this section arbitrarily build results on S. Gordon, B. Mazaheri, Leonard J Schulman, et al., 2020 in Chapter 4 22 and Allman, Matias, and Rhodes, 2009 in Chapter 5, but our methods easily extend to stronger identifiability conditions with modifications in the sparsity requirements. The requirements on 𝑛 imply a duality in the scope of 𝑈: while variables that are unaffected by 𝑈 are useful for the removal of 𝑈’s influence, variables which are affected by 𝑈 help identify that influence. So long as the power of 𝑈 is limited (i.e. limited cardinality in the case of discrete 𝑈), it is likely we will find ourselves in one of the two scenarios. Mixtures as a Tool The solutions to 𝑘-MixProd make up the main tool which we use for the remainder of this section. In Chapter 4 we explain how to leverage these results in Bayesian networks by solving instances of mixtures of products in conditional probability distributions. Our solution will require knowledge of the DAG on the visible variables, for which we give a structure learning algorithm in Chapter 5. 23 Chapter 4 CONFOUNDER IDENTIFICATION This chapter will introduce the first algorithm for identifying mixtures of Bayesian network distributions, which allows for the deconfounding of latent classes. These results are published in S. Gordon, B. Mazaheri, Yuval Rabani, et al., 2023. 4.1 Problem Statement A Bayesian network is a directed acyclic graph 𝔊 = (V, E), on a set of |V| = 𝑛 random variables. A corresponding Bayesian network distribution (BND) is a probability distribution on the random variables that is Markovian on the graph. That is to say, Î𝑛 the joint distribution on the variables can be factored as 𝑖=1 Pr(𝑉𝑖 = 𝑣 𝑖 | pa(𝑉𝑖 )) where pa(𝑉𝑖 ) is the assignment to the parents of 𝑉𝑖 . A 𝑘-MixBND on 𝔊 is a convex combination, or “mixture”, of 𝑘 BNDs. We represent this situation graphically by a single unobservable random variable 𝑈 with edges to each of the variable 𝑉 ∈ 𝐺. Here, 𝑈 is referred to as a “source” variable with range 1, . . . , 𝑘 and the variables in 𝔊 are referred to as the “observables.” The main complexity parameter of the problem is 𝑘, representing the number of mixture constituents or “sources.” In this chapter we study the identification problem for 𝑘-MixBNDs. Specifically, given the graph 𝔊, and given a joint distribution Pr(V) on the variables (vertices), recover (a) the mixture weights (probability of each source), up to a permutation of the constituents, and (b) for every mixture source and for every vertex 𝑉, its conditional distribution given each possible setting to the parents of 𝑉. This task identifies the joint probability distribution Pr(𝑈, V) up to the 𝑘! permutations in the label 𝑈. Identification will be shown by giving an algorithm that reduces the 𝑘-MixBND problem into a series of calls to a 𝑘-MixProd oracle. 𝑘-MixBND models are not always identifiable, as further discussed in Assumptions below. Thus, another contribution of this chapter is to establish a sufficient setting to guarantee identifiability. Assumptions The following assumptions are used throughout this chapter. 1. We have access to a 𝑘-MixProd oracle requiring 𝑁mp variables that are independent within each source. As different algorithms have different 24 requirements for the number of independent variables, we will keep our results agnostic to these requirements. The most efficient published algorithm is given in S. Gordon, B. H. Mazaheri, et al., 2021, which requires 𝑁mp = 3𝑘 − 3 variables and time complexity exp(𝑘 2 ). Recent work improves the complexity bound to exp(𝑘 log 𝑘) Spencer L. Gordon et al., 2023. 2. The observable variables in our BND are binary and discrete. While a number of papers have focused on continuous or large-alphabet settings, we restrict our focus to the simplest setting of binary, discrete variables. 3. The mixture is supported on ≤ 𝑘 sources. If the hidden variable 𝑈 has unrestricted range (Specifically, range 𝑘 = 2𝑛 would be enough), the model is rich enough to describe any probability distribution on V, making identification impossible. The question is therefore one of trading 𝑘 against the sample and computational complexity of an algorithm (and the degree of the network). 4. The underlying Bayesian DAG is sufficiently sparse. In order to reduce 𝑘-MixBND to 𝑘-MixProd we need sufficiently many variables that can be separated from each other by conditioning on disjoint Markov boundaries (example in Fig. 4.1, definition in Sec. 4.2). As a result, the complexity of the algorithm is exponential in the size of a Markov boundary. Both for complexity and in order to keep 𝑛 small, a bound on the maximum degree Δ is required. We require 𝑛 ≥ (Δ + 1) 4 𝑁mp .1 5. The resulting product mixtures are non-degenerate. Even in mixtures of graphs with sparse structure (in particular the empty graph—the 𝑘-MixProd problem), the 𝑘-MixBND can be unidentifiable if the mixture components are insufficiently distinct, (e.g., trivially, a mixture of identical sources generates the same statistics as a single source.) Past work has used conditions such as 𝜁-separation in S. Gordon, B. H. Mazaheri, et al., 2021 to ensure that matrices representing the parameters for each source are well-conditioned. These are not always necessary conditions; characterizing necessary conditions is a difficult question tackled in part in S. L. Gordon and L. J. Schulman, 2022. 6. The DAG structure representing conditional independence properties within each source, or a common supergraph of these structures, is known. It is often the case that domain knowledge provides an understanding of the causal DAG. 1 If the skeleton of 𝔊 happens to be a path, then we only need a milder condition that 𝑛 ≥ 2𝑁 For details see S. Gordon, B. Mazaheri, Yuval Rabani, et al., 2023. mp . 25 𝑈 𝑉2 𝑉1 𝑉6 𝑉4 𝑉5 𝑉3 𝑉8 𝑉9 𝑉7 "Conditioning set"   , , , , COND := , , , 𝑉2 𝑉3 𝑉4 𝑉7 𝑉8 𝑉10 𝑉11 𝑉12 𝑉10 𝑉13 𝑉11 𝑉12 "Independent set"  I := , , , 𝑉1 𝑉6 𝑉9 𝑉13 𝑈 | COND 𝑉1 | 𝑉2, 𝑉3 𝑉6 | 𝑉4, 𝑉8 𝑉9 | 𝑉7, 𝑉10, 𝑉11 𝑉13 | 𝑉12 Figure 4.1: The reduction process of conditioning on COND to create an instance of 𝑘-MixProd. A Bayesian network with four vertices 𝑉1 , 𝑉6 , 𝑉9 , 𝑉13 and their corresponding disjoint Markov boundaries are indicated. If the causal DAG is unknown, we are faced with a different problem commonly known as “Causal Discovery” (see C. Glymour, K. Zhang, and P. Spirtes, 2019 for a recent survey.) In the setting of dataset merging, it is likely that the structure can be learned from an individual dataset. An algorithm for learning causal structure in the universal confounding/latent class setting is also given in Chapter 5. Like in Anandkumar, D. Hsu, et al., 2012, the presented algorithm will actually only require a supergraph of the true structure. Hence, some uncertainty in knowledge of the graph can be tolerated. In fact, the algorithm also works even if the different components of the 𝑘-MixBND use slightly different causal graphs. Summary of contributions Theorem 1. Our algorithm identifies a 𝑘-MixBND distribution with on a graph of 2 maximum degree Δ and of size 𝑛 ≥ Ω(𝑁MP Δ4 ), using 𝑂 (𝑛2Δ ) calls to an oracle for the 𝑘-MixProd problem. For an exact statement see Theorem 2. The algorithm will be built on the insight that conditioning on a set of Markov boundaries COND ⊂ V of I ⊂ 𝑉 induces within-source independence (that is, 𝑉𝑖 ⊥⊥ 𝑉 𝑗 | 𝑈, COND for all 𝑉𝑖 , 𝑉 𝑗 ∈ I). This describes an instance of 𝑘-MixProd for which we can identify the joint probability distribution Pr(I, 𝑈 | COND). See 26 Figure 4.1 for an illustration. Recovering Pr(I, 𝑈 | COND) for some I ⊂ V is insufficient to recover the full joint 2 probability distribution Pr(V, 𝑈). Hence, we execute a set of 𝑂 (𝑛2Δ ) “runs” of a 𝑘-MixProd oracle on differing I, COND and assignments to their Markov boundaries and synthesize information gained from these runes into the joint probability distribution. The first challenge is to handle symmetries in permutation of the output labels of 𝑈 by “aligning” the outcomes of these runs. The second challenge is to remove the conditioning of C from each run. We do this by synthesizing the results of many runs with a procedure we call “Bayesian unzipping.” Our key contributions can be summarized by these “alignment” and “unzipping” procedures, as well as the notion of a “good collection of runs” that allows for the successful application of these sub-processes. Organization The rest of the chapter is organized as follows. In Section 4.1 we outline the literature background of the problem. In Section 4.2 we give some Bayesian network notation. In Section 4.3 we formally develop the notion of a “run,” which calls a 𝑘-MixProd oracle. In Section 4.4 we explain how the output of the “runs” is combined to get the desired mixture parameters. This section details the processes of alignment and Bayesian unzipping. Section 4.5 explains what is necessary in a group of runs in order for the algorithm to succeed, which provides a framework for defining algorithms in terms of sets of runs. Background To our knowledge, the only other attempt at detailing a multiple-run reduction to 𝑘-MixProd is Anandkumar, D. Hsu, et al., 2012, which gives an algorithm for mixtures of Markov random fields—i.e., undirected graphical models. As both papers make use of boundary conditioning to induce independence and a form of “alignment,” our contribution can be thought of as both an improvement and an extension to the directed graph case. While Anandkumar, D. Hsu, et al., 2012 require a single variable that is independent from the rest of the structure for alignment, our algorithm develops the notion of “good collections of runs” to eliminate this restriction – a contribution which may have implications in the Markov random field setting as well. Additional complications arise for directed graphs because the outputs of the 𝑘-MixProd subroutine are conditioned on their Markov boundaries while the desired parameters are only conditioned on their parents. Finally, we note 27 that Anandkumar, D. Hsu, et al., 2012 only guarantees identification of second order marginal probabilities, which is insufficient for causal identification. 2 Other related work Kivva et al., 2021 contains as a special case a reduction to the 𝑘-MixProd problem. Their goal is to learn a causal graphical model with latent variables, but with a very different structure on the visible and latent variables. They allow for a DAG of latent variables with visible children (which is learned as part of their algorithm); on the other hand, they require that there be no causal relations between visible variables. In our work, the structure on the latent variables is trivial (since there is a single latent variable), but the structure on the visible variables is arbitrary. Characterizing identifiability in the generalization of both these settings in which we allow structure on both the visible and latent portion of the graph is a nice problem beyond the scope of this chapter. Another similarly motivated paper is A. Kumar and G. Sinha, 2021, which studies inference of a certain kind of MixBND, in which the structure of the Bayesian network is known, but the data collected is a mixture over some 𝑚 unknown interventional distributions. The authors give sufficient conditions for identifiability of the network and of the intervention distributions. At a technical level, the papers are not closely related. 𝑘 is not a parameter in their work, and instead what is essential is an “exclusion” assumption which says that each variable has some value to which it is not assigned by any of the interventions. Some other loosely related work includes learning hidden Markov models (D. Hsu, Kakade, and T. Zhang, 2012; Anandkumar, D. J. Hsu, and Kakade, 2012; Sharan et al., 2017), an incomparable line of work to our question, but with somewhat similar motivation. In the same vein, some papers study learning mixtures of Markov chains from observations of random paths through the state space Batu, Guha, and Kannan, 2004; Gupta, R. Kumar, and Vassilvitskii, 2016. These models, too, differ substantially from the models addressed in this chapter, and pose very different challenges. Literature on causal structure learning (P. Spirtes et al., 2000; C. Glymour, K. Zhang, and P. Spirtes, 2019) answers the question of identifying the presence of hidden confounders. Fast Causal Inference (FCI) harnesses observed conditional independence to learn causal structure, which can detect the presence of unobserved variables when the known variables are insufficient to explain the observed behavior. 2We also mention that Anandkumar, D. Hsu, et al., 2012 introduced the idea of a sparse local separator; if this can be adapted to the directed-graph case one might be able to somewhat relax assumption 4. We do not attempt this in this chapter. 28 This literature includes the MDAG problem in which the DAG structure may depend upon the hidden variable; see Thiesson et al., 1998 for heuristic approaches to this problem. Other related works study causal inference in the presence of visible “proxy” variables which are influenced by a latent confounder (Miao, Geng, and E. T. Tchetgen, 2018; Kuroki and J. Pearl, 2014; B. Mazaheri, Mastakouri, et al., 2023). This has more recently given rise to attempts at deconfounding using multiple causes (D. Heckerman, 2018; Ranganath and Perotte, 2018; Y. Wang and Blei, 2019). The initial assumptions of Y. Wang and Blei, 2019 were shown to be insufficient for deconfounding in Ogburn, Shpitser, and E. J. T. Tchetgen, 2019. This illustrates the necessity of identifying of the joint probability distribution with the confounder. 3 Finite mixture models have been the focus of intense research for well over a century, since pioneering work in the late 1800s (Newcomb, 1886; Pearson, 1894). See, e.g., the surveys Everitt and Hand, 1981; Titterington, Smith, and Makov, 1985; Lindsay, 1995; McLachlan, Lee, and Rathnayake, 2019. 4.2 Preliminaries Markov Boundaries This chapter will repeatedly use the fact that conditioning on the “Markov boundary” of a vertex MB(𝑉) makes 𝑉 conditionally independent from everything else in the graph (J. Pearl, 2014). Definition 2 (Markov Boundary). For a vertex 𝑌 in a DAG 𝔊 = (V, E), the Markov boundary of 𝑌 , denoted MB(𝑌 ), is defined by MB(𝑌 ) B PA(𝑌 ) ∪ CH(𝑌 ) ∪ PA(CH(𝑌 )) \ {𝑌 }. Lemma 4 (See J. Pearl, 2014). For any vertex 𝑉 ∈ V and subset 𝑆 ⊆ 𝑉 \ (MB(𝑉) ∪ {𝑉 }), Pr(𝑉 | MB(𝑉), 𝑆) = Pr(𝑉 | MB(𝑉)). Observation 1. For any 𝑋, 𝑌 ∈ V, 𝑋 ∈ MB(𝑌 ) ⇐⇒ 𝑌 ∈ MB(𝑋) Within-source probabilities It will be easier to write Pr𝑢 (𝑣) = Pr(𝑣 | 𝑢) to give the probability distribution within a source. Finally, here are a few more definitions that will make the upcoming sections simpler. Definition 3 (Top). We will use Top(𝑉) to denote MB(𝑉) \ CH(𝑉). 3Thanks to Betsy Ogburn for her thoughts on this topic. 29 Definition 4 (Depth of a vertex). Given a DAG 𝔊 = (V, E) and any vertex 𝑉 ∈ V, let 𝑑𝔊 (𝑉) be the depth of 𝑉 in 𝔊, i.e. the length of the shortest path from a degree-0 vertex in 𝔊. When 𝔊 is clear from context, we’ll omit the subscript. Definition 5. We’ll introduce a parameter 𝛾(𝐺) which will appear in the complexity of the identification procedure, which is defined by 𝛾(𝐺) B max𝑉 ∈V |MB(𝑉)|. 4.3 Applying a k-MixProd run Our algorithm will induce instances of 𝑘-MixProd through post-selected conditioning. A significant portion of this paper will be accounting for multiple calls (or “runs”) of a 𝑘-MixProd oracle and explaining how their results can be combined. Describing runs We will need to keep track of two crucial elements of each “run” of a 𝑘-MixProd oracle. 1. Which variables ∈ V are passed to our 𝑘-MixProd oracle as independent variables (the independent set). 2. Which variables ∈ V we have conditioned on (the conditioning set) and what values we have post-selected these variables to take. A sufficient conditioning set to induce within-source independence among the independent set is the union of their Markov boundaries. This will be further refined in Subsection 4.5. Definition 6 (Run). A run over a graph 𝔊 = (V, E) is a tuple 𝑎 = (I𝑎 , 𝑓 𝑎 ) where I𝑎 ⊆ V are variables that we will d-separate (within each source) by conditioning on assignments to the set Ø COND𝑎 B MB(𝐼). 𝐼∈I𝑎 The value of the assignment is given by 𝑓 𝑎 : COND𝑎 → {0, 1}. We’ll call I𝑎 the independent set for 𝑎, and COND𝑎 the conditioning set. We will restrict our attention to well-formed runs, i.e. runs for which I𝑎 ∩COND𝑎 = ∅. Definition 7. An individual run 𝑎 = (I𝑎 , 𝑓 𝑎 ) is 𝑁-independent if |I𝑎 | ≥ 𝑁. 30 Superscript notation We’ll write mb𝑎 (𝑉), pa𝑎 (𝑉), ch𝑎 (𝑉) to refer to the assignment to the Markov boundary of 𝑉, parents of 𝑉, and children of 𝑉 as set by run 𝑎.4 In a similar spirit, we’ll occasionally write 𝑣 0 to denote the assignment 𝑉 = 0. Definition 8 (Distribution induced by a run). For any well-formed run 𝑎, the induced distribution on the variables in I𝑎 is denoted by Pr𝑎 (·) = Pr(· | cond𝑎 ), where cond𝑎 is the assignment to COND𝑎 in keeping with our conventions. The outputs of applying a 𝑘-MixProd oracle to Pr𝑎 (I𝑎 ) are a matrix M𝑎 ∈ [0, 1] |I |×𝑘 Í and a vector of mixture weights, 𝜋 𝑎 ∈ [0, 1] 𝑘 (satisfying 𝑢 𝜋𝑢𝑎 = 1) given by 𝑎 𝑎 𝑀𝑖,𝑢 B Pr𝑎 (𝑋𝑖𝑎 = 1 | 𝑈𝑎 = 𝑢) = Pr(𝑋𝑖 = 1 | 𝑈 𝑎 = 𝑢, COND𝑎 ), 𝜋𝑢𝑎 B Pr𝑎 (𝑈 𝑎 = 𝑢) = Pr(𝑈𝑎 = 𝑢 | COND𝑎 ), ∀𝑋𝑖 ∈ I𝑎 , 𝑢 ∈ [𝑘] (4.1) ∀ 𝑗 ∈ [𝑘] where 𝑈 𝑎 is the mixture source distributed over [𝑘] according to Pr(𝑈 | COND𝑎 ). Note that because mixtures are invariant to permutations of mixture component labels, we cannot guarantee correspondence between the labels for the source variables from different runs. Hence the labels of 𝑈 𝑎 map to an unknown permutation of the labels in 𝑈. Alignment of these labels is handled in Section 4.4. 4.4 Combining Runs A single run of the 𝑘-MixProd oracle will not contain sufficient information to learn the parameters of the 𝑘-MixBND problem. Instead we must synthesize information across multiple runs. Aligning source labels across different runs Each run of the 𝑘-MixProd oracle will return Pr𝑎 (𝑉 | 𝑈𝑎 = 𝑢) for some arbitrary permutation 𝑈𝑎 of the variable. We need to align all of the outputs to the same permutation of the source, 𝑈. If the runs overlap on at least one variable with the same mixture probabilities, we can use that “alignment variable” to identify which source corresponds to which set of parameters. In our setup, we will guarantee these alignment variables exist by ensuring that runs have shared vertices in their independent sets whose Markov boundaries have identical assignments. 4Any quantities parameterized by a run will take the parameter as a superscript. 31 Definition 9. 𝑋 ∈ V is separated if for all 𝑢𝑖 ≠ 𝑢 𝑗 ∈ [𝑘], Pr𝑢𝑖 (𝑥) ≠ Pr𝑢 𝑗 (𝑥). Definition 10 (Aligned runs). A pair of runs 𝑎, 𝑏 over independent sets I𝑎 , I𝑏 is alignable if there exists a separated 𝑋 ∈ I (𝑎) ∩ I (𝑏) such that Pr𝑢𝑎 (𝑉 (𝑎𝑏) ) = Pr𝑢𝑏 (𝑉 (𝑎𝑏) ) for all 𝑢 ∈ [𝑘]. We’ll call any such random variable 𝑋 an alignment variable, and use AV(𝑎, 𝑏) to denote the set of all alignment variables for Pr𝑎 and Pr𝑏 . We sometimes say 𝑎 and 𝑏 are “aligned at” 𝑋. Definition 11 (Alignment spanning tree). We say a set of ℓ runs is alignable if there exists an undirected spanning tree over the graph with vertices 𝑎 1 , . . . 𝑎 ℓ and an edge {𝑎𝑖 , 𝑎 𝑗 } whenever AV(𝑎𝑖 , 𝑎 𝑗 ) ≠ ∅. We call this the alignment spanning tree. The alignment step will take the output from alignable runs and permute the mixture labels until the parameters match along each alignment variable. Pseudocode for this procedure is given in Algorithm 1. Algorithm 1: Alignment Input: A set of runs A = {𝑎 0 , . . . , 𝑎 ℓ } with outputs 𝑀 𝑎 , 𝜋 𝑎 for each 𝑎 ∈ A. In addition, we have a spanning tree T = (A, E) and alignment variables (𝑖) ( 𝑗) AV(𝑎𝑖 , 𝑎 𝑗 ) ⊆ I𝑎 ∩ I𝑎 . Output: Pr𝑢𝑎 (I𝑎 ) and Pr𝑎 (𝑢) for each 𝑎 ∈ A and 𝑢 ∈ [𝑘]. 𝑎0 Let Pr𝑎0 (𝑢) ← 𝜋𝑢𝑎0 and Pr𝑢𝑎0 (I𝑎0 ) ← 𝑀−,𝑢 for an arbitrary choice of 𝑎 0 for each edge (𝑎𝑖 , 𝑎 𝑗 ) along a breadth first traversal of T from 𝑎 0 do Choose some 𝑋AV ∈ AV(𝑎𝑖 , 𝑎 𝑗 ). Let 𝑞, 𝑟 give the indices for the alignment variable, i.e. 𝑋AV = 𝑋𝑞𝑎𝑖 and 𝑎 𝑋AV = 𝑋𝑟 𝑗 . 𝑎 𝑎𝑖 Find 𝜎, the permutation on the sources that minimizes 𝑀𝑞,− − 𝜎𝑀𝑟,−𝑗 ∞ . 𝑎 Assign Pr𝑎 𝑗 (𝑈) ← 𝜎𝜋 𝑎 𝑗 and Pr𝑢 𝑗 (I𝑎 ) ← 𝜎𝑀 𝑎 𝑗 . end Bayesian unzipping: recovering parameters per source Recall that our algorithm conditions on Markov boundaries to induce independent variables. Hence, after aligning the sources in runs of the 𝑘-MixProd oracle we will have access to Pr𝑢 (𝑌 | MB(𝑌 )) for each 𝑌 ∈ 𝑉. Our goal is to obtain Pr𝑢 (V), which is described by the parameters Pr𝑢 (𝑌 | PA(𝑌 )) for each 𝑌 ∈ V. Note that Pr𝑢 (𝑦 1 | mb(𝑌 )) = Pr𝑢 (𝑦 1 , mb(𝑌 )) . Pr𝑢 (𝑦 1 , mb(𝑌 )) + Pr𝑢 (𝑦 0 , mb(𝑌 )) (4.2) 32 The terms in this fraction are all of the same form and can be factored according to the DAG into Ö Pr𝑢 (𝑦, mb𝑎 (𝑌 )) = Pr𝑢 (top(𝑌 )) Pr𝑢 (𝑦 | pa𝑎 (𝑌 )) Pr𝑢 (𝑣 𝑎 | 𝑓 𝑎 (PA(𝑉) \ {𝑌 }), 𝑦) . 𝑉 ∈CH(𝑌 ) | {z } Pr𝑢 (ch𝑎 (𝑌 )|top𝑎 (𝑌 ),𝑦) See Figure 4.2 for a concrete example of this decomposition. After substituting this factorization into Equation (4.2) we see that Pr𝑢 (top(𝑌 )) appears in both the numerator and denominator because it is independent of the assignment to 𝑌 . Simplification leaves only the following terms: 1. Pr𝑢 (𝑦 0 | pa𝑎 (𝑌 )) and Pr𝑢 (𝑦 1 | pa𝑎 (𝑌 )), which must sum to 1. 2. Pr𝑢 (ch𝑎 (𝑌 ) | top𝑎 (𝑌 ), 𝑦 0 ) and Pr𝑢 (ch𝑎 (𝑌 ) | top𝑎 (𝑌 ), 𝑦 1 ) which are both the product of the desired parameters of variables later in the topological ordering. We can ensure we have access to these terms by solving for the parameters of 𝑉 ∈ V in a reverse-topological ordering.5 We can substitute 1 − Pr𝑢 (𝑦 1 | pa𝑎 (𝑌 )) for Pr𝑢 (𝑦 0 | pa𝑎 (𝑌 )) in the expanded version of Equation (4.2) to obtain a single equation with only Pr𝑢 (𝑦 1 | pa𝑎 (𝑌 )) as an unknown, which we can then solve. The pseudocode for this process is given in Algorithm 2. 𝑉1 𝑉2 𝑉1 𝑉1 𝑉2 𝑉2 𝑉3 𝑌 𝑉3 𝑌 𝑉5 𝑉4 𝑉3 𝑌 𝑉5 𝑉4 𝑉5 𝑉1 𝑉4 𝑉2 𝑉3 𝑌 𝑉5 𝑉4 Figure 4.2: We can decompose Pr𝑢 (𝑣 1 , 𝑣 2 , 𝑣 3 , 𝑦, 𝑣 4 , 𝑣 5 ) = Pr𝑢 (𝑣 1 , 𝑣 2 , 𝑣 3 ) Pr𝑢 (𝑦 | 𝑣 1 , 𝑣 2 ) Pr𝑢 (𝑣 4 | 𝑦, 𝑣 3 ) Pr𝑢 (𝑣 5 | 𝑦, 𝑣 4 ). 𝑈 and any other variables in the graph are omitted for clarity. Recovering the distribution on sources Now consider some arbitrary run 𝑎 with conditioning cond𝑎 . Since Pr𝑢 (V) = Î 𝑉 ∈V Pr𝑢 (𝑉 | PA(𝑉)), knowing Pr𝑢 (𝑉 | PA(𝑉)) grants us full access to the withinsource probability distribution Pr𝑢 (V) after Bayesian unzipping. From this we can 5We will want to ensure that we only need to unzip parameters from vertices of a bounded depth, which bounds the iterations of this step. Details on how this is done appear in Section 4.5. 33 Algorithm 2: Bayesian Unzipping 2 Input: A collection of runs A of size at most 2𝑂 (Δ ) and their aligned output. For each 𝑉𝑖 and assignment to its parents pa(𝑉𝑖 ) there must be some run with 𝑉𝑖 in its independent set with parents conditioned to pa(𝑉𝑖 ). e 𝑢 (𝑌 | PA(𝑌 )) Output: Pr Fix a topological ordering on the vertices in V, ⟨𝑋1 , 𝑋2 , . . . , 𝑋𝑛 ⟩; for 𝑖 = 𝑛, 𝑛 − 1, . . . , 1 do for each assignment pa(𝑋𝑖 ) to PA(𝑋𝑖 ) do Let 𝑎 be a run in A with pa𝑎 (𝑋𝑖 ) = pa(𝑋𝑖 ).; for 𝑢 = 1, . . . , 𝑘 do for 𝑏 = 0, 1 do if 𝑋𝑖 is a bottom vertex then e 𝑢 (𝑥 𝑏 | pa𝑎 (𝑋𝑖 )) ← Pr e 𝑢𝑎 (𝑥 𝑏 ).; Set Pr 𝑖 𝑖 else e 𝑢 (𝑥 𝑏 | pa(𝑋𝑖 )) ← Set Pr 𝑖 e 𝑢 (ch𝑎 (𝑋𝑖 )|top𝑎 (𝑋𝑖 ),𝑥 1−𝑏 ) e 𝑢𝑎 (𝑥 𝑏 ) Pr Pr 𝑖 𝑖 ; 𝑎 𝑏 𝑎 e 𝑢 (𝑥 ) Pr e 𝑢 (ch (𝑋𝑖 )|top𝑎 (𝑋𝑖 ),𝑥 1−𝑏 )+Pr e 𝑢𝑎 (𝑥 1−𝑏 ) Pr e 𝑢 (ch𝑎 (𝑋𝑖 )|top𝑎 (𝑋𝑖 ),𝑥 𝑏 ) Pr 𝑖 𝑖 𝑖 𝑖 end end end end end we obtain Pr𝑢 (cond𝑎 ) = Pr(cond𝑎 | 𝑢). The 𝑘-MixProd oracle will also return Pr𝑎 (𝑈) = Pr(𝑈 | cond𝑎 ) when run on 𝑎 (after source alignment). Finally, Pr(cond𝑎 ) is directly observable. Combining these terms in Bayes’ rule lets us compute the distribution on 𝑈 (under the assumption of positivity), Pr(𝑢 | cond𝑎 ) Pr(cond𝑎 ) Pr(𝑢) = . Pr(cond𝑎 | 𝑢) Outline of the combination process Combining a set of runs A has four steps. 1. Use a 𝑘-MixProd oracle on Pr(I𝑎 | COND𝑎 ) for each run 𝑎 ∈ A to compute 𝑃(𝑉 | MB(𝑉), 𝑈𝑎 = 𝑢) for all variables 𝑉 ∈ V. 2. Align the parameters obtained from the previous step to ensure that 𝑈 means the same thing across different runs, giving Pr𝑢 (𝑉 | MB(𝑌 )). 3. Recover Pr𝑢 (𝑉 | PA(𝑉)) for each vertex 𝑉 ∈ V via Bayesian unzipping. 34 4. Compute Pr(𝑈) by applying Bayes’ law. The full procedure appears as Algorithm 3. Algorithm 3: The 𝑘-MixBND algorithm 2 Input: A good collection of runs A of size at most 2𝑂 (Δ ) . e 𝑢 (𝑌 | PA(𝑌 )) and Pr(𝑈). e Output: Pr 𝑎 𝑎 e Estimate Pr (I ) for all runs 𝑎 ∈ A. ; for each run 𝑎 ∈ A do Set (𝑀 𝑎 , 𝜋 𝑎 ) ← LearnProductMixture(Pr𝑎 (I𝑎 )); end Run Algorithm 1 to align the sources in the output of all the runs in A.; Run Algorithm 2 to unzip the parameters.; Fix any run 𝑎 ∈ A.; for 𝑢 = 1, . . . , 𝑘 do 𝑎 e 𝑎 e ) e Set Pr(𝑢) ← Pr(𝑢|cond ) Pr(cond .; 𝑎 e 𝑢 (cond ) Pr end 4.5 Collections of runs With the main concepts of source alignment and Bayesian unzipping now defined, our algorithm will primarily consist of finding a good collection of these runs so that these subroutines can be successfully applied to recover the 𝑘-MixBND mixture. Observation 2. Two runs 𝑎, 𝑏 are aligned at 𝑋 ∈ V if and only if 1. 𝑋 ∈ I𝑎 ∩ I𝑏 , 2. mb𝑎 (𝑋) = mb𝑏 (𝑋), i.e, 𝑓 𝑎 (MB(𝑋)) = 𝑓 𝑏 (MB(𝑋)), and 3. 𝑋 is separated given mb𝑎 (𝑋) (equivalently, given mb𝑏 (𝑋)). Definition 12. A collection of runs A covers 𝑋 ∈ V if for every assignment pa(𝑋) to PA(𝑋) there exists a run 𝑎 ∈ A with 𝑋 ∈ I𝑎 and pa(𝑋) = pa𝑎 (𝑋). Definition 13 (A good collection of runs). A collection of well-formed runs A is good if it is (i) alignable via an alignment spanning tree, (ii) every run is 𝑁mp -independent, and (iii) the collection covers every vertex in V. The following is our main result on good collections of runs: 35 Theorem 2. Given a graph with max degree Δ satisfying 𝑛 ≥ 𝑁mp × 𝑂 (Δ4 ), we  can find a set of centers X = 𝑋1 , . . . , 𝑋𝑁mp ⊆ V of size 𝑁mp and depth at most 3𝑁mp , such that Algorithm 4 succeeds in finding a good collection of runs A of size 2 𝑂 (2Δ 𝑛). While any good collection of runs will suffice for our algorithm, Theorem 2 represents conditions under which we can provably obtain such a collection of runs. A generic good collection of runs To prove Theorem 2, we will sketch the construction of a good collection of runs before giving its details. To ensure alignment is possible, we will construct a set of central runs, A𝐶 which we can align to each other and which all other runs will be alignable to.  Definition 14 (Centers, Central Runs). A set of vertices X = 𝑋1 , . . . , 𝑋𝑁mp ⊆ V will be called centers if the Markov boundaries of the vertices in X are disjoint. Given a set of centers X, a run 𝑎 is called a central run if I𝑎 = X. To build these central runs, we will start with a set of 𝑁mp vertices  X = 𝑋1 , 𝑋2 , . . . 𝑋𝑁mp with disjoint Markov boundaries and a maximum depth of 3𝑁mb , whose existence is implied by our degree bounds. An example of four such vertices is given in Figure 4.1. First, we fix a run 𝑎 0 with I𝑎0 = X and mb𝑎0 (X) being chosen arbitrarily where MB(X) B ∪ 𝑋𝑖 ∈X MB(𝑋𝑖 ). We will refer to this assignment mb𝑎0 (X) as the default assignment. Each run in 𝑎 ∈ A𝐶 will have the same independent set I𝑎 = X and will agree with 𝑎 0 on the assignment to all of the conditioning set other than the Markov boundary of some 𝑋𝑖 ∈ X, i.e. 𝑓 𝑎 (𝑉) = 𝑓 𝑎0 (𝑉) ∀𝑉 ∈ MB(X) \ MB(𝑋𝑖 ). (4.3) The central runs will span over all assignments mb(𝑋𝑖 ) to MB(𝑋𝑖 ) for each 𝑋𝑖 ∈ X. We’ll write each such run as 𝑎 0 [mb(𝑋𝑖 )]. n o Definition 15. A𝐶 B {𝑎 0 } ∪ 𝑎 0 [mb(𝑋𝑖 )] : 𝑖 ∈ [3𝑘 − 3], mb(𝑋𝑖 ) ∈ {0, 1}MB(𝑋𝑖 ) . 36 0 1 1 0 𝑋1 𝑋2 𝑋1 𝑋2 0 0 0 0 MB(𝑋1 ) MB(𝑋2 ) Al 2 MB(𝑋1 ) MB(𝑋2 ) 0 1 1 0 𝑋1 𝑋2 𝑋1 𝑋2 1 0 MB(𝑋1 ) MB(𝑋2 ) 0 0 ign ed a Default Assignment Run 𝑎 0 t𝑋 0 𝑋1 𝑋2 0 0 1 MB(𝑋1 ) MB(𝑋2 ) Al Aligned at 𝑋1 0 1 MB(𝑋1 ) MB(𝑋2 ) ed ign 𝑋 at e ign l A 0 1 t𝑋 a d Aligned at 𝑋2 ign ed a t𝑋 0 0 𝑋1 𝑋2 𝑋1 𝑋2 0 1 1 0 MB(𝑋1 ) MB(𝑋2 ) MB(𝑋1 ) MB(𝑋2 ) Al 2 Figure 4.3: An alignment spanning tree of the default assignment 𝑎 0 (COND𝑎0 arbitrarily assigns all Markov boundaries to 0) and six other central runs. The runs on the left cover all possible assignments to MB(𝑋2 ) ∈ {(0, 0), (0, 1), (1, 0)}, while maintaining the default assignment to MB(𝑋1 ) to allow alignment with 𝑎 0 . The right runs similarly cover all possible assignments to MB(𝑋1 ), aligned at 𝑋2 . See Figure 4.3 for an example of a set of central runs and a visualization of how they are alignable. The central runs provide a backbone for easily guaranteeing alignment. The runs in A𝑌 made up of the following two types of perturbations to the independent set (with COND𝑎 always defined as the union of the Markov boundaries of the independent set, as in Definition 6): 1. For each 𝑌 ∈ MB(𝑋𝑖 ) for some 𝑋𝑖 ∈ X, we exclude 𝑋𝑖 from the independent set to form I𝑎𝑌 = X ∪ {𝑌 } \ {𝑋𝑖 }.6 2. For each 𝑌 ∉ MB(X) ∩ X, I𝑎 = X ∪ {𝑌 }. 6This is a well-formed run since 𝑌 ∈ MB(𝑋 ) 𝑖 =⇒ 𝑋 𝑗 ∉ MB(𝑌 ) for any 𝑗 ≠ 𝑖 by Observation 1. 37 For either independence set we will form 2|PA(𝑌 )| runs each associated with a single assignment to pa𝑎 (𝑌 ), with the remaining variables in COND𝑎 ∩ COND𝑎0 conditioned on their defaults given by 𝑓 𝑎0 . Any other assignments to variables in COND𝑎 can be chosen arbitrarily. Algorithm 4 gives the proceedure for explicitly constructing a good set of runs from 𝑁MP centers. Algorithm 4: Building a good collection of runs Input: Vertices X = {𝑋1 , . . . , 𝑋3𝑘−3 } ⊆ V having disjoint Markov boundaries with maximum depth 3𝑁mp . Output: A good collection of runs A. Let 𝑎 0 be a run with I𝑎0 = {𝑋1 , . . . , 𝑋3𝑘−3 } and COND𝑎0 chosen arbitrarily.; Set A ← {𝑎 0 }.; for 𝑖 = 1, . . . , 3𝑘n − 3 do o A ← A ∪ 𝑎 0 [mb(𝑋𝑖 )] : mb(𝑋𝑖 ) ∈ {0, 1}MB(𝑋𝑖 ) .; for 𝑌 ∈ V \ X do if 𝑌 ∈ MB(𝑋) for some 𝑋 ∈ X then I𝑎 ← X ∪ {𝑌 } \ {𝑋𝑖 }; end else I𝑎 ← X ∪ {𝑌 }; end for pa(𝑌 ) ∈ {0, 1} |PA(𝑌 )| do if 𝑌 ∈ AN(I𝑎 − 𝑌 ) then COND𝑎 ← MB(I𝑎 ); end else COND𝑎 ← MB(I𝑎 \ {𝑌 }) ∪ PA(𝑌 ); end COND𝑎 ← MB(I𝑎 ); 𝑓 𝑎 (PA(𝑌 )) ← pa(𝑌 ); Defaults ← COND𝑎0 ∩ COND𝑎 \ PA(𝑌 ); 𝑓 𝑎 ( Defaults) ← 𝑓 𝑎0 ( Defaults); 𝑓 𝑎 (COND𝑎 \ COND𝑎0 \ PA(𝑌 )) are chosen arbitrarily.; A ← A ∪ {𝑎}, where 𝑎 is given by 𝑎 = (I𝑎 , 𝑓 𝑎 ).; end end end  Lemma 5. Given a set X = 𝑋1 , . . . , 𝑋𝑁mp ⊆ V of 𝑁mp variables with depth at most 3𝑁mp and disjoint Markov boundaries, Algorithm 4 finds a good collection of 38 runs A, satisfying |A| = 𝑂 (2𝛾 𝑛) where 𝛾 = max𝑉 ∈V |MB(𝑉)|. Claim 1. By construction, A𝐶 covers X and is alignable. Claim 2. 𝐴𝑌 covers V \ X and each run in A𝑌 can be aligned with some run in A𝐶 . Proof. The fact that A𝑌 covers V \ X follows immediately from A𝑌 containing a run assigning for independent variable 𝑌 ∈ V \ X each possible assignment pa(𝑌 ). Fix any run 𝑎 ∈ A𝑌 with I𝑎 = X − 𝑋𝑖 + 𝑌 or I𝑎 = X + 𝑌 depending on whether 𝑌 overlaps with MB(X). Now if MB(𝑌 ) ∩ MB(𝑋 𝑗 ) = ∅ for any 𝑗 ≠ 𝑖, 𝑎 and 𝑎 0 are aligned at 𝑋 𝑗 . If instead MB(𝑌 ) ∩ MB(𝑋 𝑗 ) ≠ ∅ for all 𝑗 ≠ 𝑖, pick any 𝑗 and consider the central run 𝑎 0 [mb𝑎 (𝑋 𝑗 )] ∈ A𝐶 . Clearly, 𝑎 and 𝑎 0 [mb𝑎 (𝑋 𝑗 )] are aligned at 𝑋 𝑗 . In either case, we’ve aligned 𝑎 to a run in A𝐶 . □ Claim 3. Every run 𝑎 ∈ A is at least 𝑁mp -independent. Proof of Lemma 5. This follows immediately from Claims 1, 2, and 3 □ Degree bounds We can ensure that 𝑁mp centers can be found on certain degree-bounded graphs, which in turn bound 𝛾. Let Δin upper bound on the in-degree of any vertex in 𝔊 and let Δout upper bound the out-degree. Then 𝛾 ≤ Δin + Δout + Δout (Δin − 1) = Δin + Δout Δin . If we have a bound, Δ, on the degree of the undirected skeleton of 𝔊, we get that 𝛾 ≤ Δ(Δ − 1) = Δ2 − Δ. Corollary 1. If either of the following conditions hold, we can find 𝑁mp centers for 𝔊 with depth at most 3𝑁mp : 1. 𝑛 ≥ 𝑁mp (Δ2in + 2Δout Δin + Δ2out Δ2in − Δin − Δout + 1) = 𝑁mp · 𝑂 (Δ2out Δ2in ). 2. 𝑛 ≥ 𝑁mp (Δ4 − 2Δ3 + Δ + 1) = 𝑁mp · 𝑂 (Δ4 ). Proof of Lemma 2. This follows immediately from Corollary 1 and Lemma 5. □ 39 Limiting the depth of unzipping As currently given, our algorithm may require Bayesian unzipping steps equal to the depth of the graph. We can bound accumulated errors from this process by limiting the depth of the vertices that need to be unzipped. Recall that the goal of the conditioning set of each run is to d-separate each of the vertices in the independent set. For a topological ordering on the independence set, notice that we need not condition on the descendants of the deepest vertices in order to d-separate them from the others. Conveniently, avoiding conditioning on these vertices descendants leaves the output of the 𝑘-MixProd oracle in the desired form. We call these deepest vertices “bottom” vertices. Definition 16 (Bottom vertices in a run). Given vertices I, we’ll define the set of bottom vertices of the run to be the subset BOT(I) ⊆ I of vertices with maximal depth among the vertices in I. That is 𝑑 (𝐵) = max 𝐼∈I 𝑑 (𝐼) for all 𝐵 ∈ BOT(I). We can now update the conditioning sets for our definition of runs: Ø Ø COND𝑎 B MB(𝐼) PA(𝐵) 𝐼∈I𝑎 \BOT(I𝑎 ) (4.4) 𝐵∈BOT(I𝑎 ) We append two additional requirements for a good collection of runs (Definition 13). • no vertex appears both as a bottom vertex and a non-bottom vertex, and • every non-bottom vertex has depth at most 3𝑁mp . Note that because we only need independent sets of size 𝑁mp , it is trivial to limit the depth of our non-bottom vertices to 3𝑁mp . 4.6 Conclusion We have developed the first algorithm for identifying the parameters of 𝑘-MixBND mixtures. This algorithm allows us to access the probability distributions within each source — equivalent to the probability distribution conditioned on a universal and unobserved confounder. With access to this conditional distribution, the confounding of 𝑈 can be adjusted for, opening up the opportunity for causal inference despite universal confounding. The algorithm presented here is intended as an identifiability result. We hope this algorithm and framework can serve as a springboard for understanding how solutions to the 𝑘-MixProd and 𝑘-MixBND problems are intimately related. 40 The alignment process is highly nontrivial, explaining why papers such as Anandkumar, D. Hsu, et al., 2012 made crude assumptions (such as a conveniently independent variable) to simplify this process. The formal development of a notion of a run, while tedious, will allow for further improvements to give better “good sets of runs.” Graph-specific sets of runs can also be optimized further, as demonstrated in the published version of this chapter, S. Gordon, B. Mazaheri, Yuval Rabani, et al., 2023. 41 Chapter 5 STRUCTURE LEARNING UNDER GLOBAL CONFOUNDING This chapter will present an algorithm for causal discovery under latent global confounding using the LLC assumption. This work is not yet published. 5.1 Introduction Structural Causal Models Many modern approaches to studying causal systems use structural causal models (SCMs) to graphically model causal relationships in a directed acyclic graph (DAG) (Judea Pearl, 2009; Peters, Janzing, and Schölkopf, 2017). In an SCM, 𝐴 → 𝐵 indicates “𝐴 has a direct effect on 𝐵.” These graphical models provide a systematic way of determining covariate adjustments to indentify the effects of interventions. “Causal discovery” is the task of recovering a causal DAG from data. The simplest algorithms for causal discovery make use of a correspondence between the conditional independence of the observed variables and graphical properties of the underlying SCM. These “d-separation rules”1 give graphical criteria for independence and dependence under an assumption known as faithfulness. For example, 𝐴 ̸⊥⊥ 𝐵 and 𝐴 ⊥⊥ 𝐶 | 𝐵 is sufficient to conclude that there is no arrow between 𝐴 and 𝐶. We say that 𝐵 is a “separating set” for 𝐴, 𝐶 and conclude that there are three possible structures: 𝐴 → 𝐵 → 𝐶, 𝐴 ← 𝐵 ← 𝐶 or 𝐴 ← 𝐵 → 𝐶. Together, we say these three structures form a “Markov equivalence class,” which can be described using a CPDAG, i.e. the undirected skeleton of the true graph and partial orientation of the edges. 𝔊′ 𝑈 𝔊 𝑉1 𝑉2 𝑉3 𝑉4 𝑉5 𝑉6 Figure 5.1: The goal is to learn the graph structure 𝔊 without observing 𝑈. 1 See Judea Pearl (2009) for a review of d-separation or Chapter 2 for a summary of important results. 42 Problem Statement Suppose we augment a DAG 𝔊 = (V, E) with an unobserved 𝑈 that has 𝑘 “latent classes,” each of which leaves a distinctive signal on all the observed variables V. The result is a mixture-model whose DAG, 𝔊′ = (V ∪ {𝑈}, E′) includes additional arrows from the “universal confounder” to every 𝑉 ∈ V. See Figure 5.1 for an example. We will refer to 𝔊 as the observed sub-graph and we will use Δ := max𝑉 ∈V deg𝔊 (𝑉) to denote its maximum (in plus out) degree. The goal is to to uncover the observed sub-graph’s structure 𝔊 up to a Markov equivalence class (i.e. a CPDAG) using statistics gained from the “observed probability distribution,” i.e. Pr(V) marginalized over 𝑈. Notice that 𝑈 confounds all pairs of variables in V, which requires it to be in every separating set. This means that the correspondence between conditional independence and d-separation is no longer adequate to make any structural deductions about the graph. Assumptions We will assume (1) that the distribution is faithful2 with respect to 𝔊′, and (2) that 𝑈 is discrete with a known number of latent classes 𝑘. 𝑘 represents the “complexity” of unobserved confounding, as more latent classes are capable of exerting “more complex” signals. Since our deductions rely upon each value of U having distinctive effects upon the observables, the most difficult case of the problem is Bernoulli observable variables. Compositions of such variables are adequate to express any finite-range variables. Hence we focus entirely upon the case of Bernoulli observables. Causal discovery with unobserved confounding The PC algorithm was the first causal discovery algorithm to make use of conditional independence, outlined in (P. Spirtes et al., 2000). Many causal discovery algorithms have been developed since, summarized in Squires and Uhler (2023). A number of these algorithms address the presence of latent confounding using one of two assumptions that “limit” 𝑈, neither of which apply to our setting. The first type of assumption involves limiting the degree of latent confounding. For example, the FCI algorithm can detect the presence of unobserved confounders that act on only two observed variables (Peter Spirtes, Clark Glymour, Scheines, Peter Spirtes, et al., 1993; Peter Spirtes, 2001). Richardson and Peter Spirtes (2002)’s seminal work introduced ancestral graphs for the general study of this 2The precise assumption is a slight extension of faithfulness to the mixture setting, discussed later. 43 setting. Unfortunately, this assumption cannot be applied to a global or “pervasive” confounder. The second type of assumption is on the parametric nature of the structural equations. For example, Frot, Nandy, and Maathuis, 2019 was able to show superior performance in settings with large-degree confounders with linear relationships and additive subGaussian or elliptical noise. Other settings include linear structural equations with non-Gaussian additive noise (Cai et al., 2023) and non-linear structural equations within a finite-dimensional Hilbert space (Agrawal et al., 2023). These approaches are incapable of addressing discrete 𝑈, V, which have no restriction on their noise or structural equations. Motivation Heterogeneous Data Whenever data spans multiple populations, environments, or laboratories, the relationships within are subject to “global” or “pervasive” confounding (S. Gordon, B. Mazaheri, Yuval Rabani, et al., 2023). In these settings, true causal relationships become entangled with spurious indicators of population (or lab source, etc.) membership. While these data-sources are sometimes observed, naive adjustments for global confounding require conditioning on the data-source, which limits the power of a joined dataset. Recoverying a smaller subset of latent “classes” of data-sources is therefore essential to the problem of data-fusion (Castanedo, 2013). This task involves learning a mixture model, ie. uncovering “mixture weights” Pr(𝑈) and “source distributions” Pr(V | 𝑈). Mixture Models Most work on mixture models deals with parametric distributions. The simplest example is a Gaussian mixture, which relies on the parametric assumption of Gaussianity to recover clusters of points. Without this assumption, there is no way to tell the difference between a mixture of two mono-modal (one hump) distributions versus a single bi-modal (two humps) distribution. Discrete data poses an interesting challenge because categorical distributions are non-parametric. Notice that the statistics gathered from a single unbiased coin are exactly the same as those gathered from two biased coins with the same average bias (e.g. 1/3 and 2/3), so long as a we are limited to a single sample from each choice of coin. Almost all of the research in this setting leverages an assumption of mutually independent observed variables within the source distributions (Cryan, L. Goldberg, and P. Goldberg, 2001; Freund and Mansour, 1999; Chaudhuri and Rao, 2008; J. Feldman, R. O’Donnell, and R. A. Servedio, 2008; Y. Rabani, L. J. Schulman, and 44 Swamy, 2014; Chen and A. Moitra, 2019; S. Gordon, B. H. Mazaheri, et al., 2021; Spencer L. Gordon et al., 2023) (see also the earlier seminal work of Kearns et al. (1994)). 𝑘-MixProd amounts to a graphical assumption, i.e. that the causal model within each source is an empty graph. Early work by Anandkumar, D. Hsu, et al. (2012) and S. Gordon, B. Mazaheri, Yuval Rabani, et al. (2023) broadened this class of independence assumptions for mixture identifiability, exploring Markov random fields and Bayesian networks respectively. A key assumption of (S. Gordon, B. Mazaheri, Yuval Rabani, et al., 2023) and 𝑘-MixProd is that the Bayesian DAG structure within each mixture component is known. To date there is no result showing if and when the within-source DAG structure can be identified from the observed data of a mixture model. If we are capable of recovering graphical structure in this setting, then we can use that structure to recover the mixture without ever making parametric or structural assumptions. Other Related Work Other work has investigated a different setting in which different DAG structures are mixed (Saeed, Panigrahi, and Uhler, 2020). This work relies on the preservation of some local conditional independence properties to learn a “union graph.” Main Result This chapter will give the first proof of the identifiability of causal structures within the discrete mixture setting by developing the first known algorithm for this problem. Theorem 3. Consider 𝔊 = (V, E) with Ω(log(𝑘)Δ3 ) vertices, mixture source 𝑈 ∈ {1, . . . , 𝑘 } and degree bound Δ. 𝔊 is generically3 identifiable up to its Markov equivalence class. Using an oracle that can solve 𝑘-MixProd in time 𝜏 and an oracle that can solve for 2 non-negative rank in time 𝜌 we give an algorithm that runs in time |V| O(Δ log(𝑘)) 𝜌 + 2 O(𝑘 |E| 2Δ )𝜏. The non-negative rank of a 𝑘 + 1 × 𝑘 + 1 matrix (as used for our algorithm) can be 2 solved in time 𝑘 O(𝑘 ) (Ankur Moitra, 2016). In the absence of non-negative rank tests, Anandkumar, D. Hsu, et al., 2012 demonstrated that regular rank tests generally work well in place of non-negative rank tests in practice. We will develop a hypothesis 3with Lebesgue measure 1 in the space of observed moments. 45 test for matrix-rank that requires O(𝑘 6 ) operations to invert a (𝑘 + 1) 2 × (𝑘 + 1) 2 covariance matrix of the estimated elements of our 𝑘 + 1 × 𝑘 + 1 matrix. Our solution also requires solving 𝑘-MixProd on 3 variables of cardinality O(𝑘), which corresponds to decomposing a O(𝑘) × O(𝑘) × O(𝑘) tensor into rank 1 components. When the rank of such a tensor is known to be linear in 𝑘, this decomposition can be solved in 𝑂 (𝑘 6.05 ) (J. Ding et al., 2022), though S. Gordon, B. H. Mazaheri, et al., 2021 showed that the problem generally suffers from instability, with sample complexity exponential in 𝑘. This step may be considered optional, as it is used to refine a small number of “false-positive” adjacencies confined to a provably small subset of the DAG. Our algorithm will build on two key ideas. The first of these ideas is that the rank of a matrix formed by the joint probabilities of two variables contains information about the graphical structure on those variables under latent class confounding. We observe that the joint probability distribution of two independent variables of cardinality ℓ forms an ℓ × ℓ matrix that can be written as a rank 1 outer-product. Hence, marginalizing over a 𝑘-mixture gives us a linear combination of these rank 1 matrices, which will generically be rank 𝑘. Of course, finite data only affords a stochastic perturbation of the true matrix of joint probabilities. A naive approach to testing the rank of the underlying joint probability distribution involves thresholding singular values as in Anandkumar, D. Hsu, et al. (2012). Such an approach is unstable, with many practical difficulties associated with selecting the correct threshold. Instead, we modify a statistical test based on Ratsimalahelo (2001) to obtain a hypothesis test for the rank of an estimated joint probability matrix. We demonstrate the superiority of this approach and note that this test may be of fundamental use in other mixture settings. The idea of using matrix rank is limited by the cardinality of our variables — the joint probability distribution of two binary observables forms a 2 × 2 matrix whose rank is no larger than 2. This problem leads to the second idea: We can “coarsen” variables by joining sets of variables. These “super variables” take on values in the Cartesian product of their components’ alphabets. For example, three Bernouli variables can be combined into a single variable of cardinality 8. This chapter develops the notion of testing rank on these coarsened variables for causal discovery. 46 Outline of Algorithm The PC-algorithm works in two phases. The first phase begins with a complete graph (i.e. all possible edges) and uses conditional independence tests to find non-adjacencies and “separating sets” that d-separate them. When a non-adjacency is found, the corresponding edge is removed and its separating set is stored. The second phase uses the separating sets from the first phase to orient immoralities (as well as further propagation of edge orientation via Meek rules). Our algorithm will mirror the PC Algorithm, but will split the first phase into two parts, yielding three total phases. Phase I will again begin with a complete graph and remove edges between variables when we find evidence of non-adjacency (this time using rank tests). Phase I will only test independence on groupings of variables, so its termination will not guarantee that we have discovered all possible non-adjacencies. Instead, a provably small subset of the graph will have false positive edges. In Phase II, we will make use of the structure we have uncovered so far to induce instances of 𝑘-MixProd within conditional probability distributions. A 𝑘-MixProd solver will then identify the joint probability distribution between subsets of V and the latent class 𝑈. Access to this joint probability distribution enables a search for separating sets that include 𝑈, meaning that the rest of the structure can be resolved using traditional conditional independence tests (following the standard steps of the PC algorithm). Phase III mirrors the last phase of the PC algorithm: identifying immoralities using non-adjacencies and separating sets, and then propagating orientations according to Meek rules. This phase is no different from the PC algorithm, so we will not discuss this phase in detail. 5.2 Additional Background Judea Pearl, 1988 uses structural causal models to justify the local Markov condition, which means that d-separation always implies independence and allows DAG structures to be factorized. It is possible that two d-connected variables by chance exhibit some unexpected statistical independence. This complication is often assumed away using “faithfulness” (Peter Spirtes, Clark Glymour, Scheines, and David Heckerman, 2000), which ensures that d-connectedness implies statistical dependence. Together, the local Markov condition and faithfulness give a correspondence between statistical dependence and the graphical conditions of the causal DAG which can be leveraged for causal structure learning. 47 The following fact will be useful when building separating sets. Lemma 6 (Judea Pearl, 2009). If vertices 𝑉𝑖 , 𝑉 𝑗 are nonadjacent in 𝔊, either PA𝔊 (𝑉𝑖 ) or PA𝔊 (𝑉 𝑗 ) are a valid separating set for 𝑉𝑖 , 𝑉 𝑗 . We will often want to bound the cardinality of separating sets relative to the degree bound of the graph (Δ). When dealing with a separating set between two vertices 𝑉𝑖 , 𝑉 𝑗 , Lemma 6 implies a simple upper bound of Δ. Separating sets for sets (or coarsenings) of vertices are significantly more complicated because conditioning may d-separate some pairs of vertices while d-connecting others. To unify the treatment of separating sets, we will make use of moral graphs, which can be thought of as undirected equivalents of DAGs (Lauritzen et al., 1990). We will denote the moral graph of 𝔊 as 𝔊 (𝑚) . To transform 𝔊 into 𝔊 (𝑚) , we add edges between all immoralities, i.e. nonadjacent vertices with a common child, sometimes called an unsheilded collider. After this, we change all directed edges to undirected edges. A very useful fact from Lauritzen et al., 1990 (also Eq. 1 in Acid and De Campos, 1996) is that all separating sets C ⊆ V for S, S′ ⊆ V in 𝔊 are also separating sets in (𝔊[AN+ (S ∪ S′ ∪ C)]) (𝑚) . This transforms complicated active path analysis into simple connectedness arguments on undirected moral graphs (of special subgraphs of 𝔊). A convenient consequence of this transformation, which we will use throughout the paper, is Lemma 7. Lemma 7. If 𝔊 = (V, E) has degree bound Δ, then the size of a separating set between S, S′ ⊆ V is no larger than min(|S| , |S′ |)Δ2 . Lemma 7 is a consequence of the maximum increase in the degree of the moral graph. Proof. A key observation is that the moral graph of a subgraph of 𝔊 has no additional edges relative to 𝔊 (𝑚) . That is, if 𝔊[W] = (W, F) is a subgraph of 𝔊 = (V, E) then the corresponding edge-sets of the moral graphs obey F (𝑚) ⊆ E (𝑚) because adding vertices cannot have invalidated any previously contained immoralities. Abbreviate (𝔊[AN+ (S ∪ S′ ∪ C)]) (𝑚) as 𝔊C(𝑚) . Even though we do not know what C is, we know that the 1-neighborhood of S in 𝔊C(𝑚) suffices as a separating set. 𝔊 (𝑚) has all of the edges of 𝔊C(𝑚) , so (𝑚) 𝔊 NB1 C (S) ⊆ NB𝔊 1 (𝑚) (S). (5.1) 48 (𝑚) Note that NB𝔊 (S) is not necessarily a separating set for S, S′ in 𝔊 (𝑚) , in fact it 1 may include some vertices in S itself. However, the size of the separating set is (𝑚) bounded by NB𝔊 (S) , which is no larger than |S| Δ2 . As we chose S arbitrarily, 1 this bound also holds for S′. □ Another helpful result from moral graphs is Lemma 8, which will help us when we prove the existence of separating sets. Lemma 8 (Corollary of Theorem 1 in Acid and De Campos, 1996). For DAG 𝔊 = (V, E), and S, S′ ⊆ V, separating sets S, S in (𝔊[AN+ (S ∪ S′)]) (𝑚) are also separating sets in 𝔊. 5.3 Rank Tests This section will develop “rank tests” which will serve as a replacement for conditional independence tests as a test for d-separation or d-connectedness. Checking for 𝑘-Mixture Independence To determine non-adjacency, we will take advantage of a signature 𝑈 leaves on the marginal probability distributions of variables which are independent conditional on 𝑈. First, we interpret the marginal probability distribution as a matrix. Definition 17. Given two discrete variables 𝑋, 𝑌 ∈ V each with |𝑋 | = |𝑌 | = 𝑚, define the “probability matrix” M[𝑋, 𝑌 ] ∈ [0, 1] 𝑚×𝑚 to be M[𝑋, 𝑌 ] 𝑥,𝑦 := Pr(𝑥, 𝑦), (5.2) where 𝑥, 𝑦 both range from 1, . . . , 𝑚. Similarly, for C ⊆ V, define M[𝑋, 𝑌 | c] 𝑥,𝑦 := Pr(𝑥, 𝑦 | c). (5.3) We now notice that we can decompose the probability matrix into a linear combination of conditional probability matrices for each source, for which Lemma 9 gives an upper bound on rank. Lemma 9. Given a mixture of Bayesian network distributions that are Markovian in 𝔊, if 𝑋 ⊥⊥𝔊 𝑑 𝑌 | C, then for all c, rk+ (M[𝑋, 𝑌 | c]) ≤ 𝑘 4. 4 rk + (M) will denote the non-negative rank of matrix M. 49 Proof. We can decompose M[𝑋, 𝑌 | c] as follows: ∑︁ M[𝑋, 𝑌 | c] = Pr(𝑢)M[𝑋, 𝑌 | c, 𝑢]. (5.4) 𝑢 𝑋 ⊥⊥ 𝑌 | C, 𝑈, so M[𝑋, 𝑌 | c, 𝑢] can be written as the outer product of two vectors describing the probabilities of each variable. Therefore, we conclude that rk+ (M[𝑋, 𝑌 | c, 𝑢]) = 1. If 𝑈 ∈ [𝑘], then rk+ (M[𝑋, 𝑌 | c]) ≤ 𝑘. □ Having shown that d-separation in 𝔊 upper bounds the rank of probability matrices, we now seek a lower bound on the rank in the case of d-connectedness. This will require a “faithfulness-like” assumption that the dependence exerts some noticeable effect between the two variables. Lemma 10 shows that such a condition holds generically. Lemma 10. Given a mixture of Bayesian network distributions, each of which is |𝑋 | = 𝑛, |𝑌 | = 𝑚 with 𝑛, 𝑚 > 𝑘, then for all c, faithful to 𝔊. If 𝑋 ̸⊥⊥𝔊 𝑑 𝑌 | C and rk+ (M[𝑋, 𝑌 | c]) > 𝑘 with Lebesgue measure 1. The proof of Lemma 10 (see Section 5.8) involves applying faithfulness to each component of the decomposition in Equation 5.4. Lemma 9 and Lemma 10 provide a generically necessary and sufficient condition for detecting 𝑉𝑖 ⊥⊥𝔊 𝑑 𝑉𝑗 . Lemma 11 (Rank Test). For 𝑉𝑖 , 𝑉 𝑗 with cardinality > 𝑘, 𝑉𝑖 ⊥⊥𝔊 𝑑 𝑉 𝑗 | C if and only if (generically) rk+ (M[𝑉𝑖 , 𝑉 𝑗 | C]) ≤ 𝑘. Hypothesis test for rank We can build a hypothesis test for the null-hypothesis that A B M[𝑉𝑖 , 𝑉 𝑗 | C] is has rank ≤ 𝑘. First, decompose A according to the SVD A = VDU⊤ . If A ∈ R𝑚×𝑚 is rank 𝑘, we will have U, V ∈ R 𝑘×𝑚 . When this decomposition is done on the empirical matrix, Â, we define U2 and V2 to be the 𝑘 + 1th to 𝑚th “extra” rows of U and V respectively, and L to be the diagonal matrix with the 𝑘 + 1th through 𝑚th singular values. Vectorize L to 𝑙ˆ by stacking the columns of L. Let Σ be the covariance matrix of the entries of A with similar stacking, i.e. Σ𝑖+ 𝑗 𝑚,𝑖 ′ + 𝑗 ′ 𝑚 B Cov(A𝑖 𝑗 , A𝑖 ′ 𝑗 ′ ). (5.5) Now, with Σ̂† indicating the Moore-Penrose pseudoinverse of Σ̂ and ⊗ indicating the Kronecker product, define ⊤ † Q̂† B (V⊤ 2 ⊗ U2 ) Σ̂ (V2 ⊗ U2 ). (5.6) 50 According to Ratsimalahelo (2001), if 𝑓 is the rank of Σ, then 𝑁 𝑙ˆ⊤ Q̂† 𝑙ˆ converges to 𝜒2𝑓 under 𝑁 samples. We can use this test statistic with a simplified Σ̂ by noting that  for a single data-point is 0 for all but a single entry, which is 1. This means that the variance of an entry is given by the variance of a Bernoulli random variable and the covariance is given by the expanding into three cases: both are 0, the first entry is 1, the second entry is 1. This simplifies to Cov(A𝑖 𝑗 , A𝑖 ′ 𝑗 ′ ) =    A𝑖 𝑗 (1 − A𝑖 𝑗 )  if 𝑖 = 𝑖′, 𝑗 = 𝑗 ′   −A𝑖 𝑗 A𝑖 ′ 𝑗 ′  otherwise. (5.7) Using this estimate for Σ̂ gives us a hypothesis test that is specifically designed to implement the rank test from this section. This approach is tested and compared to thresholding the 𝑘 + 1th singular value in Section 5.7. 5.4 Algorithm Phase I We will now outline the first two phases of our algorithm, leaving out the final phase of orienting edges with respect to Meek’s rules. The first phase of our algorithm involves coarsening sets of variables and applying rank tests. We then analyze the output of the first phase 𝔊1 which has removed most but not all of the missing edges. We define FP edges as these “leftovers” and show that they are contained within a subset of bounded size. This fact allows us to resolve all of the non-adjacencies in 𝔊 by setting up instances of 𝑘-MixProd in Phase II. Coarsened Rank Tests Lemma 11 allows for a simple generalization of classical structure-learning algorithms (such as the PC algorithm) provided that our probability matrix M[𝑋, 𝑌 ] is large enough. Unfortunately, categorical variables ranging over smaller alphabets (such as the binary alphabets addressed by this chapter) do not contain sufficient information to detect non-adjacency in cases of larger 𝑘. We resolve this problem by coarsening sets of small-alphabet (binary) variables into supervariables of larger cardinality. Definition 18. Consider DAG 𝔊 = (V, E), 𝑉𝑖 , 𝑉 𝑗 ∈ V, and sets S𝑖 , S 𝑗 ⊆ V \ {𝑉𝑖 , 𝑉 𝑗 }. We call the ordered pair (S𝑖+ , S+𝑗 ) = (S𝑖 ∪{𝑉𝑖 }, S 𝑗 ∪{𝑉 𝑗 }) an independence preserving augmentation (IPA) of (𝑉𝑖 , 𝑉 𝑗 ) if, for some IPA conditioning set C ⊂ V, + S𝑖+ ⊥⊥𝔊 𝑑 S 𝑗 | C. 51 The creation of supervariables allows us to use conditional rank tests in the place of conditional independence tests. This leads to a modified version of the PC algorithm that searches over pairs of supervariable coarsenings instead of pairs of vertices, given in Algorithm 5. Algorithm 5: Phase I Input: The marginal probability distribution Pr(V), marginalized over 𝑈. Output: An undirected graph 𝔊1 = (V, E1 ) and a separating set C𝑖 𝑗 for each detected non-adjacency. Begin with a complete undirected graph 𝔊1 = (V, V × V) and 𝑑max ← |V| − 1. for ℓ = 0 to ℓ = 𝑑max do for C ⊂ V and |C| = ℓ do for S, S′ ⊆ V \ C, with |S| = |S′ | = ⌈lg(𝑘)⌉ + 1 do if arbitrary assignment c has rk+ (M[S, S′ | c]) ≤ 𝑘 then Remove all edges between S and S′ in 𝔊1 . C𝑖, 𝑗 ← C for each 𝑉𝑖 ∈ S, 𝑉 𝑗 ∈ S′ Update 𝑑max to the maximum degree of 𝔊1 . end end end end 2 Lemma 12. Algorithm 5 utilizes |V| O(Δ log(𝑘)) non-negative rank tests. Proof. Lemma 7 tells us that the maximum size of a separating set, is 𝛼 :=
|V|
|V|
( ⌈lg(𝑘)⌉ + 1)Δ2 , so we need to check |V| 𝛼 + 𝛼−1 + . . . + 1 possible separating 2 sets, which is |V| O(Δ log(𝑘)) . We must iterate over all possible supervariables for each
|V| separating sets, which is upper bounded by 2( ⌈lg(𝑘)⌉+1) , which is |V| O(log(𝑘)) . □ FP Edges Phase I of our algorithm removes an edge between two non-adjacent variables through a rank test so long as there exists an IPA for the non-adjacency. Not all non-adjacencies will contain an IPA, so the adjacency graph 𝔊1 contains a superset of the true adjacencies. Definition 19. E1 \ E are false positive (FP) edges. To see why FP edges exist, we will introduce the notion of immoral descendants. 52 Definition 20 (Immoral Descendants). For non-adjacent 𝑉𝑖 , 𝑉 𝑗 the immoral descendants of 𝑉𝑖 and 𝑉 𝑗 are their co-children (often called immoralities (Judea Pearl, 2009)) and those children’s descendants. FB(𝑉𝑖 , 𝑉 𝑗 ) := CH(𝑉𝑖 , 𝑉 𝑗 ) ∪ DE(CH(𝑉𝑖 , 𝑉 𝑗 )) (5.8) Observation 3. Any set C such that 𝑉𝑖 ⊥⊥𝑑 𝑉 𝑗 | C must be disjoint from FB(𝑉𝑖 , 𝑉 𝑗 ). Clearly, an IPA conditioning set C will need to avoid the immoral descendants FB(𝑉𝑖 , 𝑉 𝑗 ) in order to preserve 𝑉𝑖 ⊥⊥ 𝑉 𝑗 | C. Lemma 13 will show that S𝑖 , S 𝑗 must also avoid FB(𝑉𝑖 , 𝑉 𝑗 ). Lemma 13. All IPAs for 𝑉𝑖 , 𝑉 𝑗 are disjoint from the FB(𝑉𝑖 , 𝑉 𝑗 ). That is, FB(𝑉𝑖 , 𝑉 𝑗 ) ∩ S𝑖+ = ∅ and FB(𝑉𝑖 , 𝑉 𝑗 ) ∩ S+𝑗 = ∅ for all IPAs (S𝑖+ , S+𝑗 ) of (𝑉𝑖 , 𝑉 𝑗 ). 𝑉𝑖 𝑉𝑗 FB(𝑉𝑖 , 𝑉 𝑗 ) 𝑉1 𝑉2 DE(𝑉1 ) DE(𝑉2 ) Figure 5.2: An illustration of an FP edge after Phase I due to a large set of immoral descendants. The population variable 𝑈 is omitted to avoid clutter. While 𝑉𝑖 and 𝑉 𝑗 are d-separated by C = ∅ no IPA can be made because all of the leftover vertices are immoral descendants. This illustrates that FP edges can occur as pairs of vertices with too many immoral descendants, leaving no vertices to form IPAs (shown in Figure 5.2). Using non-descendants to form IPAs Notice that the immoral descendants of a pair of vertices are always descendants of both 𝑉𝑖 and 𝑉 𝑗 . To simplify our analysis, we focus on the existence of IPAs that are disjoint from the entire set of descendants. This allows us to show that FP edges only occur between “early vertices.” Definition 21. We define the early vertices, H := {𝑉 ∈ V s.t. DE(𝑉) < (2 + Δ2 )(⌈lg(𝑘)⌉ + 1)}. Lemma 14. After Phase I (Algorithm 5), all of the false positive edges lie within the early vertices. More formally, E1 \ E ⊆ H × H. 53 Observation 4. |H| ≤ (2 + Δ2 ) ⌈lg(𝑘)⌉ and the maximum degree of 𝔊1 is bounded by |H|. 5.5 Phase II: Handle FP Edges Recall that the marginal probability distribution cannot use independence tests to discover non-adjacency because latent variable 𝑈 confounds all of the independence properties. An important observation is that the within-source distribution Pr(V | 𝑢) would not suffer from this limitation because it would allow us to condition on separating sets that include unobserved 𝑈. Phase II will make use of this observation by selecting subsets of variables T ⊆ V on which to obtain Pr(T | 𝑢) using techniques from discrete mixture models. We will then apply regular conditional independence tests on the recovered Pr(T | 𝑢) to detect FP edges. We will use a separate T𝑖 𝑗 ∋ 𝑉𝑖 , 𝑉 𝑗 coarsening to verify each edge (𝑉𝑖 , 𝑉 𝑗 ) ∈ E1 , though this process can likely be optimized further. The primary result on mixture model identifiability is given by Allman, Matias, and Rhodes, 2009 as a direct consequence of a result by Kruskal, 1977. Lemma 15 (Allman, Matias, and Rhodes, 2009). Consider the discrete mixture source 𝑈 ∈ {1, . . . , 𝑘 } and discrete variables 𝑋1 , 𝑋2 , 𝑋3 with cardinality 𝜅1 , 𝜅2 , 𝜅3 respectively and 𝑋1 ⊥⊥ 𝑋2 ⊥⊥ 𝑋3 | 𝑈. The mixture is generically identifiable (with Lebesgue measure 1 on the parameter space) if min(𝜅 1 , 𝑘) + min(𝜅2 , 𝑘) + min(𝜅3 , 𝑘) ≥ 2𝑘 + 2. We can again use coarsening to form 𝑋𝑖 with large enough 𝜅𝑖 . The conditions for identifiability are therefore quite mild — Phase I only needs to uncover enough sparsity to d-separate three sufficiently large independent coarsenings, one of which will be T𝑖 𝑗 . Conveniently, the constrained nature of our FP edges means that the graph 𝔊1 is sufficiently sparse to allow the construction of this setting. T𝑖 𝑗 must be designed to include enough information to discover a nonadjaceny between 𝑉𝑖 , 𝑉 𝑗 . In other words, we need to ensure that T𝑖 𝑗 contains a separating set C ⊂ T𝑖 𝑗 such that 𝑉𝑖 ⊥⊥𝔊 𝑑 𝑉 𝑗 | C. It turns out that augmenting 𝑉𝑖 , 𝑉 𝑗 with their distance-1 neighborhood is enough to guarantee this requirement. Definition 22. Given vertices 𝑉𝑖 , 𝑉 𝑗 , let T𝑖 𝑗 be the set containing 𝑉𝑖 , 𝑉 𝑗 and all vertices that are distance 1 in 𝔊1 from 𝑉𝑖 or 𝑉 𝑗 . 54 Lemma 16. If vertices 𝑉𝑖 , 𝑉 𝑗 are nonadjacent, the set T𝑖 𝑗 contains a valid separating set C ⊆ T𝑖 𝑗 such that 𝑉𝑖 ⊥⊥𝑑 𝑉 𝑗 | C. Lemma 16 guarantees that the conditional probability distribution Pr(T𝑖 𝑗 | 𝑢) has sufficient information to verify or falsify the adjacency of 𝑉𝑖 and 𝑉 𝑗 . 5.6 Utilizing 𝔊1 to set up 𝑘-MixProd The rest of the construction of the 𝑘-MixProd instances is left to Section 5.6. Generally, it involves ensuring that the recovered 𝔊1 from Phase I is sparse enough to 𝑑-separate all T𝑖 𝑗 from two other supervariables of sufficient cardinality. The procedure is outlined by Algorithm 6 and then Algorithm 7 performs the actual correction with the statistics recovered from 𝑘-MixProd oracles. Lemma 17 summarizes the results proved in Section 5.6. Lemma 17. Phase II requires Ω(Δ3 log(𝑘)) vertices and solves 𝑘-MixProd no more 2 than O(𝑘 |E| 2Δ ) times. The first step to recovering Pr(T𝑖 𝑗 | 𝑢) will be to select some Z𝑖 𝑗 and recover Pr(T𝑖 𝑗 | 𝑢, z𝑖 𝑗 ) using instances of 𝑘-MixProd induced on the conditional probability distribution Pr(V | z𝑖 𝑗 ). Recall that 𝑘-MixProd requires three independent variables of sufficient cardinality. Hence, we must find X1 , X2 , T𝑖 𝑗 which are sufficiently large, and d-separated from each other by Z𝑖 𝑗 in 𝔊. See Figure 5.3 for an example. 𝑈′ T𝑖 𝑗 X1 X2 𝑉1 𝑉3 𝑉5 𝑉7 𝑉9 𝑉11 𝑉13 𝑉2 𝑉4 𝑉6 𝑉8 𝑉10 𝑉12 𝑉14 FB(𝑉3 , 𝑉4 ) Figure 5.3: The given graph has an FP edge between 𝑉3 and 𝑉4 , indicated by a dashed line, caused by a large set of immoral descendants (shown in red). Conditioning on 𝑉7 , 𝑉11 , 𝑉12 creates an instance of 𝑘-MixProd on T𝑖 𝑗 , X1 , X2 . Notice that 𝑉7 , 𝑉11 , 𝑉12 are all in FB(𝑉3 , 𝑉4 ), which means that the Pr(T𝑖 𝑗 | 𝑉7 , 𝑉11 , 𝑉12 , 𝑢′) recovered by 𝑘-MixProd will not be sufficient for detecting the FP edge. This obstacle will be solved in Subsection 5.6. Of course we have access to 𝔊1 , not 𝔊. 𝔊1 contains no orientations5 and may 5 It is, in principle, possible to orient immoralities within 𝔊 complexity improvements. 1 at this stage, but this gives no 55 contain extra false-positive edges. We will need to build a conditioning set Z𝑖 𝑗 that achieves a guaranteed instance of 𝑘-MixProd nonetheless. While Markov boundaries cannot be computed without 𝔊, we can easily use 𝔊1 to find a superset that contains the Markov bounary of a given vertex. Lemma 18. The 2-neighborhood of X ⊆ V in 𝔊1 contains MB𝔊 (X). Proof. The distance between 𝑋 ∈ X, and 𝑉 ∈ V in 𝔊1 is less than or equal to the distance in 𝔊, because E1 ⊇ E. This means that the 2-neighborhood of X in 𝔊1 includes the 2-neighborhood of X in 𝔊. Furthermore because all vertices in MB𝔊 (X) are distance ≤ 2 from at least one 𝑋 ∈ X, we have that MB𝔊 (X) is contained in the 2-neighborhood of X in 𝔊1 . □ The sparsity of 𝔊1 will dictate the number of necessary vertices to successfully set up 𝑘-MixProd. We will want to limit the size of Z𝑖 𝑗 as much as possible. Fortunately, the bounds on the size of H mean that most of 𝔊1 is degree bounded by Δ. We will avoid large degree vertices in H by strategically selecting X1 , X2 with the smallest 2-neighborhoods. Z𝑖 𝑗 = MB(X1 ) ∪ MB(X2 ) will be sufficient to d-separate all three vertices, so we need not worry about potentially large degree in T𝑖 𝑗 . This process is described by Algorithm 6. Algorithm 6: Formation of 𝑘-MixProd Instances Input: Two vertices 𝑉𝑖 , 𝑉 𝑗 ∈ V and 𝔊1 = (V, E1 ) from the output of Algorithm 5. Output: T𝑖 𝑗 , X1 , X2 and Z𝑖 𝑗 such that T𝑖 𝑗 ⊥⊥𝑑 X1 ⊥⊥𝑑 X2 | Z𝑖 𝑗 . 𝔊1 1 Let T𝑖 𝑗 = {𝑉𝑖 , 𝑉 𝑗 } ∪ NB𝔊 1 (𝑉𝑖 ) ∪ NB1 (𝑉 𝑗 ). V′ ← V \ (T𝑖 𝑗 ∪ NB2 (T𝑖 𝑗 )) X1 , X2 all begin as empty sets. while 2|X1 | + 2|X2 | < 2𝑘 + 2 − min(𝑘, 2 | T𝑖 𝑗 | ) do Add 𝑉 ∈ V′ \ (NB2 (X2 ) ∪ X2 ) to X1 . Add 𝑉 ∈ V′ \ (NB2 (X1 ) ∪ X1 ) to X2 . end Z𝑖 𝑗 ← NB2 (X1 ) ∪ NB2 (X2 ). Lemma 19. Algorithm 6 terminates successfully (without running out of vertices in V′) with Ω(Δ3 log(𝑘)) vertices. Proof. The algorithm designates vertices in V into the following sets and succeeds so long as those sets are disjoint. 56 1. X1 and X2 2. NB2 (X1 ), NB2 (X2 ) 3. T𝑖 𝑗 4. NB1 (T𝑖 𝑗 ) T𝑖 𝑗 ≥ 2 and 𝑘 ≥ 2, so the number of vertices added to X1 , X2 in the loop of Algorithm 6 is at most ⌈lg(2𝑘 + 2 − 2)⌉ < lg(𝑘) + 2. To bound the 2-neighborhood, we notice that we cannot easily apply our degree bound of Δ because H could be a clique in 𝔊1 (from FP edges). Instead, we bound |NB2 (X1 ) ∪ NB2 (X2 ) ∪ X1 ∪ X2 | ≤ Δ2 |X1 ∪ X2 | + Δ |H| (5.9) because the distance 1 neighborhood could include all of H but all additional neighborhoods are bounded by Δ. |H| is O(Δ2 log(𝑘)) by Observation 4, so this bound is O(Δ3 log(𝑘)). The size of T𝑖 𝑗 is the largest when including 𝑉𝑖 or 𝑉 𝑗 in H, for which NB1 (𝑉𝑖 ) could be all of H and NB(𝑉𝑖 ) then necessarily falls outside of H. This worst case gives T𝑖 𝑗 ≤ |H| + Δ2 , (5.10) which is O(Δ2 log(𝑘)). Expanding to the 1 neighborhood picks up another factor of Δ, bringing us again to O(Δ3 log(𝑘)). □ Aligning multiple 𝑘-MixProd runs 𝑘-MixProd distributions are symmetric with respect to the 𝑘! permutations on the label of their source. For this reason, there is no guarantee that multiple calls to a 𝑘-MixProd solver will return the same permutation of source labels. To solve this, S. Gordon, B. Mazaheri, Yuval Rabani, et al., 2023 noticed that any two solutions to 𝑘-MixProd problems that share the same conditional probability distribution for at least one “alignment variable” can be “aligned” by permuting the source labels until the distributions on that variable match up. We will only need alignment along runs for different assignments to each Z𝑖 𝑗 , used in the next section. Explicitly, two assignments z𝑖 𝑗 and z′𝑖 𝑗 , need least one X∗ ∈ {T𝑖 𝑗 , X1 , X2 } such that mbz𝑖 𝑗 (X∗ ) and mbz𝑖′ 𝑗 (X∗ ) are the same, in order for alignability to be satisfied. 57 To align sets of 𝑘-MixProd results which are not all pairwise alignable, Chapter 4 introduced the concept of an “alignable set of runs” for which chains of alignable pairs create allow alignability. We re-use this idea in the following lemma. Lemma 20. The set of 𝑘-MixProd instances on the same T𝑖 𝑗 , X1 , X2 with all possible assignments z𝑖 𝑗 to Z𝑖 𝑗 is alignable. Proof. Any two runs with assignments z𝑖 𝑗 and z′𝑖 𝑗 that differ in their assignment to only one variable are alignable. Therefore, any two non-alignable runs can be aligned using a chain of Hamming-distance one alignments. □ Recovering the unconditioned within-source distribution After all our calls to the 𝑘-MixProd oracle, we have access to Pr(T𝑖 𝑗 | 𝑢, z𝑖 𝑗 ) and Pr(𝑢 | z𝑖 𝑗 ) for every assignment z𝑖 𝑗 and 𝑢. Pr(T𝑖 𝑗 | 𝑢, z𝑖 𝑗 ) is insufficient to determine the adjacency of 𝑉𝑖 , 𝑉 𝑗 because Z𝑖 𝑗 may contain vertices in the immoral descendants of 𝑉𝑖 , 𝑉 𝑗 , prohibiting the discovery of a separating set within T𝑖 𝑗 . Instead, we must recover Pr(T𝑖 𝑗 | 𝑢), which is not conditioned on Z𝑖 𝑗 . To do this, we can apply the law of total probability over all possible assignments to Z𝑖 𝑗 . Pr(T𝑖 𝑗 | 𝑢) = ∑︁ Pr(z𝑖 𝑗 | 𝑢) Pr(T𝑖 𝑗 | z𝑖 𝑗 , 𝑢) (5.11) z𝑖 𝑗 We can obtain Pr(z𝑖 𝑗 | 𝑢) by using Bayes rule on the 𝑘-MixProd output, Pr(𝑢 | z𝑖 𝑗 ). Pr(z𝑖 𝑗 | 𝑢) = Pr(𝑢 | z𝑖 𝑗 ) Pr(z𝑖 𝑗 ) . Pr(𝑢) (5.12) Pr(z𝑖 𝑗 ) can be obtained by counting the frequency of z𝑖 𝑗 in the data. In addition, Í Pr(𝑢) = z𝑖 𝑗 Pr(z𝑖 𝑗 ) Pr(𝑢 | z𝑖 𝑗 ) is computable by the law of total probability after the runs for each assignment z𝑖 𝑗 , have been aligned. Equivalently, we can normalize Í such that z𝑖 𝑗 Pr(z𝑖 𝑗 | 𝑢) = 1. Lemma 21. We can compute Pr(T𝑖 𝑗 | 𝑢) using known quantities, Í z𝑖 𝑗 Pr(𝑢 | z𝑖 𝑗 ) Pr(z𝑖 𝑗 ) Pr(T𝑖 𝑗 | z𝑖 𝑗 , 𝑢) Í Pr(T𝑖 𝑗 | 𝑢) = . z𝑖 𝑗 Pr(z𝑖 𝑗 ) Pr(𝑢 | z𝑖 𝑗 ) Pr(T𝑖 𝑗 | 𝑢) is a completely deconfounded distribution on which we can run the PCalgorithm. The full procedure is given in Algorithm 7, in which we use Algorithm 6 58 Algorithm 7: Phase II: Detection and correction of FP edges. Input: Pr(V) marginalized over 𝑈, a black box solver for 𝑘-MixProd, and 𝔊1 = (V, E1 ) from the output of Algorithm 5. Output: 𝔊2 = (V, E2 ), an undirected skeleton of 𝔊 and separating sets for nonadjacenceis (vertices not in E2 ) . Start with E2 ← E1 . for each {𝑉𝑖 , 𝑉 𝑗 } ∈ E1 do Retrieve T𝑖 𝑗 , X1 , X2 , Z𝑖 𝑗 from Algorithm 6. for each assignment z𝑖 𝑗 do Run the 𝑘-MixProd solver on T𝑖 𝑗 , X1 , X2 on Pr(V | z𝑖 𝑗 ). end Perform alignment of the 2z𝑖 𝑗 runs to retrieve Pr(T𝑖 𝑗 | Z𝑖 𝑗 , 𝑈). Calculate Pr(T𝑖 𝑗 | 𝑢) for every 𝑢 using Lemma 21. Run PC or any other structure learning algorithm on Pr(T𝑖 𝑗 | 𝑢) to find a separating set C𝑖 𝑗 (or verify adjacency) for 𝑉𝑖 , 𝑉 𝑗 . If 𝑉𝑖 ⊥⊥ 𝑉 𝑗 | C𝑖 𝑗 , 𝑢 for all 𝑢, remove {𝑉𝑖 , 𝑉 𝑗 } from E2 and store C𝑖 𝑗 . end followed by alignment and Lemma 21 in order to remove all of the false-positive edges from 𝔊1 . 2 Lemma 22. Algorithm 7 requires solving 𝑘-MixProd O(𝑘 |E| 2Δ ) times. Proof. This algorithm requires running 𝑘-MixProd for every possible assignment to the conditioning set D𝑖 𝑗 , for which we have D𝑖 𝑗 ≤ (lg(𝑘) + 2)Δ2 total binary 2 variables. This gives an upper bound of 2𝑘2Δ runs of 𝑘-MixProd for each edge. □ 5.7 Empirical Results The algorithm is successful when enough data is gathered, as proved by our theoretical results. We now employ three empirical tests to show the superiority of our derived hypothesis-based rank test as well as investigate the sensitivity of Phase I. Structural Equation Setup SCMs are made up of a graphical structure and accompanying structural equations. We focus our tests primarily on varying the graphical structure, using a standard set of structural equations on these graphs.Our 𝑈 are generated using a fair coin (𝑘 = 2), and all other vertices are Bernoulli random variables with bias 𝑝𝑉 determined by 𝑉’s parents (including 𝑈): Í 1 + 𝑊 ∈PA𝔊′ (𝑉) 𝑊 𝑝𝑉 = . (5.13) ′ PA𝔊 (𝑉) + 2 59 Singular Value Rank Test P-value k + 1th Singular Value 0.8 0.08 0.06 0.04 0.6 0.4 0.2 0.02 0.00 Rank Hypothesis Test 1.0 0.10 Connected Split 0.0 1000 2000 3000 4000 5000 6000 7000 8000 9000 Number of Data Samples Connected Split 1000 2000 3000 4000 5000 6000 7000 8000 9000 Number of Data Samples Figure 5.4: The results of Test 1. Structural equations of this form have a reasonable strength between vertices that is decreased relative to in-degree. Test 1: Rank Hypothesis Test vs. Singular Values We begin with a comparison of the test developed in Section 5.3 to a naive thresholding of singular values as in Anandkumar, D. Hsu, et al. (2012). To study the differences between these tests, we generate data from two observed subgraphs. 1. “Connected” 𝔊𝑐 : 𝑉1 → 𝑉2 → 𝑉3 → 𝑉4 2. “Split” 𝔊𝑠 : 𝑉1 → 𝑉2 𝑉3 → 𝑉4 If we coarsen our vertices into S𝑖+ = {𝑉1 , 𝑉2 } and S+𝑗 = {𝑉3 , 𝑉4 }, then these two DAGs 𝑐 + + ⊥𝔊𝑠 S+ . We note that this is a challenging test, as differ in that S𝑖+ ̸⊥⊥𝔊 𝑗 𝑑 S 𝑗 and S𝑖 ⊥ 𝑑 the connection between the two partitions is only driven by the 𝑋2 → 𝑋3 arrow. We varied the number of samples from these distributions from 1000 to 9000 and studied the distributions of the two reported p-values and 𝑘 + 1th singular values across 200 runs. The results are reported in Figure 5.4, showing that the hypothesis test has significant difference in p-values beyond 4000 samples (relative to the difference in the 𝑘 + 1th singular values). The hypothesis test does appear to remove edges more aggressively in the low-data regime — i.e. the blue curves overlap with much of the orange curve in the low-data regime. However, it is worth noting that it is almost impossible to choose a threshold for the 𝑘 +1th singular value ahead of time, whereas hypothesis tests give a meaningful interpretation to significance. 60 Test 2: Recovering a “Y” Graph The second test is a simple recovery test on a 7 vertex graph in which two disconnected parents form a child, with a series of descendants. We sample 10, 000 data points and test with a p-value of .0005 — i.e. we remove edges when we get a p-value of .9995 in our rank test (a value that is motivated by knock-on effects and the concentration of the “split” graph results in Test 1). We report our results from 100 different tests (and implicitly show the tested graph) in Figure 5.5. This test illustrates a few things that are not revealed in our theoretical results. The first is that edges which are far-enough apart often appear independent even before conditioning on their separating sets, presumably due to their weak dependence. In these cases our algorithm will “incorrectly” remove edges between vertices too early, but still give the correct result (as with any other independence-based algorithm). As is the case with most causal discovery algorithms, the frequency of false-positive edges tends to increase with the size of the separating set between the vertices. Vertices with a large separating set require a rank tests for each assignment to that separating set, leading to more “accidentally” dependence. These “knock-on effects” are often handled using p-value adjustments, suggesting that a smaller p-value thresholds would serve a similar purpose for our algorithm. It is worth emphasizing this effect is dependent on the separating set for the IPA rather than the two vertices themselves. For the graph tested in Figure 5.5 conditioning on 𝑉5 is sufficient to induce independence between 𝑉4 and 𝑉6 in the unconfounded setting. In the presence of a global counfounder, however, we require an additional vertex to be coarsened with 𝑉6 and independent from 𝑉4 . As 𝑉6 has no descendants, we must obtain this vertex from 𝑉0 , . . . , 𝑉2 , requiring an additional vertex to be conditioned on. For this reason, false-positive edges are especially likely to occur at the end of our chain. Test 3: Many Graphs with Varying Density In our third test, we explore the role that graph density plays in accurately detecting graph adjacency. For this test, we sample random Erdös-Renyi undirected graph structures on 7 vertices and orient them according to a random permutation of the edges. We vary the probability of edge-occurrence in our graphs from .1 to .9 in .1 increments, sampling 20 graph structures for each. Among these graphs we draw 10, 000 datapoints and study the role of maximum in-degree and total number of edges on the percentage of correctly recovered edges (p-value .0005 again). The 61 𝑉0 𝑉1 𝑉2 𝑉6 𝑉3 𝑉4 𝑉5 𝑉6 𝑉0 𝑉2 𝑉3 𝑉4 𝑉5 𝑉1 𝑉0 0.0 0.14 0.79 0.1 0.04 0.05 0.05 𝑉1 0.14 0.0 0.81 0.03 0.01 0.03 0.03 𝑉2 0.79 0.81 0.0 0.85 0.17 0.09 0.11 𝑉3 0.1 0.03 0.85 0.0 0.82 0.1 0.15 𝑉4 0.04 0.01 0.17 0.82 0.0 0.72 0.26 𝑉5 0.05 0.03 0.09 0.1 0.72 0.0 0.79 𝑉6 0.05 0.03 0.11 0.15 0.26 0.79 0.0 Figure 5.5: Results from Test 2. In blue, we show correctly returned edges. In red, we show edges that were returned which are not in the true model. The opacity of the lines show the percentage of the time that the edge was returned (ideally, we would want faint red lines and strong blue lines). To the right of the graph, we show a table of the frequency of returning the edge colored according to the same scheme. Graph Density and Edge Accuracy 0.8 0.6 0.4 0.2 Edges Missing Edges 0.0 0.1 0.2 0.3 1.0 Fraction correctly identified Fraction correctly identified 1.0 0.4 0.5 0.6 0.7 0.8 0.9 Graph density (probability of adding each edge) Maximum In-Degree and Edge Accuracy 0.8 0.6 0.4 0.2 Edges Missing Edges 0.0 0 1 2 3 4 Maximum in-degree 5 6 Figure 5.6: The results of Test 3. Horizontal ticks represent the median accuracy for recovering edges (blue) or lack of edges (orange). A violin plot is also shown, representing the density of results over 20 iterations at each 𝑝 (probability of adding an edge). results are given in Figure 5.6. For 𝑝 = 0.1 to 𝑝 = 0.3 the medians of both true positive and true negative edge reconstruction are at 100%, with the distributions showing the occasional error. As the density of the graph increases, we fail to detect edges (lower blue marks), and incorrectly return edges where there are none (lower orange marks). At high densities, our accuracy for detecting true edges returns to higher levels, but at the cost of occasionally adding false positive edges. The second figure shows that these false positive edges may be due to the larger in-degree of more dense networks. We see very good recovery for networks with limited in-degree, and significantly more error with larger in-degrees. 5.8 Deferred Proofs Proof of Lemma 14 To prove this lemma we first define the entire set of descendants D𝑖 𝑗 ⊇ FB(𝑉𝑖 , 𝑉 𝑗 ). 62 Definition 23. Let the set D𝑖 𝑗 := DE(𝑉𝑖 , 𝑉 𝑗 ) be the descendants of both vertices and A𝑖 𝑗 := V \ D𝑖 𝑗 . Recall that a separating set for any two sets of variables exists as a separating set in the moral graph of their ancestors (Lemma 8). By restricting our focus to S𝑖 , S 𝑗 ⊆ A𝑖 𝑗 will also have AN+ (S𝑖+ , S+𝑗 ) ⊆ A𝑖 𝑗 which guarantees that our separating sets will not overlap with FB(𝑉𝑖 , 𝑉 𝑗 ). Lemma 23 will tell us how large we need A𝑖 𝑗 to be in order to be guaranteed an IPA. Lemma 23. An IPA S𝑖+ , S+𝑗 for 𝑉𝑖 , 𝑉 𝑗 ∈ V exists so long as A𝑖 𝑗 ≥ (2 + Δ2 )(⌈lg(𝑘)⌉ + 1). Proof. We can form S𝑖+ out of 𝑉𝑖 and ⌈lg(𝑘)⌉ arbitrary other vertices from A𝑖 𝑗 . Now, let the separating set be C := MB𝔊[A𝑖 𝑗] (S𝑖+ ) and note that C d-separates S𝑖+ from all other elements of A𝑖 𝑗 . Since we know |𝐶 | ≤ Δ2 (⌈lg(𝑘)⌉ + 1), we have at least ⌈lg(𝑘)⌉ vertices in A𝑖 𝑗 left to join with 𝑉 𝑗 and make S+𝑗 . □ We are now ready to prove Lemma 14. Proof. A convenient consequence of Lemma 23 is that it guarantees the existance of IPAs everywhere except within a small subset of vertices. Let DE(𝑉) := V \ DE(𝑉) be the “non-descendants” of 𝑉. Note that A𝑖 𝑗 = DE(𝑉𝑖 ) ∪ DE(𝑉 𝑗 ). This implies that A𝑖 𝑗 ≥ max( DE(𝑉𝑖 ) , DE(𝑉 𝑗 ) ). (5.14) Hence, so long as at least one vertex has enough non-descendants, A𝑖 𝑗 will be large enough to form an IPA. This set of vertices with enough non-descendants corresponds to the complement of the early vertices. □ Proof of Lemma 10 Proof. We will drop the conditioning on c in this proof for simplicity. Consider the sum 𝑗 ∑︁ 𝜎 𝑗 := Pr(𝑢𝑖 )M[𝑋, 𝑌 | 𝑢𝑖 ], (5.15) 𝑖=1 and note that 𝜎𝑘 = M[𝑋, 𝑌 ]. Faithfulness with respect to 𝔊′ tells us that there is some assignment, which we call 𝑢 1 wlog, such that 𝑋 ̸⊥⊥ 𝑌 | 𝑢 1 . Hence rk+ (𝜎1 ) > 1. Now, we show inductively that rk+ (𝜎𝑖 ) = rk+ (𝜎𝑖−1 ) + 1 is a measure 1 event for 𝑖 = 1, . . . , 𝑘. Denote M[𝑋, 𝑌 | 𝑢𝑖 ] = 𝑣 𝑖 𝑤 𝑖⊤ with column space 𝑣 𝑖 drawn from a 63 subspace with non-zero measure on R𝑛 . The column space of 𝜎𝑖−1 is rank ≤ 𝑖 −1 < 𝑚, so it has measure zero on R𝑚 . Hence, 𝑣 𝑖 being in the column space of 𝜎𝑖−1 is a measure 0 event. We conclude that rk+ (𝜎𝑖−1 + M𝑢𝑖 [𝑋, 𝑌 ]) = rk+ (𝜎𝑖−1 ) + 1 with measure 1. Inducting on 𝑖 gives rk+ (𝜎𝑘 ) > 𝑘 with measure 1. □ Proof of Lemma 13 Proof. Suppose for contradiction that some vertex 𝐵 ∈ FB(𝑉𝑖 , 𝑉 𝑗 ) ∩ S+𝑗 . 𝐵 ∈ FB(𝑉𝑖 , 𝑉 𝑗 ) implies that there is a directed path P ⊆ FB(𝑉𝑖 , 𝑉 𝑗 ) from 𝑉𝑖 to 𝐵. By the definition of an IPA, 𝐵 ∈ S+𝑗 means that there must be some C with 𝐵 ⊥⊥𝑑 𝑉𝑖 | C. We conclude that C must contain some 𝐶 ∈ P in order to block P from being an active path. However, this also means that 𝐶 ∈ FB(𝑉𝑖 , 𝑉 𝑗 ), which contradicts Observation 3. The same argument holds for 𝐵 ∈ S𝑖+ . □ Proof of Lemma 16 Proof. T𝑖 𝑗 contains both PA(𝑉𝑖 ) and PA(𝑉 𝑗 ), so Lemma 6 tells us that we contain a separating set. □ Part III Level 4: Wisdom 64 65 Chapter 6 GRAPHICALLY MODELED CONTEXTS Recall that the goal of Level 4 knowledge is to infer universal relationships that are invariant among possible worlds. When seeking to generalize models outside of their domain, it is natural to focus on shifts in data due to interventions and changes in exogenous variables. It is not immediately obvious what role dataset-specific biases play in universality. The focus of Part III will be to argue that sampling bias and restricted contexts play an essential role in Level 4 knowledge in the following ways: 1. Context must be understood when adjusting for shifts in distributions, or else conclusions develop paradoxes. 2. Context can be used to isolate information based on its causal functions. 6.1 The Domain Expertise Paradox We will begin our discussion of Level 4 knowledge by introducing a paradox that arises when the context of data is misunderstood. The context we will specifically explore is that of domain expertise, in which experts or ML models are capable of classifying only a subset of the total possible classes. We will also refer to this as an “omitted label context.” For example, “dogs vs. cats” is omitted label context, but “dogs vs. non-dogs” is not. While the relative probabilities of classes within this subset are maintained, data from all other labels are unobserved. More precisely, 𝑝(𝑦 ∗1 )/𝑝(𝑦 ∗2 ) = 𝑞(𝑦 ∗1 )/𝑞(𝑦 ∗2 ) for 𝑦∗1 , 𝑦∗2 ∈ Y∗ , but 𝑝(𝑦′) = 0 if 𝑦′ ∉ Y∗ . Within the scope of this paper, we will restrict our focus to |Y∗ | = 2. Omitted label contexts are motivated by a few real-life scenarios within medicine and epidemiology. The first is “immortal time bias” (Suissa, 2008), which famously reversed the perceived risks of postmenopausal hormone treatment. While initial observational studies suggested this treatment could decrease in cardiovascular issues (Grodstein, Joann E Manson, and Stampfer, 2006), a followup clinical study eventually showed the opposite (Michels and JoAnn E Manson, 2003).1 This discrepancy can be attributed to the observational study’s focus on current users 1 Grodstein, Joann E Manson, and Stampfer (2006) was initially published before Michels and JoAnn E Manson (2003), but later updated. 66 of the therapy (Hernán, Alonso, et al., 2008). More specifically, the backtracking nature of the observational study excluded a group of vulnerable women who had not survived treatment long enough to participate – i.e. all participants were “immortal” from the inception of their treatment to the beginning of the study. Exclusion of an outcome (in this case, death before the study) constitutes an example of an omitted label context. Omitted label contexts are also extremely common in the study of rare conditions. For example, a census genome sequencing of the US population would be an impractical and financially infeasible task. Instead, databases like TCGA (Tomczak, Czerwińska, and Wiznerowicz, 2015) allow focused access to patients with specific (and often rare) cancers. In study designs, investigators may opt for an omitted label context or induce further label shift by working with a uniform distribution on the labels of interest. Omitted labels are a form of sampling bias – a topic that has been studied in detail within the causal inference literature (Bareinboim and Tian, 2015; Correa, Tian, and Bareinboim, 2019). Bareinboim and Tian (2015) calls a causal effect “recoverable” if it can be computed in the presence of a selection mechanism. An important difficulty within omitted label contexts is that they are what we will call “irreversible.” Zero-probability labels cannot be “weighted-up” to transform the distribution to the that of the general population. With respect to covariate adjustments, this leads to incompatible quantities that make the causal effects unrecoverable. 6.2 Simpson’s Paradox We will ease into our dissection of errors in causal quantities by discussing Simpson’s paradox. This discussion will rely on hypothetical observational data on a treatment 𝑇 and it’s outcome 𝑌 , given in Table 6.1 (a). The example is motivated by the effect of illness severity on the probability of treatment prescription. Patients improve with treatment within both severe and mild cases, but treatment is primarily given to more severe illnesses that have a lower overall rate of improvement. As a result, treated patients have lower rates of improvement than untreated patients. The driver for Simpson’s paradox is the difference in severity between those who did and did not receive treatment, and the effect of this severity on patient outcomes. That is to say the treatment and control groups are not exchangeable. A natural solution to this is to reweight the rows of our table to exchangeability by emphasizing the severely ill patients who did not receive treatment and the mildly ill patients who did receive 67 𝑇 𝑇 𝑡 (0) 𝑡 (0) 𝑡 (1) 𝑡 (1) 𝑋 𝑥 (0) 𝑥 (1) 𝑥 (0) 𝑥 (1) 𝑦 (0) 𝑦 (1) 3 1 0 7 7 0 1 3 (a) A specification of counts for Simpson’s Paradox. 𝑇 𝑡 (0) 𝑡 (0) 𝑡 (1) 𝑡 (1) 𝑋 𝑥 (0) 𝑥 (1) 𝑥 (0) 𝑥 (1) 𝑦 (0) 𝑦 (1) 𝑦 (2) 𝑡 (1) 3 1 0 7 7 0 1 3 0 99 99 0 𝑡 (1) 𝑡 (1) 𝑡 (0) 𝑡 (0) 𝑡 (0) 𝑋 𝑥 (0) 𝑥 (1) 𝑥 (2) 𝑥 (0) 𝑥 (1) 𝑥 (2) 𝑦 (0) 2 0 1 0 2 1 𝑦 (1) 1 2 0 1 0 2 𝑦 (2) 0 1 2 2 1 0 (b) An augmentation of (a) with a third column that shifts (c) A specification of counts that mimics Condorcet’s parathe distribution of 𝑋. dox. Table 6.1: Three tables discussed in this paper. treatment. This is accomplished by reweighting datapoints (𝑡, 𝑥, 𝑦) according to the inverse propensity of receiving the treatment that they got, 𝑤(𝑡, 𝑥, 𝑦) = 1/Pr(𝑡 | 𝑥), sometimes referred to as “Inverse Propensity weighting” (IPW) (G. W. Imbens and Rubin, 2015). For Table 6.1 (a), this corresponds to weighting up the second and third rows by a factor of 10. When this reweighting is interpreted as a synthetic study on 40 participants (20 treated and 20 control, each with a 10 : 10 split on severity), the new apparent treatment effect is 13/20 − 7/20 = 30%. An alternative perspective is that the causal effect of the treatment lies in the outcome changes within each severity group. By separately considering the severe and mild patients, we can average outcomes according to the marginal probability distribution of severity. Following this intuition, the “backdoor adjustment” (Judea Pearl, 2009) calculates the probability distribution of 𝑌 = 𝑦 (𝑖) under an intervention of 𝑇 = 𝑡 ( 𝑗) : ∑︁ Pr(𝑦 (𝑖) | do(𝑡 ( 𝑗) )) ≔ Pr(𝑥) Pr(𝑦 (𝑖) | 𝑥, 𝑡 ( 𝑗) ). 𝑥 The difference between the two possible interventions gives the “average treatment effect” (ATE) ATE = Pr(𝑦 (1) | do(𝑡 (1) )) − Pr(𝑦 (1) | do(𝑡 (0) )) = 1/1 + 3/10 2 − 7/10 + 0/1 2 = .3. Notice that the marginal probability distribution of 𝑋 is uniform, corresponding to an equal weighting of the 𝑥 (0) and 𝑥 (1) rows in Table 6.1 (a). In fact, both IPW and backdoor approaches result in the same weightings of the rows of the table because Pr(𝑡, 𝑥)/Pr(𝑡 | 𝑥) = Pr(𝑥). Simpson’s paradox has been the subject of a long list of works for which it would be impossible to do a full justice to. Judea Pearl (2022) and Hernán, Clayton, and 68 Keiding (2011) describe Simpson’s paradox as “solved” by causal modeling because the confounding role of 𝑋 tells the researcher how to proceed, namely that they must separately consider outcome changes for each assignment of 𝑥. We will focus on one key takeaway: the choice of how to re-weight sub-cases (rows of our table) plays a key role in the conclusion of a study, sometimes reversing the apparent relationship (as in Simpson’s Paradox). An important observation that there is a geometry to the way in which these errors occur. Notice that the reversal in Table 6.1 (a) would be maximized by further increasing the probability of rows 1 and 4, e.g. by changing the 3, 7 counts to 300, 700. This reweighting strengthens the dependence of 𝑇 on 𝑋, resulting in an unadjusted treatment effect that approaches Pr(𝑦 (1) | 𝑡 (1) , 𝑥 (1) ) − Pr(𝑦 (1) | 𝑡 (0) , 𝑥 (0) ) = .4. While we will not dive further into the geometry of Simpson’s paradox, the existence of this structure stands as motivation for the structures we will study in Chapter 7. 6.3 Omitted Label Contexts Now that we understand the potential effects of reweighting distributions on covariates, we will move our focus to the study of omitted label contexts. Recall that these contexts involves the removal of some labels while preserving the relative probabilities of the non-removed labels. This removal can shift the apparent distribution of any variable that is associated with 𝑌 , including both treatment 𝑇 and covariates 𝑋. Causality within Omitted Label Contexts Consider a second hypothetical dataset that augments Table 6.1 (a) with an additional column, shown in Table 6.1 (b). We will focus on the context that excludes the deceased (𝑦 (2) ) label, meaning that the observed dataset is equivalent to Table 6.1 (a), which we recall has a .3 ATE on the outcome of 𝑦 (1) . Although the full context has the exact same (uniform) marginal probability distribution on 𝑋, we see a reversal of the ATE on 𝑦 (1) : ATE = 1/100 + 3/10 2 − 7/10 + 0/100 2 = −.195. (6.1) The correct adjustment comes down to a loss of datapoints. The goal is to shift to exchangeable treatment and control distributions in the overall population, which involves weights 𝑤(𝑡 (0) , 𝑥 (0) , 𝑦) = 𝑤(𝑡 (1) , 𝑥 (1) , 𝑦) = 10 and 𝑤(𝑡 (0) , 𝑥 (1) , 𝑦) = 𝑤(𝑡 (1) , 𝑥 (0) , 𝑦) = 1, or any other rescaling. Notice that this reweighting differs from the reweighting suggested by IPW and the backdoor adjustment in the left table. Instead of scaling up the (𝑡 (1) , 𝑥 (0) ) and (𝑡 (0) , 𝑥 (1) ) rows to make up for a bias towards 69 𝑋 (a) 𝑇 𝑋 𝑌 (b) 𝑇 𝑋 𝑌 (c) 𝑇 𝑌 𝐶 Figure 6.1: (a) A causal DAG depicting confounding from a common cause 𝑋. (b) The causal DAG that “severs” 𝑋 → 𝑇 by reweighting for exchangeability. (c) The causal DAG depicting the effect of a omitted label context 𝐶 which has been conditioned on. giving treatment to more severe cases, the correct reweighting does the opposite, resulting in a seemingly less exchangeable distribution. This effect can be understood by graphically modeling the selection bias as in Bareinboim and Tian (2015), shown in Figure 6.1. Figure 6.1 (a) shows the graph describing a confounding variable 𝑋 causing both 𝑇 and 𝑌 . The goal of IPW and the backdoor adjustment is to reweigh the distribution to fit the DAG in Figure 6.1 (b), i.e. the distribution of 𝑋 is exchangeable in both 𝑡 (0) and 𝑡 (1) or equivalently 𝑋 ⊥⊥ 𝑇. Figure 6.1 (c) shows the effect of restricting the labels of 𝑌 within a dataset context (such as with omitted label contexts), which involves conditioning on a child of 𝑌 . 𝑋 and 𝑇 are not d-separated2, because conditioning on a variable that is causally downstream of both 𝑋 and 𝑇 can induce a spurious correlation. The effect we have outlined darkens the outlook for causal inference in omitted label contexts: We are no longer guided by the principle of exchangeability in our observed data and finding the correct adjustment requires knowledge of the context-induced distribution Pr(𝑋, 𝑇, 𝑌 | 𝑦 ∈ {𝑦 (𝑖) , 𝑦 (𝑖) }). This distribution cannot be obtained without extending the study to all labels. To progress to Level 4 knowledge, now investigate what can be learned from many conclusions under different omitted label contexts. Such a network is not a replacement for a single study on all of the potential labels, but is a realistic setting for patching omitted label bias. We study networks of conclusions that ignore the limitations of their omitted labels and perform standard adjustments. We will see that these networks have limitations, much like the limitations to Simpson’s paradox. 6.4 Networks of Contexts Before we discuss the structures within networks of omitted label contexts, we will introduce another paradox from social choice theory, known as the Condorcet Paradox (Nicolas et al., 1785). We will see that this paradox and its structure are 2 see Judea Pearl (2009) for the full rules of d-separation 70 𝑦 (1) © ­ 2 ­ 3 ­ ­ ­ 𝑦 (3) « 2 3 𝑦 (1) ª © ® ­ 2 0 ® 1­ 3 = ® 3­ ® ­ ­ 𝑦 (3) 𝑦 (2) ® ¬ « 1 𝑦 (1) ª © ® ­ 1 1 ® 1­ + ® 3­ ® ­ ­ 𝑦 (3) 𝑦 (2) ® ¬ « 1 𝑦 (1) ª © ® ­ 0 1 ® 1­ + ® 3­ ® ­ ­ 𝑦 (3) 𝑦 (2) ® ¬ « 1 0 ª ® ® ® ® 𝑦 (2) ® ¬ Figure 6.2: The Condorcet paradox as an aggregation of rankings. deeply related to the networks we will study. The Condorcet Paradox The Condorcet paradox works as follows: three voters each have preferences 𝑦 (0) → 𝑦 (1) → 𝑦 (2) , 𝑦 (1) → 𝑦 (2) → 𝑦 (0) , and 𝑦 (2) → 𝑦 (0) → 𝑦 (1) , with 𝑎 → 𝑏 indicating a preference of 𝑎 over 𝑏. The key to these preferences is that the order has been rotated three times, meaning that each candidate is preferred to its successor mod 2. That is, 𝑦 (𝑖) → 𝑦 (𝑖+1 mod 2) ) in two out of the three voters. The result is an aggregate cycle of preference 𝑦 (0) → 𝑦 (1) → 𝑦 (2) → 𝑦 (0) with frequencies of 2/3 voters for each edge. This paradox can be generalized into what we will call an “aggregation of rankings” (AR) – a complete directed-graph3 on the set of labels Y with weights on each 𝑦 (𝑖) → 𝑦 ( 𝑗) corresponding to the fraction of voters who prefer 𝑦 (𝑖) to 𝑦 ( 𝑗) . AR structures are a convex combination of total orderings (i.e. graphs with edge weights of 0 or 1), with component weights corresponding to the fraction of voters carrying each total ordering. See Figure 6.2 for an illustration of this perspective for the Condorcet paradox. As a result, the space occupied by all possible AR structures is known as the “linear ordering polytope,” which has been the subject of extensive study (P. C. Fishburn, 1992; Alon, 2002). The preferences of voters in the Condorcet paradox can be embedded into a table of frequencies, with each voter becoming a specific value for covariate 𝑋. Table 6.1 (c) demonstrates this using the counts 2 > 1 > 0 to induce high, medium, and low preference. Notice that the order of preferences for each 𝑥 in the 𝑡 (1) half of the table (first three rows) exactly correspond to the order of preferences given by the voters in the Condorcet paradox, starting with 𝑦 (0) → 𝑦 (1) → 𝑦 (2) for 𝑥 (0) and cycling the order with the other values of 𝑋. 3These graphs are always complete, but we use graph terminology as in B. Mazaheri, Jain, and Bruck (2021) in order to reference properties that are dependent on cycles. 71 𝑈 (a) 𝑋 𝑈 𝑌 (b) 𝐼 𝑋 𝑈 𝑌 (c) 𝑋 𝑌 Figure 6.3: (a) 𝑈 exerts unobserved influence on 𝑌 , but not on the covariate 𝑋, meaning an auxiliary training task predicting 𝑌 using 𝑋 can remove the effect of 𝑈. (b) shows a DAG of an instrumental variable setup. (c) shows an unobserved active path for which an auxiliary training task predicting 𝑌 using 𝑋 can capture and isolate information about 𝑈. The 𝑡 (0) half of the table complements the 𝑡 (1) half so that the counts for (𝑡 (0) , 𝑥 (𝑖) , 𝑦 ( 𝑗) ) and (𝑡 (1) , 𝑥 (𝑖) , 𝑦 ( 𝑗) ) always sum to three. As a result, restricting our table to any two columns still yields a uniform probability distribution on 𝑋, i.e. Pr(𝑥 (0) | 𝑦 ∈ {𝑦 (𝑖) , 𝑦 ( 𝑗) }) = Pr(𝑥 (1) | 𝑦 ∈ {𝑦 (𝑖) , 𝑦 ( 𝑗) }) = Pr(𝑥 (2) | 𝑦 ∈ {𝑦 (𝑖) , 𝑦 ( 𝑗) }). This is the distribution that a naive study would average over when applying a backdoor adjustment, meaning that Pr(𝑦 (0) | do(𝑡 (1) ), 𝑦 ∈ {𝑦 (0) , 𝑦 (1) } = 2/3 + 0/2 + 1/1 = 5/9, 3 0/3 + 2/2 + 1/3 Pr(𝑦 (0) | do(𝑡 (0) ), 𝑦 ∈ {𝑦 (0) , 𝑦 (1) } = = 4/9. 3 (6.2) The calculations in 6.2 conclude that the ATE on 𝑦 (0) in the 𝑦 ∈ {𝑦 (0) , 𝑦 1) } context is +1/9. These calculations are the same for the ATE on 𝑦 (1) for 𝑦 ∈ {𝑦 (1) , 𝑦 (2) } and the ATE on 𝑦 (2) for 𝑦 ∈ {𝑦 (2) , 𝑦 (0) } due to the cyclic shifting of columns. Hence, the studies separately conclude that the treatment increases the relative frequency of all three labels, which is clearly impossible. The embedding of the Condorcet paradox into causal conclusions implies a correspondence between aggregations of rankings and backdoor adjustments4. This correspondence will lead to the study of “expert graphs” in Chapter 7. 6.5 Context-Based Features The paradoxes we have presented establish that exchangeability (as it is understood for causal inference) is an insufficient criterion for understanding data contexts – especially sampling bias and omitted label contexts. Chapter 8 will discuss contextbased (counterfactual) features, which represent specifically engineered scenarios in which prediction tasks contain desirable information. 72 Training Tasks as Filters To filter information, we will draw on and expand upon insight from instrumental variables. Consider Figure 6.3 (a) and notice that the scope of 𝑈 is limited to just 𝑌 and therefore opens up no new causal pathways between variables in the observed system. As a result, we can interpret 𝑈 as an additional source of independent noise for 𝑌 . Such unobserved influence cannot ever be incorporated in a model for 𝑌 with the data at hand. This distinction is related to epistemic uncertainty (i.e. “knowable” randomness) versus and aleatoric (“unknowable” randomness) (Hacking, 2001). Variation due to independent noise (whether or not the uncertainty is epistemic or aleatoric) can be removed using prediction tasks. For example, in Figure 6.3 (a) if I were to train a model 𝑌ˆ (·) which predicts 𝑌 using 𝑋, the output of such a model 𝑌ˆ (𝑋) would be independent of 𝑈 by virtue of it being a function of 𝑋, which is also independent of 𝑈. This can also be thought of as retrieving a response function that is indexed and marginalized over the unobserved noise, 𝑌ˆ (𝑋) ≈ E𝑈 (Γ𝑢 (𝑋)). (6.3) Such an observation is useful even when the scope of 𝑈 extends beyond 𝑌 . For example, consider Figure 6.3 (b), which graphically models the conditions of an instrumental variable. 𝑈 confounds both 𝑋 and 𝑌 , but 𝑋ˆ (𝐼) is independent of 𝑈 by virtue of 𝐼’s independence from 𝑈. Therefore, 𝑋ˆ (𝐼) can be used to isolate 𝑋’s impact on 𝑌 from its correlation via the 𝑈 backdoor path. The effectiveness of this strategy is, of course, dependent on the unaffected variable’s (i.e. 𝐼’s) ability to accurately predict the cause we wish to study (i.e. 𝑋). This can be quantified using information theory. For example, in Figure 6.3 (b), 𝐼 ⊥⊥ 𝑌 | 𝑋, 𝑈, which lets us apply the data processing inequality: I( 𝑋ˆ (𝐼) : 𝑌 ) ≤ I( 𝑋ˆ (𝐼) : 𝑋, 𝑈) = I( 𝑋ˆ (𝐼) : 𝑋) ≤ I(𝐼 : 𝑋). (6.4) Hence, we cannot determine causal relationships that exceed the predictive power of our instrumental variable (𝐼) on our cause (𝑋). Controlling Filtration with Active Paths The key to further engineering these training tasks involves studying the active paths between the label and its covariates. This is formalized by Lemma 24. 4 or any other case-based weighting 73 Lemma 24 (Unobserved common-cause information). Given a causal DAG 𝔊 = (V, E), for any 𝑈, 𝑉𝑖 , 𝑉 𝑗 ∈ V where 𝑈 d-separates 𝑉𝑖 , 𝑉 𝑗 (i.e. 𝑉𝑖 ⊥⊥ 𝑉 𝑗 | 𝑈 by the causal Markov condition), we have that I(𝑉𝑖 , 𝑉 𝑗 : 𝑈) ≥ I(𝑉𝑖 : 𝑉 𝑗 ). Proof. A visualization of this proof is given in Figure 6.4. Colors are added to the equations in the proof to match this figure. Begin with the definition of mutual information: I(𝑉𝑖 , 𝑉 𝑗 : 𝑈) ≔ H(𝑉𝑖 , 𝑉 𝑗 ) − H(𝑉𝑖 , 𝑉 𝑗 | 𝑈). (6.5) We can expand the joint entropy of both terms as follows, H(𝑉𝑖 , 𝑉 𝑗 ) =H(𝑉𝑖 | 𝑉 𝑗 ) + H(𝑉 𝑗 | 𝑉𝑖 ) + I(𝑉𝑖 : 𝑉 𝑗 ) (6.6) H(𝑉𝑖 , 𝑉 𝑗 | 𝑈) =H(𝑉𝑖 | 𝑉 𝑗 , 𝑈) + H(𝑉 𝑗 | 𝑉𝑖 , 𝑈) + I(𝑉𝑖 : 𝑉 𝑗 | 𝑈) . | {z } (6.7) G =0 because 𝑉𝑖 ⊥ ⊥𝑑 𝑉 𝑗 |𝑈 Together, Equations 6.6 and 6.7 give: I(𝑉𝑖 , 𝑉 𝑗 : 𝑈) =H(𝑉𝑖 | 𝑉 𝑗 , 𝑈)+H(𝑉 𝑗 | 𝑉𝑖 )+I(𝑉𝑖 : 𝑉 𝑗 ) −H(𝑉𝑖 | 𝑈, 𝑉 𝑗 )−H(𝑉 𝑗 | 𝑈, 𝑉𝑖 ) =I(𝑉𝑖 : 𝑈 | 𝑉 𝑗 )+I(𝑉 𝑗 : 𝑈 | 𝑉𝑖 )+I(𝑉𝑖 : 𝑉 𝑗 ) ≥I(𝑉𝑖 : 𝑉 𝑗 ). □ 𝐻 (𝑈) I(𝑉𝑖 : 𝑈 | 𝑉 𝑗 ) H(𝑉𝑖 ) I(𝑉 𝑗 : 𝑈 | 𝑉𝑖 ) I(𝑉𝑖 : 𝑉 𝑗 ) H(𝑉 𝑗 ) Figure 6.4: A visual proof for Lemma 24. Now, consider the DAG in Figure 6.3 (c). We can conclude from Lemma 24 that I(𝑋, 𝑌 : 𝑈) ≥ I(𝑋 : 𝑌 ). (6.8) Similarly, if we apply the lemma to 𝑌ˆ (𝑋), I(𝑌ˆ (𝑋), 𝑌 : 𝑈) ≥ I(𝑌ˆ (𝑋) : 𝑌 ). (6.9) 74 Later in Chapter 8 we will argue that I(𝑌ˆ (𝑋) : 𝑌 ) ≈ I(𝑋 : 𝑌 ), if the model is trained “ideally.” Generally, this indicates that a variable 𝑌 and its prediction from 𝑋 captures information about the variables along active paths from 𝑋 to 𝑌 . With this insight comes the potential to isolate information by activating and deactivating different paths through conditioning. The predictors within these contexts become estimates of counterfactuals. Chapter 8 uses this idea to isolate robustness-improving information. Path-specific information has also been used in fairness contexts (Chiappa, 2019; Dutta, Venkatesh, et al., 2020; Dutta and Hamman, 2023), but with restrictive sufficiency constraints. A broader interpretation of training tasks as information filters may therefore also provide advances in these areas. 75 Chapter 7 EXPERT GRAPHS In this section we give a detailed study of the paradox presented in Chapter 6. This work has been partially published in B. Mazaheri, Jain, and Bruck, 2021, with unpublished components in B. Mazaheri, Jain, Cook, et al., 2023. 7.1 Motivation AI has made considerable progress towards methods for training machine learning models, but privacy laws and data ownership severely limit many consumers’ access to the data necessary for these techniques. In the absence of high quality data, many practitioners rely on pre-trained third-party classifiers and regressors. In order to fully harness these “off the shelf” products, knowledge from different training tasks is put together to address new goals. This process is generally known as decision fusion (Castanedo, 2013). As we have seen in Chapter 6, intuition often fails in this setting. Given an A vs. B classifier that prefers A and a B vs. C classifier that prefers B, one might assume that A is preferable to C. This assumption of transitivity is incorrect for any set of classifiers with decision boundaries that do not meet at a single point (see Figure 7.1). In more than 2 dimensions it is even worse; the three (𝑛 − 1)-dimensional class boundary manifolds need to be aligned on an entire (𝑛 − 2)-dimensional manifold. Barring perfect high-dimensional classifiers, nontransitivity is bound to occur. Nontransitivity has been explored in two related settings: probability theory and voter preferences. In probability, sets of dice with nontransitive winning probabilities (such as A beats B beats C beats A) have been a source of considerable interest (Savage Jr, 1994). Voting theory has studied the Condorcet paradox, where pairwise elections of candidates yield nontransitive preferences. The “linear ordering polytope” generalization corresponds to all possible pairwise election networks in a population of ranked preferences (Alon, 2002; P. C. Fishburn, 1992; McGarvey, 1953; Cohen and Falmagne, 1990; Saari, 2000), which we call Aggegations of Rankings (ARs). Related work focuses on the “Condorcet domain,” which studies conditions necessary for transitivity (Saari, 2009; P. Fishburn, 1996; Monjardet, 2006). Such previous work has only focused on complete graphs of all pairwise elections. 76 Decision Boundaries of Linear SVMs Decision Boundaries of Nonlinear SVMs 10.0 4 7.5 2 5.0 2.5 0 0.0 2.5 2 5.0 7.5 4 4 2 0 2 4 10.0 10 5 0 5 10 Figure 7.1: Examples of decision boundaries from classifiers trained on pairs of differently colored Gaussians. Regions of nontransitivity are shaded in grey. In the case on the right, this includes both the center and the outer region of all outliers. The classifiers on the left are trained with sklearn’s linear SVM, and on the right they are trained with sklearn’s nonlinear kernel SVM. The level of nontransitivity is limited in these frameworks. For example, it is possible to construct a population of voters where 2/3 prefer A to B to C to A, but impossible to create such a cycle with 100% preference. This effect is known as the triangle inequality or, as we call it, the curl condition. Other properties of this framework have also been studied (P. C. Fishburn, 1992; Reinelt, 1985; Grötschel, Jünger, and Reinelt, 1985; Gilboa, 1990). This chapter will present an investigation into the parallels between ARs, machine learning classifiers and, more generally, human experts, particularly with respect to the paradoxes discussed in Chapter 6. These expert graphs expand our understanding of nontransitivity and empower further cross-pollination between these fields, such as bounding unknown edges and protocols for deciding on a transitive ordering in a nontransitive region. Additional Notation The following notations are used throughout the chapter. • [ℓ] is used to denote the set {1, 2, . . . , ℓ} for any ℓ ∈ N. • 1 [𝑐] will be used for an indicator function which is 1 if condition 𝑐 is met and 0 otherwise. • 1ℓ denotes an all 1 vector of size ℓ. 77 • △ℓ will be used to denote vectors of length ℓ which are probability distributions. That is, 𝜆 ∈ △ℓ iff 𝜆 ∈ [0, 1] ℓ and 1⊤𝜆 = 1. • We use ≺, ≻, ⪯, ⪰ to denote element-wise inequality. For example, we say w ⪰ v if 𝑤 𝑖 ≥ 𝑣 𝑖 ∀ 𝑖 ∈ [ℓ]. • We will use Co(𝑆) to denote the open convex hull of 𝑆, Co(𝑆) to denote the closed convex hull, and Bo(·) to denote the boundary. • While graphs in other chapters have vertices that are random variables, the vertices in this chapter represent assignments to 𝑌 . Hence, they appear as lowercase letters. Sets and ordered lists of labels are denoted using calligraphic font. 7.2 Aggregations of Rankings and Soft Rankings When setting up the paradox of nontransitivity presented in Chapter 6, Table 6.1 (c) used counts of 2, 1, 0 to induce preference between labels in each row. As this system is effectively cardinal, these preferences differ from those in the Condorcet paradox in that they can be any frequencies between [0, 1]. For this reason, we refer to the induced preferences in each row of our tables as a “soft ranking.” We will now be formal about both aggregations of rankings (ARs) and aggregations of soft rankings (ASRs). Definition 24 (Ranking). A ranking of Y is a function 𝐴 : Y × Y → {0, 1} generated by a total ordering. We use 𝐴(𝑦 (𝑖) , 𝑦 ( 𝑗) ) = 1 to denote preference 𝑦 (𝑖) → 𝑦 ( 𝑗) and 𝐴(𝑦 (𝑖) , 𝑦 ( 𝑗) ) = 1 for 𝑦 (𝑖) ← 𝑦 ( 𝑗) . Definition 25 (Aggregation of Rankings (AR)). An aggregation of rankings is specified by a set of rankings A and a corresponding weight function 𝛼 ∈ △ |A| (indexed by 𝐴 ∈ A). Definition 26 (Aggregate Preference). An aggregation preference in an AR between 𝑦 (𝑖) , 𝑦 ( 𝑗) ∈ Y is defined to be ∑︁ 𝑅A,𝛼 (𝑦 (𝑖) , 𝑦 ( 𝑗) ) ≔ 𝛼 𝐴 𝐴(𝑦 (𝑖) , 𝑦 ( 𝑗) ). 𝐴∈A Corresponding to rankings, ARs, and aggregate preferences 𝑅A,𝛼 , we will have soft rankings, ASRs, and aggregate probabilities 𝐹B,𝛽 . 78 Definition 27 (Soft Rankings). A soft ranking on Y is a function 𝐵 : Y × Y → [0, 1] generated by a categorical probability distribution on Y, p ∈ △ |Y| : 𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ) ≔ 𝑝𝑖 . 𝑝𝑖 + 𝑝 𝑗 Definition 28 (Aggregation of Soft Rankings (ASR)). An aggregation of soft rankings is specified by a set of soft rankings B and a corresponding weight function 𝛽 ∈ △ |B| (indexed by 𝐵 ∈ B). Definition 29 (Aggregate Probability). An aggregate probability in an ASR between 𝑦 (𝑖) , 𝑦 ( 𝑗) ∈ Y is ∑︁ (𝑖) ( 𝑗) 𝐹B,𝛽 (𝑦 , 𝑦 ) ≔ 𝛽 𝐵 𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ). 𝐵∈B Observation 5. Suppose the probability distribution for a covariate adjustment on X (e.g. 𝑋 in our previous examples), Pr(X | 𝑌 ∈ {𝑦 (𝑖) , 𝑦 ( 𝑗) }) is the same for all pairs of labels {𝑦 (𝑖) , 𝑦 ( 𝑗) }. The treatment effects computed for each label pair then correspond to the difference between the aggregate probabilities in two ASRs with a 𝐵 for each assignment of X = x and 𝛽 𝐵 = Pr(x | 𝑌 ∈ {𝑦 (𝑖) , 𝑦 ( 𝑗) }). We will now show that ARs and ASRs on the same cardinality |Y| = 𝑛 can hold the exact same vectors of weights. To make this statement precise, we will denote A as the set of {0, 1}𝑛(𝑛−1) vectors associated with the output values of some 𝐴 in a total ordering and Co(A) as its convex hull. Note that Co(A) is the space of possible vectors of aggregate preferences 𝑅(𝑦 (𝑖) , 𝑦 ( 𝑗) ). Similarly denote B as the set of [0, 1] 𝑛(𝑛−1) vectors generated by some categorical distribution and note its convex hull Co(B) is the space of possible aggregate probability vectors. Theorem 4. Co(A) and Co(B) are the same. It is not difficult to see how soft rankings can be made “harder” by simply increasing the relative difference in counts. That is, replacing 2, 1, 0 in Table 6.1 (c) with 100, 1, 0 more closely simulates an absolute preference. Showing that any set of aggregate probabilities from an ASR can be realized with aggregate preferences from an AR is less obvious. We will prove this direction by using the probability table in an ASR to directly construct a corresponding AR. Probabilities can Emulate Preferences We will begin with the simpler direction, given by Lemma 25. 79 Lemma 25. Co(A) ⊂ Co(B). To prove Lemma 25, we will first show that for every 𝐴 ∈ A, there exists a 𝐵 ∈ B which is arbitrarily close to it. We will then make use of the following more general lemma. Lemma 26. Consider a set of vectors V = {v1 , . . . , v𝑡 } and with v𝑖 ∈ R𝑚 for all 𝑖. If we have Ṽ such that for every 𝜀 > 0 and v ∈ 𝑉, there exists ṽ ∈ Ṽ such that ∥ ṽ − v∥ 2 < 𝜀, then Co(V) ⊆ Co( Ṽ). Convex hulls of finite sets in Rℓ are convex polytopes, which can be expressed as ⊤ an intersection of ℎ halfspaces indexed by 𝑓 with {x : a (f) x𝑥 < 𝑏 ( 𝑓 ) } (Grünbaum ⊤ et al., 1967). Vectors a (f) can be combined as row-vectors of a matrix, 𝐴, so that any convex polytope can be expressed as (1)  (1) ⊤   © (a ) ª ©𝑏 ª  ­ .. ® ­ .. ®  {𝑥 : 𝐴x ≺ b} = x : ­ . ® x ≺ ­ . ® .  ® ­ ® ­    (h) ) ⊤ (ℎ)    𝑏 (a ¬ « ¬  «       (7.1) For convenience, the vectors a (f) , ã (f) are assumed to be unit vectors throughout. The idea behind the proof will be to analyze the movement of the boundaries of the polytope defined by V = {v1 , . . . , v𝑚 } and corresponding polytope defined by the “perturbed points” Ṽ = {ṽ1 , . . . ṽ𝑚 }. They key is to show that a point that is far enough from the boundary of Co(V will also be within Co( Ṽ), given by Lemma 27. This required distance from the boundary will be relative to the amount by which the perturbed points have moved. As we make the perturbation arbitrarily small (i.e. 𝜀 → 0, all points in the interior of the polytope will be included. Lemma 27. Let Co(V) = {x : 𝐴x ≺ b} ˜ ≺ b̃} Co( Ṽ) = {x : 𝐴x ˜ ≺ b̃. as given by Equation 7.1. If 𝐴x ≺ b − 𝜀1ℓ and ∥vi − ṽi ∥ 2 < 𝜀 ∀𝑖, then 𝐴x To prove Lemma 27, we will need to show that the boundaries of the polytopes do not move too much. We will do this using Lemma 28, which bounds how far Bo(Co(𝑉)) can be from Bo(Co( Ṽ)) along a single “face.” 80 Definition 30. Choose 𝑓 ∈ [ℎ]. Define: 𝑊 ( 𝑓 ) = {w : (a (f) ) ⊤ 𝑤 = 𝑏 ( 𝑓 ) , w ∈ 𝑉 } 𝑊˜ ( 𝑓 ) = {ṽi : vi ∈ 𝑊 ( 𝑓 ) } We restrict the size of 𝑊 ( 𝑓 ) = ℓ, which is the number of points needed to define a halfspace in Rℓ . This can be done by allowing for multiple identical af , 𝑏 𝑓 combinations corresponding to all size ℓ subsets of the 𝑣 𝑖 along the boundary. Note that Co(𝑊 ( 𝑓 ) ) describes a “face” of the polytope Co(V) indexed by 𝑓 which is perpendicular to a (f) . Co(𝑊˜ ( 𝑓 ) ) describes the perturbed face. Lemma 28. Choose 𝑓 , 𝑔 ∈ [ℎ] arbitrarily and let 𝑊 ( 𝑓 ) = {w1(f) , . . . , wℓ(f) } and 𝑊˜ ( 𝑓 ) = {w̃1(f) , . . . , w̃ℓ(f) }. For every m (f) ∈ Co(𝑊 ( 𝑓 ) ), we have ( ã (g) ) ⊤ m (f) < 𝑏˜ (𝑔) + 𝜀. Proof. Because 𝑚 ∈ Co(𝑊 ( 𝑓 ) ), there is some 𝜆 ∈ △ℓ with m (f) = ℓ ∑︁ 𝜆𝑖 wi(f) ∈ Co(𝑊 ( 𝑓 ) ) (7.2) 𝜆𝑖 w̃i(f) ∈ Co(𝑊˜ ( 𝑓 ) ) (7.3) 𝑖=1 Consider also m̃ (f) = ℓ ∑︁ 𝑖=1 Note that the norm of the difference between these two vectors is bounded: m (f) − m̃ (f) 2 = ≤ ℓ ∑︁ 𝜆𝑖 (wi(f) − w̃i(f) ) 2 𝑖=1 ℓ ∑︁ 𝜆𝑖 wi(f) − w̃i(f) < 𝜀 2 𝑖=1 | {z } (7.4) <𝜀 Also note that because m̃ (f) ∈ Co(𝑊˜ ( 𝑓 ) ) ⊆ Co( Ṽ), we have that ( ã (g) ) ⊤ m̃ (f) ≤ 𝑏˜ (𝑔) . Now, a simple application of Cauchy-Schwartz gives: ( ã (g) ) ⊤ m (f) = ( ã (g) ) ⊤ ( m̃ (f) + (m (f) − m̃ (f) )) = ( ã (g) ) ⊤ m̃ (f) +( ã (g) ) ⊤ (m (f) − m̃ (f) ) | {z } ≤ 𝑏˜ (𝑔) ≤ 𝑏˜ (𝑔) + ã (g) ˜ (𝑔) <𝑏 2 m (f) − m̃ (f) (7.5) 2 +𝜀 □ 81 With this, we are now ready to prove Lemma 27. Proof. Choose an arbitrary face 𝑔 ∈ [ℎ]. Recall we have x ∈ Co(𝑉) with (a (g) ) ⊤ x < 𝑏 − 𝜀 and we wish to show ( ã (g) ) ⊤ x < 𝑏˜ (𝑔) . Let mx(f) be the result of extending ã (g) from x to Bo(𝑉). This must hit some face with (a (f) ) ⊤ mx(f) = 𝑏 ( 𝑓 ) , so mx(f) ∈ Co(𝑊 ( 𝑓 ) ). That is, find 𝛽 such that mx(f) = 𝛽ã (g) + x ∈ Co(𝑊 ( 𝑓 ) ) (7.6) First, lets bound 𝛽. Notice that because mx(f) ∈ Co(𝑊 ( 𝑓 ) ), we have ! ℓ ∑︁ (f) (f) ⊤ (f) (f) ⊤ (a ) mx = (a ) 𝜆𝑖 wi 𝑖=1 = ℓ ∑︁ (7.7) 𝜆𝑖 (a (f) ) ⊤ wi(f) = 𝑏 ( 𝑓 ) 𝑖=1 So, we have 𝑏 ( 𝑓 ) = (a (f) ) ⊤ mx(f) = 𝛽 (a (f) ) ⊤ ã (g) + (a (f) ) ⊤ x ⇒ 𝜀 < 𝛽 | {z } | {z } ≤1 (7.8) <𝑏 ( 𝑓 ) −𝜀 Now, apply Lemma 28 ( ã (g) ) ⊤ mx(f) < 𝑏˜ (𝑔) + 𝜀 ( ã (g) ) ⊤ x + ( ã (g) ) ⊤ ã (g) 𝛽 < 𝑏˜ (𝑔) + 𝜀 (7.9) ( ã (g) ) ⊤ x < 𝑏˜ (𝑔) . Face 𝑔 ∈ [ℎ] was chosen arbitrarily, so this holds for all halfspaces in the convex polytope. Hence, we have 𝐴x ≺ b. □ We now give the proof for Lemma 25. Proof. For a given 𝐴 from a total ordering, we will show how to find a probability vector p that generates a 𝐵 with values that are arbitrarily close to the 0, 1 values of 𝐴. As already alluded to, this will involve blowing up the ratios of the probabilities in p. Let 𝑦 (0) → . . . → 𝑦 (𝑛−1) be the ordering specified without loss of generality. Let the Í 𝑖th element of p be 𝜀𝑖 /𝑧, where 𝑧 = 𝑛𝑗=1 𝜀 𝑗 is simply a normalization factor so that p remains in the simplex. Notice that this assignment gives us 𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ) = 1 1 + 𝜀 𝑗−𝑖 82 𝑦 (0) © ­ ­ ­ .2 «𝑦 (2) 𝑦 (0) .5 5 8 2 7 3 5 ª © ® ­ 0 ® = .5 ­ ® ­ .3 .4 𝑦 ¬ «𝑦 (1) (2) 𝑦 (0) .5 1 .6 3 5 𝑦 (1) ª © ® ­ ® + .3 ­ ® ­ ¬ « 𝑦 2 7 (2) 2 7 5 7 1 𝑦 (0) ª © ® ­ 0 ® + .2 ­ 1 ® ­ .4 .6 𝑦 ¬ «𝑦 (1) (2) 5 8 0 5 8 3 8 𝑦 (1) ª ® ® ® ¬ Figure 7.2: A demonstration of the inductive step in the proof for Lemma 29. The weights on the LHS are the aggregate probabilities (𝑦 (𝑖) , 𝑦 ( 𝑗) ) that we wish to generate, while the numbers within each vertex 𝑦 (𝑖) specify 𝑝𝑖 . The weights of the graphs on the RHS are given by Equation 7.11, with adjusted (re-normalized) probabilities 𝑝 [−𝑘] specified within the vertices. Three subgraphs are highlighted in red, which represent the smaller sets of labels which can be decomposed according to the inductive hypothesis. for all 𝑗 > 𝑖. Our goal is 𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ) arbitrarily close to 𝐴(𝑦 (𝑖) , 𝑦 ( 𝑗) ) = 1 for 𝑗 > 𝑖. Setting 𝜀 > 0 arbitrarily close to 0 achieves this. Finally, we can apply Lemma 26 to complete our proof. □ Preferences can Emulate Probabilities We will continue with the more difficult direction, given by Lemma 29. Lemma 29. Co(B) ⊂ Co(A). We will prove this direction by showing that every possible instance of 𝐵 is in Co(A). Convexity of Co(A) will then complete the proof. Let A (𝑖) ⊂ A denote the set of rankings for which 𝑦 (𝑖) is the “first choice.” Equivalently, A (𝑖) is defined such that we have 𝐴(𝑦 (𝑖) , 𝑦 ( 𝑗) ) = 1 for all 𝑗 ≠ 𝑖 and 𝐴 ∈ A (𝑖) . We extend this notation to multiple indices, with A (𝑖 𝑗) encoding 𝑦 (𝑖) as first choice and 𝑦 ( 𝑗) as second choice. If the number of rankings that satisfy the restriction is singular, then we remove the bold, e.g. 𝐴 (𝑖 𝑗) . Proof. We will induct on the number of labels 𝑛. The inductive hypothesis is that any 𝐵 ∈ B generated by a categorical distribution p ∈ △𝑛 over 𝑛 labels can be expressed as an AR A, 𝛼. This can easily be shown for the base case of 𝑛 = 2 by assigning 𝛼 𝐴 (01) = 𝑝 0 and 𝛼 𝐴 (10) = 𝑝 1 . Now, assuming the inductive hypothesis to be correct for all 𝐵 on 𝑛 labels, we will show how to construct an AR for a 𝐵 on 𝑛 + 1 labels. 83 First, expand 𝑅A,𝛼 , which we have not yet specified, into aggregate rankings on A (0) , . . . A (𝑛) , 𝑛 ∑︁ 𝑅A,𝛼 = 𝑝 𝑘 𝑅A (𝑘 ) ,𝛼 (𝑘 ) . (7.10) 𝑘=0 Now, consider choosing some label 𝑦 (𝑘) and constructing a new p [−𝑘] ∈ △𝑛 by setting 𝑝 𝑘[−𝑘] ← 0 and all other 𝑝𝑖[−𝑘] ← 𝑝𝑖 /(1 − 𝑝 𝑘 ). Notice that 𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ) is invariant to scaling 𝑝𝑖 , 𝑝 𝑗 (if scaled together). Therefore, this new p [−𝑘] implies a 𝐵 [−𝑘] (𝑦 (𝑖) , 𝑦 ( 𝑗) ) that matches 𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ) for all 𝑖, 𝑗 ≠ 𝑘. 𝐵 [−𝑘] also satisfies the requirements for the inductive hypothesis, so we can assume there is a set of rankings A [−𝑘] and corresponding 𝛼 [−𝑘] that forms an AR for which 𝑅A [ −𝑘 ] ,𝛼 [ −𝑘 ] = 𝐵 [−𝑘] . Each 𝐴 [−𝑘] ∈ A [−𝑘] can now be augmented with a first-choice preference of 𝑦 (𝑘) to generate the set A (𝑘) with corresponding 𝛼 (𝑘) = 𝛼 [−𝑘] . Using this assignment, we have that    𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ) if 𝑖, 𝑗 ≠ 𝑘     𝑅A (𝑘 ) ,𝛼 (𝑘 ) (𝑦 (𝑖) , 𝑦 ( 𝑗) ) = 1 (7.11) if 𝑖 = 𝑘     0 if 𝑗 = 𝑘  Applying Equation 7.11 to Equation 7.10 gives, ∑︁ 𝑝 𝑘 𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ) 𝑅A,𝛼 (𝑦 (𝑖) , 𝑦 ( 𝑗) ) = 𝑝𝑖 + 𝑘≠𝑖, 𝑗 = ( 𝑝𝑖 + 𝑝 𝑗 )𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ) + (1 − 𝑝𝑖 − 𝑝 𝑗 )𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ) = 𝐵(𝑦 (𝑖) , 𝑦 ( 𝑗) ). We chose 𝑖, 𝑗 wlog, so we have constructed an AR which emulates the the soft ranking 𝐵. This completes the inductive proof. As stated earlier, convexity of Co(A) gives the desired result. □ As the proof for Lemma 29 is rather complicated, Figure 7.2 illustrates an example inductive step. 7.3 Curl and the curl condition Not all assignments of weights to edges are realizable ARs/ASRs. Understanding these restrictions can help us bound the outputs of untrained classifiers (a task addressed in Section 7.5), which corresponds to quantifying allowed weights for nonexistent edges. A necessary condition for ARs (and therefore also ASRs) is the “triangle inequality” (P. C. Fishburn, 1992). 84 Lemma 30 (Triangle Inequality). Given an AR, 𝑅A,𝛼 , and any three 𝑦 (𝑖) , 𝑦 ( 𝑗) , 𝑦 (𝑘) , with 𝑖 ≠ 𝑗 ≠ 𝑘, we have 𝑅A,𝛼 (𝑦 (𝑖) , 𝑦 (𝑘) ) ≤ 𝑅A,𝛼 (𝑦 (𝑖) , 𝑦 ( 𝑗) ) + 𝐴(𝑦 ( 𝑗) , 𝑦 (𝑘) ). Observation 6. We can rewrite the triangle inequality in terms of the weights along a cycle. Using 𝑅A,𝛼 (𝑦 (𝑎) , 𝑦 (𝑏) ) = 1 − 𝑅A,𝛼 (𝑦 (𝑏) , 𝑦 (𝑎) ) on the triangle inequality we get 𝑅A,𝛼 (𝑦 (𝑖) , 𝑦 ( 𝑗) ) + 𝑅A,𝛼 (𝑦 ( 𝑗) , 𝑦 (𝑘) ) + 𝑅A,𝛼 (𝑦 (𝑘) , 𝑦 (𝑖) ) ≥ 1, (7.12) 𝑅A,𝛼 (𝑦 (𝑖) , 𝑦 ( 𝑗) ) + 𝑅A,𝛼 (𝑦 ( 𝑗) , 𝑦 (𝑘) ) + 𝑅A,𝛼 (𝑦 (𝑘) , 𝑦 (𝑖) ) ≤ 2. (7.13) From Observation 6, we see that the triangle inequality is in fact a statement about nontransitivity along 3-cycles. To generalize this notion to larger cycles, we define the curl for a function 𝐹, which may be a ranking, soft ranking, or aggregation of rankings or soft rankings. Definition 31 (Curl). Given any function 𝐹 on pairs of labels Y and a cycle c = (𝑐 1 , 𝑐 2 , . . . , 𝑐 ℓ ) on those labels, we define the curl: Curl(𝐹, c) ≔ 𝐹 (𝑦 (𝑐 ℓ ) ,𝑦 (𝑐 1 ) )+ (ℓ−1) ∑︁ 𝐹 (𝑦 (𝑐𝑖 ) , 𝑦 (𝑐𝑖+1 ) ). (7.14) 𝑖=1 This generalization of the triangle inequality to larger cycles is called the “curl condition.” Definition 32 (Curl Condition). Given a ranking, soft ranking, AR, or ASR 𝐹, the curl condition is satisfied if, for all cycles c of length ℓ, we have 1 < Curl(𝐹, c) < ℓ−1. If the curl condition is satisfied, we say that 𝐹 is “curl consistent.” The curl condition is obeyed for all rankings because of acyclicity. ARs are linear combinations of rankings and therefore also follow the curl condition. Finally, the equivalence between ARs and ASRs given in Theorem 4 means that the curl condition must also hold for ASRs. Therefore, the curl condition is necesarry for all of the structures we have discussed in this Chapter. 7.4 When the curl condition is sufficient It is known that the curl condition is insufficient to describe all possible ARs (i.e. there are examples of curl-consistent 𝐹 which cannot be achieved using an AR). However, the condition is sufficient in certain sparse settings. Such sparse settings will require a more graphical description of ARs and ASRs which we will call linear ordering graphs (LOGs) and Expert Graphs (EGs) respectively. 85 𝑣1 𝑣2 𝑣6 𝑣3 𝑣5 𝑣4 Figure 7.3: An example of a triangulated cycle. Definition 33. A linear ordering graph (LOG) 𝐺 = (Y, E, 𝑅A,𝛼 ) is a directed weighted graph with the weight on edge (𝑦 (𝑖) , 𝑦 ( 𝑗) ) ∈ E given by 𝑅A,𝛼 (𝑦 (𝑖) , 𝑦 (() 𝑗)) from an AR. Definition 34. An expert graph (EG) 𝐺 = (Y, E, 𝐹B,𝛽 ) is a directed weighted graph with the weight on edge (𝑦 (𝑖) , 𝑦 (() 𝑗)) ∈ E given by 𝐹B,𝛽 from an ASR. Past work has shown that the curl condition (triangle inequality) is sufficient on graphs with less than 6 vertices (Alon, 2002), a special example of which is a 3-vertex clique (a “triangle”). This result can be composed to prove sufficiency for a class of graphs we call “triangulated cycles.” We conjecture that the curl condition is sufficient to describe all planar LOGs and expert graphs. Triangulated Cycles We will now explore a class of graphs corresponding to a cycle that is triangulated with chords. Definition 35. A triangulated cycle is a planar graph with vertices V = {𝑣 1 , . . . , 𝑣 ℓ } and cyclic edges E = E𝐶 ∪ E𝑇 where E𝐶 represents the edges around the cycle and E𝑇 represents the chords. More specifically, E𝐶 = {(𝑣 1 , 𝑣 2 ), (𝑣 2 , 𝑣 3 ), . . . , (𝑣 𝑛 , 𝑣 1 )} and E𝑇 follows: 1. For all 𝑣 𝑖 , 𝑣𝑖+1 ∈ 𝑉, there exists 𝑣 𝑡 (𝑖,𝑖+1) ∈ 𝑉 such that (𝑣 𝑖 , 𝑣 𝑡 (𝑖,𝑖+1) ) ∈ E𝑇 and (𝑣 𝑖+1 , 𝑣 𝑡 (𝑖,𝑖+1) ) ∈ E𝑇 . 2. There is no (𝑣 𝑖 , 𝑣 𝑗 ) ∈ E and (𝑣 𝑎 , 𝑣 𝑏 ) ∈ E, such that the edges “cross”: 𝑖 < 𝑎 < 𝑗 < 𝑏 or 𝑎 < 𝑖 < 𝑏 < 𝑗. We begin with a lemma that helps us restrict our focus to the smaller components of the cycle. 86 Lemma 31. Consider a weighted digraph or EG/LOG 𝐺 = (𝑉, 𝐸, 𝐹) and two cycles, a = (𝑥, 𝑦, 𝑎 1 , . . . , 𝑎 ℓa −2 ) and b = (𝑦, 𝑥, 𝑏 1 , . . . , 𝑏 ℓb −2 ) which overlap at a single edge (𝑥, 𝑦), consider the “outer-cycle” c = (𝑎 1 , . . . , 𝑎 ℓa −2 , 𝑏 1 , . . . , 𝑏 ℓb −2 ). If a and b follow the curl condition, then so does c. Proof. Note that because 𝐹 (𝑥, 𝑦) = 1 − 𝐹 (𝑦, 𝑥), we have Curl(𝐹, a) + Curl(𝐹, b) − 1 = Curl(𝐹, c) If the curl condition follows for both a and b, then Curl(𝐹, c) ≥ 1 Curl(𝐹, c) ≤ ℓa + ℓb − 1. □ To prove sufficiency of the curl condition for a class of graphs, we will use Theorem 4, which equivalence between expert graphs and LOGs. That is, by showing that a class of DAGs with weights satisfying the curl condition can be decomposed into a linear combination of ranking graphs, we can conclude that it can be achieved as a LOG and therefore also be achieved as an expert graph. To apply this process, we will merge decompositions of sub-graphs, for which we will need the concept of “graph merging.” Definition 36. Consider two LOGs 𝐺 1 = (Y1 , E1 , 𝐹1 ) and 𝐺 2 = (Y2 , E2 , 𝐹2 ). If for all shared edges 𝑒 ∈ E1 ∩ E2 we have 𝐹1 (𝑒) = 𝐹2 (𝑒), then we can merge the graphs to get 𝐺 1 ∪ 𝐺 2 = (Y1 ∪ Y2 , E1 ∪, E2 , 𝐹) with 𝐹 (𝑒) ≔    𝐹1 (𝑒)  if 𝑒 ∈ E1   𝐹2 (𝑒)  if 𝑒 ∈ E2 . Lemma 32. Consider two LOGs 𝐺 1 = (Y2 , E1 , 𝐹1 ) and 𝐺 2 = (Y2 , E2 , 𝐹2 ). If 𝐺 1 and 𝐺 2 share a single edge E1 ∩ E2 = 𝑒 ∗ for which 𝐹1 (𝑒 ∗ ) = 𝐹2 (𝑒 ∗ ), then 𝐺 1 ∪ 𝐺 2 is also a LOG. Proof. By the definition of LOGs, 𝐺 1 and 𝐺 2 must have decompositions into weighted digraphs with binary weightings and no cycles. For 𝐺 1 , sort the decompositions into rankings with 𝐹1 (𝑒) = 0, which define two new LOGs and a weight 𝑤 (1) : ∗ ∗ 𝐺 1 = 𝑤 (1) 𝐺 1𝐹1 (𝑒 )=0 + (1 − 𝑤 (1) )𝐺 1𝐹1 (𝑒 )=1 . (7.15) 87 Do the same for 𝐺 2 , ∗ ∗ 𝐺 2 = 𝑤 (2) 𝐺 2𝐹2 (𝑒 )=0 + (1 − 𝑤 (2) )𝐺 2𝐹2 (𝑒 )=1 . ∗ (7.16) ∗ Note that 𝐺 1𝐹2 (𝑒 )=1 ∪ 𝐺 2𝐹2 (𝑒 )=1 is a LOG and can be merged because 𝑒 ∗ is the only shared edge and has the same weight in both graphs. Now, note that because 𝐹1 (𝑒 ∗ ) = 𝐹2 (𝑒 ∗ ), we have 𝑤 ≔ 𝑤 (1) = 𝑤 (2) . This gives, ∗ ∗ ∗ ∗ 𝐺 1 ∪ 𝐺 2 =𝑤(𝐺 1𝐹2 (𝑒 )=0 ∪ 𝐺 2𝐹2 (𝑒 )=0 ) + (1 − 𝑤)(𝐺 1𝐹2 (𝑒 )=1 ∪ 𝐺 2𝐹2 (𝑒 )=1 ). □ We are now ready to prove that the curl condition is sufficient to describe all expert graphs on triangulated cycles. Theorem 5. Any curl consistent directed graph on a triangulated cycle can be achieved as a LOG (and also an expert graph). Proof. We note that any cycle triangulation can be generated by merging a smaller cycle triangulation and a cycle of length 3 (a triangle) which share a single edge, which exactly matches the setting in Lemma 32. The base case of this recursive process is a triangle, for which we have already argued that a decomposition exists. The proof then follows from using Lemma 32 as an inductive step. □ A decomposition for a dense graph also works as a decomposition on a sparser graph that has only a subset of the dense graph’s edges. Therefore, the curl condition is also sufficient for any graph on a subset of the edges in a triangulated cycle (including a cycle of length > 3). 7.5 Synthetic experts Bounding missing edges The definition of expert graphs is given on digraphs that are not necessarily complete. If we are given an expert graph with missing edge-weights, many possible ASRs or ARs could generate the given edge-weights. Each of these possible models gives different values on the missing edge-weights. Hence, in the absence of a relevant expert, synthetic experts will bound the space of possible edge-weights. To give these bounds, we will use the curl condition.1 1We have shown the curl condition to be necessary, but not sufficient to describe the space of possible expert graphs (via equivalence with LOGs). As a result, stronger bounds may be possible. 88 For this section, we introduce new notation in which a bar indicates a reversal of a path or cycle. For example, if c = (𝑐 1 , 𝑐 2 , 𝑐 3 ) then c = (𝑐 3 , 𝑐 2 , 𝑐 1 ). Definition 37. Let b = (𝑏 1 , 𝑏 2 , · · · , 𝑏 ℓ ) denote an ordered set of vertices/classes in expert graph 𝐺 = (Y, E, 𝐹 (·)) with (𝑏𝑖 , 𝑏𝑖+1 ) ∈ E ∀ 𝑖 ∈ [ℓ − 1]. The weight of a path is given by: ℓ−1 ∑︁ Weight(𝐹, b) ≔ 𝐹 (𝑏𝑖 , 𝑏𝑖+1 ). (7.17) 𝑖=1 The path weight is very similar to the curl, but is defined on a path that does not loop back to its starting point. Lemma 33. Consider a curl consistent graph 𝐺 = (Y, E, 𝐹 (·)), for which (𝑠, 𝑡) ∉ E. Let B (𝑠𝑡) be the set of paths in 𝐺 beginning at vertex 𝑠 and ending at vertex 𝑡: B (𝑠𝑡) ≔ {b : 𝑏 1 = 𝑠, 𝑏 |b| = 𝑡}. Then, 1 − min Weight(𝐹, b) < 𝐹 (𝑠, 𝑡) < min Weight(𝐹, b). b∈B (𝑠𝑡 ) b∈B (𝑡 𝑠) (7.18) Proof. Consider the ℓ-length cycle c = (𝑠, . . . , 𝑡). The curl condition gives 1 < Curl(𝐹, c) < ℓ − 1. (7.19) Notice that, for the path version of c, b = (𝑠, . . . , 𝑡) Curl(𝐹, c) = Weight(𝐹, b) + 𝐹 (𝑠, 𝑡). Substituting this into Equation 7.19 gives 1 − Weight(𝐹, b) < 𝐹 (𝑠, 𝑡) < ℓ − 1 − Weight(𝐹, b). Now, by applying 𝐹 (𝑏𝑖 , 𝑏 𝑗 ) = 1 − 𝐹 (𝑏 𝑗 , 𝑏𝑖 ), we can simplify the upper bound to get 1 − Weight(𝐹, b) < 𝐹 (𝑠, 𝑡) < Weight(𝐹, b). (7.20) Recall that b and b were chosen without loss of generality. Obtaining the optimal bounds over all paths yields the desired result. □ A number of shortest-path algorithms for weighted directed graphs, such as FloydWarshall (Cormen et al., 2022), can be applied to find the best bounds given by Lemma 33. An example of this process is illustrated in Figure 7.4. 89 𝑦 (1) 𝐶1 0.3 0.4 < 0.7 𝑠 𝐶0 𝐶1 𝑡 > 0.1 0.4 0.5 𝐶3 𝑦 (2) Figure 7.4: The shortest path from 𝑠 → 𝑡, given by 𝑠 → 𝑦 (1) → 𝑡 gives 𝑓𝑑 (𝑠, 𝑡) < 0.7. The shortest path from 𝑡 to 𝑠, given by 𝑡 → 𝑦 (2) → 𝑆 gives 𝑓𝑑 (𝑠, 𝑡) > 1 − .9 = .1. Together, we get 𝐹 (𝑠, 𝑡) ∈ (0.1, 0.7). 𝜁-accurate synthetic experts We can restrict the possible edge-weights of a synthetic expert to arbitrary precision. Definition 38. For 𝜁 > 0 call the synthetic expert between 𝑦 (𝑖) and 𝑦 ( 𝑗) a 𝜁-accurate synthetic expert if for some lower bound 𝐿 ∈ [0, 1], 𝑓 (𝑦 (𝑖) , 𝑦 ( 𝑗) ) ∈ (𝐿, 𝐿 + 𝜁). Lemma 34. Given 𝐺 = (Y, E, 𝐹 (·)), the synthetic expert between 𝑠 and 𝑡 is 𝜁accurate if and only if there exists a cycle c = (𝑐 1 , . . . , 𝑐 ℓ ) with (𝑠, 𝑡) = (𝑐𝑖 , 𝑐𝑖+1 ) for some 𝑖 and Curl(𝐹, c) ≤ 1 + 𝜁. Proof. Choose 𝑠, 𝑡 ∈ V wlog and denote the shortest path: s (𝑠𝑡) = arg min Weight(𝐹, b). (7.21) b∈B (𝑠𝑡 ) Recall from Lemma 33 that 1 − Weight(𝐹, s (𝑡𝑠) ) < 𝑓 (𝑠, 𝑡) < Weight(𝐹, s (𝑠𝑡) ). (𝑡𝑠) (𝑠𝑡) Let  𝐿 = 1 − 𝑊 (𝐹, s  ). The desired gap between bounds is Weight(𝐹, s ) − 1 − Weight(𝐹, s (𝑡𝑠) ) ≤ 𝜁, which is true if and only if Weight(𝐹, s (𝑡𝑠) ) + Weight(𝐹, s (𝑠𝑡) ) ≤ 1 + 𝜁 . Now consider the cycle c (𝑠𝑡𝑠) = s (𝑠𝑡) s (𝑡𝑠) , which is the shortest cycle through 𝑠 and 𝑡. We have Curl(𝐺, c (𝑠𝑡𝑠) ) = Weight(𝐹, s (𝑡𝑠) ) + Weight(𝐹, s (𝑠𝑡) ) ≤ 1 + 𝜁 . □ 90 𝜁-accurate synthetic experts describe convergence to fully determining a missing opinion without access to the relevant expert. A natural question is whether any curl consistent graph can be formed entirely using 𝜁− accurate synthetic experts. Lemma 35 shows that any curl consistent network of expert opinions can be formed using 𝜁-accurate synthetic experts. Lemma 35. Given a curl consistent 𝐺 = (Y, E, 𝐹), there exists another weighted digraph 𝐺 ′ = (Y ∪ Y′, E′, 𝐹 ′) with Y′ ∩ Y = ∅ and E ∩ E′ = ∅ such that 1. All synthetic experts 𝑒 ∈ E′ in 𝐺 ′ are 𝜁-accurate. That is, we have 𝐹 (𝑒) ∈ (𝐿 (𝑒) , 𝐿 (𝑒) + 𝜁) for some lower bound 𝐿 (𝑒) . 2. 𝐺 ′ is also curl consistent. Proof. We give a constructive proof, for which an example is given in Figure 7.5. Begin with the set of classes Y and E′ = ∅. Now, for each 𝑒 = (𝑦 (𝑖) , 𝑦 ( 𝑗) ) ∈ E, do the following: 1. Add a vertex 𝑦 (𝑖 𝑗) to Y′ and assign values to 𝐹 ′ (𝑦 (𝑖) , 𝑦 (𝑖 𝑗) ) and 𝐹 ′ (𝑦 (𝑖 𝑗) , 𝑦 ( 𝑗) ) to create a path from 𝑦 (𝑖) → 𝑦 (𝑖 𝑗) → 𝑦 ( 𝑗) of weight 𝐹 (𝑒) + 2𝜁 . 2. Similarly add a vertex 𝑦 ( 𝑗𝑖) to create a path from 𝑦 ( 𝑗) → 𝑦 ( 𝑗𝑖) → 𝑦 (𝑖) of weight 1 − 𝐹 (𝑒) + 2𝜁 . By Lemma 34, 𝑒 is now a 𝜁-accurate synthetic classifier in 𝐺 ′ that includes value 𝐹 (𝑒). If this process created a cycle c′ in 𝐺 ′ with Curl(𝐹 ′, c′) ≤ 1, then there must also be a cycle through 𝐺 with Curl(𝐹, c) ≤ 1, which is a contradiction. Thus, we know our constructed 𝐺 ′ is curl consistent. □ Feasibility of networks of synthetic experts The bounds provided in Lemma 33 interact with each other when many experts are being synthesized on the same graph. For example, the bounds obtained for each edge may not be attainable simultaneously while remaining curl consistent (see Figure 7.6). While the bounds for synthetic experts may not be simultaneously attainable, each value is individually attainable, though fixing a value for a single synthetic expert will induce new bounds on the leftover synthetic experts. 91 𝑦 (1) 2 8 3 7 𝑦 (31) 5 7 𝑦 (13) 2 7 𝑦 (12) 2 8 1 7 3 8 𝑦 (21) 3 8 1 7 1 8 𝑦 (32) 1 5 𝑦 (3) 2 5 𝑦 (2) 2 5 1 5 1 5 𝑦 (23) Figure 7.5: An example of how to create 𝜁-accurate synthetic experts that make up any curl consistent graph by adding additional paths. All edge weights given in color have a + 4𝜁 added to their weight that has been omitted to reduce clutter. Here, the cycle on 𝑦 (1) , 𝑦 (2) , 𝑦 (3) is created by: (1) Adding shortest paths (shown in red on the other part of the cycle) with total weight equal to the desired edge weight 𝑓 (𝑒) + 2𝜁 . (2) Adding shortest reverse paths (shown in blue on the inner part of the cycle) with total weight equal to 1 − 𝑓 (𝑒) + 2𝜁 . 𝑦 (2) A .3 A < .7 .4 A < .7 .4 .3 A A < .7 𝑦 (1) .3 𝑦 (3) .4 A Figure 7.6: An example of how we cannot always achieve any combination of synthetic expert bounds. Here, a choice of just under .7 for all the synthetic experts (given in dashed lines) would violate the curl condition on cycle 𝑦 (1) → 𝑦 (2) → 𝑦 (3) → 𝑦 (1) . 92 7.6 Conclusion This chapter explores networks of classifiers trained on different tasks. We build intuition for understanding how these classifiers can give nontransitive results and develop a framework inspired by nontransitivity in voting theory. We proved equivalence between voting theory ARs and ASRs. We use this equivalence to further our understanding of necessary and sufficient conditions on the weights of expert graph/LOG edges, allowing us to quantify what levels of nontransitivity are allowed. We conjecture that the curl condition is sufficient to describe all possible planar LOGs/expert graphs. From insights into the role of composition in the emergence of nontransitivity, it may be possible to optimize networks of classifiers to minimize their nontransitive regions. 93 Chapter 8 CAUSAL INFORMATION SPLITTING In this section, we present a specific setup in which counterfactual features obtained from auxiliary training tasks can be used to isolate Level 3 knowledge and improve robustness to contextual shifts. This work is published in B. Mazaheri, Mastakouri, et al., 2023. 8.1 Introduction The principle assumption when building any (not necessarily causal) prediction model is access to relevant data for the task at hand. When predicting label 𝑌 from inputs X, this assumption reads that the data is drawn from a (training) probability distribution X, 𝑌 that is identical to the distribution that will generate its use-cases (target distribution). Unfortunately, the dynamic nature of real-world systems makes obtaining perfectly relevant data difficult. Data-gathering mechanisms can introduce sampling bias, yielding distorted training data. Even in the absence of sampling biases, populations, environments, and interventions give rise to distribution shifts in their own right. For example, Zech et al., 2018 found that convolutional neural networks to detect pneumonia from chest radiographs often relied on site-specific features, including the metallic tokens indicating laterality and image processing techniques. This resulted in poor generalization across sites. Understanding these inter-site breakdowns in performance is essential to safety-critical domains such as healthcare. Generalization and Invariant Sets The first attempts at handling dissociation between training and target distributions involved gathering unlabeled samples of the testing distribution. Within domain generalization (DG), covariate shift handles a shift in the distribution of X (Shimodaira, 2000) and label shift handles a shifting Pr(𝑌 ) (Schweikert et al., 2008). DG often assumes a stationary label function Pr(𝑌 | X), which is extremely limiting in real-life applications. To address these limitations, one can assume the label function is stationary for a subset of the covariates in X, called an invariant set in Muandet, Balduzzi, and Schölkopf, 2013 and Rojas-Carulla et al., 2018. 94 One approach to finding invariant sets has been to capture shifting information from a collection of datasets (Rojas-Carulla et al., 2018; Magliacane et al., 2018). Such techniques require access to a comprehensive set of datasets that represent all possible shiftings. A causal perspective developed in Storkey et al., 2009 and Judea Pearl and Bareinboim, 2011 instead uses graphical modeling via selection diagrams to model shifting mechanisms. This approach requires access to multiple datasets to learn these mechanisms, but does not require that those datasets span the entire space of possible shifting. Such approaches also allow the use of domain expert knowledge when building selection diagrams. A detailed comparison of stability in the causal and anticausal scenario is given in Schölkopf et al., 2012. Contributions The causal perspective to distribution shift is obscured when we lack direct measurements of the causes and effects of 𝑌 . Such settings arise from noisy measurements, privacy concerns, as well as abstract concepts that cannot be easily quantified (such as “work ethic” or “interests”). Instead, we will focus on a setting where we only measure proxies for the causes and effects of 𝑌 , see Fig. 8.1 for an example. All of these proxies are descendants of U — a case which is common in medicine, where the measured variables are often blood markers (or other tests) that are indicative of an underlying condition. The proxy setting is difficult to address in standard framework. While previous approaches to partially observed systems suggest restricting model inputs to those on stable paths (Subbaswamy and Saria, 2018), no observed proxies satisfy this condition in our setting. That is, the proxies of the unobserved causes are insufficient to fully block the environmental shifts of those causes. We will use concepts from causal inference and information theory to define and study the Proxy-based Environmental Robustness (PER) problem. Our framework will demonstrate that perfection is indeed the enemy of good – some variables (although with an unstable relationship to the target) should still be included as features to build a model with improved stability. A primary goal of this paper will be to distinguish between proxies that are “helpful” or “hurtful” for stability - a property that they inherit from their parents (of which they are proxies). The stability of these unobserved variables depends on their causal structure, which is unobserved. We will present a strategy for feature selection based on properties that propagate from the underlying causal structure to its observed proxies. Specifically, we will build on the observation that post-selecting on a single 95 value of the prediction label 𝑌 induces a special independence pattern, which the proxies also inherit. We use this to classify proxies from partial knowledge of a few “seeds” — a technique we call proxy bootstrapping. It is possible that some proxy variables will contain information about both stable and unstable hidden variables. We call these ambiguous proxies because it is unclear whether they will improve or worsen the model’s transportability. Inspired by node splitting (Subbaswamy and Saria, 2018), we introduce a method we call causal information splitting (CIS), which can improve stability of our models at no cost (and even some benefit) to the distribution shift robustness. CIS isolates stabilizing information using auxiliary prediction tasks that answer counterfactual questions about the covariates. While theoretical guarantees require a number of assumptions, we demonstrate the surprising ability of CIS to separate stabilizing information from ambiguous variables on synthetic data experiments with relaxed assumptions. Furthermore, we utilize CIS to enhance a prediction task on U.S. Census data that was strongly affected by the COVID-19 pandemic. While plenty of experiments have confirmed that techniques for robust models do not consistently provide benefits over empirical risk minimization (Gulrajani and Lopez-Paz, 2021), our proposed technique provides benefits for an income prediction task in the majority of tested states. 8.2 Related Work There is an increasing body of work on domain generalization, see QuinoneroCandela et al., 2008 for an overview. While we focus on proactively modeling shifts, work on invariant risk minimization (Arjovsky et al., 2019; Bellot and Schaar, 2020) has approached this problem when given access to the shifted data on which the models will be used. Recent work further generalizes to unseen environments constituting mixtures (Sagawa et al., 2019) and affine combinations (D. Krueger et al., 2021). Data from multiple environments can also be used for causal discovery (Peters, Bühlmann, and Meinshausen, 2016b; Heinze-Deml, Peters, and Meinshausen, 2018; Peters, Bühlmann, and Meinshausen, 2016a). Another line of work seeks robustness to small adversarial changes in the input that should not change the output (with attacks, e.g. Croce and Hein, 2020 and defenses, e.g. A. Sinha, Namkoong, and Duchi, 2018). Moving from small changes to potentially bigger interventions, work on counterfactual robustness and invariance, introduces additional regularization terms (Veitch et al., 2021; Quinzan et al., 2022). 96 Our work differs by allowing for interventions that change the label. We do not address the tradeoffs associated with robustness and model accuracy in this paper. Such tradeoffs are a natural consequence of restricting the input information for our model, since unstable information is still useful in unperturbed cases. This problem is generally addressed by Oberst et al., 2021; Rothenhäusler et al., 2021 via regularization. The information theoretic decomposition presented in this paper is deeply related to the Partial Information Decomposition, which has been a subject of growing interest in information theory (Bertschinger et al., 2014; Banerjee et al., 2018; Williams and Beer, 2010; Gurushankar, Venkatesh, and Grover, 2022; Venkatesh and Schamberg, 2022). The use of the PID in causal structures was pioneered by Dutta, Venkatesh, et al., 2020 for fairness (Dutta and Hamman, 2023). Our method of extracting counterfactual features using auxiliary training tasks removes information from proxies that is unique to bad features, even though those features are not observed. 8.3 Background General Notation Uppercase letters denote random variables, while lowercase letters denote assignments to those random variables. Bold letters denote sets/vectors. The paper will use concepts from information theory, with H( 𝐴) indicating the entropy of 𝐴, I( 𝐴 : 𝐵) indicating the mutual information between 𝐴, 𝐵, and I( 𝐴 : 𝐵 : 𝐶) indicating the interaction information between 𝐴, 𝐵, 𝐶. Causal Graphical Models Graphically modeling distribution shift makes use of causal DAGs. For a causal DAG 𝔊 = (V, E), the joint probability distribution factorizes according to the local Markov condition, Ö Pr(v) = Pr(𝑣 | pa𝔊 v (𝑉)). 𝑣∈v PA𝔊 (𝑉), CH𝔊 (𝑉) denote the parents and children of 𝑉 in 𝔊. Following the uppercase/lowercase convention, pav (𝑉) is an assignment to PA(𝑉) using the values in v.1 DE𝔊 (𝑉) and AN𝔊 (𝑉) denote the descendants and ancestors respectively. FM(𝑉) = PA(𝑉) ∪ CH(𝑉) denotes the “family.” We will rely on the concepts of 𝑑-separation and active paths to discuss the independence properties of Bayesian networks, which are indtroduced in Chapter 2. See Judea Pearl, 2009 for a more extensive study. 1 PA(𝑉) ⊆ V 97 Active Path Notation In addition to using 𝐴 ⊥⊥𝑑 𝐵 | 𝐶 to indicate d-separation conditioned on 𝐶, we will develop a notation to refer to sets of variables that act as “switches” for d-separation. A C B means that we have both A ̸⊥⊥𝑑 B and A ⊥⊥𝑑 B | C. Conversely, we have A C B if A ⊥⊥𝑑 B, but A ̸⊥⊥𝑑 B | C (i.e. conditioning on C renders A and B d-connected). Graphically Modeling Distribution Shift Borrowing terms from Magliacane et al., 2018, we will begin with a graphical model 𝔊 = (V ∪ U), calling U ∪ V the system variables with (un-)observed variables. In addition, we are also given a set of context variables M, which model the mechanisms that shift our distribution. The augmentation of 𝔊 with M gives what we call the distribution shift diagram (DSD), 𝔊+ = (V ∪ U ∪ M, E ∪ EM ), for which 𝔊 is a subgraph, with additional vertices M introducing shifts along EM . X ⊆ V such that Pr(𝑌 | X) = Pr(𝑌 | X, M) is called an “invariant set” because it blocks all possible influence from the mechanisms of the dataset shift. Judea Pearl and Bareinboim, 2011 shows this framework is capable of modeling sampling bias and population shift. 8.4 Setting This paper will consider the Proxy-based Environmental Robustness (PER) setting. PER focuses on the role of proxy variables in feature selection by assuming all of the causes and effects U = FM(𝑌 ) are unobserved.2 We are given access to a list of “visible proxy variables” V \ {𝑌 } which are descendants of at least one 𝑈 ∈ U. Hence, V can be thought of as the union of overlapping subsets CH(𝑈) for each 𝑈 ∈ U. We will assume that there are no edges directly within U or within V, which we call systemic sparsity. See Figure 8.1 for an example of this setting. This assumption enforces two useful independence properties: (1) 𝑉𝑖 ⊥⊥ 𝑉 𝑗 | 𝑈 for 𝑉𝑖 , 𝑉 𝑗 ∈ CH(𝑈) and (2) 𝑈𝑖 ⊥⊥ 𝑈 𝑗 | 𝑌 for 𝑈𝑖 ≠ 𝑈 𝑗 ∈ U. Systemic sparsity guarantees that a discoverable causal structure exists within the unobserved variables and simplifies the interactions between the proxies. We will build our theory on distribution shift diagrams 𝔊+ = (V ∪ U ∪ M, E ∪ EM ) with one 𝑀𝑖 ∈ M connected to a corresponding 𝑈𝑖 ∈ U. Each 𝑀𝑖 models a different shifting mechanism for each unobserved cause and effect of 𝑌 . It is common to 2This assumption is not necessary but allows us to focus on more difficult questions that have not been answered by previous work. Namely, direct causes and effects can be visible or have perfect proxies without changing the results of the paper. 98 𝑌 𝑀1 𝑀3 𝑀2 𝑈1 (a) 𝑉 𝑉2 1 𝑈3 𝑈2 𝑉3 𝑉5 𝑉4 𝑉6 𝑉7 Income Pandemic Interests (b) Employment Education Residence Commute Medicaid Eligibility Medicaid Status Figure 8.1: Examples of the 𝔊+ considered for the paper. (a) shows a generic setup where 𝑈1 is a hidden cause of 𝑌 , and 𝑈2 , 𝑈3 are hidden effects. (b) shows a plausible model explaining the success of our real-data experiment in Section 8.7. assume there is no direct shifting mechanism acting on 𝑌 — which comes without loss of generality since such a mechanism can be thought of as another unobserved cause (Judea Pearl and Bareinboim, 2011; Peters, Bühlmann, and Meinshausen, 2016b). In this setting, a perfect invariant set X in which 𝑌 ⊥⊥𝑑 M | X does not exist. PER will instead seek to minimize the influence of the context variables on our label function. Borrowing concepts from information theory, the task in PER corresponds to finding a set of features X that minimizes the conditional mutual information between the label and the environment. We call this quantity, I(𝑌 : M | X), the context sensitivity. To allow for feature engineering, we define these features to be the output of a function, X = 𝐹 (V \ {𝑌 }) which can capture higher-level representations of V \ {𝑌 }. Challenges in PER The PER setting is difficult to address using existing methods. Building a model on the causes PA(𝑌 ) as in Schölkopf et al., 2012 is impossible because all of the causes are unobserved. Furthermore, finding a separating set as in Magliacane et al., 2018; Judea Pearl and Bareinboim, 2011 is also impossible for the same reason. Proxies can contain combinations of both stable and unstable information when they are connected to multiple 𝑈 ∈ U. Introduced in Subbaswamy 99 and Saria, 2018, “node splitting” requires knowledge of the structural equations that govern a vertex to remove unstable information from ambiguous variables, which can only be learned if the causes of the split node are observed. This requirement limits node splitting’s power in the proxy setting. Invertible Dropout Functions We will demonstrate the failure of existing approaches in this setting using a counterexample built on structural equations models with cleanly interpretable entropic relationships. This construction will show the cost of restricting features to those with stable paths to the prediction variable 𝑌 , and serve as a framework for understanding the problem in general. For a discussion of relaxations, see Sec. 8.8 and for a demonstration that our method can work in real-world settings (where the assumption does not hold), see Sec.8.7. Our restricted structural equations give edges from 𝐴 to 𝐵 described by an invertible function with “dropout” noise, 𝐵 ( 𝐴) ( 𝐴) =    T𝐴,𝐵 ( 𝐴)  with probability 𝛼 𝐴,𝐵  𝜙  with probability 1 − 𝛼 . (8.1) T𝐴,𝐵 (·) is a function that is invertible, with T𝐴,𝐵 (𝜙) = 𝜙. The probability that information from the parent is preserved is given by 𝛼 𝐴,𝐵 ∈ [0, 1]. We will refer to 𝐵 ( 𝐴) ( 𝐴) ≠ 𝜙 as “transmission,” and 𝛼 𝐴,𝐵 as the “probability of transmission.”3 𝜙, called “null”, is a value that represents the dropout, or the failure of the edge to “transmit”. The structural equation for a vertex 𝐵 given its parents is a deterministic function of these 𝐵 ( 𝐴) , 𝐵 = T𝐵 ({𝐵 ( 𝐴) ( 𝐴) for 𝐴 ∈ PA(𝐵)}), (8.2) where T𝐵 is not necessarily an invertible function. For functions with many children, the probability that at least one of their children transmits is Ö 𝛼 𝐴,CH( 𝐴) := 1 − (1 − 𝛼 𝐴,𝐵 ). (8.3) 𝐵∈CH( 𝐴) 3The direction of the edge for these 𝛼 of the vertices is unimportant. 𝐴,𝐵 will sometimes be arbitrary, in which case the ordering 100 𝐴1 𝐴2 𝐵 𝐴1 𝐴2 𝐵 ( 𝐴1 ) 𝐵 ( 𝐴2 ) 𝐵 Figure 8.2: A diagram showing separability. Separability and Faithfulness If T𝐵 is invertible, we say that 𝐵 is a separable variable, which means that a child 𝐵 with more than one parent can be split into separate disconnected vertices 𝐵 ( 𝐴) for 𝐴 ∈ PA(𝐵), each with the structural equation given by Equation 8.1 (See Figure 8.2). Separable variables make up a special violation of faithfulness in that conditioning on separable colliders no longer opens up active paths, illustrated by Lemma 36. Lemma 36 (Separability violates faithfulness). If 𝑈1 then 𝑈1 ̸⊥⊥𝑑 𝑈2 | 𝑉, but 𝑈1 ⊥⊥ 𝑈2 | 𝑉. 𝑉 𝑈2 and 𝑉 is separable, The proof follows from the definition of mutual information and the fact that 𝑈1 ⊥⊥ 𝑈2 | 𝑉. Our setting will rely on the assumption of faithfulness of the sub-graph on the U ∪ {𝑌 } vertices for proxy bootstrapping, as is the case for algorithms attempting any degree of structure learning. Specifically, we will require that any active path between two proxies 𝑉𝑖 , 𝑉 𝑗 that does not travel through any other vertices in V must imply statistical dependence (we call this “partial faithfulness”). When we move to causal information splitting, we will allow specific violations of faithfulness that come from separable proxies V in order to illustrate an ideal use-case of our method. This does not contradict partial faithfulness. Transmitting Active Paths A convenient aspect of these structural equations is that 𝛼 𝐴𝐵 controls the mutual information between 𝐴 and its child 𝐵 ( 𝐴) , I( 𝐴 : 𝐵 ( 𝐴) ) =H( 𝐴) − H( 𝐴 | 𝐵 ( 𝐴) ) =H( 𝐴) − Pr(𝐵 ( 𝐴) = 𝜙)H( 𝐴 | 𝐵 ( 𝐴) = 𝜙) − Pr(𝐵 ( 𝐴) ≠ 𝜙)H( 𝐴 | 𝐵 ( 𝐴) ≠ 𝜙). An important insight is that H( 𝐴 | 𝐵 ( 𝐴) = 𝜙) = 0 and H( 𝐴 | 𝐵 ( 𝐴) ≠ 𝜙) = H( 𝐴). Applying this gives I( 𝐴 : 𝐵 ( 𝐴) ) = H( 𝐴) − (1 − 𝛼 𝐴,𝐵 )H( 𝐴) = 𝛼 𝐴,𝐵 H( 𝐴). 101 This aspect generalizes to active single paths. For a length-2 path 𝐴 → 𝐵 → 𝐶, I( 𝐴 : 𝐶) = I( 𝐴 : 𝐶 (𝐵) ) = H( 𝐴) − H( 𝐴 | 𝐶 (𝐵) ). Again, we can break up H( 𝐴 | 𝐶 (𝐵) ) into H( 𝐴 | 𝐶 (𝐵) = 𝜙) = 0 and H( 𝐴 | 𝐶 (𝐵) ≠ 𝜙) = H( 𝐴). Hence, reasoning about mutual information reduces to the task of determining the probability that one of the endpoints is null. In our setup, the dropout events of different edges are independent events. Hence, I( 𝐴 : 𝐶) = 𝛼 𝐴,𝐵 𝛼𝐵,𝐶 H( 𝐴). Conditioning adds an additional complication. Notice that transmitting active paths can “transfer” a conditioning. That is, H( 𝐴 | 𝑥) = 0 when there is only one active path between 𝐴 and 𝑋 (or 𝑋 to 𝐴) and it transmits. In the next section, we will study two cases that emerge in the PER problem: colliders and non-colliders. 8.5 Context Sensitivity We quantify robustness through the dependence on environmental mechanisms and the label function. Definition 39 (Context sensitivity). Context sensitivity of a mechanism 𝑀 ∈ M is defined as I(𝑌 : 𝑀 | X). If X d-separates M from 𝑌 , the context sensitivity is 0 and training on X to predict 𝑌 yields a model that is robust across environments M. We are usually most concerned with the success of our prediction models, something that is limited by the “relevance”, I(𝑌 : X), of our input. This concept is related to context sensitivity, and we can rewrite the sensitivity in terms of the expected relevance across environments. I(𝑌 : 𝑀 | X) = I(𝑌 : 𝑀) − I(𝑌 : 𝑀 : X) = I(𝑌 : 𝑀) − I(𝑌 : X) + I(𝑌 : X | 𝑀). Redundancy Recall that in our setting we assume that all direct causes and effects are unobserved. This unobserved set of parents gives rise to an invariant set S ⊆ U4. We seek to identify a subset of visible proxies X ⊆ V to extract information about S. Definition 40. For a specific 𝑈, we call I(𝑈 : X) = H(𝑈) − H(𝑈 | X) the redundancy between 𝑈 and X. 4The Markov boundary of 𝑌 would also give an invariant set, but could include vertices in M that are parents of effects of 𝑌 . 102 Lemma 37. In the dropout function setting, let CHX (𝑈) := CH(𝑈) ∩ X. I(𝑈 : X) = 𝛼𝑈,CHX (𝑈) H(𝑈). Redundancy in the dropout function setting is controlled by our choice of X via 𝛼𝑈,CHX (𝑈) , the probability of transmission to at least one child. Our graphical assumptions ensure that only one potential active path exists between each 𝑀 ∈ M and 𝑌 — hence each vertex acts as either a collider or a non-collider in the interaction of 𝑀 and 𝑌 (and does not do both). We now demonstrate that redundancy with stable (non-collider) variables generally improves our context sensitivity, whereas redundancy with unstable (collider) variables worsens it. “Good” U If 𝑀𝑖 and 𝑌 do not form a collider at 𝑈𝑖 ∈ U, we say 𝑈𝑖 ∈ UGOOD . From d-separation, we have that 𝑀𝑖 ⊥⊥𝑑 𝑌 | 𝑈𝑖 for all 𝑈𝑖 ∈ UGOOD . For an example, UGOOD = {𝑈1 , 𝑈3 } in Figure 8.1. Let CHX (𝑈𝑖 ) = CH(𝑈𝑖 ) ∩ X. Lemma 38 (Redundancy with UGOOD ). In the dropout function setting, for some 𝑈𝑖 ∈ U, if corresponding 𝑀𝑖 𝑈𝑖 𝑌 , then I(𝑀𝑖 : 𝑌 | X) = 𝛼 𝑀𝑖 ,𝑈𝑖 (1 − 𝛼𝑈𝑖 ,CHX (𝑈𝑖 ) )𝛼𝑈𝑖 ,𝑌 H(𝑀𝑖 ). Lemma 38 comes from multiplying the probability of transmission of each edge along the path 𝑀𝑖 , 𝑈𝑖 , 𝑌 . We also pick up a term requiring that the 𝑈𝑖 , X edges do not transmit, in which case conditioning on X would reduce the entropy of 𝑈 to nothing and close off the path. “Bad” U The inclusion of CH(𝑈𝑖 ) in X could open up active paths via colliders of the form 𝑀𝑖 → 𝑈𝑖 ← 𝑌 . We call the set of these variables UBAD . For an example, UBAD = {𝑈2 } in Figure 8.1. Lemma 39 (Redundancy with UBAD ). In the dropout function setting, 𝑈𝑖 ∈ U, X ⊆ V, if 𝑀𝑖 𝑈𝑖 𝑌 then I(𝑀𝑖 : 𝑌 | X) = 𝛼𝑈𝑖 ,CH𝑋 (𝑈𝑖 ) I(𝑀𝑖 : 𝑌 | 𝑈𝑖 ) Lemma 39 demonstrates that there are still proxies for which inclusion hurts our model’s robustness. Similar concepts can be demonstrated via upper bounds when we allow arbitrary sets of structural equations (see B. Mazaheri, Mastakouri, et al., 2023). Optimizing these upper bounds does not give a guarantee of optimality, but can still point towards a general improvement. 103 Feature Selection Implications The proxy graphical setup requires X U 𝑌 , meaning the relevance of our input is upper bounded by the redundancy with 𝑈, I(X : 𝑌 ) ≤ I(U : X). Lemma 38 shows that proxies of UGOOD help build accurate and universal models, while Lemma 39 shows that proxies of UBAD can trade universality for domainspecific accuracy. Of course, proxies need not lie neatly in these two classes — many proxies contain a combination of universally-relevant and domain-relevant features. This suggests multiple classes of proxy variables. Definition 41. VGOOD := CH(UGOOD ) \ CH(UBAD ) (8.4) VBAD := CH(UBAD ) \ CH(UGOOD ) (8.5) VAMBIG := CH(UBAD ) ∩ CH(UGOOD ) (8.6) The behavior of VGOOD in the dropout function setting shows how restricting models to invariant features fails; a high redundancy with UGOOD is beneficial for the context sensitivity even though the paths from the proxies are unstable. Inclusion of VGOOD in X improves context sensitivity even though VGOOD is not made up of direct causes (as suggested by Schölkopf et al., 2012) or invariant features (as suggested by Magliacane et al., 2018 and (Subbaswamy and Saria, 2018)). For feature selection, an obvious strategy is to choose X = VGOOD , avoid VBAD , and potentially try using some elements in VAMBIG . In the next section we will explore how we can use non-invertible functions to transform these VAMBIG into VGOOD . Proxy Bootstrapping Given the robustness implications of the different classes of 𝑉, their partitioning into good, bad, and ambiguous partitions will be important. We will now demonstrate how to harness partial information to determine these partitions and classify proxies. This step is optional if the role of each proxy is already understood (as is the case when the DAG is known). The results in this subsection will only require the graphical assumptions of the PER setting — i.e. systemic sparsity, partial faithfulness, and an independent shifting mechanism 𝑀𝑖 for each 𝑈𝑖 ∈ U. We begin with an observation about the independence structure of the conditional probability distribution on 𝑌 . 104 Lemma 40 (Linking related proxies). Within the graphical constraints of PER, if 𝑉𝑖 ̸⊥⊥𝑑 𝑉 𝑗 | 𝑌 , then either they have a shared parent (PA(𝑉𝑖 ) ∩ PA(𝑉 𝑗 ) ≠ ∅) or they both have at least one parent that is a cause of 𝑌 (i.e. PA(𝑉𝑖 ) ∩ PA(𝑌 ) ≠ ∅ and PA(𝑉 𝑗 ) ∩ PA(𝑌 ) ≠ ∅). Definition 42. For a DSD 𝔊+ = {V ∪ U ∪ M, E}, define the dependence graph 𝔊𝑌 = (V, E𝑌 ) to be an undirected graph with edges (𝑉𝑖 , 𝑉 𝑗 ) ∈ E𝑌 iff 𝑉𝑖 ̸⊥⊥𝑑 𝑉 𝑗 | 𝑌 . Lemma 40 tells us that 𝔊𝑌 will have a clique on the sets CH𝔊 (𝑈) for 𝑈 ∈ U. Furthermore, conditioning on 𝑌 links its causes, so 𝔊𝑌 has one large clique on CH𝔊 (PA(𝑌 )). This clique structure can be utilized to enhance partial knowledge of CH(UGOOD ) and CH(UBAD ). In this sense, “birds of a feather flock together” — information about each clique’s proxies can be a determined from understanding a single member of that clique. Lemma 41 (Information about seed proxies spreads). If 𝑉𝑖 ∈ VGOOD then all neighbors of 𝑉 𝑗 ∈ NB𝔊𝑌 (𝑉𝑖 ) are not in VBAD - i.e. 𝑉 𝑗 ∈ VGOOD ∩ VAMBIG . If 𝑉𝑖 ∈ VBAD then all neighbors of 𝑉 𝑗 ∈ NB𝔊𝑌 (𝑉𝑖 ) are not in VGOOD - i.e. 𝑉 𝑗 ∈ VBAD ∩ VAMBIG . Lemma 41 suggests an algorithm for bootstrapping the sets VGOOD , VBAD , VAMBIG from a set of “seed” vertices V∗ ⊆ V with known set memberships. 1. Construct 𝔊𝑌 according to Definition 42 using conditional independence tests. 2. For each 𝑉 ∗ ∈ V∗ , if 𝑉 ∗ ∈ VGOOD then add a “good” label to NB(𝑉 ∗ ). If 𝑉 ∗ ∈ VBAD then add a “bad” label to NB(𝑉 ∗ ). 3. All 𝑉 ∈ V \ V∗ with both “good” and “bad” labels receive an “ambigious” label instead. Theorem 6 (Proxy bootstrapping works). Upon termination of proxy bootstrapping all vertices with a single label are correctly described if : 1. Partial faithfulness holds. 2. V∗ has at least one 𝑉 ∗ ∈ V∗ ∩ CH(𝑈) for each 𝑈 ∈ UGOOD ∩ CH(𝑌 ). 3. V∗ has at least one 𝑉 ∗ ∈ V∗ ∩ CH(PA(𝑌 )). 4. V∗ has at least one 𝑉 ∗ ∈ V∗ for each 𝑈 ∈ UBAD . 105 8.6 Causal Information Splitting This section will expand our theory into feature engineering, which allows us to build inputs on functions of V. A main takeaway from Section 8.5 was that we should build models using proxies for UGOOD and avoid using features that are proxies for UBAD . The extension of this to engineered features is to build a model on functions of proxies for which the output of those functions is related to UGOOD and not related to UBAD . We present two lemmas to formalize this notion. g X (𝑈𝑖 ) be the children or functions of children of 𝑈𝑖 in X. Lemma 42 shows that Let CH g X (𝑈𝑖 )) building models with more redundancy with UGOOD (i.e. lower H(𝑈𝑖 | CH improves our context sensitivity in the dropout function setting.5 Lemma 42 (Engineering redundancy for UGOOD ). In the dropout function setting, if 𝑈𝑖 ∈ UGOOD then g X (𝑈𝑖 )). I(𝑀𝑖 : 𝑌 | X) = 𝛼 𝑀𝑖 ,𝑈𝑖 𝛼𝑈𝑖 ,𝑌 H(𝑈𝑖 | CH Of course, even good proxies are related to UBAD through their connection to 𝑌 , so X ⊥⊥ UBAD is impossible. Instead, Lemma 43 tells us that if we avoid redundancy with UBAD after conditioning on 𝑌 , we do not pick up any context sensitivity from the associated shifting mechanisms. Lemma 43 (Avoiding redundancy with UBAD ). For some 𝑈𝑖 ∈ UBAD , if we maintain I(𝑈𝑖 : X | 𝑌 ) = 0, then I(𝑀𝑖 : 𝑌 | X) = 0. Recall that ambiguous proxies contain information about both UGOOD and UBAD . The inclusion of an ambiguous proxy 𝑉𝐴 improves context sensitivity because of its redundancy with UGOOD via Lemma 42. This section will develop a technique for filtering 𝑉𝐴 into 𝐹 (𝑉𝐴 ), which will satisfy the conditions in Lemma 43. To do this, we will require separability. Separable Ambigious Proxies Consider the setup in Figure 8.3, where 𝑉𝐺 ∈ VGOOD , 𝑉𝐵 ∈ VBAD , and 𝑉𝐴 ∈ VAMBIG . 𝑉𝐴 is generated by invertible T𝐴 , making it a separable ambiguous proxy (SAP).6 Splitting 𝑉𝐴 into components allows us to isolate the origins of its ambiguity — the mixing of good information from 𝑉𝐴(𝐺) and bad information from 𝑉𝐴(𝐵) . 5Appendix C shows that redundancy with UGOOD lowers an upper bound on context sensitivity in more general cases 6While we may still be able to gain useful information from non-separable proxies, the tradeoffs are difficult to quantify and hence beyond the scope of this paper. 106 𝑌 𝑀𝐺 𝑀𝐵 𝑈𝐺 𝑉𝐺 𝑈𝐵 𝑉𝐴(𝐺) 𝑉𝐴(𝐵) 𝑉𝐵 𝑉𝐴 Figure 8.3: 𝑉𝐺 ∈ VGOOD , 𝑉𝐵 ∈ VBAD . 𝑉𝐴 ∈ VAMBIG is a linear transformation of two components, 𝑉𝐴(𝐺) , 𝑉𝐴(𝐵) , which are good and bad respectively. Isolation Functions We would like to isolate 𝑉𝐴(𝐺) from 𝑉𝐴 to avoid paying the penalty for 𝑉𝐴(𝐵) . We will do this using isolation functions. Definition 43. We define an isolation function of 𝑉𝑖 on 𝑉𝐴 , with optional conditioning on 𝑦, to be 𝐹ISO(𝑉𝑖 ) (𝑉𝐴 | 𝑦) := arg min H(𝐹 (𝑉𝐴 | 𝑦)) 𝐹 (8.7) such thatI(𝐹 (𝑉𝐴 ) : 𝑉𝑖 | 𝑦) = I(𝑉𝐴 : 𝑉𝑖 | 𝑦). 𝐹ISO(𝑉𝑖 ) (𝑉𝐴 | 𝑌 ) gives a vector of functions with an entry for each 𝑦 ∈ 𝑌 . Note that isolation functions are sufficient statistics for 𝑉𝑖 (Cover, 1999). Isolation involves maintaining the information about 𝑉𝑖 while removing excess noise. Recall from Lemma 43 that in order to avoid worsening context sensitivity, we want to ensure I(𝐹 (𝑉𝐴 ) : UBAD | 𝑌 ) = 0. Isolation functions on SAPs are well designed for this purpose, because they enforce the independence properties of the isolated vertex on their outputs. In order to achieve I(𝐹 (𝑉𝐴 ) : UBAD | 𝑌 ) = 0 while preserving as much information about UGOOD as possible, an optimal isolation function would be to isolate UGOOD using 𝐹ISO(UGOOD ) (𝑉𝐴 | 𝑌 ). Of course, we do not have access to UGOOD , so our next best option is to isolate VGOOD using 𝐹ISO(VGOOD ) (𝑉𝐴 | 𝑌 ), since UBAD ⊥⊥ VGOOD | 𝑌 . Lemma 44 shows that the output of 𝐹ISO(𝑉𝐺 ) (𝑉𝐴 | 𝑌 ) behaves like a good proxy if 𝑉𝐺 ∈ VGOOD and 𝑉𝐴 is a SAP. Lemma 44 (Isolating VGOOD behaves like VGOOD ). For 𝑉𝐺 ∈ VGOOD and 𝑈𝐵 ∈ UBAD and an isolation function 𝐹ISO(𝑉𝐺 ) (𝑉𝐴 | 𝑌 ), I(𝑈𝐵 : 𝐹ISO(𝑉𝐺 ) (𝑉𝐴 | 𝑌 ) | 𝑌 ) = 0. 107 The benefit from 𝐹ISO(𝑉𝐺 ) (𝑉𝐴 | 𝑌 )’s information about UGOOD is difficult to quantify for use with Lemma 42, but lower bounds can be obtained (B. Mazaheri, Mastakouri, et al., 2023). Even without a quantification of improvement, Theorem 7 shows that isolation functions can avoid worsening the context sensitivity, while certain conditions can guarantee relevance gains for predicting 𝑌 . Theorem 7 (CIS costs and benefits). Consider 𝑉𝐺 ∈ VGOOD and 𝑉𝐴 ∈ VAMBIG where 𝑉𝐴 is a SAP. Also consider the isolation function 𝐹ISO(𝑉𝐺 ) (𝑉𝐴 | 𝑌 ). We will compare the context sensitivity of inputs X := {𝑉𝐺 } and X+ := {𝑉𝐺 , 𝐹ISO(𝑉𝐺 ) (𝑉𝐴 | 𝑌 ))}. We claim that I(𝑀 : 𝑌 | X+ ) ≤ I(𝑀 : 𝑌 | X) for all 𝑀 ∈ M. Furthermore, if I(𝐹ISO(𝑉𝐺 ) (𝑉𝐴 | 𝑌 ) : 𝑉𝐺 ) < I(𝐹ISO(𝑉𝐺 ) (𝑉𝐴 | 𝑌 ) : 𝑉𝐺 | 𝑌 ), (8.8) then the relevance improves: I(𝑌 : X+ ) > I(𝑌 : X). Theorem 7 tells us that using an isolation function helps when the function is more predictive of the isolated variable in the post-selected 𝑌 distribution than it is in the full distribution. This condition is sufficient but loose because it does not take into account direct effects from I(𝑌 : 𝐹ISO(𝑉𝐺 ) (𝑉𝐴 | 𝑌 )) (for which we have no guaranteed bounds). The proof is given in Appendix E. Auxiliary Training Tasks In the infinite sample regime, consider an “optimal” model 𝐹 (·) that predicts 𝑉𝑖 using input 𝑉𝐴 . Optimal models should utilize all of the information available for prediction in their inputs, meaning 𝐼 (𝐹 (𝑉𝐴 ) : 𝑉𝑖 ) = I(𝑉𝐴 : 𝑉𝑖 ). Information theoretically, minimizing H(𝐹𝑉𝑖 (𝑉𝐴 )) corresponds to reducing the outputs of 𝐹𝑉𝑖 (𝑉𝑖 ) to equivalence classes wherein Pr(𝑉𝐴 | 𝐹𝑉𝑖 (𝑉𝑖 ) = 𝑓 ) is constant. This minimization corresponds to ensuring 𝐹𝑉𝑖 (𝑉𝑖 ) does not over-fit to the empirical values of 𝑉𝐴 using noise that is orthogonal to PA(𝑉𝐴 ). Auxiliary training tasks can therefore be used in place of isolation functions: we can get an approximate isolation function, 𝐹˜ISO(𝑉𝑖 ) (𝑉𝐴 ), by training a model to predict 𝑉𝑖 using input 𝑉𝐴 . We do not give any theoretical results beyond intuition for this interpretation, but will support our claims with experiments in the next section. Equation 8.8 in Theorem 7 also has a nice interpretation within the training context — the accuracy of the predictor must degrade when moving from the post-selected data 108 to the full dataset. More precisely, the conditions for improvement now translate to ∑︁ minE[Error(𝐹 (𝑉𝐴 ), 𝑉𝐺 )] > Pr(𝑦) min (E[Error(𝐹 (𝑉𝐴 ), 𝑉𝐺 ) | 𝑦]), (8.9) 𝐹 𝐹 𝑦 which can easily be checked on our training data. Suggested Overall Procedure We propose the following procedure for building robust (low context-sensitivity) models in the PER problem. 1. Partition the data into constant 𝑌 = 𝑦 and determine cliques of dependence. 2. Using domain knowledge, identify seeds in VGOOD , VBAD for proxy bootstrapping (Sec. 8.5). 3. Perform CIS on VAMBIG (Sec. 8.6). 4. Build a prediction model for 𝑌 using VGOOD and the CIS-engineered VAMBIG . 8.7 Experiments We will now demonstrate the effectiveness of these methods on synthetic and real world data. Full code for both of these experiments is available at https: //zenodo.org/badge/latestdoi/651823136. Experiments on Synthetic Data We generate data for the DAG in Figure 8.3 based on normal distributions, see details of the setup in Section 8.10. We vary the standard deviations of normally distributed 𝑀𝐺 and 𝑀𝐵 . The training data is drawn from 𝜎(𝑀𝐺 ) = 𝜎(𝑀𝐵 ) = 1, while the testing data varies both quantities and thus the influence of the context. We measure the accuracy of our feature engineering based on CIS, 𝑌ˆ (3) (𝑉𝐺 , 𝐹˜ISO(𝑉𝐺 ) (𝑉𝐴 )), that utilizes the auxiliary task approximation to isolate 𝑉𝐴 ’s predictive information about 𝑉𝐺 . We compare it to 𝑌ˆ (1) (𝑉𝐺 , 𝑉𝐴 ) trained on VGOOD ∪ VAMBIG and 𝑌ˆ (2) (𝑉𝐺 ) trained on only VGOOD . For a theoretical limit of CIS we also compare to 𝑌ˆ (4) (𝑉𝐺 , 𝑉𝐴(𝐺) ) although access to 𝑉𝐴(𝐺) is usually not possible. Results When comparing feature selection approaches, we observe in Figure 8.4 that including 𝑉𝐴 results in higher accuracy of 𝑌ˆ (1) over 𝑌ˆ (2) when the shift acts on UGOOD (a) or is small for UBAD (b). However, the accuracy of 𝑌ˆ (2) deteriorates with bigger shifts in UBAD . (a) 0.94 0.93 0.92 0.91 0.90 0.89 0.88 0.87 Varying (MG) Y(1)(VG, VA) Y(2)(VG) Y(3)(VG, FISO(VG|Y = 0)(VA), FISO(VG|Y = 1)(VA)) Y(4)(VG, VA(G)) 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 (MG) Accuracy Accuracy 109 (b) 0.92 0.90 0.88 0.86 0.84 0.82 0.80 0.78 Varying (MB) 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 (MB) Figure 8.4: Results from our experiments on synthetic data. Single standard deviation confidence intervals are shaded in the corresponding colors. Our proposed method based on causal information splitting offers a middle ground. 𝑌ˆ (3) is able to maintain the same robustness as 𝑌ˆ (2) while taking advantage of some of the gains enjoyed by 𝑌ˆ (1) in (a). In fact, 𝑌ˆ (3) performs very similarly to 𝑌ˆ (4) , which had a-priori knowledge of the SAP components and used only 𝑉 (𝐺) . 𝐴 These improvements were achieved despite not meeting the sufficient condition for increasing relevance in Theorem 7. Experiments on Census Data We use US Census data processed through folktables F. Ding et al., 2021 to predict whether the income of a person exceeds 50k following Dua and Graff, 2017. To test out-of-domain generalization, prediction models were built on 2019 pre-pandemic data and evaluated on 2021 data during the pandemic.7 As model inputs, we consider commute time (coded as JWMNP in the dataset), a flag whether the person received Medicaid, Medical Assistance, or any kind of government-assistance plan for those with low incomes or a disability (coded as HINS4) and education level (SCHL). This small feature set was purposefully selected to see a starker effect of including/excluding individual features, including a feature with relatively stable predictive power (education level) and two features heavily affected by the pandemic through increased work-from-home and medicaid’s continuous enrollment provision. Our auxiliary task from Sec. 8.6, referred to as engineered features, does not use HINS4 and JWMNP directly as input features to predict the income level. Instead it uses HINS4 and JWMNP to train two models predicting the education-level: One trained on examples with high income and one trained on examples with low income. These predictions based on HINS4 and JWMNP together with the actual education-level serve as input features to the final model. We compare the model 7We ignored the experimental release of 2020 data to ensure a starker distribution shift. 110 built on these engineered features to ones using all three features directly (all features) or using just the stable education feature (limited features). We use logistic regression from sklearn with l1 regularization to build models based on the different feature sets that the three methods created. l1 regularization yielded better generalization than l2 regularization. Table 8.1: Comparison of out-of-domain (2021) performance via mean of accuracy. State CA FL GA IL NY NC OH PA TX avg All Features 0.712 ± 0.0011 0.683 ± 0.0012 0.689 ± 0.0025 0.662 ± 0.0026 0.707 ± 0.0022 0.691 ± 0.0031 0.689 ± 0.0022 0.672 ± 0.0017 0.690 ± 0.0029 0.688 Engineered Features 0.711 ± 0.0014 0.678 ± 0.0018 0.707 ± 0.0055 0.689 ± 0.0033 0.702 ± 0.0025 0.684 ± 0.0034 0.703 ± 0.0040 0.695 ± 0.0023 0.712 ± 0.0028 0.698 Limited Features 0.692 ± 0.0014 0.680 ± 0.0013 0.709 ± 0.0029 0.684 ± 0.0019 0.687 ± 0.0080 0.683 ± 0.0030 0.696 ± 0.0029 0.688 ± 0.0022 0.712 ± 0.0027 0.692 Results Table 8.1 reports the mean and standard deviation of accuracies for 10 different test splits. For the F1 scores of the same experiment, see Section 8.10. Using all features leads to the best in-domain performance (see Section 8.10), but not necessarily the best out-of-domain performance. Dropping the ambiguous features hurts predictive power in limited feature models, but helps with robustness varies across the states: these limited models even perform better on 2021 data. Our proposed feature engineering using CIS achieves the best of both worlds, with the best mean out-of-domain accuracy of 0.698. It also achieves close to the best out-of-domain accuracy for 8 out of 9 states. 8.8 Discussion In this paper we studied the challenging problem of building models that are robust to distribution shift when causes and effects of the target variable are unmeasured. Among the observed noisy proxies, we showed how to perform feature selection based on conditional independence tests and knowledge about some seed nodes. After bootstrapping, we often have a significant number of ambiguous proxies, which have components that are both helpful and hurtful to our model’s robustness. Through CIS, however, we showed how to isolate robust predictive power from these ambiguous proxies using auxiliary learning tasks. We proved that including these engineered features safely increases robustness in our setting, while also improving 111 accuracy. In our experiments on real census data under shifts due to the pandemic, we showed that the engineered features provided benefits for most states over using the ambiguous features directly or completely ignoring them. While our theoretical framework is involved, these experiments demonstrate improvements outside of our assumptions. Relaxation of Assumptions A number of our assumptions can be softened. One softening of systemic sparsity would involve allowing edges within U so long as their dependence is relatively weak. Such a relaxation would involve using mutual information (or correlation) thresholds instead of independence tests. Sparsity assumptions may also be relaxed by building on ideas from mixtures of DAG structures from S. Gordon, B. Mazaheri, Yuval Rabani, et al. (2023). The strongest assumption is that of separable ambiguous proxies. Under a softening of the separability assumption, we cannot guarantee that we have isolated only robust information from our ambiguous proxy — some unstable information associated with UBAD may slip through. However, degrees of separability may still guarantee the benefit of the engineered feature. While separability corresponds to invertability with linear functions, there are many examples of nonlinear that are separable. For example, when the effects of two causes have significantly different magnitudes they can be easily disentangled, such as fine and hyper-fine structures in atomic energy levels. Work on data fission (Leiner et al., 2022) may provide valuable insights to help understand the degrees of separability for different choices of functions. 8.9 Deferred Proofs Proof of Lemma 36 Proof. We can rewrite the conditional mutual information making use of 𝑈1 ⊥⊥ 𝑈2 as follows. I(𝑈1 : 𝑈2 | 𝑉) = −I(𝑈1 : 𝑈2 : 𝑉) = −I(𝑈1 : 𝑉) + I(𝑈1 : 𝑉 | 𝑈2 ) = −I(𝑈1 : 𝑉 (𝑈1 ) ) + I(𝑈1 : 𝑉 (𝑈2 ) | 𝑈2 ) = −I(𝑈1 : 𝑉 (𝑈1 ) ) + I(𝑈1 : 𝑉 (𝑈1 ) ) = 0. □ 112 Proof of Lemma 37 Proof. I(𝑈 : X) = H(𝑈) − H(𝑈 | X). If at least one child of 𝑈 is conditioned on and transmits (i.e. in 𝑉 ∈ CH(𝑈) ∩ X and 𝑣 x ≠ 𝜙), then H(𝑈 | x) = 0. Otherwise, H(𝑈 | X) = H(𝑈) because all 𝑋 ∈ X \ CH(𝑈) with active paths to 𝑈 must go through colliders in CH(𝑈) — all of which are not in X or not transmitting. □ Proof of Lemma 38 Í Proof. I(𝑀𝑖 : 𝑌 | X) = X I(𝑀𝑖 : 𝑌 | X) is zero unless (𝑀𝑖 , 𝑈𝑖 ) and (𝑈𝑖 , 𝑌 ) both transmit. Furthermore, the DPI gives I(𝑀𝑖 : 𝑌 | X) ≤ H(𝑈𝑖 | X), which is zero if any of the edges from 𝑈𝑖 → 𝑋 for 𝑋 ∈ X transmit. □ Proof of Lemma 39 Í Proof. I(𝑀𝑖 : 𝑌 | X) = X I(𝑀𝑖 : 𝑌 | X) is zero unless both (𝑀𝑖 , 𝑈𝑖 ) and (𝑈𝑖 , 𝑌 ) transmit and at least one of (𝑈𝑖 , 𝑋) transmits for 𝑋 ∈ X, in which case I(𝑀𝑖 : 𝑌 | x) = I(𝑀𝑖 : 𝑌 | 𝑈𝑖 ). □ Proof of Lemma 40 Proof. (⇒) We will prove this with the contrapositive. If there is no shared parent between 𝑉𝑖 and 𝑉 𝑗 , then all active paths must go through 𝑌 . However, because at least one of 𝑉𝑖 , 𝑉 𝑗 is not connected to a cause, all paths between 𝑉𝑖 and 𝑉 𝑗 cannot have a collider at 𝑌 (through two causes). This means conditioning on 𝑌 blocks the remaining paths. (⇐) If there is a shared parent 𝑈 between 𝑉𝑖 and 𝑉 𝑗 , then 𝑉𝑖 ← 𝑈 → 𝑉 𝑗 is an active path, d-connecting the two vertices. If 𝑉𝑖 and 𝑉 𝑗 each have corresponding parents 𝑈𝑖 , 𝑈 𝑗 ∈ PA(𝑌 ), then 𝑉𝑖 ← 𝑈𝑖 → 𝑌 ← 𝑈 𝑗 → 𝑉 𝑗 is an active path conditioned on 𝑌. □ Proof of Lemma 41 Proof. The proof follows from Lemma 40. Adjacent edges in 𝔊𝑌 either indicate shared parents or that both vertices have a (potentially different) cause of 𝑌 as their parent. If 𝑉𝑖 and 𝑉 𝑗 share a parent, then PA(𝑉𝑖 ) ⊆ UGOOD implies UGOOD ∩PA(𝑉 𝑗 ) ≠ ∅, so 𝑉 𝑗 has at least one “good” parent. The symmetric argument holds for PA(𝑉𝑖 ) ⊆ UBAD . 113 If 𝑉𝑖 and 𝑉 𝑗 both have at least one causal parent, then we know 𝑉𝑖 , 𝑉 𝑗 ∉ VBAD . We know both vertices have at least one good 𝑈 as a parent, so it trivially follows that both are either in VGOOD or VAMBIG . □ Proof of Theorem 6 Proof. The only potential conditioned collider is 𝑌 , which is not allowed to be separable by partial faithfulness. Hence, partial faithfulness guarantees that we can construct 𝔊𝑌 from conditional independence tests because d-connection implies dependence between 𝑉. The requirements on V∗ given by the theorem ensure that every 𝑉 ∈ V has at least one label from an adjacency to a 𝑉 ∗ ∈ V∗ in 𝔊𝑌 . The algorithm adds “good” labels to all vertices with a known good parent and “bad” labels to all vertices with a known bad parent. Therefore, all “ambigious” vertices are correctly labeled. We now only need to guarantee that that the “good” and “bad” vertices are not ambiguous. If 𝑉 were ambiguous, it would be connected to a 𝑈 of the opposite label (i.e. a “good” vertex would be connected to a bad 𝑈). Such a 𝑈 would have at least one 𝑉 ∗ ∈ V∗ ∩ CH(𝑈) which would be adjacent to 𝑉 and have given 𝑉 the label of 𝑈, a contradiction. □ Proof of Lemma 42 Proof. I(𝑀𝑖 : 𝑌 | X) = Í x∈X Pr(x)I(𝑀𝑖 : 𝑌 | x). Now, we have I(𝑀𝑖 : 𝑌 | x) = H(𝑀𝑖 | x) = H(𝑈𝑖 | x) if both (𝑀𝑖 , 𝑈𝑖 ) and (𝑈𝑖 , 𝑌𝑖 ) edges transmit, which occurs with probability 𝛼 𝑀𝑖 ,𝑈𝑖 𝛼𝑈𝑖 ,𝑌 . Pulling this coefficient outside of the sum gives I(𝑀𝑖 : 𝑌 | X) = 𝛼 𝑀𝑖 ,𝑈𝑖 𝛼𝑈𝑖 ,𝑌 H(𝑈𝑖 | x). □ Proof of Lemma 43 Proof. 𝑈𝑖 ∈ UBAD means 𝑀𝑖 𝑈𝑖 𝑌 , so I(𝑀𝑖 : 𝑌 ) = 0. I(𝑀𝑖 : 𝑌 | X) = −I(𝑀𝑖 : 𝑌 : X) ≤ I(𝑀𝑖 : X | 𝑌 ) ≤ I(𝑈𝑖 : X | 𝑌 ) = 0 (8.10) 114 The final inequality comes from the data processing inequality. □ Proof of Lemma 44 Proof. Consider the function 𝐹𝐶 (𝑉𝐴 ) = 𝐹𝐶 (𝐺, 𝐵)) = 𝐹ISO(𝑉𝐺 ) (𝐺). By definition, I(𝐹ISO(𝑉𝐺 ) (𝐺) : 𝑉𝐺 | 𝑌 ) = I(𝐺 : 𝑉𝐺 | 𝑌 ) (8.11) = I(𝑉𝐴 : 𝑉𝐺 | 𝑌 ). (8.12) Hence, 𝐹𝐶 is in the feasible set of the optimization function defining isolation functions. Furthermore, 𝐹𝐶 (𝑉𝐴 ) is only a function of 𝐺 and 𝐺 ⊥⊥ 𝑈𝐵 | 𝑌 , so we can also conclude that 𝐹𝐶 (𝑉𝐴 ) ⊥⊥ 𝑈𝐵 . This means H(𝐹𝐶 (𝑉𝐴 ) | 𝑌 ) = H(𝐹𝐶 (𝑉𝐴 ) | 𝑈𝐵 , 𝑌 ) ≤H(𝐹ISO(𝑉𝐺 ) (𝑉𝐴 ) | 𝑈𝐵 , 𝑌 ) ≤H(𝐹ISO(𝑉𝐺 ) (𝑉𝐴 ) | 𝑌 ) − I(𝐹ISO(𝑉𝐺 ) (𝑉𝐴 ) : 𝑈𝐵 | 𝑌 ). Hence, if I(𝐹ISO(𝑉𝐺 ) (𝑉𝐴 ) : 𝑈𝐵 | 𝑌 ) > 0, then H(𝐹𝐶 (𝑉𝐴 ) | 𝑌 ) < H(𝐹ISO(𝑉𝐺 ) (𝑉𝐴 ) | 𝑌 ), contradicting the minimality of 𝐹ISO(𝑉𝐺 ) (𝑉𝐴 ). □ Proof of Theorem 7 Proof. To shorten some equations, we will use F(𝑉𝐴 ) := 𝐹ISO(𝑉𝐺 ) (𝑉𝐴 | 𝑌 ). We first show that Equation 8.8 is sufficient for an improvement in relevance. We can expand the relevance of X+ as follows: I(𝑌 : X+ ) =I(𝑌 : F(𝑉𝐴 )) + I(𝑌 : 𝑉𝐺 | F(𝑉𝐴 )) ≥ I(𝑌 : 𝑉𝐺 | F(𝑉𝐴 )) (8.13) ≥ I(𝑌 : 𝑉𝐺 ) − I(𝑌 : 𝑉𝐺 : F(𝑉𝐴 )). So, for guaranteed improvement in relevance (I(𝑌 : X+ ) > I(𝑌 : 𝑉𝐺 )), we need negative I(𝑌 : 𝑉𝐺 : F(𝑉𝐴 )) < 0. Expanding, I(𝑌 : 𝑉𝐺 : F(𝑉𝐴 )) = I(F(𝑉𝐴 ) : 𝑉𝐺 ) − I(F(𝑉𝐴 ) : 𝑉𝐺 | 𝑌 ). (8.14) Thus, Equation 8.8 gives us the exact condition needed for negative interaction information, guaranteeing improvement. 115 We can show that the context sensitivity is no worse by separately considering the context sensitivity with PA(UBAD ) and PA(UGOOD ). We begin with UBAD . Applying Lemma 44, I((𝑉𝐺 , F(𝑉𝐴 )) : UBAD | 𝑌 ) = 0, (8.15) which satisfies the conditions for Lemma 43 to ensure us that I(PA(UBAD ) : 𝑌 | X+ ) = 0. Now, consider an arbitrary for 𝑀𝐺 = PA(𝑈𝐺 ) ∈ PA(UGOOD ). Lemma 42 tells us that g X+ (𝑈𝐺 )). I(𝑀𝐺 : 𝑌 | X+ ) = 𝛼 𝑀𝐺 :𝑈𝐺 𝛼𝑈𝐺 ,𝑌 𝐻 (𝑈𝐺 | CH g X+ (𝑈𝐺 )) ≤ 𝐻 (𝑈𝐺 | X) because entropy is We then observe that 𝐻 (𝑈𝐺 | CH submodular, which leads us to conclude I(𝑀𝐺 : 𝑌 | X+ ) ≤ I(𝑀𝐺 : 𝑌 | X). This completes the proof. 8.10 □ Experimental Details Synthetic experimental setup 𝑀𝐺 and 𝑀𝐵 are drawn from normal distributions with mean 0 and variable standard deviations. All other vertices (other than 𝑌 ) are the average of their parents plus additional Gaussian noise 𝑁 (0, .2). 𝑇𝐴 ∈ R2 is generated by applying a rotation matrix to (𝑇𝐴(𝐺) , 𝑇𝐴𝐵 )𝑇 8. 𝑌 indicates whether its parents sum to a positive number with a 5% probability of flipping randomly. F1 scores for real world experiment We give the F1 scores for the experiment described in Section 8.7 in Table 8.2. Real world experiment in-domain performance Here we provide the results of the in-domain accuracy for the experiment described in Sec. 8.7. Recall, that we use US Census data and consider distributions shifts across time as suggested by F. Ding et al., 2021. Table 8.3 shows the accuracy on 2019 data on a held-out dataset (separate from the training split). We repeated the experiment 10 times on different training/testing splits and report the mean and standard deviation of the accuracy for the largest states in the U.S. As expected, using all features has the most predictive power for in-domain tasks. 8 Many rotations were tried in our experiments with identical results, so we display results from a 45 degree rotation. 116 Table 8.2: Comparison of out-of-domain (2021) performance on predicting high income via F1 scores. State CA FL GA IL NY NC OH PA TX avg All Features 0.684 0.459 0.541 0.563 0.688 0.475 0.519 0.531 0.554 0.557 Engineered Features 0.683 0.388 0.626 0.630 0.690 0.410 0.581 0.608 0.619 0.582 Limited Features 0.676 0.388 0.624 0.628 0.662 0.410 0.580 0.606 0.619 0.577 Table 8.3: Comparison of in-domain (2019) performance on predicting high income via Accuracies. State CA FL GA IL NY NC OH PA TX avg All Features 0.713 ± 0.0010 0.700 ± 0.0014 0.708 ± 0.0025 0.689 ± 0.0023 0.705 ± 0.0024 0.713 ± 0.0020 0.717 ± 0.0029 0.702 ± 0.0028 0.708 ± 0.0019 0.706 Engineered Features 0.710 ± 0.0012 0.693 ± 0.0020 0.708 ± 0.0036 0.690 ± 0.0039 0.698 ± 0.0022 0.703 ± 0.0049 0.716 ± 0.0042 0.701 ± 0.0027 0.705 ± 0.0025 0.703 Limited Features 0.691 ± 0.0011 0.694 ± 0.0017 0.707 ± 0.0036 0.685 ± 0.0021 0.687 ± 0.0076 0.700 ± 0.0028 0.712 ± 0.0033 0.695 ± 0.0026 0.706 ± 0.0022 0.697 Part IV Discussion 117 118 Chapter 9 RETHINKING DIMENSIONALITY AND ERRORS Through a detailed discussion of Level 3 and 4 knowledge, this thesis has introduced a number of important ideas on which future work can be built. 9.1 Diverse Data One recurring lesson is the nuanced problems associated with data from multiple sources, contexts, and populations. While the many-source conundrum can be addressed using the LLC and mixture models, significant additional work is required to handle the analogous settings with continuous variables. Further study is also needed to understand the effects of sampling bias on subsets of variables. A natural complement to the universal confounder is a universal “sink” which applies different levels of sampling bias to the entire system. With an analogous limit to the power of this sampling bias, we can study the identifiability of the relationships and causal structure. 9.2 The Necessity (and Blessing?) of Dimensionality High dimensional models often overfit, leading to what is known as the “curse of dimensionality.” This thesis presents two algorithms for a previously unsolved setting of causal identification and causal discovery. The main requirements for these algorithms are a minimum span of confounding influence relative to the sparsity of the graph — i.e. the unobserved 𝑈 must point to enough variables to be identifiable. This implicitly requires a high dimensional setting. The necessity of dimensionality may go further. Response functions which are indexed by latent variables, such as Γ𝑢 (𝑋), are generally unidentifiable because we cannot draw multiple samples from the same response function without resampling 𝑢. This information deficiency parallels the need for “exchangable” samples for Level 3 knowledge. To navigate heterogeneous data, our approach in Chapter 4 utilizes the observation that each additional observed variable contains new information about the heterogeneity. Concretely, this corresponds to creating “synthetic copies” of variables, which use linear combinations of new vertices to mimic the statistical properties of a disjoint 119 vertex. These synthetic copies allow us to simulate multiple samplings from the same distribution with constant 𝑈. The details of this “method of synthetic bits” are given in S. Gordon, B. H. Mazaheri, et al., 2021. The need for exponential sample complexity in the 𝑘-MixProd algorithm partially derives from restricting ourselves to a linear number of independent vertices. However, with a quadratic number of independent vertices it is possible to form a quadratic number of synthetic copies of each vertex. At this threshold, we expect to gain enough information to begin to label each data-point with a specific 𝑈, significantly simplifying the problem to the identification of a categorical distribution on 𝑈. As a result, it is possible that access to higher-dimensional data contains the cure to the (often prohibitive) data demands of deconfounding and causal inference. 9.3 The Information Theoretical Value of Errors Causal structures represent conditional independence properties. These structures fail when these independence properties are violated, either due to an incorrect model or the presence of unobserved confounding. As we saw in Chapters 4 and 5, the breakdown of these properties provides key information that is used to learn the effects of confounding. A similar phenomenon is observed at Level 4 in Chapter 8, where pairing a variable with its (potentially incorrect) prediction can help isolate information from variables along active paths, thereby enhancing stability to distribution shift. Again, we see that errors play a key roll in augmenting knowledge. In an field that has placed an increasing emphasis on incremental improvements to accuracy, it can be easy to forget that science is built on the breakdown of models. This thesis treats statistical models more broadly than the naïve tasks for which they have been trained by harnessing their failures. Following down this path may yield exciting new tools to ascend the hierarchy of knowledge. 120 BIBLIOGRAPHY Acid, Silvia and Luis M De Campos (1996). “An algorithm for finding minimum d-separating sets in belief networks”. In: Proceedings of the Twelfth international conference on Uncertainty in artificial intelligence, pp. 3–10. Agrawal, Raj et al. (July 2023). “The DeCAMFounder: nonlinear causal discovery in the presence of hidden variables”. In: Journal of the Royal Statistical Society Series B: Statistical Methodology, qkad071. issn: 1369-7412. doi: 10.1093/ jrsssb/qkad071. eprint: https://academic.oup.com/jrsssb/advancearticle-pdf/doi/10.1093/jrsssb/qkad071/50874769/qkad071.pdf. url: https://doi.org/10.1093/jrsssb/qkad071. Allman, E. S., C. Matias, and J. A. Rhodes (2009). “Identifiability of parameters in latent structure models with many observed variables”. In: Ann. Statist. 37.6A, pp. 3099–3132. doi: 10.1214/09-AOS689. Alon, Noga (2002). “Voting paradoxes and digraphs realizations”. In: Advances in Applied Mathematics 29.1, pp. 126–135. Anandkumar, A., D. Hsu, et al. (2012). “Learning high-dimensional mixtures of graphical models”. In: arXiv preprint arXiv:1203.0697. Anandkumar, A., D. J. Hsu, and S. M. Kakade (2012). “A Method of Moments for Mixture Models and Hidden Markov Models”. In: Proc. 25th Ann. Conf. on Learning Theory - COLT. Vol. 23. JMLR Proceedings, pp. 33.1–33.34. url: http: //proceedings.mlr.press/v23/anandkumar12/anandkumar12.pdf. Angrist, Joshua and Guido Imbens (1995). Identification and estimation of local average treatment effects. Angrist, Joshua D and Alan B Krueger (2001). “Instrumental variables and the search for identification: From supply and demand to natural experiments”. In: Journal of Economic perspectives 15.4, pp. 69–85. Arjovsky, Martin et al. (2019). “Invariant risk minimization”. In: arXiv preprint arXiv:1907.02893. Banerjee, Pradeep Kr et al. (2018). “Unique informations and deficiencies”. In: 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, pp. 32–38. Bareinboim, Elias and Jin Tian (2015). “Recovering causal effects from selection bias”. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 29. 1. Batu, T., S. Guha, and S. Kannan (2004). “Inferring Mixtures of Markov Chains”. In: Proc. 17th Conf. on Learning Theory, pp. 186–199. doi: 10.1007/978-3-54027819-1_13. 121 Bellot, Alexis and Mihaela van der Schaar (2020). “Generalization and invariances in the presence of unobserved confounding”. In: arXiv preprint arXiv:2007.10653 4. Bertschinger, Nils et al. (2014). “Quantifying unique information”. In: Entropy 16.4, pp. 2161–2183. Cai, Ruichu et al. (2023). “Causal Discovery with Latent Confounders Based on Higher-Order Cumulants”. In: arXiv preprint arXiv:2305.19582. Castanedo, Federico (2013). “A review of data fusion techniques”. In: The scientific world journal 2013. Chaudhuri, K. and S. Rao (2008). “Learning Mixtures of Product Distributions Using Correlations and Independence”. In: Proc. 21st Ann. Conf. on Learning Theory - COLT. Omnipress, pp. 9–20. url: http://colt2008.cs.helsinki. fi/papers/7-Chaudhuri.pdf. Chen, S. and A. Moitra (2019). “Beyond the low-degree algorithm: mixtures of subcubes and their applications”. In: Proc. 51st Ann. ACM Symp. on Theory of Computing, pp. 869–880. doi: 10.1145/3313276.3316375. Chiappa, Silvia (2019). “Path-specific counterfactual fairness”. In: Proceedings of the AAAI conference on artificial intelligence. Vol. 33. 01, pp. 7801–7808. Chikritzhs, Tanya, Kaye Fillmore, and Tim Stockwell (2009). “A healthy dose of scepticism: four good reasons to think again about protective effects of alcohol on coronary heart disease”. In: Drug and alcohol review 28.4, pp. 441–444. Cohen, Michael and Jean-Claude Falmagne (1990). “Random utility representation of binary choice probabilities: a new class of necessary conditions”. In: Journal of Mathematical Psychology 34.1, pp. 88–94. Cormen, Thomas H et al. (2022). Introduction to algorithms. MIT press. Correa, Juan D, Jin Tian, and Elias Bareinboim (2019). “Identification of causal effects in the presence of selection bias”. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. 01, pp. 2744–2751. Cover, Thomas M (1999). Elements of information theory. John Wiley & Sons. Croce, Francesco and Matthias Hein (2020). “Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse Parameter-Free Attacks”. In: Proceedings of the 37th International Conference on Machine Learning. ICML. Cryan, M., L. Goldberg, and P. Goldberg (2001). “Evolutionary trees can be learned in polynomial time in the two state general Markov model”. In: SIAM J. Comput. 31.2, pp. 375–397. doi: 10.1137/S0097539798342496. Ding, Frances et al. (2021). “Retiring Adult: New Datasets for Fair Machine Learning”. In: Advances in Neural Information Processing Systems 34. Ding, Jingqiu et al. (2022). “Fast algorithm for overcomplete order-3 tensor decomposition”. In: Conference on Learning Theory. PMLR, pp. 3741–3799. 122 Dua, Dheeru and Casey Graff (2017). UCI Machine Learning Repository. url: http://archive.ics.uci.edu/ml. Dutta, Sanghamitra and Faisal Hamman (2023). “A Review of Partial Information Decomposition in Algorithmic Fairness and Explainability”. In: Entropy 25.5, p. 795. Dutta, Sanghamitra, Praveen Venkatesh, et al. (2020). “An information-theoretic quantification of discrimination with exempt features”. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. 04, pp. 3825–3833. Everitt, B. S. and D. J. Hand (1981). “Mixtures of discrete distributions”. In: Finite Mixture Distributions. Dordrecht: Springer Netherlands, pp. 89–105. Feldman, J., R. O’Donnell, and R. A. Servedio (2008). “Learning Mixtures of Product Distributions over Discrete Domains”. In: SIAM J. Comput. 37.5, pp. 1536–1564. doi: 10.1137/060670705. Feldman, Jon, Ryan O’Donnell, and Rocco A Servedio (2008). “Learning mixtures of product distributions over discrete domains”. In: SIAM Journal on Computing 37.5, pp. 1536–1564. Fishburn, Peter (1996). “Acyclic sets of linear orders”. In: Social choice and Welfare 14.1, pp. 113–124. Fishburn, Peter C (1992). “Induced binary probabilities and the linear ordering polytope: A status report”. In: Mathematical Social Sciences 23.1, pp. 67–80. Fisher, Ronald Aylmer and John Wishart (1930). “The arrangement of field experiments and the statistical reduction of the results”. In. Freund, Y. and Y. Mansour (July 1999). “Estimating a mixture of two product distributions”. In: Proc. 12th Ann. Conf. on Computational Learning Theory, pp. 53–62. doi: 10.1145/307400.307412. Frot, Benjamin, Preetam Nandy, and Marloes H Maathuis (2019). “Robust causal structure learning with some hidden variables”. In: Journal of the Royal Statistical Society Series B: Statistical Methodology 81.3, pp. 459–487. Gilboa, Itzhak (1990). “A necessary but insufficient condition for the stochastic binary choice problem”. In: Journal of Mathematical Psychology 34.4, pp. 371–392. Glymour, C., K. Zhang, and P. Spirtes (2019). “Review of Causal Discovery Methods Based on Graphical Models”. In: Frontiers in Genetics 10, p. 524. doi: 10.3389/ fgene.2019.00524. Gordon, S. L. and L. J. Schulman (2022). “Hadamard extensions and the identification of mixtures of product distributions”. In: IEEE Transactions on Information Theory. to appear. Gordon, Spencer, Bijan Mazaheri, Yuval Rabani, et al. (2023). “Causal inference despite limited global confounding via mixture models”. In: 2nd Conference on Causal Learning and Reasoning. 123 Gordon, Spencer, Bijan Mazaheri, Leonard J Schulman, et al. (2020). “The sparse Hausdorff moment problem, with application to topic models”. In: arXiv preprint arXiv:2007.08101. Gordon, Spencer, Bijan H Mazaheri, et al. (2021). “Source identification for mixtures of product distributions”. In: Conference on Learning Theory. PMLR, pp. 2193– 2216. Gordon, Spencer L. et al. (2023). Identification of Mixtures of Discrete Product Distributions in Near-Optimal Sample and Time Complexity. arXiv: 2309.13993 [cs.LG]. Grodstein, Francine, Joann E Manson, and Meir J Stampfer (2006). “Hormone therapy and coronary heart disease: the role of time since menopause and age at hormone initiation”. In: Journal of Women’s Health 15.1, pp. 35–44. Grötschel, Martin, Michael Jünger, and Gerhard Reinelt (1985). “Facets of the linear ordering polytope”. In: Mathematical programming 33.1, pp. 43–60. Grünbaum, Branko et al. (1967). Convex polytopes. Vol. 16. Springer. Gulrajani, Ishaan and David Lopez-Paz (2021). “In Search of Lost Domain Generalization”. In: International Conference on Learning Representations. url: https://openreview.net/forum?id=lQdXeXDoWtI. Gupta, R., R. Kumar, and S. Vassilvitskii (2016). “On Mixtures of Markov Chains”. In: Advances in Neural Information Processing Systems. Vol. 29. url: https:// proceedings.neurips.cc/paper/2016/file/8b5700012be65c9da25f49408d959ca0Paper.pdf. Gurushankar, Keerthana, Praveen Venkatesh, and Pulkit Grover (2022). “Extracting unique information through markov relations”. In: 2022 58th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, pp. 1–6. Hacking, Ian (2001). An introduction to probability and inductive logic. Cambridge university press. Heckerman, D. (2018). “Accounting for hidden common causes when inferring cause and effect from observational data”. (NIPS 2017 causal inference workshop). url: https://arxiv.org/abs/1801.00727. Heinze-Deml, Christina, Jonas Peters, and Nicolai Meinshausen (2018). “Invariant causal prediction for nonlinear models”. In: Journal of Causal Inference 6.2. Hernán, Miguel A, Alvaro Alonso, et al. (2008). “Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease”. In: Epidemiology 19.6, pp. 766–779. 124 Hernán, Miguel A, David Clayton, and Niels Keiding (Mar. 2011). “The Simpson’s paradox unraveled”. In: International Journal of Epidemiology 40.3, pp. 780–785. issn: 0300-5771. doi: 10.1093/ije/dyr041. eprint: https://academic. oup . com / ije / article - pdf / 40 / 3 / 780 / 18481353 / dyr041 . pdf. url: https://doi.org/10.1093/ije/dyr041. Hsu, D., S. M. Kakade, and T. Zhang (Sept. 2012). “A Spectral Algorithm for Learning Hidden Markov Models”. In: J. Comput. Syst. Sci. 78.5, pp. 1460–1480. doi: 10.1016/j.jcss.2011.12.025. Imbens, G. W. and D. B. Rubin (2015). Causal inference in statistics, social, and biomedical sciences. Cambridge University Press. Jain, Siddharth et al. (2019). “Short tandem repeats information in tcga is statistically biased by amplification”. In: BioRxiv, p. 518878. – (2021). “Glioblastoma signature in the DNA of blood-derived cells”. In: Plos one 16.9, e0256831. Kearns, M. et al. (1994). “On the learnability of discrete distributions”. In: Proc. 26th Ann. ACM Symp. on Theory of Computing, pp. 273–282. doi: 10.1145/ 195058.195155. Kingma, Diederik P and Max Welling (2013). “Auto-encoding variational bayes”. In: arXiv preprint arXiv:1312.6114. Kivva, Bohdan et al. (2021). “Learning latent causal graphs via mixture oracles”. In: Advances in Neural Information Processing Systems. Ed. by A. Beygelzimer et al. url: https://openreview.net/forum?id=f9mSLa07Ncc. KPFRS, LIII (1901). “On lines and planes of closest fit to systems of points in space”. In: Proceedings of the 17th ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems (SIGMOD), p. 19. Krueger, David et al. (2021). “Out-of-distribution generalization via risk extrapolation (rex)”. In: International Conference on Machine Learning. PMLR, pp. 5815–5826. Kruskal, Joseph B (1977). “Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics”. In: Linear algebra and its applications 18.2, pp. 95–138. Kumar, A. and G. Sinha (July 2021). “Disentangling mixtures of unknown causal interventions”. In: Proc. Thirty-Seventh Conference on Uncertainty in Artificial Intelligence. Ed. by C. de Campos and M. H. Maathuis. Vol. 161. Proc. Machine Learning Research. PMLR, pp. 2093–2102. url: https://proceedings.mlr. press/v161/kumar21a.html. Kuroki, M. and J. Pearl (2014). “Measurement bias and effect restoration in causal inference”. In: Biometrika 101.2, pp. 423–437. doi: 10.1093/biomet/ast066. Lauritzen, Steffen L et al. (1990). “Independence properties of directed Markov fields”. In: Networks 20.5, pp. 491–505. 125 Leiner, James et al. (2022). “Data fission: splitting a single data point”. In: arXiv preprint arXiv:2112.11079. Li, J. et al. (2015). “Learning Arbitrary Statistical Mixtures of Discrete Distributions”. In: Proc. 47th Ann. ACM Symp. on Theory of Computing, pp. 743–752. doi: 10.1145/2746539.2746584. Lindsay, B. G. (1995). Mixture models: theory, geometry and applications, pp. i–163. Magliacane, Sara et al. (2018). “Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions”. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. NIPS’18. Montréal, Canada: Curran Associates Inc., pp. 10869–10879. Mazaheri, Bijan, Siddharth Jain, and Jehoshua Bruck (2021). “Synthesizing New Expertise via Collaboration”. In: 2021 IEEE International Symposium on Information Theory (ISIT), pp. 2447–2452. doi: 10.1109/ISIT45174.2021.9517822. Mazaheri, Bijan, Siddharth Jain, Matthew Cook, et al. (2023). Omitted Labels in Causality: A Study of Paradoxes. arXiv: 2311.06840 [cs.LG]. Mazaheri, Bijan, Atalanti Mastakouri, et al. (July 2023). “Causal information splitting: Engineering proxy features for robustness to distribution shifts”. In: Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence. Ed. by Robin J. Evans and Ilya Shpitser. Vol. 216. Proceedings of Machine Learning Research. PMLR, pp. 1401–1411. url: https://proceedings.mlr.press/ v216/mazaheri23a.html. McGarvey, David C (1953). “A theorem on the construction of voting paradoxes”. In: Econometrica: Journal of the Econometric Society, pp. 608–610. McLachlan, G. J., S. X. Lee, and S. I. Rathnayake (2019). “Finite Mixture Models”. In: Annual Review of Statistics and Its Application 6.1, pp. 355–378. doi: 10. 1146/annurev-statistics-031017-100325. Miao, W., Z. Geng, and E. T. Tchetgen (2018). “Identifying Causal Effects With Proxy Variables of an Unmeasured Confounder”. In: Biometrika 105.4, pp. 987–993. doi: 10.1093/biomet/asy038. Michels, Karin B and JoAnn E Manson (2003). Postmenopausal hormone therapy: a reversal of fortune. Moitra, Ankur (2016). “An almost optimal algorithm for computing nonnegative rank”. In: SIAM Journal on Computing 45.1, pp. 156–173. Monjardet, Bernard (2006). “Condorcet domains and distributive lattices”. In. Muandet, Krikamol, David Balduzzi, and Bernhard Schölkopf (2013). “Domain generalization via invariant feature representation”. In: International conference on machine learning. PMLR, pp. 10–18. Naimi, Timothy S et al. (2005). “Cardiovascular risk factors and confounders among nondrinking and moderate-drinking US adults”. In: 28.4, pp. 369–373. 126 Newcomb, S. (1886). “A Generalized Theory of the Combination of Observations so as to Obtain the Best Result”. In: American Journal of Mathematics 8.4, pp. 343–366. Nicolas, Jean Antoine et al. (1785). Essai sur l’application de l’analyse à la probabilité des decisions rendues à la pluralité des voix. Par m. le marquis de Condorcet,... de l’Imprimerie Royale. Oberst, Michael et al. (2021). “Regularizing towards causal invariance: Linear models with proxies”. In: International Conference on Machine Learning. PMLR, pp. 8260–8270. Ogburn, Elizabeth L, Ilya Shpitser, and Eric J Tchetgen Tchetgen (2019). “Comment on “blessings of multiple causes””. In: Journal of the American Statistical Association 114.528, pp. 1611–1615. Pearl, J. (2014). Probabilistic reasoning in intelligent systems: networks of plausible inference. Elsevier. Pearl, Judea (1988). Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan kaufmann. – (2009). Causality. Cambridge university press. – (2022). “Comment: understanding Simpson’s paradox”. In: Probabilistic and causal inference: The works of judea Pearl, pp. 399–412. Pearl, Judea and Elias Bareinboim (2011). “Transportability of causal and statistical relations: A formal approach”. In: Twenty-fifth AAAI conference on artificial intelligence. Pearson, K. (1894). “Contributions to the mathematical theory of evolution III”. In: Philosophical Transactions of the Royal Society of London (A.) 185, pp. 71–110. Peters, Jonas, Peter Bühlmann, and Nicolai Meinshausen (2016a). “Causal inference by using invariant prediction: identification and confidence intervals”. In: Journal of the Royal Statistical Society. Series B (Statistical Methodology), pp. 947–1012. – (2016b). “Causal inference by using invariant prediction: identification and confidence intervals”. In: Journal of the Royal Statistical Society. Series B (Statistical Methodology) 78.5, pp. 947–1012. issn: 13697412, 14679868. url: http://www.jstor.org/stable/44682904 (visited on 02/13/2023). Peters, Jonas, Dominik Janzing, and Bernhard Schölkopf (2017). Elements of causal inference: foundations and learning algorithms. The MIT Press. Peters, Jonas, Joris M Mooij, et al. (2014). “Causal discovery with continuous additive noise models”. In. Quinonero-Candela, Joaquin et al. (2008). Dataset shift in machine learning. Mit Press. 127 Quinzan, Francesco et al. (2022). Learning Counterfactually Invariant Predictors. doi: 10.48550/ARXIV.2207.09768. url: https://arxiv.org/abs/2207. 09768. Rabani, Y., L. J. Schulman, and C. Swamy (2014). “Learning mixtures of arbitrary distributions over large discrete domains”. In: Proc. 5th Conf. on Innovations in Theoretical Computer Science, pp. 207–224. doi: 10.1145/2554797.2554818. Ranganath, R. and A. Perotte (2018). “Multiple causal inference with latent confounding”. url: https://arxiv.org/abs/1805.08273. Ratsimalahelo, Zaka (2001). Rank test based on matrix perturbation theory. Tech. rep. EERI Research Paper Series. Reinelt, Gerhard (1985). The linear ordering problem: algorithms and applications. Vol. 8. Heldermann. Richardson, Thomas and Peter Spirtes (2002). “Ancestral graph Markov models”. In: The Annals of Statistics 30.4, pp. 962–1030. Rojas-Carulla, Mateo et al. (2018). “Invariant models for causal transfer learning”. In: The Journal of Machine Learning Research 19.1, pp. 1309–1342. Rothenhäusler, Dominik et al. (2021). “Anchor regression: Heterogeneous data meet causality”. In: Journal of the Royal Statistical Society Series B: Statistical Methodology 83.2, pp. 215–246. Saari, Donald G (2000). “Mathematical structure of voting paradoxes”. In: Economic Theory 15.1, pp. 1–53. – (2009). “Condorcet domains: A geometric perspective”. In: The Mathematics of Preference, Choice and Order. Springer, pp. 161–182. Saeed, Basil, Snigdha Panigrahi, and Caroline Uhler (2020). “Causal structure discovery from distributions arising from mixtures of dags”. In: International Conference on Machine Learning. PMLR, pp. 8336–8345. Sagawa, Shiori et al. (2019). “Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization”. In: arXiv preprint arXiv:1911.08731. Savage Jr, Richard P (1994). “The paradox of nontransitive dice”. In: The American Mathematical Monthly 101.5, pp. 429–436. Schlesselman, James J (1982). Case-control studies: design, conduct, analysis. Vol. 2. Oxford university press. Schmidhuber, Juergen (2022). “Annotated History of Modern AI and Deep Learning”. In: arXiv preprint arXiv:2212.11279. Schölkopf, Bernhard et al. (2012). “On causal and anticausal learning”. In: arXiv preprint arXiv:1206.6471. 128 Schweikert, Gabriele et al. (2008). “An empirical analysis of domain adaptation algorithms for genomic sequence analysis”. In: Advances in neural information processing systems 21. Sharan, V. et al. (2017). “Learning Overcomplete HMMs”. In: Advances in Neural Information Processing Systems, pp. 940–949. url: https://arxiv.org/abs/ 1711.02309. Shimodaira, Hidetoshi (2000). “Improving predictive inference under covariate shift by weighting the log-likelihood function”. In: Journal of statistical planning and inference 90.2, pp. 227–244. Sinha, Aman, Hongseok Namkoong, and John Duchi (2018). “Certifiable Distributional Robustness with Principled Adversarial Training”. In: International Conference on Learning Representations. url: https://openreview.net/ forum?id=Hk6kPgZA-. Spirtes, P. et al. (2000). “Constructing Bayesian network models of gene expression networks from microarray data”. Spirtes, Peter (2001). “An anytime algorithm for causal inference”. In: International Workshop on Artificial Intelligence and Statistics. PMLR, pp. 278–285. Spirtes, Peter, Clark Glymour, Richard Scheines, and David Heckerman (2000). Causation, prediction, and search. MIT press. Spirtes, Peter, Clark Glymour, Richard Scheines, Peter Spirtes, et al. (1993). “Discovery algorithms for causally sufficient structures”. In: Causation, prediction, and search, pp. 103–162. Squires, Chandler and Caroline Uhler (2022). “Causal structure learning: A combinatorial perspective”. In: Foundations of Computational Mathematics, pp. 1– 35. – (2023). “Causal structure learning: A combinatorial perspective”. In: Foundations of Computational Mathematics 23.5, pp. 1781–1815. Storkey, Amos et al. (2009). “When training and test sets are different: characterizing learning transfer”. In: Dataset shift in machine learning 30, pp. 3–28. Subbaswamy, Adarsh and Suchi Saria (2018). “Counterfactual Normalization: Proactively Addressing Dataset Shift Using Causal Mechanisms.” In: UAI, pp. 947– 957. Sugiyama, Masashi (2015). Introduction to statistical machine learning. Morgan Kaufmann. Suissa, Samy (2008). “Immortal time bias in pharmacoepidemiology”. In: American journal of epidemiology 167.4, pp. 492–499. 129 Tahmasebi, B., S. A. Motahari, and M. A. Maddah-Ali (2018). “On the Identifiability of Finite Mixtures of Finite Product Measures”. (Also in “On the identifiability of parameters in the population stratification problem: A worst-case analysis,” Proc. ISIT’18 pp. 1051-1055). url: https://arxiv.org/abs/1807.05444. Thiesson, B. et al. (1998). “Learning Mixtures of DAG Models”. In: Proc. 14th Conf. on Uncertainty in Artificial Intelligence, pp. 504–513. doi: 10.5555/2074094. 2074154. Titterington, D. M., A. F. M. Smith, and U. E. Makov (1985). Statistical Analysis of Finite Mixture Distributions. John Wiley and Sons, Inc. Tomczak, Katarzyna, Patrycja Czerwińska, and Maciej Wiznerowicz (2015). “Review The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge”. In: Contemporary Oncology/Współczesna Onkologia 2015.1, pp. 68–77. Uhler, Caroline et al. (2013). “Geometry of the faithfulness assumption in causal inference”. In: The Annals of Statistics, pp. 436–463. Veitch, Victor et al. (2021). “Counterfactual Invariance to Spurious Correlations in Text Classification”. In: Advances in Neural Information Processing Systems. Ed. by A. Beygelzimer et al. url: https://openreview.net/forum?id= BdKxQp0iBi8. Venkatesh, Praveen and Gabriel Schamberg (2022). “Partial information decomposition via deficiency for multivariate gaussians”. In: 2022 IEEE International Symposium on Information Theory (ISIT). IEEE, pp. 2892–2897. Wang, Y. and D. M. Blei (2019). “The Blessings of Multiple Causes”. In: Journal of the American Statistical Association 114.528, pp. 1574–1596. doi: 10.1080/ 01621459.2019.1686987. Weiss, Karl, Taghi M Khoshgoftaar, and DingDing Wang (2016). “A survey of transfer learning”. In: Journal of Big data 3.1, pp. 1–40. Williams, Paul L and Randall D Beer (2010). “Nonnegative decomposition of multivariate information”. In: arXiv preprint arXiv:1004.2515. Zech, John R. et al. (Nov. 2018). “Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study”. In: PLOS Medicine 15.11, pp. 1–17. doi: 10.1371/journal.pmed.1002683. url: https://doi.org/10.1371/journal.pmed.1002683.