[{"data":1,"prerenderedAt":275},["ShallowReactive",2],{"content-query-0RVogPMgKp":3},{"_path":4,"_dir":5,"_draft":6,"_partial":6,"_locale":7,"title":8,"description":9,"date":10,"cover":11,"type":12,"body":13,"_type":269,"_id":270,"_source":271,"_file":272,"_stem":273,"_extension":274},"/technology-blogs/en/3132","en",false,"","Federated Causality Learning with Explainable Adaptive Optimization","Research on causality discovery surpasses association-based machine learning, yet faces challenges due to data limitations, proposing a solution that integrates local and global optimizations for improved causal modeling.","2024-05-09","https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/05/31/56b8d5e1b7b948068e8e952cb2f9270c.png","technology-blogs",{"type":14,"children":15,"toc":266},"root",[16,24,34,38,46,51,59,70,78,87,100,105,113,118,123,131,136,141,149,154,159,167,172,177,182,190,195,202,207,214,219,224,231,236,243,248,256,261],{"type":17,"tag":18,"props":19,"children":21},"element","h1",{"id":20},"federated-causality-learning-with-explainable-adaptive-optimization",[22],{"type":23,"value":8},"text",{"type":17,"tag":25,"props":26,"children":27},"p",{},[28],{"type":17,"tag":29,"props":30,"children":31},"strong",{},[32],{"type":23,"value":33},"Paper Title:",{"type":17,"tag":25,"props":35,"children":36},{},[37],{"type":23,"value":8},{"type":17,"tag":25,"props":39,"children":40},{},[41],{"type":17,"tag":29,"props":42,"children":43},{},[44],{"type":23,"value":45},"Source:",{"type":17,"tag":25,"props":47,"children":48},{},[49],{"type":23,"value":50},"AAAI 2024",{"type":17,"tag":25,"props":52,"children":53},{},[54],{"type":17,"tag":29,"props":55,"children":56},{},[57],{"type":23,"value":58},"URL:",{"type":17,"tag":25,"props":60,"children":61},{},[62],{"type":17,"tag":63,"props":64,"children":68},"a",{"href":65,"rel":66},"https://ojs.aaai.org/index.php/AAAI/article/view/29566",[67],"nofollow",[69],{"type":23,"value":65},{"type":17,"tag":25,"props":71,"children":72},{},[73],{"type":17,"tag":29,"props":74,"children":75},{},[76],{"type":23,"value":77},"Code URL:",{"type":17,"tag":25,"props":79,"children":80},{},[81],{"type":17,"tag":63,"props":82,"children":85},{"href":83,"rel":84},"https://www.sdu-idea.cn/codes.php?name=FedCausal",[67],[86],{"type":23,"value":83},{"type":17,"tag":25,"props":88,"children":89},{},[90,92,98],{"type":23,"value":91},"As an open-source AI framework, MindSpore supports ultra-large-scale AI pre-training and brings excellent experience of device-edge-cloud synergy, simplified development, ultimate performance, and security and reliability for researchers and developers. Since it was open sourced on March 28th, 2020, MindSpore has been downloaded for more than 7 million times. It has also been the subject of more than a thousand papers presented at premier AI conferences. Furthermore, it has a large community of developers and has been introduced to over 100 top universities and 5000 commercial applications. Being widely used in scenarios such as AI computing centers, finance, smart manufacturing, cloud, wireless, datacom, energy, \"1+8+",{"type":17,"tag":93,"props":94,"children":95},"em",{},[96],{"type":23,"value":97},"N",{"type":23,"value":99},"\" consumers, and smart automobiles, MindSpore has emerged as the leading open-source software on Gitee. Here in this open source community, you are welcome to make contributions such as development kits, idea pooling for modeling, industry innovations and applications, algorithm innovations, academic collaborations, AI book collaborations, and your own application cases in the cloud, device, edge, security fields, and more.",{"type":17,"tag":25,"props":101,"children":102},{},[103],{"type":23,"value":104},"Thanks to the support from scientific, industry and academic circles, MindSpore-based papers account for 7% of all papers based on all AI frameworks as of 2023, ranking No. 2 in the world for two consecutive years. The MindSpore community supports analysis on top-level conference papers and promotes original AI achievements. In this blog, I'd like to share the paper of the team led by Guoxian Yu and Prof. Jun Wang, School of Control Science and Engineering, Shandong University.",{"type":17,"tag":25,"props":106,"children":107},{},[108],{"type":17,"tag":29,"props":109,"children":110},{},[111],{"type":23,"value":112},"01 Research Background",{"type":17,"tag":25,"props":114,"children":115},{},[116],{"type":23,"value":117},"Most existing machine learning algorithms focus on associations between variables behind events. However, associations cannot accurately represent potential generation relationships in data. Causality discovery, the practice of uncovering causal structures and generation relationships from observational data, has become a hotbed of research across diverse fields like education, economics, and biomedicine. Causality allows researchers to not only explore the root causes of real-world events and identify necessary interventions, but also predict potential outcomes based on the causes.",{"type":17,"tag":25,"props":119,"children":120},{},[121],{"type":23,"value":122},"However, causality discovery algorithms heavily depend on the quantity and quality of data. Researchers often need to collect data from multiple organizations or regions for causal structure learning. Currently, the growing privacy awareness prohibits internal data of organizations or regions from being externally exposed, and it is difficult to accurately model causality from dispersed and limited data. In addition, data of different organizations and regions has different distributions. A unified causal structure that satisfies all dataset distributions cannot be learned by simply using aggregated data. To solve these problems, the paper proposes an approach that unifies the local and global optimizations into the learning process of a complete directed acyclic graph (DAG) with a consistent optimization objective. At the same time, the authors prove that the optimization objective of the algorithm can be flexibly interpreted as two forms to adaptively process homogenous and heterogeneous data distributed across organizations or regions.",{"type":17,"tag":25,"props":124,"children":125},{},[126],{"type":17,"tag":29,"props":127,"children":128},{},[129],{"type":23,"value":130},"02 Team Introduction",{"type":17,"tag":25,"props":132,"children":133},{},[134],{"type":23,"value":135},"Dezhi Yang, the paper's lead author, is a master's student at School of Control Science and Engineering, Shandong University, specializing in biological data mining and causal structure discovery. He is mentored by Prof. Jun Wang.",{"type":17,"tag":25,"props":137,"children":138},{},[139],{"type":23,"value":140},"Prof. Jun Wang, a doctoral mentor at the Joint SDU-NTU Centre for Artificial Intelligence Research, focuses on developing trustworthy and interpretable AI theories for biomedical big data analysis. His research results have been widely recognized and cited by researchers in different fields such as information technologies and life sciences.",{"type":17,"tag":25,"props":142,"children":143},{},[144],{"type":17,"tag":29,"props":145,"children":146},{},[147],{"type":23,"value":148},"03 Introduction to the Paper",{"type":17,"tag":25,"props":150,"children":151},{},[152],{"type":23,"value":153},"When faced with dispersed data, current causal discovery algorithms fail to learn a correct and unified causal graph from multiple datasets that are heterogeneously distributed. To solve this problem, the paper aims to extend the existing causal discovery algorithms to the federated learning framework. Previous distributed algorithms either directly share local model parameters globally, causing serious privacy data leakage, or impose strict constraints on local models, resulting in poor fitting due to ineffective updates.",{"type":17,"tag":25,"props":155,"children":156},{},[157],{"type":23,"value":158},"The approach proposed in this paper decomposes a local causal model into a structural model and a parameter model. By sharing a small fraction of structural parameters, the approach aggregates the global causal structure while effectively avoiding privacy leakage risks. In addition, by allowing local parameters related to causal mechanisms to freely participate in training and implementing secondary constraint optimization during global parameter aggregation, the approach learns a global causal graph while allowing local models to adapt to local heterogeneous data distribution. By combining local and global optimization objectives, this paper proves that the federated causal discovery objective can be flexibly interpreted homogeneously and heterogeneously, and is consistent with the optimization objective of the conventional causal discovery algorithms.",{"type":17,"tag":25,"props":160,"children":161},{},[162],{"type":17,"tag":163,"props":164,"children":166},"img",{"alt":7,"src":165},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/05/31/e18f87a1b60d46939100bb7e997b8420.png",[],{"type":17,"tag":25,"props":168,"children":169},{},[170],{"type":23,"value":171},"Figure 1 Framework of FedCausal",{"type":17,"tag":25,"props":173,"children":174},{},[175],{"type":23,"value":176},"This paper proposes FedCausal, a new federated causal discovery algorithm with explainable adaptive optimization. Considering the causal model differences caused by data distribution heterogeneity and the requirements of federal privacy protection, the algorithm divides the local causal model into two parts: structural model and parameter model (as shown in Figure 1), which are related to the causal structure and causal mechanism, respectively. The local model trains a complete local causal model based on local data, and uploads only parameters of the structural model to the server. The server returns the aggregated structural model and replaces the local structural model. In this way, we can easily ensure that all organizations jointly learn a consistent causal structure, that is, an organizational model, in a distributed setting. In addition, a local model complies with local data distribution due to a freely trained parameter model.",{"type":17,"tag":25,"props":178,"children":179},{},[180],{"type":23,"value":181},"The paper also proposes a secondary optimization strategy for global structural model aggregation. Due to the heterogeneity of data distribution, the structural models uploaded by different local models may not be consistent. Simple average parameter aggregation may cause the structural model to deviate from the real causal structure. Therefore, constraint optimization is required to ensure that the aggregated structural model satisfies the causal condition, that is, the structural model should be mapped to a DAG. The approach in this paper re-optimizes a global structural model when the server aggregates the global model, so that the average aggregated structure of the global and local structural models is constrained to approximate and satisfy the acyclicity term. The globally optimized structural model will be broadcast and replace the local structural models. Through local and global iterative optimization, when the global model converges, we can extract the global unified causal graph from the global structure model.",{"type":17,"tag":25,"props":183,"children":184},{},[185],{"type":17,"tag":29,"props":186,"children":187},{},[188],{"type":23,"value":189},"04 Experiment Results",{"type":17,"tag":25,"props":191,"children":192},{},[193],{"type":23,"value":194},"The paper describes experiments in scenarios with homogeneous and heterogeneous data distribution. 10 clients are set in each scenario, and each client contains 200 samples. The causal graphs are set to have 10, 20, 40, and 80 parameters, respectively. The evaluation indicators include the structural hamming distance (SHD), true positive rate (TPR) and false discovery rate (FDR) of the edges contained in the discovered graph structure compared with the edges contained in the real graph structure. The superiority of the proposed algorithm is proved by comparing the results with those of state-of-the-art (SOTA) distributed or federated causal discovery methods.",{"type":17,"tag":25,"props":196,"children":197},{},[198],{"type":17,"tag":163,"props":199,"children":201},{"alt":7,"src":200},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/05/31/586a84b0e13449409809c371dd9a8fe2.png",[],{"type":17,"tag":25,"props":203,"children":204},{},[205],{"type":23,"value":206},"Figure 2 Result comparison of FedCausal and other methods on homogeneous data",{"type":17,"tag":25,"props":208,"children":209},{},[210],{"type":17,"tag":163,"props":211,"children":213},{"alt":7,"src":212},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/05/31/add49e5cfcea41f7afc639289fd0ec02.png",[],{"type":17,"tag":25,"props":215,"children":216},{},[217],{"type":23,"value":218},"Figure 3 Result comparison of FedCausal and other methods on heterogeneous data",{"type":17,"tag":25,"props":220,"children":221},{},[222],{"type":23,"value":223},"Experimental results show that FedCausal has significantly higher comprehensive performance compared to SOTA distributed or federated causal discovery methods. In addition, experiments show that the local causal graph output by FedCausal is equally excellent and the output global graph fully satisfies the causal condition (DAG). In this paper, homogeneous and heterogeneous data distribution can be flexibly used, and the proposed global aggregation optimization effectively aggregates local structure models and improves the performance of global causal discovery while ensuring that the results meet causal constraints. Finally, the experimental results on real data also demonstrate the effectiveness and stability of FedCausal on real problems.",{"type":17,"tag":25,"props":225,"children":226},{},[227],{"type":17,"tag":163,"props":228,"children":230},{"alt":7,"src":229},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/05/31/f7345c6637994678bb07fc3452f7b07d.png",[],{"type":17,"tag":25,"props":232,"children":233},{},[234],{"type":23,"value":235},"Table 1 Global acyclic constraint term and metrics of local causal graphs output by FedCausal",{"type":17,"tag":25,"props":237,"children":238},{},[239],{"type":17,"tag":163,"props":240,"children":242},{"alt":7,"src":241},"https://obs-mindspore-file.obs.cn-north-4.myhuaweicloud.com/file/2024/05/31/843a696ee394472f81848f02cf5ef2bb.png",[],{"type":17,"tag":25,"props":244,"children":245},{},[246],{"type":23,"value":247},"Table 2 Experimental results of FedCausal on real data",{"type":17,"tag":25,"props":249,"children":250},{},[251],{"type":17,"tag":29,"props":252,"children":253},{},[254],{"type":23,"value":255},"05 Summary and Prospects",{"type":17,"tag":25,"props":257,"children":258},{},[259],{"type":23,"value":260},"This paper introduces a federated approach to learning unified global causal graphs from dispersed heterogeneous data. FedCausal uses explainable adaptive optimization to coordinate clients, optimize local causal graphs based on client data, and learn a global causal graph that satisfies a DAG. Our analysis shows that the optimization objective of FedCausal under statistically homogeneous data is consistent with that of causal discovery algorithms for centralized data, and FedCausal can flexibly learn DAGs from decentralized heterogeneous data. Experimental results have verified the effectiveness, generality and reliability of FedCausal.",{"type":17,"tag":25,"props":262,"children":263},{},[264],{"type":23,"value":265},"In addition, MindSpore simplifies development with easy to read framework code. However, while the MindSpore framework has a strong foundation, there is an opportunity to expand MindSpore's ecosystem and foster a more vibrant developer community. By enriching the documentation and tutorial resources, MindSpore can empower beginners to grasp core concepts and embark on their MindSpore development journey more swiftly. We encourage everyone to actively participate in the MindSpore community by sharing their expertise, collaborating on problem-solving, proposing improvements, and contributing code. By working together, we can cultivate a richer and more diversified MindSpore ecosystem, empowering the growth and potential of MindSpore.",{"title":7,"searchDepth":267,"depth":267,"links":268},4,[],"markdown","content:technology-blogs:en:3132.md","content","technology-blogs/en/3132.md","technology-blogs/en/3132","md",1776506110563]