{"id":14427,"date":"2024-03-01T16:05:06","date_gmt":"2024-03-01T15:05:06","guid":{"rendered":"https:\/\/www.digitalfutures.kth.se\/?page_id=14427"},"modified":"2024-03-01T16:50:41","modified_gmt":"2024-03-01T15:50:41","slug":"fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees","status":"publish","type":"page","link":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/postdoc-fellowships\/postdoc-fellowships-ongoing-projects\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\/","title":{"rendered":"Fast Distributed Learning based on Adaptive Gradient Coding with Convergence Guarantees"},"content":{"rendered":"<p>January 2024 \u2013 December 2025<\/p>\n<p><em>Objective<br \/>\n<\/em>This project aims to propose innovative distributed learning methods based on adaptive gradient coding techniques. Within this framework, workers&#8217; participation is fluidly adjusted in real-time during training to enhance learning performance under practical constraints. We will offer rigorous theoretical proofs to ensure the convergence of the proposed methods, solidifying their reliability. We will also test the performance of the proposed methods on both simulated and actual datasets in real-world scenarios. This evaluation will benchmark the effectiveness of our techniques and underscore their superiority over current practices.<\/p>\n<p><em>Background<br \/>\n<\/em>In the framework of distributed learning, a central server aggregates computational results from various workers to update the trained model. However, in practical scenarios, &#8220;stragglers&#8221;\u2014workers who are slow or unresponsive\u2014can significantly impede overall training time. Addressing these slowdowns is crucial for real-time processing requirements in the healthcare and smart transportation sectors. While current distributed learning methods employ gradient coding to mitigate the effects of stragglers, they rely on a fixed number of the fastest workers throughout the entire training process, which have limited flexibility in balancing training time and loss. Based on that, our research question is how to transcend the limitations inherent in existing distributed learning methods and to reduce the training time required to achieve a specified training loss.<em><br \/>\n<\/em><\/p>\n<p><em>About the Digital Futures Postdoc Fellow<\/em><br \/>\n<strong>Chengxi Li<\/strong> received a PhD in 2022 from the Department of Electronic Engineering at Tsinghua University and a bachelor&#8217;s degree in 2018 from the University of Electronic Science and Technology of China. Her research interests lie in distributed learning, federated learning, signal processing and information theory.<\/p>\n<p><em>Main supervisor<\/em><br \/>\n<strong>Mikael Skoglund<\/strong>, Professor, Head of Department, Division of Information Science and Engineering, EECS, KTH.<\/p>\n<p><em>Co-supervisor<\/em><br \/>\n<strong>Ming Xiao<\/strong>, Professor, Division of Information Science and Engineering, EECS, KTH.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>January 2024-December 2025: Chengxi Li received a PhD in 2022 from the Department of Electronic Engineering at Tsinghua University&#8230;<\/p>\n","protected":false},"author":46,"featured_media":0,"parent":13339,"menu_order":211,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-14427","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.8 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Fast Distributed Learning based on Adaptive Gradient Coding with Convergence Guarantees &#8212; Digital Futures<\/title>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Fast Distributed Learning based on Adaptive Gradient Coding with Convergence Guarantees &#8212; Digital Futures\" \/>\n<meta property=\"og:description\" content=\"January 2024-December 2025: Chengxi Li received a PhD in 2022 from the Department of Electronic Engineering at Tsinghua University...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/postdoc-fellowships\/postdoc-fellowships-ongoing-projects\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\/\" \/>\n<meta property=\"og:site_name\" content=\"Digital Futures\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-01T15:50:41+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/research\\\/postdoc-fellowships\\\/postdoc-fellowships-ongoing-projects\\\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\\\/\",\"url\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/research\\\/postdoc-fellowships\\\/postdoc-fellowships-ongoing-projects\\\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\\\/\",\"name\":\"Fast Distributed Learning based on Adaptive Gradient Coding with Convergence Guarantees &#8212; Digital Futures\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/#website\"},\"datePublished\":\"2024-03-01T15:05:06+00:00\",\"dateModified\":\"2024-03-01T15:50:41+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/research\\\/postdoc-fellowships\\\/postdoc-fellowships-ongoing-projects\\\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\\\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/research\\\/postdoc-fellowships\\\/postdoc-fellowships-ongoing-projects\\\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/research\\\/postdoc-fellowships\\\/postdoc-fellowships-ongoing-projects\\\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Research\",\"item\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/research\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Postdoc fellowships\",\"item\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/research\\\/postdoc-fellowships\\\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Ongoing Postdoc fellowships\",\"item\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/research\\\/postdoc-fellowships\\\/postdoc-fellowships-ongoing-projects\\\/\"},{\"@type\":\"ListItem\",\"position\":5,\"name\":\"Fast Distributed Learning based on Adaptive Gradient Coding with Convergence Guarantees\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/#website\",\"url\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/\",\"name\":\"Digital Futures\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/#organization\",\"name\":\"Digital Futures\",\"url\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"\\\/wp-content\\\/uploads\\\/sites\\\/7\\\/2020\\\/11\\\/df_black_hires.png\",\"contentUrl\":\"\\\/wp-content\\\/uploads\\\/sites\\\/7\\\/2020\\\/11\\\/df_black_hires.png\",\"width\":5870,\"height\":856,\"caption\":\"Digital Futures\"},\"image\":{\"@id\":\"https:\\\/\\\/wpmu-tris.sys.kth.se\\\/digitalfutures\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Fast Distributed Learning based on Adaptive Gradient Coding with Convergence Guarantees &#8212; Digital Futures","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_GB","og_type":"article","og_title":"Fast Distributed Learning based on Adaptive Gradient Coding with Convergence Guarantees &#8212; Digital Futures","og_description":"January 2024-December 2025: Chengxi Li received a PhD in 2022 from the Department of Electronic Engineering at Tsinghua University...","og_url":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/postdoc-fellowships\/postdoc-fellowships-ongoing-projects\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\/","og_site_name":"Digital Futures","article_modified_time":"2024-03-01T15:50:41+00:00","twitter_card":"summary_large_image","twitter_misc":{"Estimated reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/postdoc-fellowships\/postdoc-fellowships-ongoing-projects\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\/","url":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/postdoc-fellowships\/postdoc-fellowships-ongoing-projects\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\/","name":"Fast Distributed Learning based on Adaptive Gradient Coding with Convergence Guarantees &#8212; Digital Futures","isPartOf":{"@id":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/#website"},"datePublished":"2024-03-01T15:05:06+00:00","dateModified":"2024-03-01T15:50:41+00:00","breadcrumb":{"@id":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/postdoc-fellowships\/postdoc-fellowships-ongoing-projects\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/postdoc-fellowships\/postdoc-fellowships-ongoing-projects\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/postdoc-fellowships\/postdoc-fellowships-ongoing-projects\/fast-distributed-learning-based-on-adaptive-gradient-coding-with-convergence-guarantees\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/"},{"@type":"ListItem","position":2,"name":"Research","item":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/"},{"@type":"ListItem","position":3,"name":"Postdoc fellowships","item":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/postdoc-fellowships\/"},{"@type":"ListItem","position":4,"name":"Ongoing Postdoc fellowships","item":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/research\/postdoc-fellowships\/postdoc-fellowships-ongoing-projects\/"},{"@type":"ListItem","position":5,"name":"Fast Distributed Learning based on Adaptive Gradient Coding with Convergence Guarantees"}]},{"@type":"WebSite","@id":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/#website","url":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/","name":"Digital Futures","description":"","publisher":{"@id":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/#organization","name":"Digital Futures","url":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/#\/schema\/logo\/image\/","url":"\/wp-content\/uploads\/sites\/7\/2020\/11\/df_black_hires.png","contentUrl":"\/wp-content\/uploads\/sites\/7\/2020\/11\/df_black_hires.png","width":5870,"height":856,"caption":"Digital Futures"},"image":{"@id":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/wp-json\/wp\/v2\/pages\/14427","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/wp-json\/wp\/v2\/users\/46"}],"replies":[{"embeddable":true,"href":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/wp-json\/wp\/v2\/comments?post=14427"}],"version-history":[{"count":5,"href":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/wp-json\/wp\/v2\/pages\/14427\/revisions"}],"predecessor-version":[{"id":14460,"href":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/wp-json\/wp\/v2\/pages\/14427\/revisions\/14460"}],"up":[{"embeddable":true,"href":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/wp-json\/wp\/v2\/pages\/13339"}],"wp:attachment":[{"href":"https:\/\/wpmu-tris.sys.kth.se\/digitalfutures\/wp-json\/wp\/v2\/media?parent=14427"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}