Federated learning is a privacy preserving method of distributed machine learning over heterogeneous user data. However, its current clients selection technique is a source of bias as it discriminates against slow clients. For starters, it selects clients that satisfy certain network and system specific criteria, thus ignores slow clients. Even when such clients are included in the training process, they either straggle the training or are altogether dropped from the round for being too slow.
This projects presents FedSS, federated learning with smart selection of clients, which reaches the best trade off between training performance and data heterogeneity. The main idea is that the FedSS server groups clients with similar training time together based on online profiling. Then, it applies round-robin to choose a group for training every round. The round-robin approach can avoid bias while the effect of straggler is greatly alleviated since clients in a group are similar. Our major contribution are:
- We proposes an online profiling approach to estimate the training time of clients based on network environment, computation capabilities, etc.
- We proposes a clients clustering algorithm for FedSS, and an approach to get optimal number of clusters.
- We implemented FedSS based on LEAF by building a RPC-based communication protocol and scheduling system.
- We evaluated the effectiveness of FedSS, which improves the accuracy the slow clients by 10% with only 40% more training time compared with FedCS.
