Federated Learning (FL), a revolutionized privacy-preserving distributed Machine Learning (ML), enables models to learn from data distributed across multiple devices at the edge, empowering Edge Intelligence (EI) applications. However, a significant challenge within FL is the issue of data skew, where data distribution across devices varies significantly, potentially impairing model performance. This paper investigates this challenge by exploring the application of FL in a complex facial ethnicity classification, including blurry label boundaries and non-IID data distribution. The paper systematically examines the effects of data skews on FL aggregation algorithms over five algorithms and three different datasets using multiple scenarios. In particular, in scenarios involving sensitive non-IID data such as facial attributes. Our approach involves a novel methodology that adapts aggregation techniques to handle better the heterogeneous data distributions typical of real-world FL environments, demonstrating the potential for more robust and equitable model performance across diverse edge devices. Key findings highlight the importance of FL in preserving data privacy while facilitating model improvement, exemplifying its potential in diverse fields beyond biometrics.