Human-sensing applications such as activity recognition, fall detection, and health monitoring have been revolutionized by advancements in artificial intelligence (AI) and machine learning technologies. These applications can significantly impact health management by monitoring human behavior and providing critical data for health assessments. However, due to the variability in individual behaviors, environmental factors, and the physical placement of devices, the performance of generic AI models is often hindered. This is particularly problematic when such models encounter distribution shifts in sensory data, as the variations cause a mismatch between training and testing conditions. Personalization is thus necessary to adapt these models to specific user patterns, making them more effective and reliable for real-world use.
The core issue that researchers aim to address is the challenge of adapting AI models to individual users when there is limited data available or when the data collected exhibits variability due to changes in external conditions. While capable of generalizing across broader populations, generic models tend to falter when faced with unique user-specific variations such as changes in movement patterns, speech characteristics, or health indicators. This issue is exacerbated in healthcare scenarios where data scarcity is common, and unique patient traits are often underrepresented in the training data. Furthermore, the intra-user variability across different scenarios leads to a lack of generalizability, which is critical for applications like health monitoring, where physiological conditions may change significantly over time due to disease progression or treatment interventions.
Various methods have been proposed to personalize models, including continuous and static personalization techniques. Continuous personalization involves updating the model based on newly acquired data. However, obtaining ground truths for such data in healthcare applications can be labor-intensive and require constant clinical supervision, making this method infeasible for real-time or large-scale deployments. On the other hand, static personalization occurs during user enrollment using a limited initial data set. While this reduces computational overhead and minimizes user engagement, it typically results in models that do not generalize well to contexts not seen during the initial enrollment phase.
Researchers from Syracuse University and Arizona State University introduced a new approach called CRoP (Context-wise Robust Static Human-Sensing Personalization). This method leverages off-the-shelf pre-trained models and adapts them using pruning techniques to address the intra-user variability challenge. The CRoP approach is unique in its use of model pruning, which involves removing redundant parameters from the personalized model and replacing them with generic ones. This technique helps maintain the personalized model’s ability to generalize across different unseen contexts while ensuring high performance for the context in which it was trained. Using this method, the researchers can create static personalized models that perform robustly even when the user’s external conditions change significantly.
The CRoP approach begins by finetuning a generic model using the limited data collected during a user’s initial enrollment. This personalized model is then pruned to identify and remove redundant parameters that do not contribute significantly to model inference for the given context. Next, the pruned parameters are replaced with corresponding parameters from the generic model, effectively restoring the model’s generalizability. The final step involves further fine-tuning the mixed model on the available user data to optimize performance. This three-step process ensures that the personalized model retains the capacity to generalize across unseen contexts without compromising its effectiveness in the context in which it was trained.
The researchers tested the method on four human-sensing datasets: the PERCERT-R clinical speech therapy dataset, the WIDAR WiFi-based activity recognition dataset, the ExtraSensory mobile sensing dataset, and a stress-sensing dataset collected via wearable sensors. The results show that CRoP achieved a 35.23% increase in personalization accuracy compared to generic models and a 7.78% improvement in generalization compared to conventional finetuning methods. Specifically, on the WIDAR dataset, CRoP improved accuracy from 63.90% to 87.06% in the primary context while maintaining a lower performance drop in unseen contexts, demonstrating its robustness in adapting to varied user scenarios. Similarly, on the PERCEPT-R dataset, CRoP yielded a 67.81% accuracy in the initial context and maintained a performance stability of 13.81% in unseen scenarios.
The research demonstrates that CRoP models outperform conventional methods such as SHOT, PackNet, Piggyback, and CoTTA in personalization and generalization. For example, while PackNet achieved only a 26.05% improvement in personalization and a -1.39% drop in generalization, CRoP provided a 35.23% improvement in personalization and a positive 7.78% gain in generalization. This indicates that CRoP’s method of integrating pruning and restoration techniques is more effective in handling the distribution shifts common in human-sensing applications.
Key Takeaways from the research:
- CRoP increases personalization accuracy by 35.23% compared to generic models.
- Generalization improvement of 7.78% is achieved using CRoP over conventional finetuning.
- In most datasets, CRoP outperforms other state-of-the-art methods like SHOT and CoTTA by 9-20%.
- The method maintains high performance across diverse contexts with minimal additional computational overhead.
- The approach is particularly effective for health-related applications, where changes in user conditions are frequent and challenging to predict.
In conclusion, CRoP offers a novel solution for tackling the limitations of static personalization. Leveraging off-the-shelf models and incorporating pruning techniques effectively balances the trade-off between intra-user personalization and generalization. This approach addresses the need for personalized models that perform well across different contexts, making it particularly suitable for sensitive applications like healthcare, where robustness and adaptability are crucial.
Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..
Don’t Forget to join our 52k+ ML SubReddit.
We are inviting startups, companies, and research institutions who are working on small language models to participate in this upcoming ‘Small Language Models’ Magazine/Report by Marketchpost.com. This Magazine/Report will be released in late October/early November 2024. Click here to set up a call!
The post CRoP: A Context-wise Static Personalization Method for Robust and Scalable Human-Sensing AI Models in Healthcare and Real-World Scenarios appeared first on MarkTechPost.