This research project explores the emerging potential of large language models (LLMs) to interpret, summarize, and contextualize human behavior from multimodal activity data. As wearable sensors and smart environments increasingly track physical, physiological, and contextual signals, such as movement patterns, environmental cues, and interaction logs, there is a growing opportunity to generate rich, narrative-level insights into daily life. LLMs, with their strong capabilities in pattern abstraction, contextual inference, and natural language generation, offer a powerful means to transform low-level activity traces into meaningful, human-centered representations. These capabilities can enable applications such as automated life journaling, behavior-aware digital assistants, and personalized health and productivity insights. However, using LLMs in this context also introduces novel risks, including biased interpretation of activity patterns, unintended behavioral profiling, and privacy violations from inferred context. This project aims to develop a foundational framework for using LLMs in human activity recognition (HAR), with an emphasis on interpretability, personalization, and responsible deployment. Objectives of this project are follows.

1. Design LLM-augmented models that semantically enrich and interpret multimodal sensor data for accurate, personalized activity recognition and context understanding;

2. Construct and utilize paired datasets that link structured behavioral data with natural language summaries, enabling training and evaluation of narrative-level HAR systems;

3. Develop privacy-aware, bias-mitigated methods for transforming activity traces into language-based reflections, supporting applications in life logging, wellness analytics, and assistive technologies. 

LLM-Based Human Activity Recognition Multimodal Behavior Understanding Narrative Generation from Sensor Data  Privacy-Preserving Life Logging  Personalized Contextual AI   

The proliferation of wearable technologies in health monitoring, activity recognition, and intelligent transportation systems has introduced a growing reliance on machine learning (ML) models for real-time, context-aware inference. However, the integration of ML into resource-constrained, distributed, and privacy-sensitive wearable ecosystems exposes the underlying models to a range of sophisticated vulnerabilities, including data poisoning, inference-time attacks, and energy-latency sponge attacks, that can undermine system reliability, compromise user safety, and degrade trust in AI-assisted decision-making.

This research project investigates the full-spectrum vulnerability landscape of wearable ML systems, from data collection and training to deployment and inference. We aim to develop principled, efficient, and context-aware defense mechanisms capable of securing these systems under adversarial and uncertain conditions. Building on our prior work in federated learning, differential privacy, spatiotemporal poisoning, and trust-aware inference on wearables, this project systematically addresses both known and emerging threats across the ML lifecycle. Our objectives are threefold:

1. Characterize attack surfaces and failure modes in wearable ML systems, including spatiotemporal data poisoning, label-flipping attacks, inference-time adversarial perturbations, and energy-latency sponge attacks targeting resource exhaustion and performance degradation;

2. Develop mitigation techniques tailored for wearable and edge environments, such as privacy-preserving robust training methods, anomaly-aware aggregation in federated learning, runtime inference filtering, and trust-calibrated fusion of multi-sensor data streams;

3. Quantitatively evaluate defense strategies under real-world constraints, focusing on adversarial robustness, energy efficiency, computational overhead, detection latency, and overall system utility in applications including activity recognition, cognitive state monitoring, and behavior-driven interventions. 

Adversarial Machine Learning in Wearables   Data Poisoning Attacks and Defenses   Energy-Latency Sponge Attacks   Federated Learning Security   Robust Human Activity Recognition