OS-Kairos: Adaptive Interaction for MLLM-Powered GUI Agents

Pengzhou Cheng, Zheng Wu, Zongru Wu, Aston Zhang, Zhuosheng Zhang, Gongshen Liu

• By predicting confidence levels at each interaction step, we are releasing an adaptive GUI agent (OS-Kairos) that alleviates the over-execution of autonomous GUI agents, and significantly improves task success rates in complex scenarios.

Smoothing Grounding and Reasoning for MLLM-Powered GUI Agents with Query-Oriented Pivot Tasks

Zongru Wu, Pengzhou Cheng, Zheng Wu, Tianjie Ju, Zhuosheng Zhang, Gongshen Liu

• We propose a pivot task name query inference to smooth coordinate-oriented grounding and action-oriented reasoning, enhancing the comprehension of user intent and imporoving the performance of MLLM-powered GUI agents in resource-constrained scenarios.

Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Review

Pengzhou Cheng, Zongru Wu, Wei Du, Haodong Zhao, Wei Lu, Gongshen Liu

• We provide the natural language processing (NLP) community with a timely review of backdoor attacks and countermeasures.

SynGhost: Invisible and Universal Task-agnostic Backdoor Attack via Syntactic Transfer

Pengzhou Cheng, Wei Du, Zongru Wu, Fengwei Zhang, Libo Chen, Zhuosheng Zhang, Gongshen Liu

• We propose maxEntropy, an entropy-based poisoning filter that accurately detects existing task-agnostic backdoors. To further expose the vulnerabilities in PLMs, we propose SynGhost, an invisible and universal task-agnostic backdoor.

Gracefully Filtering Backdoor Samples for Generative Large Language Models without Retraining

Zongru Wu, Pengzhou Cheng, Lingyong Fang, Zhuosheng Zhang, Gongshen Liu

• We reveal a distinct separation between the gradients of backdoor and clean samples in the frequency space. Based on this, we propose Gradient Clustering in the Frequency Space for Backdoor Sample Filtering (GraCeFul), which leverages sample-wise gradients in the frequency space to effectively identify backdoor samples without requiring retraining LLMs.

Acquiring Clean Language Models from Backdoor Poisoned Datasets by Downscaling Frequency Space

Zongru Wu, Zhuosheng Zhang, Pengzhou Cheng, Gongshen Liu

• We conduct Fourier Analysis to the clean and backdoor mapping and reveal the low-frequency bias on backdoor mapping, which results in the faster convergence of backdoor mapping. Based on this phenomenon, we propose a novel approach that encourages language models to prioritize clean mapping by downscaling in the frequency space, thus efficiently reducing the average success rate of backdoor attacks.