Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
10-2023
Abstract
Existing methods of privacy-preserving action recognition (PPAR) mainly focus on frame-level (spatial) privacy removal through 2D CNNs. Unfortunately, they have two major drawbacks. First, they may compromise temporal dynamics in input videos, which are critical for accurate action recognition. Second, they are vulnerable to practical attacking scenarios where attackers probe for privacy from an entire video rather than individual frames. To address these issues, we propose a novel framework STPrivacy to perform video-level PPAR. For the first time, we introduce vision Transformers into PPAR by treating a video as a tubelet sequence, and accordingly design two complementary mechanisms, i.e., sparsification and anonymization, to remove privacy from a spatio-temporal perspective. In specific, our privacy sparsification mechanism applies adaptive token selection to abandon action-irrelevant tubelets. Then, our anonymization mechanism implicitly manipulates the remaining action-tubelets to erase privacy in the embedding space through adversarial learning. These mechanisms provide significant advantages in terms of privacy preservation for human eyes and action-privacy trade-off adjustment during deployment. We additionally contribute the first two large-scale PPAR benchmarks, VP-HMDB51 and VP-UCF101, to the community. Extensive evaluations on them, as well as two other tasks, validate the effectiveness and generalization capability of our framework.
Keywords
Privacy, Data privacy, Computer vision, Benchmark testing, Transformers, Information filtering, Task analysis
Discipline
Graphics and Human Computer Interfaces
Research Areas
Intelligent Systems and Optimization
Areas of Excellence
Digital transformation
Publication
Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 2023 October 1-6
First Page
5106
Last Page
5115
ISBN
9798350307191
Identifier
10.1109/ICCV51070.2023.00471
Publisher
IEEE
City or Country
Piscataway, NJ
Citation
LI, Ming; XU, Xiangyu; FAN, Hehe; ZHOU, Pan; LIU, Jun; LIU, Jia-Wei; LI, Jiahe; KEPPO, Jussi; SHOU, Mike Zheng; and YAN, Shuicheng.
STPrivacy: Spatio-temporal privacy-preserving action recognition. (2023). Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 2023 October 1-6. 5106-5115.
Available at: https://ink.library.smu.edu.sg/sis_research/8985
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/ICCV51070.2023.00471