Introduction
- The structured segment networks framework (SSN) is a framework for temporal action detection in untrimmed videos.
- SSN performs accurate temporal action detection. This is achieved by efficiently and effectively modeling temporal structures of activity instances in an end-to-end manner.
- SSN achieves leading performance on THUMOS14 and ActivityNet benchmarks.
Experiments
Action Proposals with TAG
| Dataset | Average Number of Proposal | Average Recall |
|---|---|---|
| THUMOS14 | 200 | 48.9% |
| ActivityNet v1.2 Val. | 100 | 67.3% |
Temporal Action Detection with SSN
| THUMOS14 | |||||||
|---|---|---|---|---|---|---|---|
| IOU | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 |
| mAP | 66.0% | 59.4% | 51.9% | 41.0% | 29.8% | 19.6% | 10.7% |
| ActivityNet v1.3 Testing Server | ||||
|---|---|---|---|---|
| IOU | 0.5 | 0.75 | 0.95 | Average |
| mAP | 43.26% | 28.70% | 5.63% | 28.28% |
Visualize Detection Results
- Blue: groundtruth instances.
- Green: valid detection results.
- Red: over-length detections.
- Yellow: in-complete detections.
Citation
Please kindly cite the following paper if you find this project helpful.
@inproceedings{SSN2017ICCV,
author = {Yue Zhao and Yuanjun Xiong and Limin Wang
and Zhirong Wu and Xiaoou Tang and Dahua Lin},
title = {Temporal Action Detection with Structured Segment Networks},
booktitle = {ICCV},
year = {2017},
}
Related Projects
-
Temporal Segment Networks (TSN)
Current state-of-the-art action recognition framework featuring efficient end-to-end modeling of long-range temporal information.
[Project Site] [Github Link] [Paper] -
A Pursuit of Temporal Accuracy in General Activity Detection
We provide in-depth analysis of the problem of temporal action detection in this tech report. It is the predecessor of the SSN framework.
[Tech Report] -
CES-STAR@ActivityNet 2016
We secured the first place of untrimmed video classification task in ActivityNet Large Scale Action Recognition Challenge 2016, held in conjunction with CVPR'16. The method and models of our submissions are released for research use.
[Github Link] [Notebook Paper] [Challenge Results] -
Caffe
Our modified version of the famous Caffe toolbox featuring MPI-based parallel training and Video IO support. We also introduced the cross-modality training of optical flow networks in this work.
[Github Link] [Tech Report] -
Enhanced MV for Real-Time Action Recognition
Enhanced MV-CNN is a real-time action recognition algorithm. It uses motion vector to achieve real-time processing speed and knowledge transfer techniques to improve recognition performance.
[CVPR16 Paper] [Project Page] -
Trajectory-Pooled Deep Descriptors (TDD)
The state-of-the-art approach for action recognition before TSN.
[CVPR15 Paper] [Github Link]
Contact
- For questions and inquiries, please contact:
- Yue Zhao: thuzhaoyue@gmail.com
- Yuanjun Xiong: yjxiong@ie.cuhk.edu.hk
- Limin Wang: lmwang.nju@gmail.com