Home | Behavioral | Applications | Datasets

Click on each entry below to see additional information.

Action anticipation

Pardo-Decimavilla et al., Do You Act Like You Talk? Exploring Pose-based Driver Action Classification with Speech Recognition Networks, IV, 2024 | paper | code

Drive&Act

@inproceedings{2024_IV_Pardo-Decimavilla,
    author = "Pardo-Decimavilla, Pablo and Bergasa, Luis M and Montiel-Mar{\'\i}n, Santiago and Antunes, Miguel and Llamazares, {\'A}ngel",
    booktitle = "2024 IEEE Intelligent Vehicles Symposium (IV)",
    organization = "IEEE",
    pages = "1395--1400",
    title = "Do You Act Like You Talk? Exploring Pose-based Driver Action Classification with Speech Recognition Networks",
    year = "2024"
}

Tanama et al., Quantized Distillation: Optimizing Driver Activity Recognition Models for Resource-Constrained Environments, IROS, 2023 | paper | code

Drive&Act

@inproceedings{2023_IROS_Tanama,
    author = "Tanama, Calvin and Peng, Kunyu and Marinov, Zdravko and Stiefelhagen, Rainer and Roitberg, Alina",
    booktitle = "2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)",
    organization = "IEEE",
    pages = "5479--5486",
    title = "Quantized Distillation: Optimizing Driver Activity Recognition Models for Resource-Constrained Environments",
    year = "2023"
}

Peng et al., TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration, IROS, 2022 | paper | code

Drive&Act

@inproceedings{2022_IROS_Peng,
    author = "Peng, Kunyu and Roitberg, Alina and Yang, Kailun and Zhang, Jiaming and Stiefelhagen, Rainer",
    booktitle = "2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)",
    organization = "IEEE",
    pages = "278--285",
    title = "TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration",
    year = "2022"
}

Jain et al., Recurrent Neural Networks for Driver Activity Anticipation via Sensory-Fusion Architecture, ICRA, 2016 | paper | code

Brain4Cars

@inproceedings{2016_ICRA_Jain,
    author = "Jain, Ashesh and Singh, Avi and Koppula, Hema S and Soh, Shane and Saxena, Ashutosh",
    booktitle = "ICRA",
    title = "Recurrent neural networks for driver activity anticipation via sensory-fusion architecture",
    year = "2016"
}

Scene gaze

Kotseruba et al., SCOUT+: Towards Practical Task-Driven Drivers’ Gaze Prediction, IV, 2024 | paper | code

DR(eye)VE

BDD-A

SCOUT

@inproceedings{2024_IV_Kotseruba_2,
    author = "Kotseruba, Iuliia and Tsotsos, John K",
    booktitle = "Intelligent Vehicles Symposium (IV)",
    title = "{SCOUT+: Towards Practical Task-Driven Drivers' Gaze Prediction}",
    year = "2024"
}

Kotseruba et al., Data Limitations for Modeling Top-Down Effects on Drivers’ Attention, IV, 2024 | paper | code

@inproceedings{2024_IV_Kotseruba_1,
    author = "Kotseruba, Iuliia and Tsotsos, John K",
    booktitle = "Intelligent Vehicles Symposium (IV)",
    title = "Data Limitations for Modeling Top-Down Effects on Drivers' Attention",
    year = "2024"
}

Kotseruba et al., Understanding and Modeling the Effects of Task and Context on Drivers’ Gaze Allocation, IV, 2024 | paper | code

DR(eye)VE

SCOUT

@inproceedings{2024_IV_Kotseruba,
    author = "Kotseruba, Iuliia and Tsotsos, John K",
    booktitle = "2024 IEEE Intelligent Vehicles Symposium (IV)",
    organization = "IEEE",
    pages = "1337--1344",
    title = "Understanding and modeling the effects of task and context on drivers’ gaze allocation",
    year = "2024"
}

Deng et al., Driving Visual Saliency Prediction of Dynamic Night Scenes via a Spatio-Temporal Dual-Encoder Network, Trans. ITS, 2023 | paper | code

DrFixD-night

DR(eye)VE

@article{2023_T-ITS_Deng,
    author = "Deng, Tao and Jiang, Lianfang and Shi, Yi and Wu, Jiang and Wu, Zhangbi and Yan, Shun and Zhang, Xianshi and Yan, Hongmei",
    journal = "IEEE Transactions on Intelligent Transportation Systems",
    publisher = "IEEE",
    title = "Driving Visual Saliency Prediction of Dynamic Night Scenes via a Spatio-Temporal Dual-Encoder Network",
    year = "2023"
}

Bhagat et al., Driver Gaze Fixation and Pattern Analysis in Safety Critical Events, IV, 2023 | paper | code

Dataset(s): SHRP2

@inproceedings{2023_IV_Bhagat,
    author = "Bhagat, Hirva and Jain, Sandesh and Abbott, Lynn and Sonth, Akash and Sarkar, Abhijit",
    booktitle = "2023 IEEE Intelligent Vehicles Symposium (IV)",
    organization = "IEEE",
    pages = "1--8",
    title = "Driver gaze fixation and pattern analysis in safety critical events",
    year = "2023"
}

Zhao et al., Gated Driver Attention Predictor, ITSC, 2023 | paper | code

DADA-2000

BDD-A

@inproceedings{2023_ITSC_Zhao,
    author = "Zhao, Tianci and Bai, Xue and Fang, Jianwu and Xue, Jianru",
    booktitle = "2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC)",
    organization = "IEEE",
    pages = "270--276",
    title = "Gated Driver Attention Predictor",
    year = "2023"
}

Zhu et al., Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding, ICCV, 2023 | paper | code

DR(eye)VE

BDD-A

DADA-2000

@inproceedings{2023_ICCV_Zhu,
    author = "Zhu, Pengfei and Qi, Mengshi and Li, Xia and Li, Weijian and Ma, Huadong",
    booktitle = "Proceedings of the IEEE/CVF International Conference on Computer Vision",
    pages = "8558--8568",
    title = "Unsupervised self-driving attention prediction via uncertainty mining and knowledge embedding",
    year = "2023"
}

Li et al., Adaptive Short-Temporal Induced Aware Fusion Network for Predicting Attention Regions Like a Driver, Trans. ITS, 2022 | paper | code

BDD-A

DADA-2000

TrafficSaliency

@article{2022_T-ITS_Li,
    author = "Li, Qiang and Liu, Chunsheng and Chang, Faliang and Li, Shuang and Liu, Hui and Liu, Zehao",
    journal = "IEEE Transactions on Intelligent Transportation Systems",
    number = "10",
    pages = "18695--18706",
    publisher = "IEEE",
    title = "Adaptive short-temporal induced aware fusion network for predicting attention regions like a driver",
    volume = "23",
    year = "2022"
}

Fang et al., DADA: Driver Attention Prediction in Driving Accident Scenarios, Trans. ITS, 2021 | paper | code

TrafficSaliency

DR(eye)VE

DADA-2000

@article{2022_T-ITS_Fang,
    author = "Fang, Jianwu and Yan, Dingxin and Qiao, Jiahuan and Xue, Jianru and Yu, Hongkai",
    journal = "IEEE Transactions on Intelligent Transportation Systems",
    number = "6",
    pages = "4959--4971",
    publisher = "IEEE",
    title = "DADA: Driver attention prediction in driving accident scenarios",
    volume = "23",
    year = "2021"
}

Araluce et al., ARAGAN: A dRiver Attention estimation model based on conditional Generative Adversarial Network, IV, 2022 | paper | code

BDD-A

DADA-2000

@inproceedings{2022_IV_Araluce,
    author = "Araluce, Javier and Bergasa, Luis M and Oca{\\textasciitilde n}a, Manuel and Barea, Rafael and L{\'o}pez-Guill{\'e}n, Elena and Revenga, Pedro",
    booktitle = "2022 IEEE Intelligent Vehicles Symposium (IV)",
    organization = "IEEE",
    pages = "1066--1072",
    title = "ARAGAN: A dRiver Attention estimation model based on conditional Generative Adversarial Network",
    year = "2022"
}

Kasahara et al., Look Both Ways: Self-Supervising Driver Gaze Estimation and Road Scene Saliency, ECCV, 2022 | paper | code

LBW

@inproceedings{2022_ECCV_Kasahara,
    author = "Kasahara, Isaac and Stent, Simon and Park, Hyun Soo",
    booktitle = "Computer Vision--ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XIII",
    organization = "Springer",
    pages = "126--142",
    title = "Look Both Ways: Self-supervising Driver Gaze Estimation and Road Scene Saliency",
    year = "2022"
}

Gopinath et al., MAAD: A Model and Dataset for “Attended Awareness” in Driving, ICCVW, 2021 | paper | code

MAAD

@inproceedings{2021_ICCVW_Gopinath,
    author = "Gopinath, Deepak and Rosman, Guy and Stent, Simon and Terahata, Katsuya and Fletcher, Luke and Argall, Brenna and Leonard, John",
    booktitle = "Proceedings of the IEEE/CVF International Conference on Computer Vision",
    pages = "3426--3436",
    title = {MAAD: A Model and Dataset for" Attended Awareness" in Driving},
    year = "2021"
}

Baee et al., MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning, ICCV, 2021 | paper | code

Dataset(s): Eyecar

@inproceedings{2021_ICCV_Baee,
    author = "Baee, Sonia and Pakdamanian, Erfan and Kim, Inki and Feng, Lu and Ordonez, Vicente and Barnes, Laura",
    booktitle = "ICCV",
    title = "MEDIRL: Predicting the visual attention of drivers via maximum entropy deep inverse reinforcement learning",
    year = "2021"
}

Deng et al., How Do Drivers Allocate Their Potential Attention? Driving Fixation Prediction via Convolutional Neural Networks, Trans. ITS, 2020 | paper | code

TrafficSaliency

@article{2020_T-ITS_Deng,
    author = "Deng, Tao and Yan, Hongmei and Qin, Long and Ngo, Thuyen and Manjunath, BS",
    journal = "IEEE Transactions on Intelligent Transportation Systems",
    number = "5",
    pages = "2146--2154",
    publisher = "IEEE",
    title = "{How do drivers allocate their potential attention? Driving fixation prediction via convolutional neural networks}",
    volume = "21",
    year = "2019"
}

Pal et al., “Looking at the right stuff” - Guided semantic-gaze for autonomous driving, CVPR, 2020 | paper | code

DR(eye)VE

BDD-A

@inproceedings{2020_CVPR_Pal,
    author = "Pal, Anwesan and Mondal, Sayan and Christensen, Henrik I",
    booktitle = "CVPR",
    title = {{" Looking at the Right Stuff"-Guided Semantic-Gaze for Autonomous Driving}},
    year = "2020"
}

Palazzi et al., Predicting the Driver’s Focus of Attention: the DR(eye)VE Project, PAMI, 2018 | paper | code

DR(eye)VE

@article{2018_PAMI_Palazzi,
    author = "Palazzi, Andrea and Abati, Davide and Solera, Francesco and Cucchiara, Rita and others",
    journal = "IEEE TPAMI",
    number = "7",
    pages = "1720--1733",
    title = "{Predicting the Driver's Focus of Attention: the DR (eye) VE Project}",
    volume = "41",
    year = "2018"
}

Xia et al., Predicting Driver Attention in Critical Situations, ACCV, 2018 | paper | code

BDD-A

@inproceedings{2018_ACCV_Xia,
    author = "Xia, Ye and Zhang, Danqing and Kim, Jinkyu and Nakayama, Ken and Zipser, Karl and Whitney, David",
    booktitle = "ACCV",
    title = "Predicting driver attention in critical situations",
    year = "2018"
}

Ohn-Bar et al., Are all objects equal? Deep spatio-temporal importance prediction in driving videos, Pattern Recognition, 2017 | paper | code

Dataset(s): KITTI

@article{2017_PR_Ohn-Bar,
    author = "Ohn-Bar, Eshed and Trivedi, Mohan Manubhai",
    journal = "Pattern Recognition",
    pages = "425--436",
    title = "Are all objects equal? Deep spatio-temporal importance prediction in driving videos",
    volume = "64",
    year = "2017"
}

Palazzi et al., Learning Where to Attend Like a Human Driver, IV, 2017 | paper | code

DR(eye)VE

@inproceedings{2017_IV_Palazzi,
    author = "Palazzi, Andrea and Solera, Francesco and Calderara, Simone and Alletto, Stefano and Cucchiara, Rita",
    booktitle = "IV",
    title = "Learning where to attend like a human driver",
    year = "2017"
}

Deng et al., Where Does the Driver Look? Top-Down-Based Saliency Detection in a Traffic Driving Environment, Trans. ITS, 2016 | paper | code

Dataset(s): private

@article{2016_T-ITS_Deng,
    author = "Deng, Tao and Yang, Kaifu and Li, Yongjie and Yan, Hongmei",
    journal = "IEEE Transactions on Intelligent Transportation Systems",
    number = "7",
    pages = "2051--2062",
    publisher = "IEEE",
    title = "Where does the driver look? Top-down-based saliency detection in a traffic driving environment",
    volume = "17",
    year = "2016"
}

Johnson et al., Predicting human visuomotor behaviour in a driving task, Philosophical Transactions of the Royal Society: B, 2013 | paper | code

Dataset(s): private

@article{2013_RSTB_Johnson,
    author = "Johnson, Leif and Sullivan, Brian and Hayhoe, Mary and Ballard, Dana",
    journal = "Philosophical Transactions of the Royal Society B: Biological Sciences",
    number = "1636",
    pages = "20130044",
    title = "Predicting human visuomotor behaviour in a driving task",
    volume = "369",
    year = "2014"
}

Borji et al., Probabilistic Learning of Task-Specific Visual Attention, CVPR, 2012 | paper | code

3DDS

@inproceedings{2012_CVPR_Borji,
    author = "Borji, Ali and Sihite, Dicky N and Itti, Laurent",
    booktitle = "CVPR",
    title = "Probabilistic learning of task-specific visual attention",
    year = "2012"
}

In-vehicle gaze

Cheng et al., What Do You See in Vehicle? Comprehensive Vision Solution for In-Vehicle Gaze Estimation, CVPR, 2024 | paper | code

IVGaze

@inproceedings{2024_CVPR_Cheng,
    author = "Cheng, Yihua and Zhu, Yaning and Wang, Zongji and Hao, Hongquan and Liu, Yongwei and Cheng, Shiqing and Wang, Xi and Chang, Hyung Jin",
    booktitle = "Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition",
    pages = "1556--1565",
    title = "What Do You See in Vehicle? Comprehensive Vision Solution for In-Vehicle Gaze Estimation",
    year = "2024"
}

Rangesh et al., Driver Gaze Estimation in the Real World: Overcoming the Eyeglass Challenge, IV, 2020 | paper | code

Dataset(s): LISA v3

@inproceedings{2020_IV_Rangesh,
    author = "Rangesh, Akshay and Zhang, Bowen and Trivedi, Mohan M",
    booktitle = "IV",
    title = "Driver gaze estimation in the real world: Overcoming the eyeglass challenge",
    year = "2020"
}

Stappen et al., X-AWARE: ConteXt-AWARE Human-Environment Attention Fusion for Driver Gaze Prediction in the Wild, ICMI, 2020 | paper | code

DGW

@inproceedings{2020_ICMI_Stappen,
    author = {Stappen, Lukas and Rizos, Georgios and Schuller, Bj{\"o}rn},
    booktitle = "ICMI",
    title = "X-AWARE: ConteXt-AWARE Human-Environment Attention Fusion for Driver Gaze Prediction in the Wild",
    year = "2020"
}

Jokinen et al., Multitasking in Driving as Optimal Adaptation Under Uncertainty, Human Factors, 2020 | paper | code

Dataset(s): private

@article{2020_HumanFactors_Jokinen,
    author = "Jokinen, Jussi PP and Kujala, Tuomo and Oulasvirta, Antti",
    journal = "Human factors",
    number = "8",
    pages = "1324--1341",
    publisher = "Sage Publications Sage CA: Los Angeles, CA",
    title = "Multitasking in driving as optimal adaptation under uncertainty",
    volume = "63",
    year = "2021"
}

Distraction detection

Yang et al., Quantitative Identification of Driver Distraction: A Weakly Supervised Contrastive Learning Approach, Trans. ITS, 2024 | paper | code

3MDAD

AUCD2

SAM-DD

@article{2024_T-ITS_Yang,
    author = "Yang, Haohan and Liu, Haochen and Hu, Zhongxu and Nguyen, Anh-Tu and Guerra, Thierry-Marie and Lv, Chen",
    journal = "IEEE Transactions on Intelligent Transportation Systems",
    publisher = "IEEE",
    title = "Quantitative identification of driver distraction: A weakly supervised contrastive learning approach",
    year = "2023"
}

Hasan et al., Vision-Language Models Can Identify Distracted Driver Behavior From Naturalistic Videos, Trans. ITS, 2024 | paper | code

@article{2024_T-ITS_Hasan,
    author = "Hasan, Md Zahid and Chen, Jiajing and Wang, Jiyang and Rahman, Mohammed Shaiqur and Joshi, Ameya and Velipasalar, Senem and Hegde, Chinmay and Sharma, Anuj and Sarkar, Soumik",
    journal = "IEEE Transactions on Intelligent Transportation Systems",
    publisher = "IEEE",
    title = "Vision-language models can identify distracted driver behavior from naturalistic videos",
    year = "2024"
}

Ma et al., ViT-DD: Multi-Task Vision Transformer for Semi-Supervised Driver Distraction Detection, IV, 2024 | paper | code

SFDDD

AUCD2

@inproceedings{2024_IV_Ma,
    author = "Ma, Yunsheng and Wang, Ziran",
    booktitle = "2024 IEEE Intelligent Vehicles Symposium (IV)",
    organization = "IEEE",
    pages = "417--423",
    title = "Vit-dd: Multi-task vision transformer for semi-supervised driver distraction detection",
    year = "2024"
}

Sonth et al., Explainable Driver Activity Recognition Using Video Transformer in Highly Automated Vehicle, IV, 2023 | paper | code

Dataset(s): VTTIMLP01, SHRP2

@inproceedings{2023_IV_Sonth,
    author = "Sonth, Akash and Sarkar, Abhijit and Bhagat, Hirva and Abbott, Lynn",
    booktitle = "2023 IEEE Intelligent Vehicles Symposium (IV)",
    organization = "IEEE",
    pages = "1--8",
    title = "Explainable Driver Activity Recognition Using Video Transformer in Highly Automated Vehicle",
    year = "2023"
}

Zhou et al., Multi View Action Recognition for Distracted Driver Behavior Localization, CVPRW, 2023 | paper | code

AI CITY NDAR

@inproceedings{2023_CVPRW_Zhou,
    author = "Zhou, Wei and Qian, Yinlong and Jie, Zequn and Ma, Lin",
    booktitle = "Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition",
    pages = "5375--5380",
    title = "Multi view action recognition for distracted driver behavior localization",
    year = "2023"
}

Driver awareness

Zhou et al., Using Eye-Tracking Data to Predict Situation Awareness in Real Time During Takeover Transitions in Conditionally Automated Driving, Trans. ITS, 2021 | paper | code

Dataset(s): private

@article{2022_T-ITS_Zhou,
    author = "Zhou, Feng and Yang, X Jessie and de Winter, Joost CF",
    journal = "IEEE Transactions on Intelligent Transportation Systems",
    number = "3",
    pages = "2284--2295",
    publisher = "IEEE",
    title = "Using eye-tracking data to predict situation awareness in real time during takeover transitions in conditionally automated driving",
    volume = "23",
    year = "2021"
}

Self-driving

Chitta et al., NEAT: Neural Attention Fields for End-to-End Autonomous Driving, ICCV, 2021 | paper | code

Dataset(s): CARLA

@inproceedings{2021_ICCV_Chitta,
    author = "Chitta, Kashyap and Prakash, Aditya and Geiger, Andreas",
    booktitle = "ICCV",
    title = "NEAT: Neural Attention Fields for End-to-End Autonomous Driving",
    year = "2021"
}

Prakash et al., Multi-Modal Fusion Transformer for End-to-End Autonomous Driving, CVPR, 2021 | paper | code

Dataset(s): CARLA

@inproceedings{2021_CVPR_Prakash,
    author = "Prakash, Aditya and Chitta, Kashyap and Geiger, Andreas",
    booktitle = "CVPR",
    title = "Multi-Modal Fusion Transformer for End-to-End Autonomous Driving",
    year = "2021"
}

Xia et al., Periphery-Fovea Multi-Resolution Driving Model Guided by Human Attention, WACV, 2020 | paper | code

BDD-X

BDD-A

DR(eye)VE

@inproceedings{2020_WACV_Xia,
    author = "Xia, Ye and Kim, Jinkyu and Canny, John and Zipser, Karl and Canas-Bajo, Teresa and Whitney, David",
    booktitle = "WACV",
    title = "Periphery-fovea multi-resolution driving model guided by human attention",
    year = "2020"
}

Mittal et al., AttnGrounder: Talking to Cars with Attention, ECCVW, 2020 | paper | code

Dataset(s): Talk2Car

@inproceedings{2020_ECCVW_Mittal,
    author = "Mittal, Vivek",
    booktitle = "ECCV",
    title = "Attngrounder: Talking to cars with attention",
    year = "2020"
}

Kim et al., Advisable Learning for Self-driving Vehicles by Internalizing Observation-to-Action Rules, CVPR, 2020 | paper | code

BDD-X

@inproceedings{2020_CVPR_Kim,
    author = "Kim, Jinkyu and Moon, Suhong and Rohrbach, Anna and Darrell, Trevor and Canny, John",
    booktitle = "CVPR",
    title = "Advisable learning for self-driving vehicles by internalizing observation-to-action rules",
    year = "2020"
}

Kim et al., Textual Explanations for Self-Driving Vehicles, ECCV, 2018 | paper | code

BDD-X

@inproceedings{2018_ECCV_Kim,
    author = "Kim, Jinkyu and Rohrbach, Anna and Darrell, Trevor and Canny, John and Akata, Zeynep",
    booktitle = "ECCV",
    title = "Textual explanations for self-driving vehicles",
    year = "2018"
}

Bojarski et al., Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car, arXiv, 2017 | paper | code

Dataset(s): private

@article{2017_arXiv_Bojarski,
    author = "Bojarski, Mariusz and Yeres, Philip and Choromanska, Anna and Choromanski, Krzysztof and Firner, Bernhard and Jackel, Lawrence and Muller, Urs",
    journal = "arXiv:1704.07911",
    title = "Explaining how a deep neural network trained with end-to-end learning steers a car",
    year = "2017"
}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

papers_with_code.md

papers_with_code.md

Home | Behavioral | Applications | Datasets

Action anticipation

Scene gaze

In-vehicle gaze

Distraction detection

Driver awareness

Self-driving

Files

papers_with_code.md

Latest commit

History

papers_with_code.md

File metadata and controls

Home | Behavioral | Applications | Datasets

Action anticipation

Scene gaze

In-vehicle gaze

Distraction detection

Driver awareness

Self-driving