Overview

• Citation
• Survey
• Image Classification / Backbone
  • Replace Conv w/ Attention
    • Pure Attention
    • Conv-stem + Attention
    • Conv + Attention
  • Vision Transformer
    • General Vision Transformer
    • Efficient Vision Transformer
    • Conv + Transformer
    • Training + Transformer
    • Robustness + Transformer
    • Model Compression + Transformer
  • Attention-Free
    • MLP-Series
    • Other Attention-Free
  • Analysis for Transformer
• Detection
  • Object Detection
  • 3D Object Detection
  • Multi-Modal Detection
  • HOI Detection
  • Salient Object Detection
  • Other Detection Tasks
• Segmentation
  • Semantic Segmentation
  • Depth Estimation
  • Object Segmentation
  • Other Segmentation Tasks
• Video (High-level)
  • Action Recognition
  • Action Detection/Localization
  • Action Prediction/Anticipation
  • Video Object Segmentation
  • Video Instance Segmentation
  • Other Video Tasks
• References

------ (The following papers are moved to README_multimodal.md) ------

• Multi-Modality
  • Visual Captioning
  • Visual Question Answering
  • Visual Grounding
  • Multi-Modal Representation Learning
  • Multi-Modal Retrieval
  • Multi-Modal Generation
  • Prompt Learning/Tuning
  • Visual Document Understanding
  • Other Multi-Modal Tasks

------ (The following papers are moved to README_2.md) ------

• Other High-level Vision Tasks
  • Point Cloud / 3D
  • Pose Estimation
  • Tracking
  • Re-ID
  • Face
  • Scene Graph
  • Neural Architecture Search
• Transfer / X-Supervised / X-Shot / Continual Learning
• Low-level Vision Tasks
  • Image Restoration
  • Video Restoration
  • Inpainting / Completion / Outpainting
  • Image Generation
  • Video Generation
  • Transfer / Translation / Manipulation
  • Other Low-Level Tasks
• Reinforcement Learning
  • Navigation
  • Other RL Tasks
• Medical
  • Medical Segmentation
  • Medical Classification
  • Medical Detection
  • Medical Reconstruction
  • Medical Low-Level Vision
  • Medical Vision-Language
  • Medical Others
• Other Tasks
• Attention Mechanisms in Vision/NLP
  • Attention for Vision
  • NLP
  • Both
  • Others

Citation

If you find this repository useful, please consider citing this list:

@misc{chen2022transformerpaperlist,
    title = {Ultimate awesome paper list: transformer and attention},
    author = {Chen, Min-Hung},
    journal = {GitHub repository},
    url = {https://github.com/cmhungsteve/Awesome-Transformer-Attention},
    year = {2022},
}

Survey

• "A Survey on Multimodal Large Language Models for Autonomous Driving", WACVW, 2024 (Purdue). [Paper][GitHub]
• "Efficient Multimodal Large Language Models: A Survey", arXiv, 2024 (Tencent). [Paper][GitHub]
• "From Sora What We Can See: A Survey of Text-to-Video Generation", arXiv, 2024 (Newcastle University, UK). [Paper][GitHub]
• "When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models", arXiv, 2024 (Oxford). [Paper][GitHub]
• "Foundation Models for Video Understanding: A Survey", arXiv, 2024 (Aalborg University, Denmark). [Paper][GitHub]
• "Vision Mamba: A Comprehensive Survey and Taxonomy", arXiv, 2024 (Chongqing University). [Paper][GitHub]
• "Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond", arXiv, 2024 (GigaAI, China). [Paper][GitHub]
• "Video Diffusion Models: A Survey", arXiv, 2024 (Bielefeld University, Germany). [Paper][GitHub]
• "Unleashing the Power of Multi-Task Learning: A Comprehensive Survey Spanning Traditional, Deep, and Pretrained Foundation Model Eras", arXiv, 2024 (Lehigh + UPenn). [Paper]
• "Hallucination of Multimodal Large Language Models: A Survey", arXiv, 2024 (NUS). [Paper][GitHub]
• "A Survey on Vision Mamba: Models, Applications and Challenges", arXiv, 2024 (HKUST). [Paper][GitHub]
• "State Space Model for New-Generation Network Alternative to Transformers: A Survey", arXiv, 2024 (Anhui University). [Paper][GitHub]
• "Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions", arXiv, 2024 (IIT Patna). [Paper]
• "From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Models", arXiv, 2024 (UIUC). [Paper][GitHub]
• "Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey", arXiv, 2024 (Northeastern). [Paper]
• "Sora as an AGI World Model? A Complete Survey on Text-to-Video Generation", arXiv, 2024 (Kyung Hee University). [Paper]
• "Controllable Generation with Text-to-Image Diffusion Models: A Survey", arXiv, 2024 (Beijing University of Posts and Telecommunications). [Paper][GitHub]
• "Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models", arXiv, 2024 (Lehigh University, Pennsylvania). [Paper][GitHub]
• "Large Multimodal Agents: A Survey", arXiv, 2024 (CUHK). [Paper][GitHub]
• "Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey", arXiv, 2024 (BIGAI). [Paper][GitHub]
• "Vision-Language Navigation with Embodied Intelligence: A Survey", arXiv, 2024 (Qufu Normal University, China). [Paper]
• "The (R)Evolution of Multimodal Large Language Models: A Survey",