Repositories list

• InternVL

  Public
  [CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型

  image-classificationgptmulti-modal+ 9semantic-segmentationvideo-classificationimage-text-retrievalllmvision-language-modelgpt-4vvit-6b+ 2

  Python

  •

  MIT License

  •465•6k•136•5•Updated Nov 15, 2024Nov 15, 2024

• GUI-Odyssey

  Public
  GUI Odyssey is a comprehensive dataset for training and evaluating cross-app navigation agents. GUI Odyssey consists of 7,735 episodes from 6 mobile devices, spanning 6 types of cross-app tasks, 201 apps, and 1.4K app combos.

  Python

  •3•66•2•0•Updated Nov 12, 2024Nov 12, 2024

• Vision-RWKV

  Public
  Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures

  Python

  •

  Apache License 2.0

  •14•369•12•0•Updated Oct 31, 2024Oct 31, 2024

• .github

  Public
  1•0•0•0•Updated Oct 30, 2024Oct 30, 2024

• PIIP

  Public
  [NeurIPS 2024 Spotlight ⭐️] Parameter-Inverted Image Pyramid Networks (PIIP)

  computer-visionimage-classificationobject-detection+ 3semantic-segmentationinstance-segmentationvision-transformer

  Python

  •

  MIT License

  •2•57•0•0•Updated Oct 28, 2024Oct 28, 2024

• OV-OAD

  Public
  This repo takes the initial step towards leveraging text learning for online action detection without explicit human supervision.

  1•1•0•0•Updated Oct 28, 2024Oct 28, 2024

• OmniCorpus

  Public
  OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text

  Python

  •6•271•0•0•Updated Oct 25, 2024Oct 25, 2024

• InternVL-MMDetSeg

  Public
  Train InternViT-6B in MMSegmentation and MMDetection with DeepSpeed

  object-detectionsemantic-segmentationvision-foundation

  Jupyter Notebook

  •4•55•1•0•Updated Oct 25, 2024Oct 25, 2024

• PhyGenBench

  Public
  The code and data of Paper: Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation

  Python

  •1•69•3•0•Updated Oct 25, 2024Oct 25, 2024

• MM-NIAH

  Public
  [NeurIPS 2024] Needle In A Multimodal Haystack (MM-NIAH): A comprehensive benchmark designed to systematically evaluate the capability of existing MLLMs to comprehend long multimodal documents.

  benchmarklong-contextvision-language-model+ 1multimodal-large-language-models

  Python

  •6•100•1•0•Updated Oct 22, 2024Oct 22, 2024

• VisionLLM

  Public
  VisionLLM Series

  object-detectionlarge-language-modelsgeneralist-model

  Python

  •

  Apache License 2.0

  •26•915•12•0•Updated Oct 18, 2024Oct 18, 2024

• VideoMAEv2

  Public
  [CVPR 2023] VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking

  video-understandingaction-detectionself-supervised-learning+ 4temporal-action-detectionfoundation-modelcvpr2023action-recognition

  Python

  •

  MIT License

  •61•519•12•0•Updated Oct 8, 2024Oct 8, 2024

• EfficientQAT

  Public
  EfficientQAT: Efficient Quantization-Aware Training for Large Language Models

  Python

  •17•224•3•0•Updated Oct 8, 2024Oct 8, 2024

• OmniQuant

  Public
  [ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

  quantizationlarge-language-modelsllm

  Python

  •

  MIT License

  •56•729•24•1•Updated Oct 8, 2024Oct 8, 2024

• STM-Evaluation

  Public
  Python

  •

  MIT License

  •6•69•1•0•Updated Oct 6, 2024Oct 6, 2024

• InternVideo

  Public
  [ECCV2024] Video Foundation Models & Data for Multimodal Understanding

  benchmarkaction-recognitionvideo-understanding+ 17video-dataself-supervisedmultimodalvideo-datasetopen-set-recognitionvideo-retrievalvideo-question-answering+ 10

  Python

  •

  Apache License 2.0

  •85•1.4k•87•6•Updated Sep 23, 2024Sep 23, 2024

• MMIU

  Public
  MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

  Python

  •2•53•2•0•Updated Sep 14, 2024Sep 14, 2024

• ChartAst

  Public
  [ACL 2024] ChartAssistant is a chart-based vision-language model for universal chart comprehension and reasoning.

  Python

  •

  Other

  •8•107•6•0•Updated Sep 7, 2024Sep 7, 2024

• Ask-Anything

  Public
  [CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

  chatvideogradio+ 10big-modelvideo-understandingcaptioning-videosvideo-question-answeringfoundation-modelslarge-modellarge-language-models+ 3

  Python

  •

  MIT License

  •252•3.1k•58•5•Updated Sep 5, 2024Sep 5, 2024

• EgoExoLearn

  Public
  [CVPR 2024] Data and benchmark code for the EgoExoLearn dataset

  Python

  •

  MIT License

  •0•47•1•0•Updated Sep 3, 2024Sep 3, 2024

• InternGPT

  Public
  InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)

  samclickvqa+ 17image-captioningllamagptgradiohuskymultimodalvideo-generation+ 10

  Python

  •

  Apache License 2.0

  •232•3.2k•17•1•Updated Aug 20, 2024Aug 20, 2024

• all-seeing

  Public
  [ICLR 2024 & ECCV 2024] The All-Seeing Projects: Towards Panoptic Visual Recognition&Understanding and General Relation Comprehension of the Open World"

  datasetall-seeingregion-text

  Python

  •15•459•10•0•Updated Aug 9, 2024Aug 9, 2024

• Diffree

  Public
  Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model

  Python

  •

  Apache License 2.0

  •13•234•2•0•Updated Aug 6, 2024Aug 6, 2024

• InternImage

  Public
  [CVPR 2023 Highlight] InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions

  backbonesemantic-segmentationdeformable-convolution+ 2foundation-modelobject-detection

  Python

  •

  MIT License

  •234•2.5k•183•8•Updated Aug 1, 2024Aug 1, 2024

• MMT-Bench

  Public
  ICML'2024 | MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

  Python

  •3•95•5•0•Updated Jul 18, 2024Jul 18, 2024

• ControlLLM

  Public
  ControlLLM: Augment Language Models with Tools by Searching on Graphs

  Python

  •8•186•6•0•Updated Jul 15, 2024Jul 15, 2024

• HumanBench

  Public
  This repo is official implementation of HumanBench (CVPR2023)

  Python

  •

  MIT License

  •11•232•15•1•Updated Jul 8, 2024Jul 8, 2024

• VideoMamba

  Public
  [ECCV2024] VideoMamba: State Space Model for Efficient Video Understanding

  Python

  •

  Apache License 2.0

  •60•839•42•0•Updated Jul 6, 2024Jul 6, 2024

• LORIS

  Public
  Long-Term Rhythmic Video Soundtracker, ICML2023

  music-generationpytorch-implementationmulti-modality+ 2diffusion-modelsaigc

  Python

  •

  MIT License

  •1•54•1•0•Updated Jul 5, 2024Jul 5, 2024

• EgoVideo

  Public
  [CVPR 2024 Champions] Solutions for EgoVis Chanllenges in CVPR 2024

  Jupyter Notebook

  •3•107•5•0•Updated Jul 5, 2024Jul 5, 2024