Skip to content

Commit

Permalink
website
Browse files Browse the repository at this point in the history
  • Loading branch information
XindiWu committed Jan 9, 2025
1 parent 29b39e6 commit 96d66d7
Show file tree
Hide file tree
Showing 2 changed files with 48 additions and 8 deletions.
48 changes: 44 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,48 @@
# icons
# ICONS

ICONS: Influence Consensus for Vision-Language Data Selection
**ICONS: Influence Consensus for Vision-Language Data Selection**

Xindi Wu, Mengzhou Xia, Rulin Shao, Zhiwei Deng, Pang Wei Koh, Olga Russakovsky
[[paper](https://arxiv.org/abs/2501.00654)][[website](https://princetonvisualai.github.io/icons/)][[dataset](https://huggingface.co/datasets/xindiw/LLAVA-ICONS-133K)]

Authors: [Xindi Wu](https://xindiwu.github.io/), [Mengzhou Xia](https://xiamengzhou.github.io/), [Rulin Shao](https://rulinshao.github.io/), [Zhiwei Deng](https://lucas2012.github.io/), [Pang Wei Koh](https://koh.pw/), [Olga Russakovsky](https://www.cs.princeton.edu/~olgarus/)

We propose ICONS, a method for selecting vision-language data that optimizes training efficiency by identifying and prioritizing data samples that are consistently valuable across multiple tasks.

## News 🔥
- [01/25] We have released the LLAVA-ICONS-133K dataset on [Hugging Face](https://huggingface.co/datasets/xindiw/LLAVA-ICONS-133K) for public use.
- [12/24] We have released the paper [ICONS](https://arxiv.org/abs/2501.00654).

## Table of Contents

- [Installation](#installation)
- [Usage](#usage)
- [Configuration](#configuration)
- [Contributing](#contributing)
- [License](#license)

## Installation

To set up the environment for ICONS, you can use the provided `environment.yml` file to create a Conda environment.

```bash
conda env create -f environment.yml
conda activate icons
```

## Usage
Stage 1: Specialist

Stage 2: Generalist

## Citation
If you find this repository useful for your research, please cite with the following BibTeX entry:
```
@article{wu2024icons,
title={ICONS: Influence Consensus for Vision-Language Data Selection},
author={Wu, Xindi and Xia, Mengzhou and Shao, Rulin and Deng, Zhiwei and Koh, Pang Wei and Russakovsky, Olga},
journal={arXiv preprint arXiv:2501.00654},
year={2024}
}
```


Under construction 🚧
8 changes: 4 additions & 4 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -236,10 +236,9 @@ <h1 class="title">ICONS: Influence Consensus for Vision-Language Data Selection<
<a href="https://github.com/princetonvisualai/icons" class="button">
<i class="fab fa-github"></i> Code
</a>
<a href="https://huggingface.co/datasets/xdiwu/LLaVA-ICONS-133K" class="button">
<a href="https://huggingface.co/datasets/xindiw/LLAVA-ICONS-133K" class="button">
<span class="huggingface-small">🤗</span> Dataset
</a>
<p style="margin-top: 15px; color: #666;">Website under construction 🚧</p>
</div>
</div>
</div>
Expand Down Expand Up @@ -746,7 +745,7 @@ <h3>Ablation Studies</h3>
<!-- Different Aggregation Approaches -->
<div style="flex: 1; padding: 20px; background: #f8f9fa; border-radius: 10px; overflow: hidden;">
<h4>Different Aggregation Approaches</h4>
<p>To understand how different methods of combining task-specific influence scores affect the performance, we compare our majority voting approach (<strong>Vote</strong>) with four aggregation approaches: (1) <strong>Mean</strong>, (2) <strong>Max</strong>, (3) <strong>Rank</strong>, (4) <strong>Norm</strong>.</p>
<p>To understand how different methods of combining task-specific influence scores affect the performance, we compare our majority voting approach (<strong>Vote</strong>) with four aggregation methods: (1) <strong>Mean</strong>, (2) <strong>Max</strong>, (3) <strong>Rank</strong>, (4) <strong>Norm</strong>.</p>
<table style="width: 100%; border-collapse: collapse;">
<thead>
<tr style="background-color: #e9ecef;">
Expand Down Expand Up @@ -855,7 +854,8 @@ <h4>Different Aggregation Approaches</h4>
<!-- Consensus-aware Selection vs. Direct-merge Selection -->
<div style="flex: 1; padding: 20px; background: #f8f9fa; border-radius: 10px; overflow: hidden;">
<h4>Consensus-aware Selection vs. Direct-merge Selection</h4>
<p>To validate our choice of a consensus-aware approach versus a direct-merge approach, we compare the performance of both approaches.</p>
<p>To validate our choice of a consensus-aware approach versus a direct-merge approach (directly
computing influence using a combined validation set), we compare the performance of both approaches.</p>
<table style="width: 100%; border-collapse: collapse;">
<thead>
<tr style="background-color: #e9ecef;">
Expand Down

0 comments on commit 96d66d7

Please sign in to comment.