Skip to content

Latest commit

 

History

History
105 lines (93 loc) · 16.3 KB

README.md

File metadata and controls

105 lines (93 loc) · 16.3 KB

Awesome-Pretraining-for-Graph-Neural-Networks

PRs Welcome Stars

This repository contains a curated list of papers on (or related to) pre-training for graph neural networks (Pre-train4GNN), which are categorized based on their published years, graph types, pre-training strategies, tuning strategies, and applications.

Continuously updating!

Papers

Paper Title Venue Graph Type Pre-training Strategy Tuning Strategy Application Paper Link Code Link
Beyond Redundancy: Information-aware Unsupervised Multiplex Graph Structure Learning NeurIPS 2024 Multiplex Contrastive N/A General [PDF] [Code]
GraphPro: Graph Pre-training and Prompt Learning for Recommendation WWW 2024 Dynamic Generative Prompt Recommendation [PDF] [Code]
Empowering Dual-Level Graph Self-Supervised Pretraining with Motif Discovery AAAI 2024 Static MTL N/A Biology [PDF] [Code]
Unified Pretraining for Recommendation via Task Hypergraphs WSDM 2024 Hypergraph Generative N/A Recommendation [PDF] [Code]
GraphFM: A Scalable Framework for Multi-Graph Pretraining Arxiv 2024 Multi-graphs MTL N/A General [PDF] N/A
Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training CIKM 2023 KG MTL N/A Recommendation [PDF] N/A
Self-Contrastive Graph Diffusion Network MM 2023 Static Contrastive N/A General [PDF] N/A
All in One: Multi-Task Prompting for Graph Neural Networks KDD 2023 Static Generative Prompt General [PDF] [Code]
Automated 3D Pre-Training for Molecular Property Prediction KDD 2023 Static Generative N/A Chemistry&Biology [PDF] N/A
What’s Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders KDD 2023 Static Generative N/A General [PDF] [Code]
When to Pre-Train Graph Neural Networks? From Data Generation Perspective! KDD 2023 Static Generative (Quantification) N/A General [PDF] [Code]
GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks WWW 2023 Static Contrastive Prompt General [PDF] [Code]
GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner WWW 2023 Static Generative N/A General [PDF] [Code]
Structure Pretraining and Prompt Tuning for Knowledge Graph Transfer WWW 2023 KG Generative Prompt General [PDF] [Code]
Protein Representation Learning by Geometric Structure Pretraining ICLR 2023 Static Contrastive N/A Biology [PDF] [Code]
Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting SIGIR 2023 Text Graph Contrastive Prompt Text Classification [PDF] [Code]
Self-Supervised Graph Structure Refinement for Graph Neural Networks WSDM 2023 Static Contrastive GSL General [PDF] [Code]
Multi-task Item-attribute Graph Pre-training for Strict Cold-start Item Recommendation RecSys 2023 Static MTL N/A Recommendation [PDF] [Code]
MVRACE: Multi-view Graph Contrastive Encoding for Graph Neural Network Pre-training CogSci 2023 Static Contrastive N/A General [PDF] N/A
Curriculum Pre-training Heterogeneous Subgraph Transformer for Top-N Recommendation TOIS 2023 Heterogeneous Curriculum N/A Recommendation [PDF] N/A
GPPT: Graph Pre-training and Prompt Tuning to Generalize Graph Neural Networks KDD 2022 Static Generative Prompt General [PDF] [Code]
GraphMAE: Self-Supervised Masked Graph Autoencoders KDD 2022 Static Generative N/A General [PDF] [Code]
Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries KDD 2022 KG Generative Generative General [PDF] [Code]
KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Prediction KDD 2022 GT Generative N/A Biology [PDF] [Code]
Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting KDD 2022 ST-Graph Generative N/A Time Series Forecasting [PDF] [Code]
Self-supervised Heterogeneous Graph Pre-training Based on Structural Clustering NIPS 2022 Heterogeneous Clustering N/A General [PDF] [Code]
Does GNN Pretraining Help Molecular Representation? NIPS 2022 Molecular Evaluation N/A General [PDF] N/A
Pre-training Molecular Graph Representation with 3D Geometry ICLR 2022 Static Generative+Contrastive N/A Biology [PDF] [Code]
Graph Pre-training for AMR Parsing and Generation ACL 2022 Semantic Graph Generative N/A NLP (AMR) [PDF] [Code]
Neural Graph Matching for Pre-training Graph Neural Networks SDM 2022 Static Contrastive N/A General [PDF] [Code]
Pre-train Graph Neural Networks for Brain Network Analysis Big Data 2022 Static Contrastive N/A Biology [PDF] N/A
GCCAD: Graph Contrastive Learning for Anomaly Detection TKDE 2022 Static Contrastive N/A Anomaly Detection [PDF] [Code]
An Adaptive Graph Pre-training Framework for Localized Collaborative Filtering TOIS 2022 Static Contrastive Meta-Learning Recommendation [PDF] N/A
Pre-training Graph Neural Networks for Link Prediction in Biomedical Networks Bioinformatics 2022 Static Generative N/A Biology [PDF] [Code]
Pre-Training of Equivariant Graph Matching Networks with Conformation Flexibility for Drug Binding Advanced Science 2022 Dynamic Generative N/A Biology [PDF] [Code]
Pre-training on Dynamic Graph Neural Networks Neurocomputing 2022 Dynamic Generative N/A General [PDF] [Code]
Adaptive Transfer Learning on Graph Neural Networks KDD 2021 Static N/A Meta Learning General [PDF] [Code]
Pre-training on Large-Scale Heterogeneous Graph KDD 2021 Heterogeneous Contrastive N/A General [PDF] [Code]
Self-supervised Graph-level Representation Learning with Local and Global Structure ICML 2021 Static Contrastive N/A Biology [PDF] [Code]
Pairwise Half-graph Discrimination: A Simple Graph-level Self-supervised Strategy for Pre-training Graph Neural Networks IJCAI 2021 Static Decomposition N/A General [PDF] N/A
Contrastive Pre-Training of GNNs on Heterogeneous Graphs CIKM 2021 Heterogeneous Contrastive N/A General [PDF] [Code]
Pre-Training Graph Neural Networks for Cold-Start Users and Items Representation WSDM 2021 Static Generative+RL N/A Recommendation [PDF] [Code]
Pre-training Graph Transformer with Multimodal Side Information for Recommendation MM 2021 Static Generative N/A Recommendation [PDF] [Code]
Learning to Pre-train Graph Neural Networks AAAI 2021 Static Contrastive N/A General [PDF] [Code]
GPT-GNN: Generative Pre-Training of Graph Neural Networks KDD 2020 Static Generative N/A General [PDF] [Code]
GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training KDD 2020 Static Contrastive N/A General [PDF] [Code]
Strategies for Pre-training Graph Neural Networks ICLR 2020 Static Generative N/A Chemistry&Biology [PDF] [Code]
Pre-training of Graph Augmented Transformers for Medication Recommendation IJCAI 2019 Ontology Tree Generative N/A Recommendation [PDF] [Code]

Pre-print Papers

Paper Title Venue Graph Type Pre-training Strategy Tuning Strategy Application PDF Link Code Link
GraphGPT: Graph Learning with Generative Pre-trained Transformers Arxiv 2023 Static Generative N/A General [PDF] [Code]
GraphGPT: Graph Instruction Tuning for Large Language Models Arxiv 2023 Static N/A Prompt General [PDF] [Code]
AdapterGNN: Efficient Delta Tuning Improves Generalization Ability in Graph Neural Networks Arxiv 2023 Static N/A Adapter General [PDF] N/A
Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Tasks Arxiv 2023 Static N/A AutoML General [PDF] N/A
GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning Arxiv 2023 Static Contrastive Prompt General [PDF] N/A
CPDG: A Contrastive Pre-Training Method for Dynamic Graph Neural Networks Arxiv 2023 Dynamic Contrastive Enhanced Embedding General [PDF] N/A
Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training Arxiv 2023 KG MTL N/A Recommendation [PDF] N/A
SGL-PT: A Strong Graph Learner with Graph Prompt Tuning Arxiv 2023 Static Generative Prompt General [PDF] N/A
Similarity-aware Positive Instance Sampling for Graph Contrastive Pre-training Arxiv 2022 Static Contrastive N/A General [PDF] N/A
DiP-GNN: Discriminative Pre-Training of Graph Neural Networks Arxiv 2022 Static Generative N/A General [PDF] N/A
Hypergraph Pre-training with Graph Neural Networks Arxiv 2021 Hypergraph Contrastive N/A General [PDF] N/A
GRAPH-BERT: Only Attention is Needed for Learning Graph Representations Arxiv 2020 Static Generative N/A General [PDF] [Code]
Pre-training Graph Neural Networks with Kernels Arxiv 2018 Static Kernel N/A General [PDF] N/A

Survey Papers

Paper Title Venue PDF Link Code Link
Graph Prompt Learning: A Comprehensive Survey and Beyond Arxiv 2023 [PDF] [Code]
Towards Graph Foundation Models: A Survey and Beyond Arxiv 2023 [PDF] N/A
Generative Diffusion Models on Graphs: Methods and Applications IJCAI 2023 [PDF] N/A
Graph Meets LLMs: Towards Large Graph Models Arxiv 2023 [PDF] [Code]
A Survey of Graph Prompting Methods: Techniques, Applications, and Challenges Arxiv 2023 [PDF] N/A
A Survey of Pre-training on Graphs: Taxonomy, Methods and Applications Arxiv 2022 [PDF] [Code]
Self-Supervised Learning of Graph Neural Networks: A Unified Review TPAMI 2022 [PDF] N/A
Graph Self-Supervised Learning: A Survey TKDE 2022 [PDF] N/A
Self-supervised Learning on Graphs: Contrastive, Generative, or Predictive TKDE 2021 [PDF] N/A
Self-supervised Learning on Graphs: Deep Insights and New Directions Arxiv 2020 [PDF] [Code]

Tutorials

Tutorial Title Venue PDF Link
Self-supervised Learning and Pre-training on Graphs WWW 2023 [PDF]