\begin{table}[htbp]
\vspace{-0.1in}
\caption{Statistics of datasets used in the graph classification task.}
\label{tab:graph-data-stat}
\vspace{1mm}
\small
\renewcommand\arraystretch{1}
\centering
\resizebox{0.6\textwidth}{!}{\begin{tabular*}{0.6\textwidth}{c| @{\extracolsep{\fill}} |c|c|c|c|c}
\toprule
\multirow{2}{*}{Dataset} & \multicolumn{2}{c|}{Class \#} &\multicolumn{3}{c}{Graph \#}  \\
\cmidrule{2-3} \cmidrule{4-6} &Train &Test & Training &Validation &Test  \\
\midrule
Letter-High       &11 &4 &1,330 &320 &600 \\ 
Triangles       &7 &3 &1,126 &271 &603 \\ 
Reddit-12K       &7 &4 &566 &141 &404 \\ 
Enzymes       &4 &2 &320 &80 &200 \\ 
\bottomrule
\end{tabular*}
}
\end{table}
\textbf{GFL Setting and Problem}. Let $\mathcal{C}_{base}$ and $\mathcal{C}_{novel}$ denote the base classes set and novel (new) classes set in training data $\mathcal{T}_{train}$ and testing data $\mathcal{T}_{test}$, respectively. 
Similar to the general meta-learning problem~\cite{finn2017model}, the purpose of graph few-shot learning (GFL) is to train a GNN encoder $f_\theta(\cdot)$ over $\mathcal{C}_{base}$, such that the trained GNN encoder can be quickly adapted to $\mathcal{C}_{novel}$ with few labels per class. Note that there is no overlapping between base classes and novel classes, i.e., $\mathcal{C}_{base} \cap \mathcal{C}_{novel} = \emptyset$. 
In $K$-shot setting, during the meta-training phase,
a batch of classes (tasks) is randomly sampled from $\mathcal{C}_{base}$, where $K$ labeled instances per class are sampled to form the support set $\mathcal{S}$ for model training and the remaining instances are taken as the query set $\mathcal{Q}$ for model evaluation. After sufficient training, the model is further transferred to the meta-testing phase to conduct $N$-way classification over $\mathcal{C}_{novel}$ ($N$ is the number of novel classes), where each class is only with $K$ labeled instances. GFL applies to different graph mining problems, depending on the class meaning. Each class corresponds to a node label for the node classification problem or corresponds to a graph label for the graph classification problem. In this work, we will study both node classification and graph classification problems under few-shot setting, which are formally defined as follows:

where $C = \frac{1}{2}\log (2\pi e)$. By taking Eqn.~\ref{eq:infoloss-7} into Eqn.~\ref{eq:infoloss-6}, 
we can optimize $\mathcal{L}(\bm{\sigma})$ with Adam optimizer and obtain the optimal $\bm{\sigma}$ for computing the overall discarded information $H(G|Z)$. Furthermore, we can explain GFL model's capability by comparing discarded information of different models. Ideally, we expect the GFL model to encode valid node (or graph) embeddings as much as possible and therefore discard information as small as possible. 

\subsection{Experimental Setup}
\label{sec:exp-set}
\textbf{Datasets}.
We use multiple graph datasets to conduct experiments: for the node classification task, we use ogbn-arxiv~\citep{hu2020open}, Tissue-PPI~\citep{hamilton2017inductive}, Fold-PPI~\citep{zitnik2017predicting}, Cora~\citep{sen2008collective}, and Citeseer~\citep{sen2008collective}; for the graph classification task, we use datasets in ~\citep{chauhan2020few}, i.e., Letter-High, Triangles, Reddit-12K, and Enzymes. The detailed information of datasets is illustrated in Appendix~\ref{app: dataset}. 
\\
\textbf{Baseline Methods}. 
We employ a variety of baseline methods for model comparison in two tasks. For few-shot node classification, we use node2vec~\citep{grover2016node2vec}, DeepWalk~\citep{perozzi2014deepwalk}, Meta-GNN~\citep{zhou2019meta}, FS-GIN~\citep{xu2018powerful}, FS-SGC~\citep{wu2019simplifying}, No-Finetune~\citep{triantafillou2019meta},
Finetune~\citep{triantafillou2019meta},
KNN~~\citep{triantafillou2019meta},  ProtoNet~\citep{snell2017prototypical}, MAML~\citep{finn2017model}, G-Meta~\citep{huang2020graph}, and TENT~\citep{wang2022task}. For few-shot graph classification, we utilize WL~\citep{shervashidze2011weisfeiler}, Graphlet~\citep{shervashidze2009efficient}, AWE~\citep{ivanov2018anonymous}, Graph2Vec~\citep{narayanan2017graph2vec}, Diffpool~\citep{lee2019self}, CapsGNN~\citep{xinyi2018capsule}, GIN~\citep{xu2018powerful}, GIN-KNN~\citep{xu2018powerful}, GSM-GCN~\citep{chauhan2020few}, GSM-GAT~\citep{chauhan2020few}, and AS-MAML~\citep{ma2020adaptive}. Details of baselines are illustrated in Appendix~\ref{app: implementation}. 
\\
\textbf{Experimental Settings}.
In CGFL, we use GCN~\citep{kipf2016semi} as the GNN backbone for the node classification task, including two-layer graph convolution and one-layer FC. The graph classification task adds an extra average pooling operation as a readout layer. 
In the pretraining phase, we pre-train GNN on the unlabeled graph dataset with contrastive learning. 
For graph data augmentation, the node drop rate is 15\%, the edge removing rate is 15\%; and the feature masking rate is 20\%. The mini-batch size is set to 2,048, and the learning rate is set to 0.05 with a decay factor = 0.9. Meanwhile, the $\tau$ in exponential moving average is 0.999. We consider both inductive and transductive settings. For the inductive setting (\textbf{CGFL-I}), we only use unlabeled data in the training set; for the transductive setting (\textbf{CGFL-T}), we use unlabeled data in both training data and testing data. Additionally, for pre-trained models without knowledge distillation elevation, we refer to them as \textbf{Teacher-I} and \textbf{Teacher-T} for two settings, respectively. 
In GFL phase, we employ MAML to fine-tune the model.
We implement CGFL by PyTorch and train it on NVIDIA V100 GPUs. 
The code is in the supplementary material. 

\vspace{-0.15in}
\subsection{Few-Shot Graph Classification}
\vspace{-0.1in}
\textbf{Overall Performance}.
The results of all models for 5/10-shot graph classification
are reported in Table~\ref{tab:graph-cls-1}. Similar to the findings obtained from Table~\ref{tab:node-cls-1}, according to this table: (1) CGFL outperforms all baseline models, showing its superiority for few-shot graph classification;
(2) The improvement of CGFL-I over baseline models ranges from 0.95\% to 38.36\% (5-shot) and from 0.8\% to 34.33\% (10-shot). Meanwhile, this value of CGFL-T ranges from 5.65\% to 46.21\% (5-shot) and from 2.18\% to 41.51\% (10-shot). This demonstrates the effect of contrastive pre-training and self-distillation in learning rich graph embedding from unlabeled data; (3) CGFL-T (or Teacher-T) outperforms CGFL-I (or Teacher-I), showing that unlabeled data in testing set improves model's generalization; {(4) CGFL is better than teacher model as contrastive distillation step further elevates the model.}

\textbf{Impact of Shot Number}.
In Figure~\ref{fig:graph-shot}, we report our model's result under different shot numbers (5, 10, 15, 20) compared with some selected baselines. Similar to Figure~\ref{fig:node-shot}, CGFL consistently outperforms baseline methods across different shot numbers, showing the robustness of CGFL. Note that we only show results on two datasets (e.g., Letter-High and Triangles), and the results on the other datasets are shown in Appendix~\ref{sec:app:fsg}.
The same goes for the following analyses.  
\\
\vspace{-0.1in}
\\
\textbf{Impact of Training Label Rate}.
In Figure~\ref{fig:graph-lr}, we report our model's performance under different training label rates compared with baseline models for 5-shot graph classification. The phenomenon is similar to the node task: 
our model achieves better accuracy across different label rates; the performance gaps between CGFL and baseline methods are larger when the label rate is lower. Again, this shows the significance of CGFL when labeled data is limited.  

\vspace{-0.1in}
\section{Conclusion}
\vspace{-0.1in}
\label{sec:conclusion}
In this paper, to tackle the limitations of existing GFL models, we propose a general and effective framework named CGFL - \textbf{C}ontrastive \textbf{G}raph \textbf{F}ew-shot \textbf{L}earning. CGFL leverages a self-distilled contrastive learning procedure to boost GFL, in which the GNN encoder is pre-trained with contrastive learning and further elevated with knowledge distillation in a self-supervised manner. Additionally, we introduce an information-based method to compute the amount of graph information discarded by the GFL model. Extensive experiments on multiple 
graph datasets demonstrate that CGFL outperforms state-of-the-art baseline methods for both node classification and graph classification tasks in the few-shot scenario. The discarded information value further shows the superiority of CGFL in learning node (or graph) embeddings. 

\clearpage
\appendix
\maketitle
\section{Dataset Details}
\label{app: dataset}
For few-shot node classification, we use five different datasets: ogbn-arxiv~\cite{hu2020open}, Tissue-PPI~\cite{hamilton2017inductive}, Fold-PPI~\cite{zitnik2017predicting}, Cora~\cite{sen2008collective}, and Citeseer~\cite{sen2008collective} to perform extensive empirical evaluations of different models. These datasets  vary from citation network to biochemical graph. The statistics of datasets are reported in Table~\ref{tab:node-data-stat}. 

For few-shot graph classification, we use four different datasets~\cite{chauhan2020few}: Reddit-12K, ENZYMES, Letter-High, and TRIANGLES, to perform extensive empirical evaluations of different models. These datasets vary from small average graph size (e.g., Letter-High) to large graph size (e.g., Reddit-12K). 
The statistics of datasets are reported in Table~\ref{tab:graph-data-stat}. 
{"value": "11", "type": "Data Stat.", "dataset": "Letter-High", "attribute name": ["number of Class", "Class #"], "sub-set/group name": "Train", "dataset features": {"xx": "yy"}} {"value": "4", "type": "Data Stat.", "dataset": "Letter-High", "attribute name": ["number of Class", "Class #"], "sub-set/group name": "Test", "dataset features": {"xx": "yy"}} {"value": "1,330", "type": "Data Stat.", "dataset": "Letter-High", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Training", "dataset features": {"xx": "yy"}} {"value": "320", "type": "Data Stat.", "dataset": "Letter-High", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Validation", "dataset features": {"xx": "yy"}} {"value": "600", "type": "Data Stat.", "dataset": "Letter-High", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Test", "dataset features": {"xx": "yy"}} {"value": "7", "type": "Data Stat.", "dataset": "Triangles", "attribute name": ["number of Class", "Class #"], "sub-set/group name": "Train", "dataset features": {"xx": "yy"}} {"value": "3", "type": "Data Stat.", "dataset": "Triangles", "attribute name": ["number of Class", "Class #"], "sub-set/group name": "Test", "dataset features": {"xx": "yy"}} {"value": "1,126", "type": "Data Stat.", "dataset": "Triangles", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Training", "dataset features": {"xx": "yy"}} {"value": "271", "type": "Data Stat.", "dataset": "Triangles", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Validation", "dataset features": {"xx": "yy"}} {"value": "603", "type": "Data Stat.", "dataset": "Triangles", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Test", "dataset features": {"xx": "yy"}} {"value": "7", "type": "Data Stat.", "dataset": "Reddit-12K", "attribute name": ["number of Class", "Class #"], "sub-set/group name": "Train", "dataset features": {"xx": "yy"}} {"value": "4", "type": "Data Stat.", "dataset": "Reddit-12K", "attribute name": ["number of Class", "Class #"], "sub-set/group name": "Test", "dataset features": {"xx": "yy"}} {"value": "566", "type": "Data Stat.", "dataset": "Reddit-12K", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Training", "dataset features": {"xx": "yy"}} {"value": "141", "type": "Data Stat.", "dataset": "Reddit-12K", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Validation", "dataset features": {"xx": "yy"}} {"value": "404", "type": "Data Stat.", "dataset": "Reddit-12K", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Test", "dataset features": {"xx": "yy"}} {"value": "4", "type": "Data Stat.", "dataset": "Enzymes", "attribute name": ["number of Class", "Class #"], "sub-set/group name": "Train", "dataset features": {"xx": "yy"}} {"value": "2", "type": "Data Stat.", "dataset": "Enzymes", "attribute name": ["number of Class", "Class #"], "sub-set/group name": "Test", "dataset features": {"xx": "yy"}} {"value": "320", "type": "Data Stat.", "dataset": "Enzymes", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Training", "dataset features": {"xx": "yy"}} {"value": "80", "type": "Data Stat.", "dataset": "Enzymes", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Validation", "dataset features": {"xx": "yy"}} {"value": "200", "type": "Data Stat.", "dataset": "Enzymes", "attribute name": ["number of Graph", "Graph #"], "sub-set/group name": "Test", "dataset features": {"xx": "yy"}}