\begin{table}[!htb]
\caption{{Discarded information for graph classification.}}
\vspace{1mm}
\small
\centering
\resizebox{0.5\textwidth}{!}{\begin{tabular*}{0.5\textwidth}{c|c|c|c}
\toprule
\multirow{2}{*}{Method} & \multirow{2}{*}{Layer} &\multicolumn{2}{c}{Discarded Information}  \\ 
\cmidrule{3-4} & &Reddit-12K &Enzymes \\
\midrule
       &GNN-1   &5455.46 &2455.54 \\
AS-MAML &GNN-2 &5024.15  &2243.25 \\
       &FC     &4937.99 &2125.91\\
       \midrule
       &GNN-1  &5401.30  &2273.80 \\ 
GSM-GAT &GNN-2 &5183.87  &2128.04 \\ 
       &FC     &4936.76  &1958.18\\ 
\midrule
                &GNN-1  &5323.04  &1864.98 \\ 
CGFL-I         &GNN-2  &5098.54  &1788.97 \\ 
                &FC     &4802.03  &1759.87 \\  
\midrule
                 &GNN-1 &\textbf{4823.91}  &\textbf{1787.98} \\ 
CGFL-T          &GNN-2 &\textbf{4546.23}  &\textbf{1698.35} \\ 
                   &FC  &\textbf{4329.39}  &\textbf{1585.33} \\ 
\bottomrule
\end{tabular*}
}
\label{tab:info-loss-graph-cls-app}
\end{table}
\begin{equation}
    \mathcal{L}_{\theta,\xi} = {\lVert {z_{\theta} - h_{\xi}}\rVert}_{2}^{2}=2-2\cdot \frac{{z_{\theta}, h_{\xi}}}{{\lVert {z_{\theta} }\rVert}_{2} \cdot {\lVert {h_{\xi}}\rVert}_{2}}.
\label{eq:cosine-loss}
\end{equation}

where $C = \frac{1}{2}\log (2\pi e)$. By taking Eqn.~\ref{eq:infoloss-7} into Eqn.~\ref{eq:infoloss-6}, 
we can optimize $\mathcal{L}(\bm{\sigma})$ with Adam optimizer and obtain the optimal $\bm{\sigma}$ for computing the overall discarded information $H(G|Z)$. Furthermore, we can explain GFL model's capability by comparing discarded information of different models. Ideally, we expect the GFL model to encode valid node (or graph) embeddings as much as possible and therefore discard information as small as possible. 

\subsection{Experimental Setup}
\label{sec:exp-set}
\textbf{Datasets}.
We use multiple graph datasets to conduct experiments: for the node classification task, we use ogbn-arxiv~\citep{hu2020open}, Tissue-PPI~\citep{hamilton2017inductive}, Fold-PPI~\citep{zitnik2017predicting}, Cora~\citep{sen2008collective}, and Citeseer~\citep{sen2008collective}; for the graph classification task, we use datasets in ~\citep{chauhan2020few}, i.e., Letter-High, Triangles, Reddit-12K, and Enzymes. The detailed information of datasets is illustrated in Appendix~\ref{app: dataset}. 
\\
\textbf{Baseline Methods}. 
We employ a variety of baseline methods for model comparison in two tasks. For few-shot node classification, we use node2vec~\citep{grover2016node2vec}, DeepWalk~\citep{perozzi2014deepwalk}, Meta-GNN~\citep{zhou2019meta}, FS-GIN~\citep{xu2018powerful}, FS-SGC~\citep{wu2019simplifying}, No-Finetune~\citep{triantafillou2019meta},
Finetune~\citep{triantafillou2019meta},
KNN~~\citep{triantafillou2019meta},  ProtoNet~\citep{snell2017prototypical}, MAML~\citep{finn2017model}, G-Meta~\citep{huang2020graph}, and TENT~\citep{wang2022task}. For few-shot graph classification, we utilize WL~\citep{shervashidze2011weisfeiler}, Graphlet~\citep{shervashidze2009efficient}, AWE~\citep{ivanov2018anonymous}, Graph2Vec~\citep{narayanan2017graph2vec}, Diffpool~\citep{lee2019self}, CapsGNN~\citep{xinyi2018capsule}, GIN~\citep{xu2018powerful}, GIN-KNN~\citep{xu2018powerful}, GSM-GCN~\citep{chauhan2020few}, GSM-GAT~\citep{chauhan2020few}, and AS-MAML~\citep{ma2020adaptive}. Details of baselines are illustrated in Appendix~\ref{app: implementation}. 
\\
\textbf{Experimental Settings}.
In CGFL, we use GCN~\citep{kipf2016semi} as the GNN backbone for the node classification task, including two-layer graph convolution and one-layer FC. The graph classification task adds an extra average pooling operation as a readout layer. 
In the pretraining phase, we pre-train GNN on the unlabeled graph dataset with contrastive learning. 
For graph data augmentation, the node drop rate is 15\%, the edge removing rate is 15\%; and the feature masking rate is 20\%. The mini-batch size is set to 2,048, and the learning rate is set to 0.05 with a decay factor = 0.9. Meanwhile, the $\tau$ in exponential moving average is 0.999. We consider both inductive and transductive settings. For the inductive setting (\textbf{CGFL-I}), we only use unlabeled data in the training set; for the transductive setting (\textbf{CGFL-T}), we use unlabeled data in both training data and testing data. Additionally, for pre-trained models without knowledge distillation elevation, we refer to them as \textbf{Teacher-I} and \textbf{Teacher-T} for two settings, respectively. 
In GFL phase, we employ MAML to fine-tune the model.
We implement CGFL by PyTorch and train it on NVIDIA V100 GPUs. 
The code is in the supplementary material. 

\textbf{Loss Information Comparison}.
In Section~\ref{sec:inforamtion-loss}, we propose to compute graph information discarded in the GFL model. Here, we compare results between CGFL and a selected baseline method (G-Meta) in Table~\ref{tab:info-loss-node-cls}. From this table, the amount of discarded information in each layer of CGFL is smaller than baseline models. This may be because CGFL can learn more label-irrelevant information from unlabeled graph data, which somehow shows the superiority of CGFL in learning node embeddings for node classification.  

\textbf{Loss Information Comparison}.
Finally, we compute the discarded graph information of our model and a selected baseline method (GSM-GAT), as shown in Table~\ref{tab:info-loss-graph-cls}.
Obviously, the amount of information discarded in CGFL is less than baseline models. It somehow shows the superiority of CGFL in learning graph embeddings for graph classification.

\vspace{-0.1in}
\section{Conclusion}
\vspace{-0.1in}
\label{sec:conclusion}
In this paper, to tackle the limitations of existing GFL models, we propose a general and effective framework named CGFL - \textbf{C}ontrastive \textbf{G}raph \textbf{F}ew-shot \textbf{L}earning. CGFL leverages a self-distilled contrastive learning procedure to boost GFL, in which the GNN encoder is pre-trained with contrastive learning and further elevated with knowledge distillation in a self-supervised manner. Additionally, we introduce an information-based method to compute the amount of graph information discarded by the GFL model. Extensive experiments on multiple 
graph datasets demonstrate that CGFL outperforms state-of-the-art baseline methods for both node classification and graph classification tasks in the few-shot scenario. The discarded information value further shows the superiority of CGFL in learning node (or graph) embeddings. 

\textbf{Discarded Information Comparison}. The discarded graph information of different models are shown in Table~\ref{tab:info-loss-node-cls-app}.
Obviously, the amount of information discarded in CGFL is less than baseline models. 

\textbf{Discarded Information Comparison}. The discarded graph information of different methods are reported in Table~\ref{tab:info-loss-graph-cls-app}.
According to this table, the amount of information discarded in CGFL is less than baseline models. 
{"value": "5455.46", "type": "Other"} {"value": "2455.54", "type": "Other"} {"value": "5024.15", "type": "Other"} {"value": "2243.25", "type": "Other"} {"value": "4937.99", "type": "Other"} {"value": "2125.91", "type": "Other"} {"value": "5401.30", "type": "Other"} {"value": "2273.80", "type": "Other"} {"value": "5183.87", "type": "Other"} {"value": "2128.04", "type": "Other"} {"value": "4936.76", "type": "Other"} {"value": "1958.18", "type": "Other"} {"value": "5323.04", "type": "Other"} {"value": "1864.98", "type": "Other"} {"value": "5098.54", "type": "Other"} {"value": "1788.97", "type": "Other"} {"value": "4802.03", "type": "Other"} {"value": "1759.87", "type": "Other"} {"value": "4823.91", "type": "Other"} {"value": "1787.98", "type": "Other"} {"value": "4546.23", "type": "Other"} {"value": "1698.35", "type": "Other"} {"value": "4329.39", "type": "Other"} {"value": "1585.33", "type": "Other"}