divelab
PGExplainer on BA2motifs
Hi, Thanks for sharing the library, I was trying to train pg explainer within the benchmark on ba2motifs and I was printing the accuracy of graph classification using the provided edge mask and I got 50% accuracy as it always underfits during training (always provides poor explanation for samples with house motif). I have tried various configurations but nothing has changed so far. Any help would be appreciated.
Hello, could you provide the GCN and PGExplainer model configurations? In addition, have you tried other explanation methods for this GCN model and what's their results?
GCN:
GIN_3l(model_level='graph', dim_node=10, dim_hidden=300, num_classes=2)
Explainer:
PGExplainer(model, in_channels=600, device=device, explain_graph=True, epochs=num_epochs, lr=3e-3, coff_size=0.03, coff_ent=5e-4, t0=5.0, t1=1.0, sample_bias=0.0)
Yes I have tried subgraphx, gradcam and gnnexplainer and there was no major problem, though pgexplainer masks out all edges in both house and circle-motif instances.
Hello, I am unsure about this problem. Due to the situation all the edges are masked out, it seems like the scores for all the edges are similar or the same. Therefore, I recommend you try more hyperparameters for better results although you might have already tried.
There is another paper about the reproducibility of PGExplainer on BA-2Motifs. You might find a similar situation to yours in this paper.
If you want to finetune these hyperparameters, I recommend several sets of hyperparameters here. { " lr":{"_type":"choice","_value":[0.005,0.003,0.001]}, "coff_size":{"_type":"choice","_value":[1.0,0.5,0.1,0.05]}, "coff_ent":{"_type":"choice","_value":[5e-4,1e-4,5e-5,1e-5,0]} } And coff_size is the most important hyper-parameters. In addition, different num_epochs can lead to different results.