One technique used to look at the features having the biggest impact on a prediction is to use feature attribution on a trained model.
There is a library called Captum that works well with PyTorch to accomplish this task. Can this be built into the IDE to perform feature attribution on trained models?
https://captum.ai/
