0

I am trying to put backward hooks in my code and getting the gradient of a specific layer is working. However, I can't seem to save the variable to my dictionary. Does anyone know how to do this?

 tcav = {}
    
    def backward_hook(module, grad_input, grad_output):
        #print('module:', module)
        #print('grad_input:', grad_input)
        #print('grad_output:', tuple_of_tensors_to_tensor(grad_output)[0].shape)
        return grad_output
   

    for name, layer in model.named_modules(): 
        if name.startswith(layer_name):
            print("Hook made for...", name)
            layer.register_full_backward_hook(backward_hook)
        
    model.eval()
    tcav = {}
    for ind, (img, label) in enumerate(loader):
        img.requires_grad=True
        img = img.to(device).float()
        output = model(img)
        output.mean().backward()
        tcav[ind] = output.grad
    return tcav

Here, output.grad seems to be None for all images.

rkraaijveld
  • 23
  • 4
  • What have you tried so far? Your code doesn't show the assignment of the gradient tensor onto the *dict*. Please don't forget to accept [your other question](https://stackoverflow.com/questions/72057797/is-layer-activation-register-forward-hook-the-same-as-gradient?noredirect=1#comment127367017_72057797) if you've found an answer to it before moving further down. – Ivan May 02 '22 at 14:12
  • Ah, you're right! I marked it as complete :) first week using this! I am applying backward hooks to two different models. For one of them it worked, the other is not working at the moment. – rkraaijveld May 02 '22 at 14:41
  • This is what I have done so far: ` tcav = {} gradient = {} def backward_hook(module, grad_input, grad_output): gradient[module] = grad_output for name, layer in model.named_modules(): if name in tcav_layer: layer.register_full_backward_hook(backward_hook) model.eval() for ind, (img, label) in enumerate(loader): img.requires_grad=True img = img.to(device).float() out = model(img) x = out.mean() x.requires_grad=True x.backward() tcav[ind] = gradient ` – rkraaijveld May 02 '22 at 14:41
  • Could you provide the exact code you are using (it's best if you edit your question instead of placing the code in the comment section, for easier formatting)? What are you referring to by "second model"? Are you referring to a layer inside a model or an actual second module that you attach your hook onto? – Ivan May 02 '22 at 15:12

0 Answers0