MML Discourse archived in May, 2026

Backpropagation in an expression graph

mark

Consider the expression graph G:

Perform backpropagation through this graph using the rules:

if node.op == "add":
    a, b = node.inputs
    a.grad += node.grad
    b.grad += node.grad
elif node.op == "mul":
    a, b = node.inputs
    a.grad += node.grad * b.value
    b.grad += node.grad * a.value

Comment

The backpropagation into y comes from the sum of two mul nodes. This is exactly the form that stumped me in class, which is why I've include the code defining those rules.

User 012
n0 x val=1 grad=2 n3 × val=2 grad=1 n0->n3 n1 y val=2 grad=4 n1->n3 n4 × val=6 grad=1 n1->n4 n2 3.0 val=3 n2->n4 n5 + val=8 grad=1 n3->n5 n4->n5