When I say generate a computational graph, I mean the following:
How do I express my net as a function f(x) such that f just consists of the most basic algebraic operations (add, subtraction, multiplication, division, etc.)?
Case A: Imagine my model just consists of linear layers and activation functions (say ReLU). I can break down my linear layers with the ops mult and add (because a linear layer is just A*x + b, where x = input), but how would I break down ReLU into algebraic ops?
Case B: Imagine my model is much more complicated, and now consists of Convs, MaxPools, and TanH’s (among other activation functions). How do I break down all of these into algebraic ops?
Given this, is there any way to trace the algebraic ops used in the net, starting from the input data to reach the output?
any help at all is appreciated. thanks!