In addition to what @keithmgould said, keep in mind that you are calculating the expectation of your return with respect to the trajectory distribution, and this expectation is the objective function to be maximized.
I recommend studying the first few slides of Sergey Levine’s Policy Gradient lecture notes 146 to understand how the gradient of the objective is derived.