My SRAM X3 rear derailleur will only shift to the lowest gear but won't shift back higher The rear shifter is new and the shifters work smoothly but only can shift to the bigger sprocket and no matter what I do the shifter doesn't shift back to the smaller sprocket.

I don't know if this affects shifting, but my bike's cable housing used to be in several parts but now I made only one housing.
Highly likely that friction or pinching of the cable in the housing is preventing the derailleur spring from pulling the cable when shifting from large to small sprockets.

Remove the cable from the derailleur, With the wheel out and the cable detached check that the derailleur moves freely.  Pull gently on the shift cable (you may need pliers to hold on to it) and work the shifter, on upshifts the cable should pay out smoothly with little resistance.

Check the shifter cable housing run for kinks, tight turns etc.