If I do
conda install pytorch=0.4.0 cuda90 -c pytorch

then it actually installs cuda 9.2. If I forcefully install cuda 9.0 via anaconda before I issue above command, I can’t run pytorch. It fails with a error message that, if you google it, says that the pytorch and cuda versions are incompatible.
If I try to do it with pip
pip install torch==0.4.0 -f https://download.pytorch.org/whl/cu90/stable

I get
ERROR: Could not find a version that satisfies the requirement torch==0.4.0 (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 0.4.1, 0.4.1.post2, 1.0.0, 1.0.1, 1.0.1.post2, 1.1.0)
ERROR: No matching distribution found for torch==0.4.0

Installing 0.4.1 works but then the package I want to compile does not compile. Btw if I install 0.4.0 with cuda 9.2 the software I want to compile also does not work, but hints directly that it needs cuda 9.0.
An edit to further my point:
(base) julian@thinstation:~/PycharmProjects$ conda create -n human_dynamics
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /home/julian/Software/anaconda3/envs/human_dynamics



Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate human_dynamics
#
# To deactivate an active environment, use
#
#     $ conda deactivate
(base) julian@thinstation:~/PycharmProjects$ conda activate human_dynamics
(human_dynamics) julian@thinstation:~/PycharmProjects$ conda install pytorch=0.4.0 cuda90 -c pytorch
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /home/julian/Software/anaconda3/envs/human_dynamics

  added / updated specs:
    - cuda90
    - pytorch=0.4.0


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    certifi-2019.6.16          |           py36_0         154 KB
    cudatoolkit-9.2            |                0       351.0 MB
    ------------------------------------------------------------
                                           Total:       351.2 MB

The following NEW packages will be INSTALLED:

  blas               pkgs/main/linux-64::blas-1.0-mkl
  ca-certificates    pkgs/main/linux-64::ca-certificates-2019.5.15-0
  certifi            pkgs/main/linux-64::certifi-2019.6.16-py36_0
  cffi               pkgs/main/linux-64::cffi-1.12.3-py36h2e261b9_0
  cuda90             pytorch/linux-64::cuda90-1.0-h6433d27_0
  cudatoolkit        pkgs/main/linux-64::cudatoolkit-9.2-0
  [...]

Am I doing something wrong? This should install cudatoolkit 9.0 and not 9.2, right? This looks similar to #11138 13 but his issue got solved I guess? Also his workaround does not work for me, because in my case the pytorch bundled for it actually seems to be compiled against cuda 9.2.
(Another edit to clear the logs from update warnings. I tried it with both anaconda 4.6.11 and 4.6.14, it yields the same result)
Okay, maybe I should have read the issue more carefully: "PyTorch no longer installs or depends on  cudatoolkit == 9". Hmm, so would compiling it myself solve it?