<blockquote>
<p>The following is the MDS Objective.</p>
<p><a href="https://i.sstatic.net/e4Bzz.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/e4Bzz.png" alt="MDS Objective" /></a></p>
</blockquote>
<p>Let's think of a senario where I apply MDS <strong>with/from the solution I obtained from PCA</strong>. Then I calculate the objective function on the initial PCA solution and MDS solution (after applying MDS on the former PCA solution). Then I would for sure assume that the objective function will decrease for the MDS solution compared with PCA solution. However, when I calculate the objective function respectively, MDS solution yields higher objective function value. Is this normal?</p>
<p>I am attaching my code below:</p>
<pre><code>import os
import pickle
import gzip
import argparse
import time
import matplotlib.pyplot as plt
import numpy as np
from numpy.linalg import norm

from sklearn.model_selection import train_test_split
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
from sklearn.neural_network import MLPRegressor, MLPClassifier
from sklearn.preprocessing import LabelBinarizer
from sklearn import decomposition

from neural_net import NeuralNet, stochasticNeuralNet
from manifold import MDS, ISOMAP
import utils

def mds_objective(Z,X):
    sum = 0
    n,d = Z.shape
    for i in range(n):
        for j in range(i+1,n):
            sum += (norm(Z[i,:]-Z[j,:],2)-norm(X[i,:]-X[j,:],2))**2
    return 0.5*sum

dataset = load_dataset('animals.pkl')
X = dataset['X'].astype(float)
animals = dataset['animals']
n, d = X.shape
pca = decomposition.PCA(n_components = 5)
pca.fit(X)
Z = pca.transform(X)
plt.figure()
plt.scatter(Z[:, 0], Z[:, 1])
for i in range(n):
     plt.annotate(animals[i], (Z[i,0], Z[i,1]))
utils.savefig('PCA.png')

print(pca.explained_variance_ratio_)
print(mds_objective(Z,X))


dataset = load_dataset('animals.pkl')
X = dataset['X'].astype(float)
animals = dataset['animals']
n,d = X.shape

model = MDS(n_components=2)
Z = model.compress(X)

fig, ax = plt.subplots()
ax.scatter(Z[:,0], Z[:,1])
plt.ylabel('z2')
plt.xlabel('z1')
plt.title('MDS')
for i in range(n):
       ax.annotate(animals[i], (Z[i,0], Z[i,1]))
utils.savefig('MDS_animals.png')
print(mds_objective(Z,X))
</code></pre>
<p>It prints the following:</p>
<blockquote>
<p>1673.1096816455256</p>
<p>1776.8183112784652</p>
</blockquote>

CC BY-SA 4.0
2020-03-18T22:19:13.043Z
I see. I'm not 100% sure if the code is correct. Probably I will have more chances at stack exchange?
underdisplayname
<machine-learning><principal-component-analysis>
Multiple-dimension scaling (MDS) objective for MDS and PCA
underdisplayname
# Multiple-dimension scaling (MDS) objective for MDS and PCA

(Asked by: username_0 on 2020-03-18)

The following is the MDS Objective.


Let's think of a senario where I apply MDS with/from the solution I obtained from PCA. Then I calculate the objective function on the initial PCA solution and MDS solution (after applying MDS on the former PCA solution). Then I would for sure assume that the objective function will decrease for the MDS solution compared with PCA solution. However, when I calculate the objective function respectively, MDS solution yields higher objective function value. Is this normal?
I am attaching my code below:
import os
import pickle
import gzip
import argparse
import time
import matplotlib.pyplot as plt
import numpy as np
from numpy.linalg import norm

from sklearn.model_selection import train_test_split
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
from sklearn.neural_network import MLPRegressor, MLPClassifier
from sklearn.preprocessing import LabelBinarizer
from sklearn import decomposition

from neural_net import NeuralNet, stochasticNeuralNet
from manifold import MDS, ISOMAP
import utils

def mds_objective(Z,X):
    sum = 0
    n,d = Z.shape
    for i in range(n):
        for j in range(i+1,n):
            sum += (norm(Z[i,:]-Z[j,:],2)-norm(X[i,:]-X[j,:],2))**2
    return 0.5*sum

dataset = load_dataset('animals.pkl')
X = dataset['X'].astype(float)
animals = dataset['animals']
n, d = X.shape
pca = decomposition.PCA(n_components = 5)
pca.fit(X)
Z = pca.transform(X)
plt.figure()
plt.scatter(Z[:, 0], Z[:, 1])
for i in range(n):
     plt.annotate(animals[i], (Z[i,0], Z[i,1]))
utils.savefig('PCA.png')

print(pca.explained_variance_ratio_)
print(mds_objective(Z,X))


dataset = load_dataset('animals.pkl')
X = dataset['X'].astype(float)
animals = dataset['animals']
n,d = X.shape

model = MDS(n_components=2)
Z = model.compress(X)

fig, ax = plt.subplots()
ax.scatter(Z[:,0], Z[:,1])
plt.ylabel('z2')
plt.xlabel('z1')
plt.title('MDS')
for i in range(n):
       ax.annotate(animals[i], (Z[i,0], Z[i,1]))
utils.savefig('MDS_animals.png')
print(mds_objective(Z,X))

It prints the following:

1673.1096816455256
1776.8183112784652

**Tags:** machine-learning, principal-component-analysis

> **Comments:**
> **username_0**: I see. I'm not 100% sure if the code is correct. Probably I will have more chances at stack exchange?  

---

## 0 answers

