<p>I think your code works fine for what is meant to be doing - fitting a linear regression model. The problem here is that you are using a linear model. Linear model does not have an adequate approximation capacity, it will only be able to fit data that is described by a linear function. Here, you gave a random sequence of numbers, that is very difficult for linear model to approximate. I would advise you to try 2 things:  </p>

<p>1) Try something simpler first. Instead of doing a random sequence of numbers do a linear sequence of numbers for example a function like <span class="math-container">$y = 2x$</span> or maybe affine function like <span class="math-container">$y = 2x + 5$</span>. So you would have a sequence like:  </p>

<p><span class="math-container">$2, 4, 6, 8 ...$</span> or <span class="math-container">$7, 9, 11, 13, ...$</span>  </p>

<p>If you manage to get that working try a nonlinear function like <span class="math-container">$x^2$</span> for example.  </p>

<p>2) Instead of using a linear model try a nonlinear model. For example a polynomial regression model. Especially powerful function approximators are neural networks. In theory, a neural network with single hidden layer can approximate an arbitrary continuous function under some conditions 
(<a href="https://en.wikipedia.org/wiki/Universal_approximation_theorem" rel="nofollow noreferrer">Universal approximation theorem</a>) , so you could try to see how would a neural network solve the problem, there are several open source neural network libraries that you could try.</p>

CC BY-SA 4.0
2019-05-05T11:46:53.170Z
well I found a pattern and it goes like this: we take the first element in the array and then we take it's index and next index and then the AI makes the decision which number to choose in that interval, that's how we got the number for example we got 3(2nd number in array) by applying the same logic like this [2^1(index of 2nd element), **3**(number 3 exists somewhere between this interval), 2^2(next index)] and so on and so forth for each element (his index and the next...) but since I have no previous knowledge in AI and machine learning I have no idea how to apply this
Thorvald
CC BY-SA 4.0
2019-04-27T17:27:30.323Z
2019-04-27T17:27:30.323Z
Brale
<p>Like others said you can't approximate this with a linear regression model.<br><br>
A PRM that approximates a solution could give you the following:<br>
<span class="math-container">$y = 0.948 + x + 0.00085*x^6$</span> ~ <br>
<span class="math-container">$y = 237/250 + x + (17/20000)*x^6$</span><br>
For <span class="math-container">$x = 9$</span>, <span class="math-container">$y \simeq 462$</span><br></p>

<p>or </p>

<p><span class="math-container">$y = 0.9258 + x + 0.00086*x^6$</span>
For <span class="math-container">$x = 9$</span>, <span class="math-container">$y \simeq 466.965$</span><br></p>

<p><code>UPDATE</code><br>
An approximation of course, may be in the range of:<br>
<span class="math-container">$y = 2^{(x + 1)} - 2^x$</span> -the model you propose- <br>
<code>Goodness of fit: 0.968475 and Mean Square Error = 685.111</code><br><br>
Based on this range a better approximation would be:<br>
<span class="math-container">$y = 2^x + (-1/2)*x^2$</span><br>
with <span class="math-container">$R^2$</span><br>
<code>Goodness of fit = 0.995</code> <br>
<code>Mean Square Error: 89.0278</code></p>

CC BY-SA 4.0
2019-05-09T10:14:18.820Z
I actually found a pattern it is not random the pattern goes like this: you take the element's current index and it's next index and you apply this interval : [2^index, 2^next index] for example for the number 7 in the given example above we do this [2^2, 2^3] the number 7 is somewhere between that interval, the AI must predict based on this pattern. if you have any solution proposed please be kind and guide me through!
Thorvald
CC BY-SA 4.0
2019-05-09T09:34:11.717Z
2019-05-09T10:43:17.573Z
2019-05-09T10:43:17.573Z
Panos Kalatzantonakis
<p>The AI must predict the next number in a given sequence of incremental integers (with no obvious pattern) using Python but so far I don't get the intended result!
I tried changing the learning rate and iterations but so far no luck!</p>

<p>Example sequence: [1, 3, 7, 8, 21, 49, 76, 224]</p>

<p>Expected result: 467</p>

<p>Result found : 2,795.5</p>

<p>Cost: 504579.43</p>

<p>This is what I've done so far:</p>

<pre><code>import numpy as np

# Init sequence
data =\
    [
        [0, 1.0], [1, 3.0], [2, 7.0], [3, 8.0],
        [4, 21.0], [5, 49.0], [6, 76.0], [7, 224.0]
    ]

X = np.matrix(data)[:, 0]
y = np.matrix(data)[:, 1]

def J(X, y, theta):
    theta = np.matrix(theta).T
    m = len(y)
    predictions = X * theta
    sqError = np.power((predictions-y), [2])
    return 1/(2*m) * sum(sqError)

dataX = np.matrix(data)[:, 0:1]
X = np.ones((len(dataX), 2))
X[:, 1:] = dataX

# gradient descent function
def gradient(X, y, alpha, theta, iters):
    J_history = np.zeros(iters)
    m = len(y)
    theta = np.matrix(theta).T
    for i in range(iters):
        h0 = X * theta
        delta = (1 / m) * (X.T * h0 - X.T * y)
        theta = theta - alpha * delta
        J_history[i] = J(X, y, theta.T)
     return J_history, theta
print('\n'+40*'=')

# Theta initialization
theta = np.matrix([np.random.random(), np.random.random()])

# Learning rate
alpha = 0.02

# Iterations
iters = 1000000

print('\n== Model summary ==\nLearning rate: {}\nIterations: {}\nInitial 
theta: {}\nInitial J: {:.2f}\n'
  .format(alpha, iters, theta, J(X, y, theta).item()))
print('Training model... ')

# Train model and find optimal Theta value
J_history, theta_min = gradient(X, y, alpha, theta, iters)
print('Done, Model is trained')
print('\nModelled prediction function is:\ny = {:.2f} * x + {:.2f}'
  .format(theta_min[1].item(), theta_min[0].item()))
print('Cost is: {:.2f}'.format(J(X, y, theta_min.T).item()))

# Calculate the predicted profit
def predict(pop):
    return [1, pop] * theta_min

# Now
p = len(data)
print('\n'+40*'=')
print('Initial sequence was:\n', *np.array(data)[:, 1])
print('\nNext numbers should be: {:,.1f}'
  .format(predict(p).item()))
</code></pre>

<p><strong>UPDATE</strong> Another method I tried but still giving wrong results</p>

<pre><code>import numpy as np
from sklearn import datasets, linear_model

# Define the problem
problem = [1, 3, 7, 8, 21, 49, 76, 224]

# create x and y for the problem

x = []
y = []

for (xi, yi) in enumerate(problem):
    x.append([xi])
    y.append(yi)

x = np.array(x)
y = np.array(y)
# Create linear regression object
regr = linear_model.LinearRegression()
regr.fit(x, y)

# create the testing set
x_test = [[i] for i in range(len(x), 3 + len(x))]

# The coefficients
print('Coefficients: \n', regr.coef_)
# The mean squared error
print("Mean squared error: %.2f" % np.mean((regr.predict(x) - y) ** 2))
# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % regr.score(x, y))

# Do predictions
y_predicted = regr.predict(x_test)

print("Next few numbers in the series are")
for pred in y_predicted:
    print(pred)
</code></pre>

CC BY-SA 4.0
2019-04-28T09:54:04.687Z
Cross-posted here: https://datascience.stackexchange.com/q/51033/924
Has QUIT--Anony-Mousse
CC BY-SA 4.0
2019-04-28T11:15:28.727Z
are you hunting my posts? @Anony-Mousse, I posted the same thread on multiple Stack platforms
Thorvald
CC BY-SA 4.0
2019-04-29T01:31:16.367Z
@nbro please be clear to suggest to only ask in *one* place, and not to duplicate questions. Otherwise future users will have to look for answers in many places. For the rationale of not allowing crossposting see https://meta.stackexchange.com/questions/64068/is-cross-posting-a-question-on-multiple-stack-exchange-sites-permitted-if-the-qu/64069#64069 Thank you.
Has QUIT--Anony-Mousse
<neural-networks><machine-learning><training><python>
Use Machine Learning/Artificial Intelligence to predict next number (n+1) in a given sequence of random increasing integers
Thorvald
# Use Machine Learning/Artificial Intelligence to predict next number (n+1) in a given sequence of random increasing integers

(Asked by: username_0 on 2019-04-27)

The AI must predict the next number in a given sequence of incremental integers (with no obvious pattern) using Python but so far I don't get the intended result!
I tried changing the learning rate and iterations but so far no luck!
Example sequence: [1, 3, 7, 8, 21, 49, 76, 224]
Expected result: 467
Result found : 2,795.5
Cost: 504579.43
This is what I've done so far:
import numpy as np

# Init sequence
data =\
    [
        [0, 1.0], [1, 3.0], [2, 7.0], [3, 8.0],
        [4, 21.0], [5, 49.0], [6, 76.0], [7, 224.0]
    ]

X = np.matrix(data)[:, 0]
y = np.matrix(data)[:, 1]

def J(X, y, theta):
    theta = np.matrix(theta).T
    m = len(y)
    predictions = X * theta
    sqError = np.power((predictions-y), [2])
    return 1/(2*m) * sum(sqError)

dataX = np.matrix(data)[:, 0:1]
X = np.ones((len(dataX), 2))
X[:, 1:] = dataX

# gradient descent function
def gradient(X, y, alpha, theta, iters):
    J_history = np.zeros(iters)
    m = len(y)
    theta = np.matrix(theta).T
    for i in range(iters):
        h0 = X * theta
        delta = (1 / m) * (X.T * h0 - X.T * y)
        theta = theta - alpha * delta
        J_history[i] = J(X, y, theta.T)
     return J_history, theta
print('\n'+40*'=')

# Theta initialization
theta = np.matrix([np.random.random(), np.random.random()])

# Learning rate
alpha = 0.02

# Iterations
iters = 1000000

print('\n== Model summary ==\nLearning rate: {}\nIterations: {}\nInitial 
theta: {}\nInitial J: {:.2f}\n'
  .format(alpha, iters, theta, J(X, y, theta).item()))
print('Training model... ')

# Train model and find optimal Theta value
J_history, theta_min = gradient(X, y, alpha, theta, iters)
print('Done, Model is trained')
print('\nModelled prediction function is:\ny = {:.2f} * x + {:.2f}'
  .format(theta_min[1].item(), theta_min[0].item()))
print('Cost is: {:.2f}'.format(J(X, y, theta_min.T).item()))

# Calculate the predicted profit
def predict(pop):
    return [1, pop] * theta_min

# Now
p = len(data)
print('\n'+40*'=')
print('Initial sequence was:\n', *np.array(data)[:, 1])
print('\nNext numbers should be: {:,.1f}'
  .format(predict(p).item()))

UPDATE Another method I tried but still giving wrong results
import numpy as np
from sklearn import datasets, linear_model

# Define the problem
problem = [1, 3, 7, 8, 21, 49, 76, 224]

# create x and y for the problem

x = []
y = []

for (xi, yi) in enumerate(problem):
    x.append([xi])
    y.append(yi)

x = np.array(x)
y = np.array(y)
# Create linear regression object
regr = linear_model.LinearRegression()
regr.fit(x, y)

# create the testing set
x_test = [[i] for i in range(len(x), 3 + len(x))]

# The coefficients
print('Coefficients: \n', regr.coef_)
# The mean squared error
print("Mean squared error: %.2f" % np.mean((regr.predict(x) - y) ** 2))
# Explained variance score: 1 is perfect prediction
print('Variance score: %.2f' % regr.score(x, y))

# Do predictions
y_predicted = regr.predict(x_test)

print("Next few numbers in the series are")
for pred in y_predicted:
    print(pred)

**Tags:** neural-networks, machine-learning, training, python

> **Comments:**
> **username_1**: Cross-posted here: https://datascience.stackexchange.com/q/51033/924  
> **username_0**: are you hunting my posts? @Anony-Mousse, I posted the same thread on multiple Stack platforms  
> **username_1**: @nbro please be clear to suggest to only ask in *one* place, and not to duplicate questions. Otherwise future users will have to look for answers in many places. For the rationale of not allowing crossposting see https://meta.stackexchange.com/questions/64068/is-cross-posting-a-question-on-multiple-stack-exchange-sites-permitted-if-the-qu/64069#64069 Thank you.  

---

## 2 answers

### Answer by username_2 on 2019-04-27. Score: 3

I think your code works fine for what is meant to be doing - fitting a linear regression model. The problem here is that you are using a linear model. Linear model does not have an adequate approximation capacity, it will only be able to fit data that is described by a linear function. Here, you gave a random sequence of numbers, that is very difficult for linear model to approximate. I would advise you to try 2 things:

1) Try something simpler first. Instead of doing a random sequence of numbers do a linear sequence of numbers for example a function like $y = 2x$ or maybe affine function like $y = 2x + 5$. So you would have a sequence like:

$2, 4, 6, 8 ...$ or $7, 9, 11, 13, ...$

If you manage to get that working try a nonlinear function like $x^2$ for example.

2) Instead of using a linear model try a nonlinear model. For example a polynomial regression model. Especially powerful function approximators are neural networks. In theory, a neural network with single hidden layer can approximate an arbitrary continuous function under some conditions
([Universal approximation theorem](https://en.wikipedia.org/wiki/Universal_approximation_theorem)) , so you could try to see how would a neural network solve the problem, there are several open source neural network libraries that you could try.

> **Comments:**
> **username_0**: well I found a pattern and it goes like this: we take the first element in the array and then we take it's index and next index and then the AI makes the decision which number to choose in that interval, that's how we got the number for example we got 3(2nd number in array) by applying the same logic like this [2^1(index of 2nd element), **3**(number 3 exists somewhere between this interval), 2^2(next index)] and so on and so forth for each element (his index and the next...) but since I have no previous knowledge in AI and machine learning I have no idea how to apply this  

---

### Answer by username_3 on 2019-05-09. Score: 2

Like others said you can't approximate this with a linear regression model.  
  
A PRM that approximates a solution could give you the following:  
$y = 0.948 + x + 0.00085\*x^6$ ~   
$y = 237/250 + x + (17/20000)\*x^6$  
For $x = 9$, $y \simeq 462$

or

$y = 0.9258 + x + 0.00086\*x^6$
For $x = 9$, $y \simeq 466.965$

`UPDATE`  
An approximation of course, may be in the range of:  
$y = 2^{(x + 1)} - 2^x$ -the model you propose-   
`Goodness of fit: 0.968475 and Mean Square Error = 685.111`  
  
Based on this range a better approximation would be:  
$y = 2^x + (-1/2)\*x^2$  
with $R^2$  
`Goodness of fit = 0.995`   
`Mean Square Error: 89.0278`

> **Comments:**
> **username_0**: I actually found a pattern it is not random the pattern goes like this: you take the element's current index and it's next index and you apply this interval : [2^index, 2^next index] for example for the number 7 in the given example above we do this [2^2, 2^3] the number 7 is somewhere between that interval, the AI must predict based on this pattern. if you have any solution proposed please be kind and guide me through!  

---

