Answers.Body: <p>Sure your idea makes sense to me. Yes you can give y as the reward.</p>
  <p>No need to control for the fact that the input has only discrete values 0 and 1. Technically, most RL codes would normalize the input (using the running totals, obviously), but that's an optional step that works as heuristic, it's not technically needed.</p>
  <p>Interested to hear the results of your experiment.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-25T19:56:20.713Z
Answers.LastActivityDate: 2023-01-25T19:56:20.713Z
Answers.UserDisplayName: Taw
Body: <p>Assuming a relation such that <span class="math-container">$y = f(x)$</span>, where <span class="math-container">$y$</span> represents a scalar and <span class="math-container">$x \in 20 \times 1$</span> vector consisting of zeros and ones, I want to set up a reinforcement learning model that changes the values of the elements of <span class="math-container">$x$</span> in order to maximize the <span class="math-container">$y$</span>.</p>
  <p>Let's assume that <span class="math-container">$y = f(x)$</span> is equal to</p>
  <pre><code>weights = np.random.uniform(-1, 1, 20) 
  y       = np.sum(weights*x)
  </code></pre>
  <p>How do I set up such model?</p>
  <p>In my implementation, I am using keras API and I am trying to adapt the cartpole case code (<a href="https://keras.io/examples/rl/actor_critic_cartpole/" rel="nofollow noreferrer">https://keras.io/examples/rl/actor_critic_cartpole/</a>). However, this code solves for a different problem since the model can perform only one action that can assume discrete values.</p>
  <p>Secondarily, is it appropriate to set as a reward function simply <span class="math-container">$y$</span>?. The following code reports how I would structure the architectures.</p>
  <pre><code>num_inputs = 20
  num_actions = 2
  num_hidden = 128
  
  inputs = layers.Input(shape=(num_inputs,))
  common = layers.Dense(num_hidden, activation=&quot;relu&quot;)(inputs)
  action = layers.Dense(num_actions, activation=&quot;softmax&quot;)(common)
  critic = layers.Dense(1)(common)
  
  model = keras.Model(inputs=inputs, outputs=[action, critic])
  </code></pre>
  <p>However, how can I control the fact that the input parameters could assume only discrete values (0 and 1)?</p>
  
Tags: <reinforcement-learning><keras><reward-functions>
Title: How to setup a reinforcement learning model that changes the values of $x$ to maximize $y$ in $y = f(x)$?
UserDisplayName: Fly
ThreadText: # How to setup a reinforcement learning model that changes the values of $x$ to maximize $y$ in $y = f(x)$?
  
  (Asked by: username_0 on 2023-01-24)
  
  Assuming a relation such that $y = f(x)$, where $y$ represents a scalar and $x \in 20 \times 1$ vector consisting of zeros and ones, I want to set up a reinforcement learning model that changes the values of the elements of $x$ in order to maximize the $y$.
  Let's assume that $y = f(x)$ is equal to
  weights = np.random.uniform(-1, 1, 20) 
  y       = np.sum(weights*x)
  
  How do I set up such model?
  In my implementation, I am using keras API and I am trying to adapt the cartpole case code (https://keras.io/examples/rl/actor_critic_cartpole/). However, this code solves for a different problem since the model can perform only one action that can assume discrete values.
  Secondarily, is it appropriate to set as a reward function simply $y$?. The following code reports how I would structure the architectures.
  num_inputs = 20
  num_actions = 2
  num_hidden = 128
  
  inputs = layers.Input(shape=(num_inputs,))
  common = layers.Dense(num_hidden, activation="relu")(inputs)
  action = layers.Dense(num_actions, activation="softmax")(common)
  critic = layers.Dense(1)(common)
  
  model = keras.Model(inputs=inputs, outputs=[action, critic])
  
  However, how can I control the fact that the input parameters could assume only discrete values (0 and 1)?
  
  **Tags:** reinforcement-learning, keras, reward-functions
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-01-25. Score: 1
  
  Sure your idea makes sense to me. Yes you can give y as the reward.
  
  No need to control for the fact that the input has only discrete values 0 and 1. Technically, most RL codes would normalize the input (using the running totals, obviously), but that's an optional step that works as heuristic, it's not technically needed.
  
  Interested to hear the results of your experiment.
  
  
  ---
  
  