<p dir="auto">I want to build a step activation function to use in keras.</p>
<p dir="auto">Thanks to <a href="https://stackoverflow.com/questions/39921607/how-to-make-a-custom-activation-function-with-only-python-in-tensorflow" rel="nofollow">Q&amp;A here</a>, I am able to build a working step function called <code class="notranslate">tf_stepy</code> with tensorflow. (See code below)</p>
<p dir="auto">Now my question evolves into</p>
<blockquote>
<p dir="auto">How to make use of this <code class="notranslate">tf_stepy</code> activation function created in <code class="notranslate">tensorflow</code> to work in <code class="notranslate">keras</code>?</p>
</blockquote>
<p dir="auto">I tried to the following code to utilize <code class="notranslate">tf_stepy</code> in keras, but not working. Could anyone help me make it to work in keras? Thanks</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from tensorflow_step_function import tf_stepy

def buy_hold_sell(x):
	return tf_stepy(x)

get_custom_objects().update({'custom_activation': Activation(buy_hold_sell)})"><pre class="notranslate"><code class="notranslate">from tensorflow_step_function import tf_stepy

def buy_hold_sell(x):
	return tf_stepy(x)

get_custom_objects().update({'custom_activation': Activation(buy_hold_sell)})
</code></pre></div>
<p dir="auto">Below is the step activation function created with tensorflow</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="# tensorflow_step_function.py
import tensorflow as tf
import keras.backend as K
from keras.backend.tensorflow_backend import _to_tensor
import numpy as np

def stepy(x):
	if x &lt; 0.33:
		return 0.0
	elif x &gt; 0.66:
		return 1.0
	else:
		return 0.5

import numpy as np
np_stepy = np.vectorize(stepy)

def d_stepy(x): # derivative
	if x &lt; 0.33:
		return 0.0
	elif x &gt; 0.66:
		return 1.0
	else:
		return 0.5
np_d_stepy = np.vectorize(d_stepy)

import tensorflow as tf
from tensorflow.python.framework import ops

np_d_stepy_32 = lambda x: np_d_stepy(x).astype(np.float32)

def py_func(func, inp, Tout, stateful=True, name=None, grad=None):

    # Need to generate a unique name to avoid duplicates:
    rnd_name = 'PyFuncGrad' + str(np.random.randint(0, 1E+8))

    tf.RegisterGradient(rnd_name)(grad)  # see _MySquareGrad for grad example
    g = tf.get_default_graph()
    with g.gradient_override_map({&quot;PyFunc&quot;: rnd_name}):
        return tf.py_func(func, inp, Tout, stateful=stateful, name=name)

def tf_d_stepy(x,name=None):
    with ops.op_scope([x], name, &quot;d_stepy&quot;) as name:
        y = tf.py_func(np_d_stepy_32,
                        [x],
                        [tf.float32],
                        name=name,
                        stateful=False)
        return y[0]

def stepygrad(op, grad):
    x = op.inputs[0]

    n_gr = tf_d_stepy(x)
    return grad * n_gr

np_stepy_32 = lambda x: np_stepy(x).astype(np.float32)

def tf_stepy(x, name=None):

    with ops.op_scope([x], name, &quot;stepy&quot;) as name:
        y = py_func(np_stepy_32,
                        [x],
                        [tf.float32],
                        name=name,
                        grad=stepygrad)  # &lt;-- here's the call to the gradient
        return y[0]

with tf.Session() as sess:

    x = tf.constant([0.2,0.7,0.4,0.6])
    y = tf_stepy(x)
    tf.initialize_all_variables().run()

    print(x.eval(), y.eval(), tf.gradients(y, [x])[0].eval())"><pre class="notranslate"><code class="notranslate"># tensorflow_step_function.py
import tensorflow as tf
import keras.backend as K
from keras.backend.tensorflow_backend import _to_tensor
import numpy as np

def stepy(x):
	if x &lt; 0.33:
		return 0.0
	elif x &gt; 0.66:
		return 1.0
	else:
		return 0.5

import numpy as np
np_stepy = np.vectorize(stepy)

def d_stepy(x): # derivative
	if x &lt; 0.33:
		return 0.0
	elif x &gt; 0.66:
		return 1.0
	else:
		return 0.5
np_d_stepy = np.vectorize(d_stepy)

import tensorflow as tf
from tensorflow.python.framework import ops

np_d_stepy_32 = lambda x: np_d_stepy(x).astype(np.float32)

def py_func(func, inp, Tout, stateful=True, name=None, grad=None):

    # Need to generate a unique name to avoid duplicates:
    rnd_name = 'PyFuncGrad' + str(np.random.randint(0, 1E+8))

    tf.RegisterGradient(rnd_name)(grad)  # see _MySquareGrad for grad example
    g = tf.get_default_graph()
    with g.gradient_override_map({"PyFunc": rnd_name}):
        return tf.py_func(func, inp, Tout, stateful=stateful, name=name)

def tf_d_stepy(x,name=None):
    with ops.op_scope([x], name, "d_stepy") as name:
        y = tf.py_func(np_d_stepy_32,
                        [x],
                        [tf.float32],
                        name=name,
                        stateful=False)
        return y[0]

def stepygrad(op, grad):
    x = op.inputs[0]

    n_gr = tf_d_stepy(x)
    return grad * n_gr

np_stepy_32 = lambda x: np_stepy(x).astype(np.float32)

def tf_stepy(x, name=None):

    with ops.op_scope([x], name, "stepy") as name:
        y = py_func(np_stepy_32,
                        [x],
                        [tf.float32],
                        name=name,
                        grad=stepygrad)  # &lt;-- here's the call to the gradient
        return y[0]

with tf.Session() as sess:

    x = tf.constant([0.2,0.7,0.4,0.6])
    y = tf_stepy(x)
    tf.initialize_all_variables().run()

    print(x.eval(), y.eval(), tf.gradients(y, [x])[0].eval())
</code></pre></div>
<p dir="auto">As described in the Keras handbook -<em>Deep Learning with Pyhton</em>-, for a multi-output model we need to specify different loss functions for different heads of the network. But because gradient descent requires you to minimize a scalar, you must combine these losses into a single value in order to train the model.</p>
<p dir="auto">Very imbalanced loss contributions will cause the model representations to be optimized preferentially for the task with the largest individual loss, at the expense of the other tasks. To remedy this, you can assign different levels of importance to the loss values in their contribution to the final loss. This is useful in particular if the losses’ values use different scales.</p>
<p dir="auto">Can anyone help with the following:</p>
<p dir="auto">I've got a five-output model as described in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="320318126" data-permission-text="Title is private" data-url="https://github.com/keras-team/keras/issues/10120" data-hovercard-type="issue" data-hovercard-url="/keras-team/keras/issues/10120/hovercard" href="https://github.com/keras-team/keras/issues/10120">#10120</a>.<br>
The outputs of the model are the following:</p>
<ol dir="auto">
<li>emotion (multiclass, multilabel classification)</li>
<li>valence (regression)</li>
<li>arousal (regression)</li>
<li>dominance (regression)</li>
<li>age (multiclass classification)</li>
</ol>
<p dir="auto">I am using the following :</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="    losses_list = {'EMOTIONS': 'binary_crossentropy',
                   'VALENCE': 'mse',
                   'AROUSAL': 'mse',
                   'DOMINANCE': 'mse',
                   'AGE': 'categorical_crossentropy'}"><pre class="notranslate">    <span class="pl-s1">losses_list</span> <span class="pl-c1">=</span> {<span class="pl-s">'EMOTIONS'</span>: <span class="pl-s">'binary_crossentropy'</span>,
                   <span class="pl-s">'VALENCE'</span>: <span class="pl-s">'mse'</span>,
                   <span class="pl-s">'AROUSAL'</span>: <span class="pl-s">'mse'</span>,
                   <span class="pl-s">'DOMINANCE'</span>: <span class="pl-s">'mse'</span>,
                   <span class="pl-s">'AGE'</span>: <span class="pl-s">'categorical_crossentropy'</span>}</pre></div>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="    losses_weights = {'EMOTIONS': 1.0,
                      'VALENCE': 0.025,
                      'AROUSAL': 0.025,
                      'DOMINANCE': 0.025,
                      'AGE': 0.45}"><pre class="notranslate">    <span class="pl-s1">losses_weights</span> <span class="pl-c1">=</span> {<span class="pl-s">'EMOTIONS'</span>: <span class="pl-c1">1.0</span>,
                      <span class="pl-s">'VALENCE'</span>: <span class="pl-c1">0.025</span>,
                      <span class="pl-s">'AROUSAL'</span>: <span class="pl-c1">0.025</span>,
                      <span class="pl-s">'DOMINANCE'</span>: <span class="pl-c1">0.025</span>,
                      <span class="pl-s">'AGE'</span>: <span class="pl-c1">0.45</span>}</pre></div>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="    metrics ={'EMOTIONS': 'crossentropy',
              'VALENCE': 'mse',
              'AROUSAL': 'mse',
              'DOMINANCE': 'mse',
              'AGE': 'categorical_accuracy'}"><pre class="notranslate">    <span class="pl-s1">metrics</span> <span class="pl-c1">=</span>{<span class="pl-s">'EMOTIONS'</span>: <span class="pl-s">'crossentropy'</span>,
              <span class="pl-s">'VALENCE'</span>: <span class="pl-s">'mse'</span>,
              <span class="pl-s">'AROUSAL'</span>: <span class="pl-s">'mse'</span>,
              <span class="pl-s">'DOMINANCE'</span>: <span class="pl-s">'mse'</span>,
              <span class="pl-s">'AGE'</span>: <span class="pl-s">'categorical_accuracy'</span>}</pre></div>
<p dir="auto">Can anyone comment on this ?<br>
Are those the right loss functions? Are those the right weights and are those metrics properly set?</p>