<p dir="auto">Summary/tldr:<br>
It would be nice with more String support in the Java API and a tf.reduce_sum which supports string concatenation.</p>
<p dir="auto">Body:<br>
I have successfully hacked in the custom ops of syntaxnet into the Tensorflow master as I wish to run syntaxnet from my Java NLP pipeline.</p>
<p dir="auto">I do this by saving a SavedModelBundle and load it again from Java.</p>
<p dir="auto">My project is based on a saved model of parsey_mcparseface from this branch:<br>
<a href="https://github.com/dmansfield/models/tree/documents-from-tensor">https://github.com/dmansfield/models/tree/documents-from-tensor</a><br>
By user <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/dmansfield/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/dmansfield">@dmansfield</a>.</p>
<p dir="auto">And a tensorflow build based on the custom ops from:<br>
<a href="https://github.com/tensorflow/models/tree/master/syntaxnet">https://github.com/tensorflow/models/tree/master/syntaxnet</a><br>
Hacked into master of tensorflow.</p>
<p dir="auto">Hacking in the ops, as build in ops, was necessary as the Java api through the JNI does not support user loaded ops yet (AFAIK).</p>
<p dir="auto">The code with the syntaxnet ops hacked in can be found here:<br>
<a href="https://github.com/marhlder/syntaxnet-ops-hack">https://github.com/marhlder/syntaxnet-ops-hack</a></p>
<p dir="auto">I have everything running except that the strings in the output tensor from my saved model includes junk (non-printable chars) which I think is due to the offset table described in tensor_jni.cc file.</p>
<p dir="auto">I can run the same model from Python without any issues.</p>
<p dir="auto">The Java API does currently not seem to support anything but scalar string tensors. It looks like using a scalar string tensor will solve my problem as that part of the codes seems to handle the offset table.</p>
<p dir="auto">I would therefore like to reduce the string tensor produced by my parsey mcparseface model.<br>
I then found that the tf.reduce_sum does not yet support string concatenation.</p>
<p dir="auto">There should already be an op for it:<br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="125057980" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/701" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/701/hovercard" href="https://github.com/tensorflow/tensorflow/issues/701">#701</a></p>
<p dir="auto">User <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mrry/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mrry">@mrry</a> commented that he was looking for a use case for this here:<br>
<a href="https://stackoverflow.com/questions/34247374/merge-string-tensors-in-tensorflow" rel="nofollow">https://stackoverflow.com/questions/34247374/merge-string-tensors-in-tensorflow</a></p>
<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes</li>
<li>OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Arch Linux</li>
<li>Tensorflow installed from (source or binary): pip</li>
<li>Tensorflow version (use command below): tf-nightly-gpu-2.0-preview | 2.0.0.dev20190204 |</li>
<li>Python version: 3.6.8</li>
<li>CUDA/cuDNN version: 10.0, V10.0.130</li>
<li>GPU model and memory: GeForce RTX 2018 Ti 11GB</li>
</ul>
<p dir="auto"><strong>Describe the current behavior</strong><br>
During the training of a simple "toy-example" classificator using TensorFlow 2.0 and tfds, I'm trying to use the Keras callback in order to log the results and to be able to use TensorBoard. When the first epoch of training ends I get the following error: <code class="notranslate">tensorflow.python.framework.errors_impl.NotFoundError: Resource localhost/logdir:./log/N10tensorflow22SummaryWriterInterfaceE does not exist. [Op:WriteScalarSummary] name: epoch_loss/ </code></p>
<p dir="auto"><strong>Describe the expected behavior</strong><br>
I expect to be able to use the TensorBoard callback without issues.</p>
<p dir="auto"><strong>Code to reproduce the issue</strong></p>
<p dir="auto"><strong>Classificator:</strong><br>
This is a toy example based on the inception architecture and it is written inside a <em>model.py</em> file using Keras Functional API.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="def inception(input_shape: Tuple[int, int, int]) -&gt; k.Model:
    # set the input
    input_img = k.layers.Input(shape=input_shape)

    tower_1 = k.layers.Conv2D(64, (1, 1), padding=&quot;same&quot;, activation=&quot;relu&quot;)(input_img)
    tower_1 = k.layers.Conv2D(64, (3, 3), padding=&quot;same&quot;, activation=&quot;relu&quot;)(tower_1)

    tower_2 = k.layers.Conv2D(64, (1, 1), padding=&quot;same&quot;, activation=&quot;relu&quot;)(input_img)
    tower_2 = k.layers.Conv2D(64, (5, 5), padding=&quot;same&quot;, activation=&quot;relu&quot;)(tower_2)

    tower_3 = k.layers.MaxPooling2D((3, 3), strides=(1, 1), padding=&quot;same&quot;)(input_img)
    tower_3 = k.layers.Conv2D(64, (1, 1), padding=&quot;same&quot;, activation=&quot;relu&quot;)(tower_3)

    output = k.layers.concatenate([tower_1, tower_2, tower_3], axis=3)

    output = k.layers.Flatten()(output)
    out = k.layers.Dense(10, activation=&quot;softmax&quot;)(output)

    model = k.models.Model(inputs=[input_img], outputs=[out])

    print(model.summary())

    return model"><pre class="notranslate"><code class="notranslate">def inception(input_shape: Tuple[int, int, int]) -&gt; k.Model:
    # set the input
    input_img = k.layers.Input(shape=input_shape)

    tower_1 = k.layers.Conv2D(64, (1, 1), padding="same", activation="relu")(input_img)
    tower_1 = k.layers.Conv2D(64, (3, 3), padding="same", activation="relu")(tower_1)

    tower_2 = k.layers.Conv2D(64, (1, 1), padding="same", activation="relu")(input_img)
    tower_2 = k.layers.Conv2D(64, (5, 5), padding="same", activation="relu")(tower_2)

    tower_3 = k.layers.MaxPooling2D((3, 3), strides=(1, 1), padding="same")(input_img)
    tower_3 = k.layers.Conv2D(64, (1, 1), padding="same", activation="relu")(tower_3)

    output = k.layers.concatenate([tower_1, tower_2, tower_3], axis=3)

    output = k.layers.Flatten()(output)
    out = k.layers.Dense(10, activation="softmax")(output)

    model = k.models.Model(inputs=[input_img], outputs=[out])

    print(model.summary())

    return model
</code></pre></div>
<p dir="auto"><strong>Training code:</strong><br>
The training code it is inside a <em>test.py</em> file and use the <code class="notranslate">tfds</code> to import the datasets and the Keras <code class="notranslate">fit</code> function to train.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import tensorflow_datasets as tfds
import tensorflow as tf
from tensorflow import keras as k

import multiprocessing
import model

DATADIR = &quot;/run/media/federico/XData/tensorflow_datasets&quot;
CLASSES = 10
EPOCHS = 25
BATCH = 128
LRATE = 0.01
DECAY = LRATE / EPOCHS

# this is applied to every single data passed
def process_features(feature):
    image, label = feature[&quot;image&quot;], feature[&quot;label&quot;]

    # image conversion into [0, 1]
    image = image / 255
    image = tf.cast(image, tf.float32)
    feature[&quot;image&quot;] = image

    # label conversion into one-hot
    print(label)
    label = tf.one_hot(label, CLASSES)
    feature[&quot;label&quot;] = label

    return feature

# print(tfds.list_builders())

dataset, dataset_info = tfds.load(name=&quot;cifar10&quot;, data_dir=DATADIR, with_info=True)
train_dataset, test_dataset = dataset[&quot;train&quot;], dataset[&quot;test&quot;]

# process features
train_dataset = (
    train_dataset.map(process_features, num_parallel_calls=multiprocessing.cpu_count())
    .shuffle(1000)
    .batch(BATCH)
    .repeat(EPOCHS)
    # .prefetch(10)
)


test_dataset = test_dataset.map(
    process_features, num_parallel_calls=multiprocessing.cpu_count()
).batch(dataset_info.splits[&quot;test&quot;].get_proto().statistics.num_examples)

for f in test_dataset:
    images_t, labels_t = f[&quot;image&quot;], f[&quot;label&quot;]

model = model.inception(dataset_info.features[&quot;image&quot;].shape)

# compile the model
model.compile(
    optimizer=k.optimizers.RMSprop(learning_rate=LRATE, decay=DECAY),
    loss=k.losses.categorical_crossentropy,
    metrics=[k.metrics.categorical_accuracy],
)

tbCallback = [
    k.callbacks.TensorBoard(
        log_dir=&quot;./log&quot;, histogram_freq=0, write_graph=False, write_images=False
    )
]
# tbCallback = [k.callbacks.TensorBoard(log_dir='./log')]

step = 0
for f in train_dataset:
    images, labels = f[&quot;image&quot;], f[&quot;label&quot;]
    step += 1
    model.fit(
        images,
        labels,
        validation_data=(images_t, labels_t),
        steps_per_epoch=100,
        epochs=1,
        batch_size=BATCH,
        callbacks=tbCallback,
    )
"><pre class="notranslate"><code class="notranslate">import tensorflow_datasets as tfds
import tensorflow as tf
from tensorflow import keras as k

import multiprocessing
import model

DATADIR = "/run/media/federico/XData/tensorflow_datasets"
CLASSES = 10
EPOCHS = 25
BATCH = 128
LRATE = 0.01
DECAY = LRATE / EPOCHS

# this is applied to every single data passed
def process_features(feature):
    image, label = feature["image"], feature["label"]

    # image conversion into [0, 1]
    image = image / 255
    image = tf.cast(image, tf.float32)
    feature["image"] = image

    # label conversion into one-hot
    print(label)
    label = tf.one_hot(label, CLASSES)
    feature["label"] = label

    return feature

# print(tfds.list_builders())

dataset, dataset_info = tfds.load(name="cifar10", data_dir=DATADIR, with_info=True)
train_dataset, test_dataset = dataset["train"], dataset["test"]

# process features
train_dataset = (
    train_dataset.map(process_features, num_parallel_calls=multiprocessing.cpu_count())
    .shuffle(1000)
    .batch(BATCH)
    .repeat(EPOCHS)
    # .prefetch(10)
)


test_dataset = test_dataset.map(
    process_features, num_parallel_calls=multiprocessing.cpu_count()
).batch(dataset_info.splits["test"].get_proto().statistics.num_examples)

for f in test_dataset:
    images_t, labels_t = f["image"], f["label"]

model = model.inception(dataset_info.features["image"].shape)

# compile the model
model.compile(
    optimizer=k.optimizers.RMSprop(learning_rate=LRATE, decay=DECAY),
    loss=k.losses.categorical_crossentropy,
    metrics=[k.metrics.categorical_accuracy],
)

tbCallback = [
    k.callbacks.TensorBoard(
        log_dir="./log", histogram_freq=0, write_graph=False, write_images=False
    )
]
# tbCallback = [k.callbacks.TensorBoard(log_dir='./log')]

step = 0
for f in train_dataset:
    images, labels = f["image"], f["label"]
    step += 1
    model.fit(
        images,
        labels,
        validation_data=(images_t, labels_t),
        steps_per_epoch=100,
        epochs=1,
        batch_size=BATCH,
        callbacks=tbCallback,
    )

</code></pre></div>
<p dir="auto"><strong>Other info / logs</strong></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Traceback (most recent call last):
  File &quot;/run/media/federico/XData/PycharmProjectsXData/classificator/test.py&quot;, line 105, in &lt;module&gt;
    callbacks=tbCallback,
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py&quot;, line 963, in fit
    steps_name='steps_per_epoch')
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py&quot;, line 386, in model_iteration
    callbacks.on_epoch_end(epoch, epoch_logs)
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py&quot;, line 261, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py&quot;, line 1389, in on_epoch_end
    self._write_custom_summaries(step, logs)
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py&quot;, line 1335, in _write_custom_summaries
    summary_ops_v2.scalar(name, value, step=step)
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py&quot;, line 564, in scalar
    return summary_writer_function(name, tensor, function, family=family)
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py&quot;, line 508, in summary_writer_function
    should_record_summaries(), record, _nothing, name=&quot;&quot;)
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/framework/smart_cond.py&quot;, line 54, in smart_cond
    return true_fn()
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py&quot;, line 501, in record
    with ops.control_dependencies([function(tag, scope)]):
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py&quot;, line 562, in function
    name=scope)
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/gen_summary_ops.py&quot;, line 727, in write_scalar_summary
    writer, step, tag, value, name=name, ctx=_ctx)
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/gen_summary_ops.py&quot;, line 763, in write_scalar_summary_eager_fallback
    attrs=_attrs, ctx=_ctx, name=name)
  File &quot;/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/eager/execute.py&quot;, line 66, in quick_execute
    six.raise_from(core._status_to_exception(e.code, message), None)
  File &quot;&lt;string&gt;&quot;, line 3, in raise_from
tensorflow.python.framework.errors_impl.NotFoundError: Resource localhost/logdir:./log/N10tensorflow22SummaryWriterInterfaceE does not exist. [Op:WriteScalarSummary] name: epoch_loss/"><pre class="notranslate"><code class="notranslate">Traceback (most recent call last):
  File "/run/media/federico/XData/PycharmProjectsXData/classificator/test.py", line 105, in &lt;module&gt;
    callbacks=tbCallback,
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 963, in fit
    steps_name='steps_per_epoch')
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 386, in model_iteration
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py", line 261, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py", line 1389, in on_epoch_end
    self._write_custom_summaries(step, logs)
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py", line 1335, in _write_custom_summaries
    summary_ops_v2.scalar(name, value, step=step)
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py", line 564, in scalar
    return summary_writer_function(name, tensor, function, family=family)
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py", line 508, in summary_writer_function
    should_record_summaries(), record, _nothing, name="")
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/framework/smart_cond.py", line 54, in smart_cond
    return true_fn()
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py", line 501, in record
    with ops.control_dependencies([function(tag, scope)]):
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py", line 562, in function
    name=scope)
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/gen_summary_ops.py", line 727, in write_scalar_summary
    writer, step, tag, value, name=name, ctx=_ctx)
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/ops/gen_summary_ops.py", line 763, in write_scalar_summary_eager_fallback
    attrs=_attrs, ctx=_ctx, name=name)
  File "/run/media/federico/XData/virtualenvs/python36_tf2preview/lib/python3.6/site-packages/tensorflow/python/eager/execute.py", line 66, in quick_execute
    six.raise_from(core._status_to_exception(e.code, message), None)
  File "&lt;string&gt;", line 3, in raise_from
tensorflow.python.framework.errors_impl.NotFoundError: Resource localhost/logdir:./log/N10tensorflow22SummaryWriterInterfaceE does not exist. [Op:WriteScalarSummary] name: epoch_loss/
</code></pre></div>