Cannot convert a symbolic Tensor (tower_0/cudnn_lstm/strided_slice_1:0) to a numpy array error

I have been trying to fine tune the DeepSpeech 0.9.3 with my custom data.I ahve gone through the official doc but at the training stage I get this error and I dont know how to proceed?

  /content/DeepSpeech
I0402 14:04:17.638820 140259800721280 utils.py:157] NumExpr defaulting to 2 threads.
Traceback (most recent call last):
  File "DeepSpeech.py", line 12, in <module>
    ds_train.run_script()
  File "/content/DeepSpeech/training/deepspeech_training/train.py", line 982, in run_script
    absl.app.run(main)
  File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 303, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/content/DeepSpeech/training/deepspeech_training/train.py", line 954, in main
    train()
  File "/content/DeepSpeech/training/deepspeech_training/train.py", line 484, in train
    gradients, loss, non_finite_files = get_tower_results(iterator, optimizer, dropout_rates)
  File "/content/DeepSpeech/training/deepspeech_training/train.py", line 317, in get_tower_results
    avg_loss, non_finite_files = calculate_mean_edit_distance_and_loss(iterator, dropout_rates, reuse=i > 0)
  File "/content/DeepSpeech/training/deepspeech_training/train.py", line 244, in calculate_mean_edit_distance_and_loss
    logits, _ = create_model(batch_x, batch_seq_len, dropout, reuse=reuse, rnn_impl=rnn_impl)
  File "/content/DeepSpeech/training/deepspeech_training/train.py", line 195, in create_model
    output, output_state = rnn_impl(layer_3, seq_length, previous_state, reuse)
  File "/content/DeepSpeech/training/deepspeech_training/train.py", line 133, in rnn_impl_cudnn_rnn
    sequence_lengths=seq_length)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/layers/base.py", line 548, in __call__
    outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/keras/engine/base_layer.py", line 854, in __call__
    outputs = call_fn(cast_inputs, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/autograph/impl/api.py", line 237, in wrapper
    raise e.ag_error_metadata.to_exception(e)
NotImplementedError: in converted code:

    /usr/local/lib/python3.7/dist-packages/tensorflow_core/contrib/cudnn_rnn/python/layers/cudnn_rnn.py:427 call
        initial_state = self._zero_state(batch_size)
    /usr/local/lib/python3.7/dist-packages/tensorflow_core/contrib/cudnn_rnn/python/layers/cudnn_rnn.py:452 _zero_state
        res.append(array_ops.zeros(sp, dtype=self.dtype))
    /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/ops/array_ops.py:2338 zeros
        output = _constant_if_small(zero, shape, dtype, name)
    /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/ops/array_ops.py:2295 _constant_if_small
        if np.prod(shape) < 1000:
    <__array_function__ internals>:6 prod
        
    /usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py:3031 prod
        keepdims=keepdims, initial=initial, where=where)
    /usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py:87 _wrapreduction
        return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py:736 __array__
        " array.".format(self.name))

    NotImplementedError: Cannot convert a symbolic Tensor (tower_0/cudnn_lstm/strided_slice_1:0) to a numpy array.[quote="Shalini_NA, post:1, topic:78109, full:true"]
1 Like
1 Like

(not providing support)

I’m having the same error and I looked at your past comments and it seems like you’ve eventually moved past this error. I was wondering if you could share how you wound up solving it