Accuracy issues on RPi


(Foti Dim) #1

Testing with the new binaries on RPi that @lissyx provided I am having accuracy issues. I am testing with the following:
Binaries
Sample model
2830-3980-0043.wav file which can be found here

I didn’t try the exact same files on x86 Ubuntu but using the same audio file with those model files gives accurate results.


(Lissyx) #2

Wait, I did not noticed you were still using the output_graph.pb from TaskCluster ; this is an overfitted model exclusively dedicated to test, it has no purpose and it is actually expected to perform bad on anything different than LDC93S1 :slight_smile:


(Foti Dim) #3

Is there a set of files that is supposed to work on the Pi then? Shall I wait for the next release?


(Lissyx) #4

They are on https://github.com/mozilla/DeepSpeech/releases/tag/v0.1.0


(Foti Dim) #5

But these are the files I had the error with. Did you update them in the meantime?


(Lissyx) #6

The error were related to the binaries published, not the model which should be fine: output_graph.pb.


(Lissyx) #7

@fotiDim Can you check that, please ? There may be multiple reasons for bad results, but as long as you use our model and samples you should get good results.


(Foti Dim) #8

@lissyx I am fairly sure that are exactly the files that I tried on the first place when you provided the working binary. But just to double check I will give it another go within the weekend so please bear with me until then.


(Lissyx) #9

@fotiDim I just re-tested, and I get the same result on my RPi3 and on my laptop on your same sample:
On my laptop:

alex@portable-alex:~/tmp/deepspeech$ ./deepspeech models/output_graph.pb audio/2830-3980-0043.wav models/alphabet.txt -t
2017-12-20 12:43:52.266225: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
experience proves les
cpu_time_overall=8.36112 cpu_time_mfcc=0.00293 cpu_time_infer=8.35819

RPi3:

pi@raspberrypi:~/models-release $ ~/native_client_arm/deepspeech models/output_graph.pb audio/2830-3980-0043.wav models/alphabet.txt -t
experience proves les
cpu_time_overall=82.25488 cpu_time_mfcc=0.02952 cpu_time_infer=82.22536

(Foti Dim) #10

@lissyx sorry for the delay. Life stroke in the meantime. So, there is apparently a difference with our setups.

pi@raspberrypi:~/DeepSpeech/native_client_2 $ ./deepspeech ../models/output_graph.pb ../audio/2830-3980-0043.wav ../models/alphabet.txt -t
Killed

Also you are getting experience proves les. On my PC I was getting experience proves this which happens to be the right text. Not sure why you are getting inaccuracy.

Let me know if I can run another test or trace some logs somewhere.


(Lissyx) #11

Don’t feel sorry for the delay :/. But as you can see in my post above, I don’t have any problem on RPi3, so I don’t know what is wrong on your side. Can you ensure that:

  • it’s RPi3, not something lower end?
  • you have enough free memory?
  • you have good enough PSU?

(Foti Dim) #12
  • It’s “Raspberry Pi 3 Model B Rev 1.2”.
  • On idle I have 697256 free ram. which seems good.
  • PSU is 2.5A specially made for RPi3.

Do you use Raspbian or Raspbian Lite? Is it possible to upload your image somewhere so I can give it a try?


(Lissyx) #13

697M free? That seems not that much. Check your PSU, even if it’s “specially made for RPi3”, I got funny behaviors with some poor one.

It’s Raspbian Jessie, but it should not have an impact:

pi@raspberrypi:~ $ lsb_release -a
No LSB modules are available.
Distributor ID:	Raspbian
Description:	Raspbian GNU/Linux 8.0 (jessie)
Release:	8.0
Codename:	jessie

You latest output, with “Killed” would suggest memory issue?


(Lissyx) #14

FTR:

pi@raspberrypi:~ $ cat /proc/meminfo 
MemTotal:         994232 kB
MemFree:          377192 kB
MemAvailable:     904356 kB
Buffers:           68536 kB
Cached:           505380 kB
SwapCached:         1160 kB
Active:           287996 kB
Inactive:         291256 kB
Active(anon):       3112 kB
Inactive(anon):    12696 kB
Active(file):     284884 kB
Inactive(file):   278560 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:        102396 kB
SwapFree:          89380 kB
Dirty:                24 kB
Writeback:             0 kB
AnonPages:          4696 kB
Mapped:            11040 kB
Shmem:             10472 kB
Slab:              27556 kB
SReclaimable:      17548 kB
SUnreclaim:        10008 kB
KernelStack:        1000 kB
PageTables:          604 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      599512 kB
Committed_AS:      76464 kB
VmallocTotal:    1064960 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
CmaTotal:           8192 kB
CmaFree:            6796 kB
pi@raspberrypi:~ $