Running multiple inferences in parallel on a GPU

tfmodelstate.cc is for master, it did not exist in v0.5.1. Also, make sure you’re reading the documentation for v0.5.1 as well. In the post above you linked to the master documentation. That could be why the change isn’t working for you, you followed the wrong steps maybe.