AppTek, a leader in Artificial Intelligence, Machine Learning, Automatic Speech Recognition and Machine Translation, announced that as of this week, AppTek’s Neural Network environment RETURNN supports PyTorch for efficient model training. This adds to the company’s already existing support for the open source framework Theano, which was spearheaded and supported by Yoshua Bengio and his MILA team until September 2017, and the open source framework Tensorflow, where the maintenance, support and further improvement is led and shepherded by the Google Brain team. PyTorch is an open source ML framework that is led and supported by Facebook and is seeing a fast and wide adoption by AI researchers and engineers, allowing for rapid experimentation and easy system development and deployment.
“We are happy to announce that we successfully integrated PyTorch as a third back-end into our acoustic model training software RETURNN. Just as with our Tensorflow and Theano back-end, the PyTorch version allows users to train state-of-the-art acoustic LSTM models using PyTorch modules.”
AppTek’s integration with PyTorch had a special focus on human language technology, and speech recognition in particular. The performance of the models trained on the PyTorch framework is similar or better compared to the already excellent performance of models trained with the other frameworks. The combination of RETURNN and PyTorch allows high scalability, utilizing high degrees of parallelization to either process high amounts of data simultaneously, or increase the throughput.
“We are excited to see the power of RETURNN unfold using the PyTorch back-end, we believe that RETURNN will bring benefits to scientists who do rapid product development. The need to up the rate of innovation is critical to serve practical use cases and enable the research community. We see that combining the rate of innovation in modeling on RETURNN and the rate of innovation on the core machine learning libraries of PyTorch can deliver that,” states Mudar Yaghi, CEO of AppTek.
Hermann Ney, Director of Science at AppTek and Computer Science Chair for Human Language Technology and Pattern Recognition at RWTH Aachen adds: “We are looking forward to identify ways to collaborate and push the limits even further. It proved in the past that bringing together different disciplines divulges state of the art in research. We had Machine Learning, Signal Processing, Speech Recognition, Machine Translation, Handwriting Recognition and the new initiative in speech synthesis scientists work together to build RETURNN to enable the building of best-in-class systems.”
AppTek’s leading scientist who led this effort, Patrick Doetsch, said: “We are happy to announce that we successfully integrated PyTorch as a third back-end into our acoustic model training software RETURNN. Just as with our Tensorflow and Theano back-end, the PyTorch version allows users to train state-of-the-art acoustic LSTM models using PyTorch modules.”