We use a convolutional neural network, training it on letters and non-letters so it can learn what different letters look like.īut interestingly, if we train just on very “clean”-looking letters, we risk not understanding what real-life letters look like. Second, Translate has to recognize what each letter actually is. Those are possibly letters, and if they’re near each other, that makes a continuous line we should read. It looks at blobs of pixels that have similar color to each other that are also near other similar blobs of pixels. It needs to weed out background objects like trees or cars, and pick up on the words we want translated. Here’s how.įirst, when a camera image comes in, the Google Translate app has to find the letters in the picture.

And the amazing part is it can all work on your phone, without an Internet connection. Yes, they’re good for more than just trippy art-if you're translating a foreign menu or sign with the latest version of Google's Translate app, you're now using a deep neural net. Thanks to convolutional neural networks, not only can computers tell the difference between cats and dogs, they can even recognize different breeds of dogs. Five years ago, if you gave a computer an image of a cat or a dog, it had trouble telling which was which. Neural nets have gotten a lot of attention in the last few years because they’ve set all kinds of records in image recognition. When the Word Lens team joined Google, we were excited for the opportunity to work with some of the leading researchers in deep learning. But how are we able to recognize these new languages?

So the next time you’re in Prague and can’t read a menu, we’ve got your back. Today we announced that the Google Translate app now does real-time visual translation of 20 more languages.

Posted by Otavio Good, Software Engineer, Google Translate

Google translate app