One of the categories that has Azure Cognitive Services is Language. This group let us extract meaning from unstructured text. There are different services under the language category you can see them in this link. We are going to use from Language, the Translator API.
Microsoft Translator API is a neural machine translation service.
How can we use these services?
We can use it with a simple REST API call and we are going to see some examples specifically on how to implement Translator text. It supports more than 60 languages you can see the list of them in this link.
Translator can be used to:
- Automatically detect languages (detect the language of any text string, allowing you to quickly send for translation or serve localized content).
- Transliterate into different alphabets (Display text in different alphabets to make it easier to read).
- Look up words with the bilingual dictionary (Find alternative translations for words).
- Add online and offline translation support into your Android app.
- Build customized translation systems (Customize Microsoft Translator neural translation models to improve translation quality).
Do you need to use the custom translator?
It’s important to evaluate the need of a custom translator. There are different industries that have their own vocabulary in which the custom translator is the better option since a standard implementation would not be enough. We are talking about industries that can be very technical, for example Chemistry or Automotive industries. You should have preexisting documentations (product manuals, technical documents, etc), to train the model.
Let’s create a Custom Translator.
To create it we will use the secure Custom Translator portal, you can upload training data through an intuitive interface, train systems, test them right from the portal and deploy them to a production environment. We will build our custom neural machine translation models, using our own data to fit our style and terminology. Once we can deploy our custom system, we can use it for both text and speech translator.
1 – Go to Azure portal and create a Translation Text. We are going to use the Subscription Key later.
2 – Go to https://portal.customtranslator.azure.ai/ with your Azure account.
3 – Select the Settings and include your Translation Text Subscription Key.
4 – Select now Project > Create Project.
5 – Some of the fields to fill are the following, consider the Language Pair as the language from which to want to translate. Each project is created for a single pair. Finally you can also select the Category of the translator from a variety of items.
1.Upload – 2.Train – 3.Deploy – 4.Use
Now that we have our project. We need to use previously translated documents (leaflets, webpages, documentation, etc.) to build a translation system that reflects your domain-specific terminology and style, better than a standard translation system.
When you go to Documents > Upload Files. As a first option you will see the different options of Document Type accepted.
Training: The docs you upload as Training Docs are used by the Custom Translator as the basis for building your model. When you train, the sentences in the documents are aligned or paired. *You need at least 10.000 aligned parallel sentences needed to train a translator.
Testing (Optional): Parallel documents included in the testing set are used to compute the BLEU (Bilingual Evaluation Understudy) Score, which indicates the quality of the translation system.
Tuning (Optional): Documents are used by the Custom Translator to tune the translation system for optimal results. This data is used during training to adjust all parameters and weights.
Phrase Dictionary: When you add a PD to be considered, any word or phrase is translated in the way you specify.
Sentence Dictionary: Lets you specify an exact target translation for a source sentence.
I then will start uploading a Training Parallel Document.
1. Create Model
Ok so, click on Project > And select your project. Then click on the Create Model option. As mentioned before if you do not have minimum 10.000 sentences the model can not be created.
You can see once you select the +10.000 paired sentences the training cost.
You can now Create a Model.
Then we just have to wait until our Data is processed.
Once processing is done, we will have this. As you can see we already have the Training, Tuning and Test values.
Finally when Training finishes, you will see this. The model is ready to be deployed.
3. Deploy our Custom Translator
Once the training is done, click the button Deploy under Model Action. That will popup the following window in which you can select to which region you want to deploy, in our demo we will deploy only to north america.
Click on the System test results, and that will take you to test result details page. This page shows the machine translation of sentences that were part of the test dataset.
Click the Download Translations link to download a zip file. The zip contains the machine translations of source sentences in the test data set.
4. Use the Translator
Request url: https://api.cognitive.microsofttranslator.com /translate
api version: ?api-version=3.0
destination language: to=es
category ID : category=xxxxxxxxx-512a-4322-xxxx-xx918b923165-TOOLS *You need to put this in case you are working with a Custom Translator
Any framework or programming language able to make rest ful calls can go ahead and call our custom translator.
Lets try to use our custom translator. For that download Postman.
1- Go to your Project inside the Custom Translator Portal and copu de CategoryID.
2- Use postman to test.
We will see how to include the translator in an application in the next tutorial.