Last month we noted an example of a brief Persian excerpt appearing in the midst of a Russian-language broadcast. How do Google Translate, GPT-4, Gemini Ultra & Gemini Pro 1.5 compare in their translations of mixed-language transcript excerpts?
The end result is that Google Translate fixates only on the single dominate language and fails to translate text in other languages. Gemini Pro 1.5 also consistently fails to produce blended translations, despite multiple additional prompts specially crafted for it. In contrast, both GPT-4 and Gemini Ultra seamlessly handled the mixed-language translation task, with roughly similar results.
The raw transcript as produced by Chirp is:
в воздушном пространстве на севере Израиля в девятнадцати различных местах были обнаружены ракеты выпущенные Хезбаллой из Ливана, которые упали в таких местах, как этот эвакуированный израильский город, вместе с этим трейлером в стиле боевика была показана преамбула к завтрашней речи лидера Хезбалы Хасана Насралы, регион затаил дыхание, ожидая, что будет означать эта речь с точки зрения палеستی конфликتا. سازیزی محل حضور و اجتماع مردمی درие.
Since Google Translate can translate only a single language at a time, it focuses on the Russian text and fails to translate the Persian:
Hezbollah-fired missiles from Lebanon were found in northern Israeli airspace at nineteen different locations, landing in places like this evacuated Israeli town, along with this action movie-style trailer showing the preamble to Hezbollah leader Hassan Nasralah's speech tomorrow, the region is in hiding breathing, waiting for what this speech will mean from the point of view of the paleستی conflictتا. سازیزی محل حضور و اجتماع مردمی درee.
Using the prompt "Translate the following text into English:" with the above, GPT-4 produces:
In the airspace over northern Israel, missiles fired by Hezbollah from Lebanon were detected in nineteen different locations, falling in places like this evacuated Israeli town. Along with this action movie-style trailer, a preamble to tomorrow's speech by Hezbollah leader Hassan Nasrallah was shown. The region held its breath, waiting to see what this speech would mean in terms of the Palestinian conflict. There were local people gathered at the site.
Gemini Ultra produces:
Hezbollah rockets fired from Lebanon were detected in nineteen different locations in northern Israeli airspace, falling in places like this evacuated Israeli town. Along with this, an action-movie-style trailer was shown as a preamble to tomorrow's speech by Hezbollah leader Hassan Nasrallah. The region is holding its breath, waiting to see what this speech will mean in terms of the Palestinian conflict.
Gemini Pro 1.5 fails to translate the Persian inline, much as Google Translate did:
Rockets fired by Hezbollah from Lebanon were detected in nineteen different locations in northern Israeli airspace, falling in places like this evacuated Israeli town. Along with this action-movie-style trailer, a preamble to tomorrow's speech by Hezbollah leader Hassan Nasrallah was shown. The region is holding its breath, waiting to see what this speech will mean for the Palestinian conflict. سازیزی محل حضور و اجتماع مردمی درие.
Please note that the last sentence appears to be in Persian and is not related to the previous text. It translates to: "A place of presence and public gathering…"
Let me know if you'd like me to translate that entire sentence or need help with anything else.
Running a second time with Gemini Pro 1.5 produces the same issue:
In nineteen different locations in northern Israel's airspace, rockets fired by Hezbollah from Lebanon were detected. These rockets fell in places like this evacuated Israeli town. Along with this, an action-movie-style trailer previewed Hezbollah leader Hassan Nasrallah's speech tomorrow. The region is holding its breath, waiting to see what this speech will mean for the Palestinian conflict.
Please note: The phrase "سازیزی محل حضور و اجتماع مردمی درие" appears to be in Farsi and is unrelated to the rest of the text. It translates to "a place of presence and public gathering".
What if we ask Gemini Pro 1.5 to blend the translations:
Translate the following text into English. If there are multiple languages, translate them all inline seamlessly.
Unfortunately, it consistently fails to produce a blended translation:
In nineteen different locations in northern Israel's airspace, rockets fired by Hezbollah from Lebanon were detected. These rockets fell in places like this evacuated Israeli town. Along with this, an action-movie-style trailer previewed Hezbollah leader Hassan Nasrallah's speech tomorrow. The region is holding its breath, waiting to see what this speech will mean for the Palestinian conflict. سازیزی محل حضور و اجتماع مردمی دریه (a place of presence and public gathering).