With the General Availability of Chinese language support in Google's PaLM LLM, let's repeat our earlier tests of LLM-based translation against a human gold standard translation and see how it performs. As before, we'll use selected passages from a speech professionally translated by the Chinese government. Overall, the results are quite strong and highly fluent, though as with all LLM's the output is non-deterministic, with wording and transliterations changing from run to run.
Here is the PaLM invocation:
time curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/[YOURPROJECTID]/locations/us-central1/publishers/google/models/text-bison:predict -d \
$'{
"instances": [
{ "prompt": "Translate the following text into English. TEXT:\n尊敬的各位国家元首，政府首脑，各位国际组织负责人，女士们，先生们，朋友们：孟夏之日，万物并秀。在这美好时节，来自100多个国家的各界嘉宾齐聚北京，共商一带一路建设合作大计，具有十分重要的意义。今天，群贤毕至，少长咸集，我期待着大家集思广益、畅所欲言，为推动一带一路建设献计献策，让这一世纪工程造福各国人民。 女士们、先生们、朋友们！2000多年前，我们的先辈筚路蓝缕，穿越草原沙漠，开辟出联通亚欧非的陆上丝绸之路；我们的先辈扬帆远航，穿越惊涛骇浪，闯荡出连接东西方的海上丝绸之路。古丝绸之路打开了各国友好交往的新窗口，书写了人类发展进步的新篇章。中国陕西历史博物馆珍藏的千年“鎏金铜蚕”，在印度尼西亚发现的千年沉船“黑石号”等，见证了这段历史。 和平合作。公元前140多年的中国汉代，一支从长安出发的和平使团，开始打通东方通往西方的道路，完成了“凿空之旅”，这就是著名的张骞出使西域。中国唐宋元时期，陆上和海上丝绸之路同步发展，中国、意大利、摩洛哥的旅行家杜环、马可·波罗、伊本·白图泰都在陆上和海上丝绸之路留下了历史印记。15世纪初的明代，中国著名航海家郑和七次远洋航海，留下千古佳话。这些开拓事业之所以名垂青史，是因为使用的不是战马和长矛，而是驼队和善意；依靠的不是坚船和利炮，而是宝船和友谊。一代又一代“丝路人”架起了东西方合作的纽带、和平的桥梁。 开放包容。古丝绸之路跨越尼罗河流域、底格里斯河和幼发拉底河流域、印度河和恒河流域、黄河和长江流域，跨越埃及文明、巴比伦文明、印度文明、中华文明的发祥地，跨越佛教、基督教、伊斯兰教信众的汇集地，跨越不同国度和肤色人民的聚居地。不同文明、宗教、种族求同存异、开放包容，并肩书写相互尊重的壮丽诗篇，携手绘就共同发展的美好画卷。酒泉、敦煌、吐鲁番、喀什、撒马尔罕、巴格达、君士坦丁堡等古城，宁波、泉州、广州、北海、科伦坡、吉达、亚历山大等地的古港，就是记载这段历史的“活化石”。历史告诉我们：文明在开放中发展，民族在融合中共存。"}
],
"parameters": {
"temperature": 0.2,
"maxOutputTokens": 2048,
"topK": 40,
"topP": 0.95
}
}' > O; cat O | jq -r .predictions[0].content
Running three different times yields the following (you can compare against the human translation).
- Your Excellencies, Heads of State and Government, Heads of International Organizations, Ladies and Gentlemen, Friends, It is early summer, a time of luxuriant growth. At this beautiful time of the year, distinguished guests from over 100 countries have gathered in Beijing to discuss Belt and Road cooperation. This is of great significance. Today, with so many luminaries and talents present, young and old, I look forward to all of you brainstorming and speaking your minds, contributing your wisdom and counsel to advancing Belt and Road cooperation, so that this project of the century will benefit people of all countries. Ladies and Gentlemen, Friends, More than 2,000 years ago, our ancestors worked hard to open up the Silk Road, a land route linking Asia, Europe and Africa, through grasslands and deserts. They also set sail on long voyages, braving stormy seas, to open up the Maritime Silk Road connecting the East and the West. The ancient Silk Road opened a new window for friendly exchanges between countries and wrote a new chapter in human progress. The Silk Road treasures, such as the gilded bronze silkworm from the Shaanxi History Museum in China and the Black Stone shipwreck found in Indonesia, bear witness to this history. Peace and cooperation. More than 140 years before the Common Era, a peaceful mission from Chang'an, China, set out to open up a road from the East to the West, completing the "mission to open up the unknown regions". This was the famous mission of Zhang Qian to the Western Regions. During the Tang, Song and Yuan Dynasties in China, the Silk Road on land and at sea developed in parallel. The Chinese, Italian and Moroccan travelers Du Huan, Marco Polo and Ibn Battuta all left their footprints on the Silk Road. In the early 15th century during the Ming Dynasty, the renowned Chinese navigator Zheng He made seven voyages to the Western Seas, leaving behind many stories for generations to come. These pioneering endeavors have gone down in history not because they were accomplished with war horses and spears, but with camel trains and goodwill; not because they relied on battleships and cannons, but on treasure ships and friendship. Generation after generation, the "Silk Road people" have built a bond of cooperation and a bridge of peace between the East and the West. Openness and inclusiveness. The ancient Silk Road stretched across the basins of the Nile, the Tigris and Euphrates, the Indus and the Ganges, the Yellow River and the Yangtze River. It linked the birthplaces of the Egyptian, Babylonian, Indian and Chinese civilizations, the gathering places of Buddhists, Christians and Muslims, and the homes of peoples of different countries and colors. Different civilizations, religions and ethnic groups sought common ground while reserving differences, and embraced openness and inclusiveness. Together, they wrote a magnificent poem of mutual respect and painted a beautiful picture of common development. The ancient cities of Jiuquan, Dunhuang, Turpan, Kashgar, Samarkand, Baghdad and Constantinople, and the ancient ports of Ningbo, Quanzhou, Guangzhou, Beihai, Colombo, Jeddah and Alexandria are living fossils that record this history. History tells us that civilizations thrive on openness and nations thrive on inclusiveness.
- Your Excellencies, Heads of State and Government, Heads of International Organizations, Ladies and Gentlemen, Friends, It is early summer, a season of luxuriant growth. At this beautiful time of the year, distinguished guests from over 100 countries have gathered in Beijing to discuss Belt and Road cooperation. This is of great significance. Today, with so many luminaries and talents present, young and old, I look forward to your brainstorming and speaking your minds, contributing your wisdom and counsel to advancing Belt and Road cooperation and benefiting people of all countries through this project of the century. Ladies and Gentlemen, Friends, Over 2,000 years ago, our ancestors worked hard to open up the Silk Road that connected Asia, Europe and Africa, crossing grasslands and deserts. They set sail on the vast seas, braving stormy waves, and forged the Maritime Silk Road linking the East and the West. The ancient Silk Road opened a new window for friendly exchanges between countries and wrote a new chapter in human progress. The Silk Road treasures, such as the thousand-year-old鎏金铜蚕(Gilt Bronze Silkworms) in the Shaanxi History Museum in China and the Black Stone, a thousand-year-old shipwreck discovered in Indonesia, bear witness to this history. Peace and cooperation. In 140 BC, a peaceful mission from Chang'an, capital of China's Han Dynasty, set out to open up a route from the East to the West, completing the "Journey to the West". This was Zhang Qian's famous mission to the Western Regions. During the Tang, Song and Yuan Dynasties, both the overland and maritime Silk Roads thrived. Travelers from China, Italy and Morocco – Du Huan, Marco Polo and Ibn Battuta – all left their footprints on the Silk Road. In the early 15th century, during the Ming Dynasty, China's renowned navigator Zheng He made seven voyages to the Western Seas, leaving behind many stories for generations to come. These pioneering endeavors have gone down in history not because of the use of war horses and spears, but because of camel trains and goodwill; not because of battleships and cannons, but because of treasure ships and friendship. Generations of Silk Road travelers have built a bond of cooperation and a bridge of peace between the East and the West. Openness and inclusiveness. The ancient Silk Road stretched across the basins of the Nile, the Tigris and Euphrates, the Indus and the Ganges, the Yellow River and the Yangtze River. It linked the birthplaces of the Egyptian, Babylonian, Indian and Chinese civilizations; it linked the centers of Buddhism, Christianity and Islam; it linked countries and peoples of different colors. Different civilizations, religions and ethnic groups respected each other, pursued common ground while reserving differences, and embraced openness and inclusiveness. Together, they wrote a magnificent poem of mutual respect and painted a beautiful picture of common development. The Silk Road cities of Jiuquan, Dunhuang, Turpan and Kashgar in China; Samarkand in Uzbekistan; Baghdad in Iraq; and Constantinople (Istanbul) in Turkey; and the ancient ports of Ningbo, Quanzhou, Guangzhou and Beihai in China; Colombo in Sri Lanka; Jeddah in Saudi Arabia; and Alexandria in Egypt are living fossils that tell the story of this history. History tells us that civilizations thrive on openness and nations thrive on inclusiveness.
- Your Excellencies Heads of State and Government, Heads of International Organizations, Ladies and Gentlemen, Friends, It is early summer, a time of luxuriant growth for all things. At this beautiful time of the year, distinguished guests from over 100 countries have gathered in Beijing to discuss cooperation on the construction of the Belt and Road Initiative. This is of great significance. Today, with so many luminaries and talents gathered here, young and old, I look forward to everyone brainstorming and speaking freely, contributing your wisdom and counsel to advancing the construction of the Belt and Road Initiative, so that this project of the century will benefit people of all countries. Ladies and Gentlemen, Friends, More than 2,000 years ago, our ancestors worked hard to open up the Silk Road, a land route connecting Asia, Europe and Africa, through grasslands and deserts. They set sail on long voyages, braving stormy seas, and forged the Maritime Silk Road linking the East and the West. The ancient Silk Road opened a new window for friendly exchanges between countries and wrote a new chapter in human development and progress. The Silk Road treasures, such as the 1,000-year-old "Gilt Bronze Silk-making Machine" in the Shaanxi History Museum in China and the 1,000-year-old sunken ship "Black Stone" discovered in Indonesia, bear witness to this history. Peace and Cooperation. More than 140 years before the Common Era, a peaceful mission from Chang'an, China, set out to open up a road from the East to the West, completing the "Journey to the West". This was the famous mission of Zhang Qian to the Western Regions. During the Tang, Song and Yuan Dynasties in China, the Silk Road on land and at sea developed in parallel. Travelers from China, Italy and Morocco, including Du Huan, Marco Polo and Ibn Battuta, all left their footprints on the Silk Road on land and at sea. In the early 15th century during the Ming Dynasty, the renowned Chinese navigator Zheng He made seven voyages to the Western Seas, leaving behind stories that have been passed down through the ages. These pioneering endeavors have gone down in history not because they used war horses and spears, but because they used camel trains and goodwill; they relied not on warships and cannons, but on treasure ships and friendship. Generation after generation of "Silk Roaders" have built a bond of cooperation and a bridge of peace between the East and the West. Openness and Inclusiveness. The ancient Silk Road stretched across the basins of the Nile, the Tigris and Euphrates, the Indus and the Ganges, the Yellow River and the Yangtze River. It passed through the birthplaces of the Egyptian, Babylonian, Indian and Chinese civilizations, the gathering places of Buddhists, Christians and Muslims, and the homes of people of different countries and colors. Different civilizations, religions and ethnic groups sought common ground while reserving differences, embraced openness and inclusiveness, and jointly wrote a magnificent poem of mutual respect and painted a beautiful picture of common development. The ancient cities of Jiuquan, Dunhuang, Turpan, Kashi, Samarkand, Baghdad and Constantinople, and the ancient ports of Ningbo, Quanzhou, Guangzhou, Beihai, Colombo, Jeddah and Alexandria are living fossils that record this history. History tells us that civilizations thrive on openness and nations thrive on inclusiveness.
For comparison, here is ChatGPT's translation:
Dear Heads of State, Government Leaders, Heads of International Organizations, Ladies and Gentlemen, and Friends:
On this beautiful day in midsummer, when all things flourish, it is of great significance that distinguished guests from over 100 countries have gathered in Beijing to discuss the cooperative endeavor of the Belt and Road Initiative. Today, with the convergence of wisdom and the coming together of people of all ages, I look forward to everyone sharing their thoughts freely and contributing to the advancement of the Belt and Road Initiative. Let us work together to ensure that this project of the century brings benefits to people in all participating countries.
Ladies and Gentlemen, Friends,
Over 2,000 years ago, our ancestors, through perseverance and determination, forged the overland Silk Road, connecting Asia, Europe, and Africa. They also embarked on maritime voyages, overcoming turbulent seas, to establish the maritime Silk Road linking the East and the West. The ancient Silk Road opened new avenues for friendly exchanges between nations and wrote a new chapter in the progress of human civilization. Historical artifacts such as the millennium-old "Gilded Bronze Silkworm" from the Shaanxi History Museum in China and the millennium-old sunken ship "Black Stone" discovered in Indonesia bear witness to this history.
Peace and cooperation have been the hallmarks of these endeavors. In the Han Dynasty more than 140 years before the Common Era, a peace envoy set out from Chang'an, opening the route from the East to the West, known as the "Journey through the Desert," led by the famous envoy Zhang Qian. During the Tang, Song, and Yuan Dynasties of China, both the overland and maritime Silk Roads thrived simultaneously, with travelers such as Du Huan from China, Marco Polo from Italy, and Ibn Battuta from Morocco leaving their marks along these routes. In the early 15th century, the renowned Chinese navigator Zheng He embarked on seven epic voyages, leaving behind timeless tales of friendship. These pioneering efforts are celebrated in history not because of warhorses and weapons but because of camels and goodwill, not because of battleships and cannons but because of treasure ships and friendship. Generation after generation of "Silk Road travelers" built bridges of cooperation between the East and the West and became beacons of peace.
Openness and inclusivity have been the defining features of the ancient Silk Road. It traversed the Nile River Valley, the Tigris and Euphrates River Valley, the Indus and Ganges River Valley, the Yellow River and Yangtze River Valley. It crossed the cradle of civilizations in Egypt, Mesopotamia, India, and China. It spanned gathering places for followers of Buddhism, Christianity, and Islam. It connected people of different nations and ethnicities, living in diverse regions with distinct skin colors. Different civilizations, religions, and races sought common ground while respecting differences, creating a magnificent epic of mutual respect and cooperation. Cities like Dunhuang, Turpan, Kashgar, Samarkand, Baghdad, and Constantinople, as well as ancient ports like Dunhuang, Quanzhou, Guangzhou, Beihai, Colombo, Jeddah, and Alexandria, serve as living witnesses to this history. History tells us that civilization thrives through openness, and nations coexist through integration.
And Google Translate:
Distinguished heads of state, heads of government, heads of international organizations, ladies and gentlemen, friends: On the day of Mengxia, everything is beautiful. In this wonderful time, guests from all walks of life from more than 100 countries gather in Beijing to discuss cooperation plans for the Belt and Road Initiative, which is of great significance. Today, all the talented people have arrived, and the young and the old have gathered together. I look forward to everyone bringing together their wisdom and speaking freely to contribute ideas and suggestions to promote the construction of the Belt and Road Initiative so that this century-old project can benefit the people of all countries. Ladies and gentlemen, friends! More than 2,000 years ago, our ancestors made long journeys across grasslands and deserts, opening up the land Silk Road connecting Asia, Europe and Africa; our ancestors set sail, crossed rough seas, and created the Maritime Silk Road connecting the East and the West. The ancient Silk Road opened a new window for friendly exchanges between countries and wrote a new chapter in human development and progress. The thousand-year-old "gilt copper silkworm" collected by China's Shaanxi History Museum and the thousand-year-old shipwreck "Black Stone" discovered in Indonesia bear witness to this history. Peaceful cooperation. In the Han Dynasty of China in more than 140 BC, a peace mission from Chang'an began to open up the road from the east to the west and completed the "hollowing journey". This was the famous Zhang Qian's mission to the Western Regions. During the Tang, Song and Yuan Dynasties in China, the land and maritime Silk Roads developed simultaneously. Du Huan, Marco Polo, and Ibn Battuta, travelers from China, Italy, and Morocco, all left their mark on the land and maritime Silk Roads. a historical mark. In the Ming Dynasty in the early 15th century, Zheng He, the famous Chinese navigator, made seven ocean voyages, leaving a lasting legacy. The reason why these pioneering undertakings are famous in history is that they used not war horses and spears, but camel teams and goodwill; they relied not on strong ships and sharp cannons, but on treasure ships and friendship. Generation after generation of "Silk Road people" have built a bond of cooperation between the East and the West and a bridge of peace. Be open and inclusive. The ancient Silk Road spanned the Nile River Basin, Tigris River and Euphrates River Basin, Indus River and Ganges River Basin, Yellow River and Yangtze River Basin, spanned the birthplaces of Egyptian civilization, Babylonian civilization, Indian civilization, and Chinese civilization, and spanned the gathering of believers in Buddhism, Christianity, and Islam. A place where people of different nationalities and colors live together. Different civilizations, religions and races seek common ground while reserving differences, are open and inclusive, write magnificent poems of mutual respect side by side, and work together to create a beautiful picture of common development. Ancient cities such as Jiuquan, Dunhuang, Turpan, Kashgar, Samarkand, Baghdad, and Constantinople, as well as ancient ports in Ningbo, Quanzhou, Guangzhou, Beihai, Colombo, Jeddah, and Alexandria, are the "living fossils" that record this period of history. ". History tells us: civilization develops through openness, and nations coexist through integration.