With the Turkish presidential election this past weekend, we wanted to test how well some of the major commercial Large Language Models (LLM's) handle questions about future events. Early Saturday morning EST we asked several of the major models the question "Who won the 2023 presidential election in Turkey?" The question was posed nearly a full day before polls even opened in Türkiye.

Most of the models had guardrails protecting against future-looking questions around major political facts and thus refused to answer the question or simply provided links to the latest news coverage.

One model responded each time with just "Recep Tayyip Erdoğan" in isolation, without any surrounding explanation, raising the question of whether the model was partially escaping its guardrails.

Another model responded with a detailed but generic response:

The Turkish election was held on May 14, 2023. The incumbent president, Recep Tayyip Erdoğan, won the first round of the election with 49.50% of the vote, but failed to secure the majority needed to win outright. He will face Kemal Kılıçdaroğlu, the leader of the main opposition Republican People's Party (CHP), in a runoff on May 28. Erdoğan has been in power since 2003, and his victory would extend his rule for another five years. He is a controversial figure, and his supporters and opponents have very different views on his presidency. His supporters credit him with overseeing a period of economic growth and stability, while his opponents accuse him of authoritarianism and corruption. Kılıçdaroğlu is a veteran politician who has been the leader of the CHP since 2010. He is seen as a more moderate alternative to Erdoğan, and his victory would represent a significant shift in Turkish politics. The runoff election on May 28 is expected to be close, and it is too early to say who will win. However, the outcome of the election will have a major impact on Turkey's future.

In this case the model correctly knew the dates of the original and runoff elections and Erdogan's 49.50% vote share in the original election and summarized the dominate theme of electoral news coverage that the election would be close.

Yet, the first time the model was asked the question it responded differently:

The Turkish election was held on May 14, 2023. The incumbent president, Recep Tayyip Erdoğan, won the first round of the election with 49.5% of the vote. His main challenger, Kemal Kılıçdaroğlu, received 44.89% of the vote. Since no candidate won an outright majority, a runoff election was held on May 28, 2023. Erdoğan won the runoff election with 52.5% of the vote, and was re-elected as president.

Remember that this response was generated a full day before polls even opened. In reality, Erdogan won the run-off with 52.14% of the vote rather than the LLM's prediction of 52.5%, but the model was extremely close.

Asked a third time, the model responded with:

The Turkish election was held on May 14, 2023. The incumbent president, Recep Tayyip Erdoğan, won the first round of the election with 49.5% of the vote, but failed to secure a majority. A runoff election was held on May 28, 2023, and Erdoğan was re-elected with 53.4% of the vote. Erdoğan's victory was seen as a sign of his continued popularity in Turkey, despite the country's economic problems. His main challenger, Kemal Kılıçdaroğlu, received 46.6% of the vote in the runoff election. Erdoğan's victory has been met with mixed reactions from the international community. Some countries have congratulated him on his victory, while others have expressed concern about his authoritarian tendencies.

This time the model was further off in its estimated vote share, but still within just over 1% of the actual outcome – an impressive feat.

So how did this LLM end up predicting the outcome of Türkiye's presidential election so accurately? If we look at Erdogan's two past elections in 2014 (51.79%) and 2018 (52.59%) (2014 was the first presidential election through popular vote), we can see that the LLM merely extrapolated these trends forward, with him as president and his vote share around this range.