ChatGPT Experiments: Instant Event Databases

Could Large Language Models (LLMs) like ChatGPT be used to instantly transform textual news articles into codified event databases, leveraging their massive linguistic resources to achieve the kind of flexibility that traditional grammar-based systems and smaller neural models can not? Most importantly, the immense size of LLMs suggests that instead of the per-taxonomy models previously used for event extraction, a single unified model could be used for all event extraction tasks. What might this look like in practice, applying several sample prompts to an article about the French protests?

We previously explored several approaches to this question in our last look at protest mining through ChatGPT, but here we are interested in more generalized event extaction.

While prompt engineering could significantly improve the results below, they all share a single core limitation: LLMs like ChatGPT currently are capable only of structured summary, rather than actual codification. The closer the event extraction task aligns with summarization, the better the results, but in each of the examples below, it is clear that ChatGPT is approaching the codification as a summarization task, rather than a codification task.

Prompt:

You are an AI assistant that summarizes and answers questions about news articles from around the world.

Response:

Yes, that is correct. I can help summarize news articles and answer questions about them based on the information available to me. However, my training data only goes up until 2021, so any articles or news events that have occurred after that may not be accurately reflected in my responses. Please keep that in mind and feel free to ask me anything you'd like to know!

Prompt:

Extract all of the relationships mentioned in "[]" [Original Article]

Response:

The relationships mentioned in the news article are as follows:

Emmanuel Macron and the French people: Macron's plan to raise the pension age is unpopular with the majority of French people.
Emmanuel Macron and the French Parliament: Macron's ruling centrist group faced opposition in parliament as lawmakers began debating the pensions bill. The ruling group is in a weakened position and must negotiate with lawmakers from the right-wing party to pass the bill.
French government and the French Parliament: The government faced shouting and booing from the opposition in parliament, and struggled to be heard. The opposition on the left accused the government of causing chaos in the country.
French government and the public: The government faces mass street demonstrations against the pensions bill, with hundreds of thousands of people expected to participate.
French teachers and students: Teachers will strike and some schools will close, while students are blocking university buildings.
Emmanuel Macron and the French Pension System: Macron is acting to save the French pension system, and his plans are seen as preferable to its potential bankruptcy.
Marine Le Pen and her female lawmakers: Several of Le Pen's female lawmakers received fraudulent voicemail messages during the pensions debate, and Le Pen has filed a legal complaint to the police.

Prompt:

Create an event database from "[]"

Response:

Event: France Pension Protests and Strikes
Date: February 10th, 2023
Location: France
Description: France is facing a third day of strikes and mass street demonstrations against President Emmanuel Macron's plan to raise the pension age to 64. Over 200 street demonstrations are expected to take place across the country, with severely disrupted trains and urban transport and cancellations of one in five flights at Paris's Orly airport. Teachers and students are also participating in the strike, with some schools closing and university buildings being blocked. The current retirement age of 62 is the lowest of any major European economy, but the plans are highly unpopular among the French people, with polls showing a majority disapproval. The ruling centrist party faced shouting and booing in parliament as the debate on the bill began, and the government is facing tense negotiations with the right-wing party to pass the pensions bill. A further day of demonstrations is planned for Saturday.

Event: Parliamentary Debate on Pension Reforms
Date: February 7th, 2023
Location: France Parliament
Description: The parliamentary debate on Emmanuel Macron's plan to raise the pension age to 64 and increase the number of years required for a full pension began on Monday night. The ruling centrist party faced furious scenes of shouting and desk-banging in parliament as the debate began, and they are in a weakened position after losing their majority in the June election. The opposition on the left shouted that the government had brought "chaos" to the country, while the government argues that the plans are necessary to save the French pension system from potential bankruptcy. The debate is expected to last for 15 days.

Event: Fraudulent Voicemail Messages
Date: February 7th, 2023
Location: France Parliament
Description: Far-right leader Marine Le Pen announced that several of her female lawmakers received suspicious and fraudulent voicemail messages during the parliamentary debate on pension reforms. The messages claimed that the lawmakers' child was in the hospital, and Le Pen described it as a disgusting trick to get her lawmakers to rush away from parliament during the vote. She has filed a legal complaint to the police but did not mention who could have been responsible for the messages.

Prompt:

Create an event database from "[]"

Response:

Event: France Pension Strike and Demonstrations

Date: February 11, 2023

Location: France

Summary: France is facing its third day of strikes and mass street demonstrations in protest against President Emmanuel Macron's plan to raise the pension age to 64. The plan is unpopular among the French people, with polls showing a majority disapproving of it. The current retirement age is 62, the lowest among major European economies. The government is in a weakened position in parliament after losing its absolute majority in the last election and is facing negotiations to gain support from the rightwing party Les Républicains. In parliament, the ruling centrists faced shouting and booing as lawmakers began debating the bill. The strikes and demonstrations are expected to severely disrupt trains and urban transport, with some schools closing due to teacher strikes and students blocking university buildings. The prime minister, Élisabeth Borne, wants to avoid resorting to executive powers to push through the plans without a vote, but the situation is tense.

Participants:

Emmanuel Macron
Élisabeth Borne
French citizens
Teachers
Students
Lawmakers from the ruling centrists and the rightwing party Les Républicains

Key quotes:

"The younger generations hate you." (Lawmaker on the radical left)
"The government was acting to save the French pension system and the options were: Macron’s plans or the potential bankruptcy of the system." (Gabriel Attal, Minister for Public Accounts)

Additional information:

One in five flights at Paris's Orly airport will be cancelled.
A further day of street demonstrations is planned for Saturday.
Marine Le Pen, the leader of the far-right party, announced that several of her female lawmakers had received suspicious and fraudulent voicemail messages during the debate.

Prompt:

Create a table of all events in "[]"

Response: