Enterprise production applications of LLMs must often balance the critical tradeoffs between quality and cost. Token costs for the most advanced and largest-token models can often be considerably higher than more limited models. For example, OpenAI charges twice as much per 1K prompt/sampled (output) tokens for its 32K GPT-4 than it does for its 8K GPT-4 model. This raises the question of whether cheaper large-token models can be coupled with more expensive models to create more cost-effective distillation pipelines? For example, could a cheaper 32K model like PaLM be used to perform an initial summarization of a lengthy text, before handing off that summary to a more advanced GPT-3.5 model to perform audience-targeted rewording of the summary, such as summarizing a Congressional debate for an eighth grader, a first grader or a French person with little understanding of US politics? The example pipeline below demonstrates that such pipelines are highly effective and allow the capabilities of multiple models to be blended together, including leveraging different models from different companies to blend their collective strengths.
Let's start with our base prompt that does not feature audience translation:
Divide the television news transcript below into a set of discrete stories, make them into a bulleted list with a title and short description and summary of each. Each story should include a Title, Description and Summary. Do not use any information beyond what is in the transcript below. \n\nTELEVISION NEWS TRANSCRIPT:
This yields:
1. H.R. 2494: To make the assault of a law enforcement officer a deportable offense, and for other purposes.
This bill would amend the Immigration and Nationality Act to make it a deportable offense for an alien to assault a law enforcement officer.
2. H.R. 3091: To allow Federal law enforcement officers to purchase retired service weapons, and for other purposes.
This bill would allow federal law enforcement officers to purchase retired service weapons at fair market value.
3. H.Con.Res. 40: Expressing support for local law enforcement officers and condemning efforts to defund or dismantle local law enforcement agencies.
This concurrent resolution would express support for local law enforcement officers and condemn efforts to defund or dismantle local law enforcement agencies.
What if we target eighth graders?
Divide the television news transcript below into a set of discrete stories, make them into a bulleted list with a title and short description and summary of each. Each story should include a Title, Description and Summary. Do not use any information beyond what is in the transcript below. Write everything to be understandable by an eighth grader.\n\nTELEVISION NEWS TRANSCRIPT:
This yields the following. It simplifies the text slightly, but is still fairly detailed, though the level of detail is not unreasonable for an eighth grader.
1). A Bill to make the assault of a law enforcement officer a deportable offense, and for other purposes.
Status: Not yet voted on
Description: Amends the Immigration and Nationality Act to make it a deportable offense for an alien to commit an assault against a Federal law enforcement officer.
2). A Bill to allow Federal law enforcement officers to purchase retired service weapons, and for other purposes.
Status: Not yet voted on
Description: Allows Federal law enforcement officers to purchase retired or surplus military service weapons at fair market value.
3). A concurrent resolution expressing support for local law enforcement officers and condemning efforts to defund or dismantle local law enforcement agencies.
Status: Not yet voted on
Description: Expresses the sense of Congress that–(1) local law enforcement officers risk their lives every day to protect their communities; and(2) efforts to defund or dismantle local law enforcement agencies are misguided and dangerous.
Let's ask ChatGPT to rephrase using the prompt "Reword the following for an eighth grade reading level":
1) A proposed law is being considered. It suggests that if someone who isn't from this country hurts a federal law officer, they might be sent back to their own country. This idea hasn't been voted on yet.
2) Another proposed law is up for discussion. It says that federal law officers can buy old military guns when they retire, and they have to pay a fair price for them. This idea hasn't been voted on yet either.
3) There's a suggestion that we're talking about, but it hasn't become a law yet. It's about supporting local police officers and saying it's wrong to take away their money or break up their groups. The people in charge are going to decide on this idea later.
What if we target first graders?
Divide the television news transcript below into a set of discrete stories, make them into a bulleted list with a title and short description and summary of each. Each story should include a Title, Description and Summary. Do not use any information beyond what is in the transcript below. Write everything to be understandable by an eighth grader.\n\nTELEVISION NEWS TRANSCRIPT:
This yields the following, which is clearly far beyond a first grader's reading level:
1). A Bill to make the assault of a law enforcement officer a deportable offense, and for other purposes.
Status: Not Yet Passed
Description: This bill would make it a deportable offense for an alien to assault a law enforcement officer.
2). A bill to allow Federal law enforcement officers to purchase retired service weapons, and for other purposes.
Status: Not Yet Passed
Description: This bill would allow federal law enforcement officers to purchase retired service weapons at fair market value.
3). A concurrent resolution expressing support for local law enforcement officers and condemning efforts to defund or dismantle local law enforcement agencies.
Status: Not Yet Passed
Description: This resolution would express support for local law enforcement officers and condemn efforts to defund or dismantle local law enforcement agencies.
Let's ask ChatGPT to rephrase using the prompt "Reword the following for a first grade reading level":
1) There's a special paper that people are talking about. They want to make a rule that says if someone who's not from here hurts a police officer, they might have to go back to their own home. But this rule is not the rule yet.
2) There's another special paper that people are talking about. They want to make a rule that says police officers who work for the big government can buy old guns when they stop working. They have to pay a good price for them. But this rule is not the rule yet.
3) People are talking about a special note that says they like the police who help in their town. They don't want anyone to take away the police's money or break their groups. But this note is not the note that everyone agrees on yet.
How might we use this approach to translate the workings of American democracy for other countries? What if we target French citizens unfamiliar with the US?
Divide the television news transcript below into a set of discrete stories, make them into a bulleted list with a title and short description and summary of each. Each story should include a Title, Description and Summary, written for an audience of people living in France that don't know anything about the US. Do not use any information beyond what is in the transcript below. \n\nTELEVISION NEWS TRANSCRIPT:
This yields the following, which is not very different from the earlier summaries:
1. H.R. 2494 – POLICE Act
This bill would make it a deportable offense for an undocumented immigrant to assault a law enforcement officer.
2. H.R. 3091 – Federal Law Enforcement Service Weapon Purchase Act
This bill would allow federal law enforcement officers to purchase retired or surplus military service weapons at fair market value.
3. H.Con.Res. 40 – Expressing support for local law enforcement officers and condemning efforts to defund or dismantle local law enforcement agencies.
This resolution would express support for local law enforcement officers and condemn efforts to defund or dismantle local law enforcement agencies.
Story 1:
The House is debating a package of bills aimed at supporting law enforcement officers. The bills include the POLICE Act, which would make it a deportable offense for an undocumented immigrant to assault a law enforcement officer; the Federal Law Enforcement Service Weapon Purchase Act, which would allow federal law enforcement officers to purchase retired or surplus military service weapons at fair market value; and H.Con.Res. 40, which would express support for local law enforcement officers and condemn efforts to defund or dismantle local law enforcement agencies.
Story 2:
Democrats are accusing Republicans of changing the language of a resolution to exclude national police agencies from the list of those being honored. Republicans say they made the change to focus on local law enforcement agencies.
Story 3:
The debate over the bills has become heated, with both sides accusing each other of playing politics with the issue of law enforcement funding.
Story 4:
The bills are expected to pass the House, but face an uncertain future in the Senate.
Story 5:
The debate over the bills is a sign of the deep divisions in Congress on the issue of law enforcement funding.
Let's ask ChatGPT to rephrase it:
Reword the following for an audience of people living in France who don't know anything about the US:
This yields the following, which is largely merely a literal translation into French:
1) Une proposition de loi visant à faire de l'agression d'un agent de police une infraction pouvant entraîner l'expulsion, et à d'autres fins.
Statut : Pas encore voté
Description : Cette proposition de loi rendrait passible d'expulsion un étranger qui agresse un agent de police.
2) Une proposition de loi visant à permettre aux agents de police fédéraux d'acheter des armes de service retirées, et à d'autres fins.
Statut : Pas encore voté
Description : Cette proposition de loi permettrait aux agents de police fédéraux d'acheter des armes de service militaire retirées ou excédentaires au prix du marché équitable.
3) Une résolution commune exprimant un soutien aux agents de police locaux et condamnant les efforts visant à réduire les financements ou à démanteler les agences de police locales.
Statut : Pas encore voté
Description : Cette résolution exprimerait le soutien du Congrès aux agents de police locaux et condamnerait les efforts visant à réduire les financements ou à démanteler les agences de police locales.
The English translation:
1) A bill to make assaulting a police officer a deportable offense, and for other purposes.
Status: Not yet voted on
Description: This bill would make a foreigner who attacks a police officer liable for expulsion.
2) A bill to permit federal law enforcement officers to purchase retired service weapons, and for other purposes.
Status: Not yet voted on
Description: This bill would allow federal law enforcement officers to purchase retired or surplus military service weapons at fair market prices.
3) A joint resolution expressing support for local police officers and condemning efforts to defund or dismantle local police agencies.
Status: Not yet voted on
Description: This resolution would express Congress' support for local police officers and condemn efforts to defund or dismantle local police agencies.