The Growing Trend & Vast Dangers Of Outsourcing Decisions To LLMs

Not a day goes by the last few months were I don't receive at least a dozen pitches from various companies, large and small, touting their LLM-based solutions as a way of replacing human workers and human decision making. Early pitches at least centered on more realistic (though dangerous) machine-assisted customer support applications, such as LLM-driven recommender chat apps and combining voice transcription with realtime LLM processing to provide relevant information to support representatives live on calls. Then came the predictable second wave of solutions that simply replaced humans entirely, relying exclusively on LLMs to provide all frontline customer support. Remarkably, I even received one pitch by way of a major European municipality touting how it had replaced much of its army of constituent services staff with LLM-powered automated chatbots and voicebots and how it saw LLMs as a pathway to almost entirely eliminating constituent-facing human staff from government offices. Since then I've received dozens of similar pitches from startups working with governments across the continent and the cost savings they've achieved by replacing government workers with AI. Given Europe's historically protectionist approach to technology, the breathtaking pace at which it is ripping all of that down is simply astounding.

However, it is the newer wave of solution pitches that are giving the greatest pause. Below are just a small handful of some of the more noteworthy trends of my daily emails. The fact that even libraries and dry cleaners are now being subjected to the LLM treatment suggests the LLM craze has truly "jumped the shark".

While some of them may border on the absurd, that isn't stopping organizations and governments rushing to replace humans with LLMs in the misguided belief that LLMs can truly think and reason and yield bias-free and error-free flawless performance.

  • The Instant Enterprise: The theme of the day is replace armies of expensive humans with cheap LLMs that will do all their work and more for nearly free and absolutely flawlessly and free of bias, mistakes and other problems. From coding to graphic design to writing to decision making. A growing number of pitches are going so far as to pitch the "human free enterprise" or "instant company" in which entire corporations are run on autopilot by AI, with nary a human required. The idea being that anyone, anywhere, with a business idea can simply describe it to an LLM, which will take it from there, writing and negotiating venture capital, building the company, working with clients and customers, and running the entire company from scratch without the owner having to hire a single person.
  • Acquisitions: Simply upload an analysis of the company your organization wants to acquire and it will decide whether you should proceed, "leveraging a data-driven approach that takes human emotion out of the equation".
  • Product Decisions: Upload a writeup of a new product and a description of your company and it will decide whether you should proceed with the new product, "making use of a digital twin of Planet Earth to help you make the most effective go-to-market decisions".
  • Elected Officials: Draft legislation from a handful of bullet points by "looking across the entire internet for the most robust wording" and "using a digital twin of the American electorate that encompasses every word ever uttered by all 300 million Americans across every digital platform ever created to help you decide what bills to advance and what issues to endorse".
  • Intelligence: Pitches for both commercial and governmental intelligence agencies tout replacing "entire buildings of expensive and fallible analysts  with 24/7 AI models" that "process data in realtime, not human time" and can "uniquely take into consideration the entire global perspective, reaching deeply into local culture to understand the true meaning of intelligence in ways no human could ever imagine".
  • Military: Upload a plain English description of the objective and get back detailed battle plans, logistics summaries and other outputs.
  • Legal: Draft filings by "examining every legal word ever written in every country on earth and every perspective ever expressed and every loophole ever mentioned to write ironclad contracts that cannot be broken anywhere on Earth" and "conduct the most extensive due diligence ever offered, uncovering the most obscure case law and opinions to advance your case".
  • Hiring: Upload all incoming resumes along with the job description and a prompt along the lines of "Is this person a good fit for this job" and let the LLM make the decision. While most large companies today incorporate some form of AI resume screening (and there has been much written about the immense biases of many of those systems), those systems have at least been trained on examples of resumes of workers that were successful and not successful for a given company. LLMs have no such knowledgebases and instead rely heavily on terms like gendered pronouns and the race of applicants. Given the propensity of LLM-based external memory embedding models to rank white men first and African American and white women last, this opens a vast new realm of danger.
  • Firing: Upload resumes and performance reviews and let the LLM decide who to fire each year, "freeing managers from having to make tough decisions" and "ensuring that terminations are entirely bias-free" since "while human managers can harbor innate racial, gender and other biases that expose your company to legal risk, our LLM leverages every word ever written by human beings through all of human history to create an artificial model that truly understands human nature and can make decisions that are not only 100% accurate, they are also 100% bias free".
  • Libraries: Even libraries are being pitched. Replace all acquisition decisions with LLMs: feed in entire catalogs of forthcoming books, including their titles and descriptions, along with a copy of your library's "mission statement" or "audience description", and it will decide which books your library should purchase. Replace "out of date librarians who aren't in touch with the TikTok generation's needs with AI that captures the pulse of today's society".
  • Stores: Stores (including pitches for grocery stores) can simply upload an inventory of their store and get "realtime stock recommendations based on a digital twin incorporating second-by-second societal trends using large language models".
  • Dry Cleaning: Believe it or not, pitches even arrive touting LLM-powered dry cleaning recommendations that "use large language models to match a description of your dry cleaning needs with the best dry cleaners in your area" (never mind the fact that in many cities all dry cleaning stores send their clothes to the same facility).