As generative AI becomes more capable, it becomes more potentially valuable to both organizations and each of us individually. And yet how to best use large language models and the GenAI tools based on them continues to be a puzzle.
Despite its ubiquity, generative AI is still young. So it’s no surprise that many leaders are struggling with important questions about using the tools efficiently, responsibly, and skeptically. The questions are significant: Can we trust it? What should our strategy be in deploying it and scaling success? Who should be using it? How will we measure its ROI? When should we use other types of AI, like machine learning, instead?
MIT Sloan Management Review authors on this topic — including academics, researchers, and practitioners — are reporting from the front lines about the best GenAI questions to ask and how to think about answering them.
Take our columnist Rama Ramakrishnan, professor of the practice at the MIT Sloan School of Management. He writes that many of his executive students who use GenAI wonder about a typical use case: If they include documents as part of a prompt, is there a way to ensure that the LLM will use only the provided documents when it generates the response? His answer: No.
“While careful prompting and techniques like RAG [retrieval-augmented generation] can encourage an AI model to prioritize a set of provided documents, standard LLMs cannot be forced to use only that content,” Ramakrishnan explains. “The model still has access to patterns and facts it learned during training and may blend that knowledge into its response — especially if the training data included similar content.”
Straightforward question; straightforward answer. Below, you’ll find Ramakrishnan’s article, along with seven others, chosen because they’re similarly straightforward. Use them to help ground you in what’s happening with this complex part of the technology landscape.
1. How LLMs Work: Top 10 Executive-Level Questions
Rama Ramakrishnan
“In my work at MIT Sloan School of Management, I have taught the basics of how large language models (LLMs) work to many executives during the past two years.
“Some people posit that business leaders neither want to nor need to know how LLMs and the generative AI tools that they power work — and are interested only in the results the tools can deliver. That is not my experience. …
“In this column, I share questions on 10 often-misunderstood topics that I am often asked about, along with their answers. …
“[For instance,] when does the LLM decide to give the user the final answer to a question? The decision to stop generating is determined by a combination of what the LLM predicts and the rules set by the software system running it. It is not a choice made by the LLM alone.” Read the full article »
2. How to Scale GenAI in the Workplace
Michael Wade, Konstantinos Trantopoulos, Mark Navas, and Anders Romare
“As companies move from experimentation to enterprisewide adoption, many struggle not with the tools themselves but with the organizational transformation required to integrate them meaningfully into people’s daily work. Tools will keep evolving: It is the human side of the equation that determines whether GenAI initiatives truly succeed.
“We studied one of the largest real-world generative AI deployments to date — at multinational pharmaceutical company Novo Nordisk. Its experience shows that success hinges not just on infrastructure but on how people think, adapt, and collaborate with AI. One critical lesson: While GenAI adoption and broader digital transformations have common roots, generative AI is uniquely disruptive, reshaping the nature of work itself in unprecedented ways. …
“Each employee saved 2.17 hours per week, on average, once they began using the tool. But something unexpected also happened: Those hours weren’t what employees valued most. Employee satisfaction with Copilot was three times more strongly correlated with perceived improvements in work quality than with time saved. Employees reported quality enhancements in content summarization, content creation, and ideation. Interestingly, many employees reinvested the time they saved into people interactions, strategic planning, and creative work.” Read the full article »
3. Generate Value From GenAI With ‘Small t’ Transformations
Melissa Webster and George Westerman
“Business leaders are finding ways to derive real value from large language models (LLMs) without complete replacements of existing business processes. They’re pursuing ‘small t’ transformation, even as they build the foundation for larger transformations to come. …
“Our project team interviewed the senior managers of various functions, including artificial intelligence, data science, innovation, operations, and sales, at 21 large companies. We focused on understanding what organizations with relatively early and broad GenAI adoption are doing and why. …
“Our research shows that most companies are following a more targeted approach to transforming with generative AI. While GenAI can potentially increase the speed and quality of many tasks, it also comes with a variety of risks around accuracy, security, and intellectual property management. The leaders we interviewed tend to apply the logic of a risk slope when making their decisions, attaching a higher risk to customer-facing processes than to internal ones.” Read the full article »
4. The GenAI Focus Shifts to Innovation at Colgate-Palmolive
Thomas H. Davenport and Randy Bean
“The large language models (LLMs) that Colgate-Palmolive’s teams use have been augmented with retrieval-augmented generation (RAG) content of various types — proprietary research that the company conducts, Google search trends, syndicated data sources, and more. RAG-based systems draw more on company-specific content than on public internet materials, so there is a lower likelihood of hallucinations.
“Generative AI can quickly go through such material and describe market trends and unmet consumer needs. That means that instead of downloading, reading, and notating a broad collection of market research reports when they want consumer insights, employees can just write the question they want answered in a prompt and immediately get a response. …
“Colgate-Palmolive’s teams found that they could combine one AI system that surfaces unmet consumer needs with another proprietary AI system that develops new product concepts to meet those needs. In minutes, with human guidance, it can produce copy and imagery for a new concept, such as a new flavor of toothpaste. While there are always humans in the loop to guide the workflow, using the GenAI-enhanced system is much more efficient than having humans page through market research materials. The breadth of ideas generated also creates a broader product funnel for the company to pursue.” Read the full article »
5. Bring Your Own AI: How to Balance Risks and Innovation
Nick van der Meulen and Barbara H. Wixom
“With this rise of GenAI comes a new challenge for organizational leaders: the phenomenon of Bring Your Own AI (BYOAI), which occurs when employees use unvetted, publicly available GenAI tools for work.
“While these tools promise greater productivity and creative potential, they also bring organizational security and governance risks, including data loss, intellectual property leakage, copyright violations, and security breaches. …
“Given the risks associated with BYOAI, it may seem logical for leaders to consider banning unvetted GenAI tools outright. The prospect of uncontrolled GenAI use, combined with the uncertainty around legal and regulatory exposure, can understandably make leaders cautious. However, the executives we interviewed said that banning BYOAI is neither practical nor effective. Employees — especially those already feeling stretched thin — often turn to GenAI tools to enhance their personal productivity. Restricting access only pushes them to find unofficial workarounds, potentially bypassing established governance frameworks.” Read the full article »
6. Stop Deploying AI. Start Designing Intelligence
Michael Schrage and David Kiron
“As part of our ongoing ‘Philosophy Eats AI’ exploration — the thesis that foundational philosophical clarity is essential to the future value of intelligent systems — we find that [physicist-turned-entrepreneur Stephen] Wolfram’s fundamental insights about computation have distinctly actionable, if underappreciated, uses for leaders overwhelmed by AI capabilities but underwhelmed by AI returns. …
“His life’s work now offers crucial frameworks for both understanding and applying AI in the real world. His insights aren’t clever academic flourishes; they’re imperatives for building intelligence environments that function effectively at scale. …
“With Wolfram, we explored the idea that AI leadership must shift from better adopting and integrating AI tools to designing intelligence environments, organizational architectures in which human and artificial agents proactively interact to create strategic value. Three insights from his philosophical approach to computation emerged as fundamental to this design challenge, offering a fresh perspective on why traditional approaches to AI adoption fail and what must replace them.” Read the full article »
7. The Hidden Costs of Coding With Generative AI
Edward Anderson, Geoffrey Parker, and Burcu Tan
“Organizations adopting these tools are anticipating major gains. And early research supports their optimism: GitHub has reported that programmers using Copilot are up to 55% more productive, and McKinsey has found that developers can complete tasks up to twice as fast with generative AI assistance.
“But these positive indicators come with a major caveat. The studies were conducted in controlled environments where programmers completed isolated tasks — not in real-world settings, where software must be built atop complex existing systems. When the use of AI-generated code is scaled rapidly or applied to brownfield (legacy) environments, the risks are much greater and much harder to manage. …
“When an organization rapidly introduces new software into existing systems, it can inadvertently create a tangle of dependencies that compounds its technical debt — that is, the cost of additional technological work that will be needed in the future to address shortcuts taken and quick fixes made during development. …
“Organizations must treat AI tools’ tendency to increase technical debt as a strategic risk, not just an operational nuisance.” Read the full article »
8. When to Use GenAI Versus Predictive AI
Rama Ramakrishnan
“While generative AI promises to revolutionize everything from customer service to product development, its optimal role alongside predictive AI tools (that is, machine learning and deep learning tools) remains a work in progress. That often leaves leaders asking what the right approach is for addressing a particular problem. …
“To effectively use traditional machine learning with unstructured data, the data must be manually structured — an expensive task that makes machine learning unattractive for business use cases where the input data is not tabular. …
“The inputs and outputs of generative AI systems like LLMs are typically unstructured. Most commonly, they comprise text and/or image data and, more recently, videos. Note that the text being analyzed by and created from generative AI tools encompasses an astonishing range of types, such as software code, protein sequences, music notation, mathematical expressions, and chemical formulas. …
“Let’s start with the easy case. If you have a generation problem to solve, there’s only one game in town: generative AI. Depending on the sort of output you want to generate, you may need to use multimodal LLMs, like OpenAI’s GPT-4, Anthropic’s Claude 3.7 Sonnet, or Google’s Gemini 1.5; text-to-image models, like Dall-E; or special-purpose models that have been built for audio and other domains.
“If you have a prediction problem, however, matters become more complicated.” Read the full article »
Additional Resources: Apply AI Lessons With Your Team
GenAI: The Strategy + Governance Toolkit
MIT Sloan Management Review developed its Generative AI Strategy + Governance Toolkit with five of the world’s foremost experts on generative AI leadership: Ethan Mollick (University of Pennsylvania’s Wharton School), John Sviokla (Harvard Business School and GAI Insights), John K. Thompson (Hackett Group and the University of Michigan), George Westerman (MIT Sloan School of Management), and David A. Wood (Brigham Young University’s Marriott School of Business).
The kit includes a video lesson with Westerman, a GenAI strategy planner, a GenAI strategy checklist, a GenAI governance planner, articles, and more.