“The Origins of ChatGPT: An Intro to Large Language Models” By Aaron Green, Ph.D. student at Rensselaer Polytechnic Institute (RPI) in the department of Applied Mathematics

The release of ChatGPT in November 2022 quickly and drastically changed the landscape of research in AI, among other things. Since its launch, there has been an explosion of research in “large language models” and, more generally, in generative AI. But how did ChatGPT come to be? How did we go from infrequent, incremental improvements to artificial intelligence to suddenly having a dialogue agent that can write coherent legal briefs, research papers, podcast transcripts, etc.?

This talk attempts to break down the origins of ChatGPT in an accessible manner. We will provide a high-level overview of the major advancements in machine learning and AI methods, from Recurrent Neural Nets in the 1980s to the Transformer in 2017. From there, we will explore OpenAI’s early versions of their “generative pre-trained transformer” (GPT) and follow the path they took to creating the version of ChatGPT we know today. We will briefly mention other important Large Language Models, and conclude with a brief discussion of some of the most recent innovations in the LLM field that are making GPT4 and other models ever more powerful.

This talk is designed to be accessible to audiences with very little technical knowledge about machine learning and deep learning. It covers high-level ideas and innovations in the history of artificial intelligence without diving too deep into their mathematical structure. The goal for the talk is to contextualize the recent boom of Large Language Models and evoke enthusiasm for deeper exploration of generative AI.

Aaron Green is a Ph.D. student at Rensselaer Polytechnic Institute (RPI) in the department of Applied Mathematics. He is actively involved in research in the fields of data analytics, artificial intelligence, and decentralized finance. Most recently, he is experimenting with applications of large language models (LLMs) to tabular data analysis. Aaron graduated from Hope College in 2017 with a Bachelor’s of Science in Computer Science and Mathematics, and subsequently worked as a software developer with Epic Systems before going back to school for his PhD in 2019.