TLED Guide to Generative AI

What is AI (Artificial Intelligence)?

“AI (Artificial Intelligence) is the science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to biologically observable methods” (McCarthy, n.d.). AI involves the development of intelligent programs that can perform tasks typically requiring human intelligence. Its history can be traced back to the 1950s when John McCarthy, a computer scientist, coined the term “Artificial Intelligence” (Nicholes Charles, Sidharth Venugopal, & Student, n.d.) [A Study about AI and ML: Exploring the Advancement of Generative AI Models]

According to authors Feuerriegel et al, Generative AI refers to artificial intelligence systems capable of generating new content and artifacts such as text, images, audio, and video (Stefan Feuerriegel, Jochen Hartmann, Christian Janiesch, & Patrick Zschech, 2023) [Generative AI]. The key capability is that these systems can create novel, original outputs that are not simply retrieved or rearranged from a dataset but are synthetically generated from patterns learned by the AI model. This technology, with examples like Dall-E 2 and GPT-4, is revolutionizing how we work and communicate.

The intelligence of created systems and algorithms is typically compared to human intelligence. Sometimes, LLMs (Large Language Models) and ML (Machine Learning) products can appear to have human intelligence, but it is simply the product of coding, not actual intelligence. The ethical implications of AI, especially in generative models, raise concerns about authorship and the integrity of academic work, as these technologies can produce text indistinguishable from human-written content (Hazem Zohny, J. McMillan, & M. King, 2023) [Ethics of generative AI] .

What is Generative AI (GAI)?

A subset of AI generative AI creates brand-new content using a machine learning technique known as generative adversarial networks (GANs). Each type of generative AI has its own set of applications, benefits, and challenges, and they are continually evolving with advancements in AI research and technology. (N. Anantrasirichai & D. Bull, 2020) [Artificial intelligence in the creative industries: a review]

The following are examples of what Generative AI can do: 

• • Text Generation: Systems like ChatGPT can generate human-like text such as articles, stories, code, emails, etc based on a prompt. They are trained on massive datasets of text data to learn linguistic patterns and how to respond in conversation.
  • Image Generation: Systems like DALL-E 2 and Stable Diffusion can create realistic images and art from a text description. They are trained on large datasets of image-text pairs to learn visual concepts and their relationships. Try these tools with Padlet’s “I can’t draw” feature or Adobe Express.

• • Code/Coding: Many programs generate and write software development code like HTML or Python. You can use specific engines like OpenAI’s Codex or just type it into ChatGPT. The models can also explain and answer questions about code.

• • Sound and Music: AI models can create original music compositions, and sound effects, and even simulate human voices. You can create a model based on your own voice.

• • Video and 3D animation: Create or modify movie clips and even whole films.

• • Interactive Content: AI can create interactive experiences, such as conversational agents, adaptive learning environments, or personalized gaming experiences, where the content evolves in response to user interactions.

Artificial Intelligence (AI)

A standard computer program is a set of instructions that are executed linearly. The output of the program is always the same given the same input. For example, a program that calculates the area of a circle will always output the same value for the area of a circle with a given radius.

An AI program, on the other hand, can learn and adapt its behavior based on new information, patterns, or experiences. This is because AI programs are typically built using machine learning algorithms, which allow them to identify patterns in data and make predictions based on those patterns (Opeoluwa Tosin Eluwole, Segun Akande, & O. Adegbola, 2022). [Major threats to the continued adoption of Artificial Intelligence in today’s hyperconnected world].

The ability of AI programs to learn and adapt makes them ideal for a wide variety of tasks, such as:

For example, AI could involve developing a system that can understand cat behavior, recognize different breeds of cats, or even simulate the behavior of a cat in a video game. AI in this case encompasses a broad range of techniques and approaches to enable intelligent behavior related to cats.

Generative Pre-Trained Transformer (GPT)

The “GPT” in ChatGPT stands for “Generative Pre-Trained Transformer.” Let’s break that down:

• • Generative: ChatGPT can create or generate text, like the responses you see in ChatGPT it can be any format of text from a script to a chart to a chapter of a book.
  • Pre-Trained: Before ChatGPT chats with you, it’s already learned a lot about language. As if it went to language school by reading a vast amount of text from the internet. After that, people at OpenAI worked with the program to refine how it works.
  • Transformer: This is a special type of technology used in ChatGPT. It’s advanced and different from older types of AI because it can focus on different parts of a sentence to understand the meaning better. Imagine reading a sentence and being able to pay attention to each word and how it relates to the others, even if they’re not right next to each other.

For example, in the sentence “The cat sat on the mat,” ChatGPT doesn’t just see words next to each other; it understands how “cat” relates to “sat” or “mat,” even though they are not adjacent.

• • ChatGPT has a huge number of “parameters” – these are like settings that help it decide what parts of a sentence are important. Because it has so many parameters, it can really grasp the context or the setting in which a word is used. This ability to understand context helps GPT-3 generate text that sounds fluent and natural.

The “training” part is like a massive game of “fill in the blank.” The AI is given sentences with words missing, and it has to guess the right word. It does this billions of times, getting smarter each time. It adjusts its parameters based on whether it gets these guesses right or wrong. Over time, it gets good at understanding what words fit best in different contexts.

ChatGPT is a highly advanced AI that’s been trained on a huge amount of text. It’s great at generating language because it understands how words relate to each other in different contexts, making it sound more like a human when it talks or writes.

Large Language Models (LLMs)

Think of how your brain is made up of neurons that are connected in networks, and those networks allow you to learn, think, and understand language. AI systems like large language models have artificial neural networks too. In an LLM, the neurons are simplified simulated circuits that can either fire or not fire, like binary on/off switches. And the connections between them are weighted – meaning some connections are stronger than others. These neural networks start out as random, but then researchers “train” them using huge amounts of data. For LLMs, that data is text. As the model reads words, sentences, and documents, the weights on its connections are adjusted little by little to better predict what comes next in the language (Holger Schwenk & J. Gauvain, 2005) [Training Neural Network Language Models on Very Large Corpora]

To create a large language model you give the system tons and tons of written data, like books, Wikipedia, articles, webpages, and it learns patterns between words. It can then generate new content based on the parameters set. For ChatGPT or Bard, there are billions and billions of parameters. It’s an algorithm that predicts words based on words it has seen together (D.M.E. Luitse & Wiebke Denkena, 2021) [The great Transformer: Examining the role of large language models in the political economy of AI]

Machine Learning (ML)

Machine learning is a subset of artificial intelligence (AI) that enables systems to learn and improve from experience without being explicitly programmed. It focuses on the development of computer programs that can access data and use it to learn for themselves. The learning process begins with observations or data, such as examples, direct experience, or instruction, to look for patterns in data and make better decisions in the future based on the examples provided. Machine learning algorithms create mathematical models based on sample data or training data for unpredictable prediction or decision-making (Javid Ghahremani nahr, H. Nozari, & M. E. Sadeghi, 2021) [Artificial intelligence and Machine Learning for Real-world problems (A survey)].

Key aspects of machine learning include:

• • Data: Machine Learning relies heavily on data to detect patterns. This data can be in various forms, such as images, numbers, words, etc. The more data, the better the machine can detect realistic patterns.
  • Algorithms: Machine learning uses a variety of algorithms. These algorithms are designed to learn from and make predictions or decisions based on data. They vary greatly in their complexity and application.
  • Model Training:The process involves training a model using a dataset. During training, the algorithm gradually determines the relationships between features of the data and the outcomes we’re interested in predicting or understanding. So imagine training a machine learning model using a large dataset of images of cats. The model would learn patterns and features specific to cats, such as the shape of their ears, the color of their fur, and the positioning of their whiskers. Once trained, the model can then be used to identify cats in new images it hasn’t seen before.