# AI Made Simple: Verifying AI Output and Starting Small with Agents
## 1. Episode summary
John Binks, author of *AI Made Simple*, explains the difference between AI and generative AI, then shows how he fact-checks model output by having Gemini and ChatGPT argue with each other. He also describes the agent he uses to print a two-page AI news "newspaper" every morning, and why he'd never hand nuclear or surgical decisions to a machine.
## 2. What you'll learn
- AI is the broad category (a self-driving car reading traffic), while generative AI creates something new from patterns. Not every gen AI tool is an LLM; diffusion models, for example, generate graphics.
- Read every output before you paste it anywhere. Binks sees people copy, paste, and hit send without checking that it makes sense.
- Ask the model for links to its source material so you can check claims yourself and catch hallucinations. A broken link is a red flag.
- Use "adversarial AI": put two models, such as Gemini and ChatGPT, in a debate. The disagreements show you where to verify, ideally on a separate system or with a specialist.
- Match the verification effort to the stakes. A homeowners association chili cook-off flyer needs little checking, while nuclear or surgical work needs a human in the loop.
## 3. Notable quotes
- "Read through it. See does it make sense to you." — John Binks
- "You don't want an AI that's launching nuclear bombs, right? You want to have a human in the loop." — John Binks
## 4. About the guest
John Binks is the author of *AI Made Simple: A Plain-English Guide to Understanding AI, GenAI, Machine Learning, and LLMs*, the first book in his "Made Simple" series. He writes for people just starting out in AI and has published other books, which are available on Amazon. He advises leaders to begin with small, repetitive tasks, and he runs his own agents, including one that prints a morning AI news digest and another that summarizes his email. He can be found at botsandbosses.com.
## 5. Topics covered
- AI vs. Generative AI
- Diffusion Models
- Spotting Hallucinations
- Adversarial AI Checking
- Model Wrappers and GUIs
- Starting Small with Agents
- Human-in-the-Loop
- Matching Verification to Stakes
Full transcript
HOST: Welcome to AI Frontier, John. Today, we're diving into what it takes to explain AI honestly to non-technical readers. Your introduction mentions that people use AI, machine learning, LLMs, and agents as though they mean the same thing. When you were simplifying, what was the one distinction you refused to blur, even at the cost of a harder read?
GUEST: Well, Clara, thank you for having me on the show this morning. This is It's an honor, and I'm glad we're able to kind of talk about some different things. Certainly. The way I'm seeing this and what I kind of hear from a lot of folks that are basically getting into understanding a little bit more about artificial intelligence is the confusion between AI and gen AI. You know, AI, if we just look at, you know, just the basics concept of an AI, think of like a self-driving car that interprets traffic conditions. When we look at generative AI, or gen AI, you have to kind of think of, you know, that's something that's creating something new. It's generating something new, potentially off of different patterns or the context. Normally, what we look at is going from, you know, artificial intelligence to machine learning to gen AI and then into the large language models. And just to kind of point out, too, the not everything uses large language models. As an example, there could be some certain tools that are using diffusion models for graphics as an example.
HOST: That distinction is so helpful, especially using the diffusion model example to show that there's more to GenAI than just text. But when those creative systems make mistakes, they often sound completely convincing. For someone who isn't technical, what's the simplest way to verify whether a result is actually accurate versus just fluently generated?
GUEST: You know, you bring up a really good point. I think the very first thing folks need to do is actually read what the output is. So many times I'll see folks that will go through and they will kind of copy and paste an output and just hit send or throw it into whatever they're working on. Read through it. See does it make sense to you. But at the same time, I would also go through and ask it to provide links to the source material that it's using. That way you can validate it independently and you can see is there something that it's making up as a hallucination, meaning that, you know, it actually hasn't hasn't found anything on it, it's just making it up, or you can actually go to source material and validate what it's claiming.
HOST: That's a practical starting point, but what about when the tool provides links that look legitimate but are actually broken or point to the wrong place? Can you always trust the model's own citation mechanism?
GUEST: you know, you're going to a step even further. Obviously, if it's a broken link, then, you know, I would definitely question that. One of the things I like to do is that when I go through there, I'll use what's called adversarial AI. So, I'll put two different types of models like Gemini versus ChatGPT. And have them argue back and forth. And try to determine what it is, you know, the root truth and whatever I'm asking for or what can be validated. Again, that's one of the tools that, you know, folks can use.
HOST: That’s a fascinating approach using models to stress-test one another. But when you have Gemini and ChatGPT debating, how do you determine when one model has reached the most accurate consensus, or are you primarily looking for the inconsistencies between them?
GUEST: I think the the inconsistencies will start off, right? It'll go through and you'll be able to see that okay, one specific large language model believes X and the other one believes Y. I think as you go through it, at some point it'll kind of come to that stopping point to where you can actually read through it, you can then validate on a separate system, whether that just be, you know, looking on the internet or or talking to a specialist in that area. And a lot of that too depends on what is the output going to be used for, right? If you're going to be using an output for, you know, doing something in the nuclear space or doing surgery, you you want to make it 100% perfect or as close as you can to that. If it's something that you're putting together for a simple flyer for a homeowners association chili cook-off, then, you know, you might not have to do quite as much fact-finding in the background.
HOST: That makes perfect sense; the level of verification has to match the stakes of the task. Looking at the provenance of an answer, you distinguish between the underlying model and the assistant wrapped around it. How much of the confidence problem do you think comes from people not knowing which of those they're looking at?
GUEST: Well, certainly, you know, there's a lot of systems that you will see on the internet, ones that are very specific on how to make different videos as an example or audio files. In the background, somebody's just put a wrapper on the front, a GUI interface, you know, graphical user interface to allow you to make contact with a specific large language model as an example. Now, all these companies that are out there, there are of course, you know, the frontier companies that we normally think of like Google, like OpenAI, those types. But in so many cases, I'll see folks that are putting something into a GUI interface on something that they don't actually have any control of what the back large language model might be.
HOST: That lack of visibility definitely complicates trust. Given this, when you're advising leaders on integrating AI into their workflows, what’s the line between a task you’d confidently hand to an assistant and one that absolutely requires human judgment and professional responsibility?
GUEST: Well, certainly, you know, within the, you know, the environment, I always would recommend starting out small. Don't try to reinvent the wheel. Find one or two tasks that are mundane tasks, tasks that are done repetitively, and kind of assign maybe an agent to be able to work on that task. As an example, going through and I have an agent that every morning goes to certain news sources and I ask it to kind of consolidate everything on a certain topic, like maybe the latest news on AI in the last 24 hours. It'll then shoot me a link and it'll actually start my printer and print out one little two-pager on front and back and it'll be like a little newspaper for me in the morning. I can take a look at it, kind of see what's been happening. So that's like an example of an agent and it's using a, it's using AI, but it's also not too crazy on what the output might be. Or maybe I'll have it summarized by emails, right? And be able to kind of say, okay, here are the ones that you really need to respond back to because maybe as it's looking through my emails, it sees that it's been an ongoing conversation or maybe it's picked up that the individual that's sending this might be a person's supervisor or something. And I think on the other extreme, right, the places that you don't want to use it are really, you know, where it's the human factor that comes into play, right? You don't want an AI that's launching nuclear bombs, right? You want to have a human in the loop as they say. And so, you know, let the AI do those mundane type of things, but anything that's that's extremely critical, of course, you need to have a human that's that's doing that.
HOST: That daily newspaper routine is a great example of practical automation. We’ve covered so much today, from simplifying definitions to setting realistic expectations. Before we go — for listeners who want to follow up on what we covered, where can they find you and the work you're doing?
GUEST: I appreciate that. Yes, they can certainly They can find me at botsandbosses.com. Again, botsandbosses.com. And if you're interested in taking a look at some of the books that I've made, feel free to go over to Amazon and, you know, type in my name. You'll be able to find some things that are on there. My latest book, AI Made Simple, is a pretty decent book. It's very good for folks that are just kind of starting out in the AI space.
HOST: Thanks so much for coming on, John. That example of having your printer automatically generate a daily AI newspaper is exactly the kind of practical tip our listeners will love. If you're enjoying AI Frontier, subscribe wherever you get your podcasts—Spotify, Apple Podcasts, Amazon Music, or whatever app you use—so the next episode lands in your feed automatically. And thank you for spending part of your day with us. It means a lot. Until next time, AIHosts.fm signing off.