Speaking of fact checking, it turns out that if you ask ChatGPT or Claude where you can stream a popular TV show or movie, the answer you receive is likely to be wrong 50% of the time
I've had great success with AI when it is focused on accomplishing tasks, like computer coding. I had one agent write me a program in about 30 min that would've taken me months. But when it comes to facts, AI seems to miss quite frequently and requires secondary check. For instance, I work in aerospace and needed the takeoff Mach number for a 737. Google AI came back with Mach 0.25 which is roughly accurate, but then tacked on the "fact" that most 737s cruise at a Mach around 0.25. Folks, if you are on a 737 cruising at Mach 0.25, you are falling out of the sky.
Such an important topic - thank you for providing to all of us!
When I wrote my book on my high school's football team from 1974, I was able to interview a number of the players. I would use AI to upload an interview, and ask it to summarize and pull out specific quotes to use as notes. HOWEVER, when I would upload one interview with one person, the AI would sometimes provide quotes and information from people and topics which had nothing to do with topic or interview. Sometimes, the quotes were from people who did not even exist, and/or never had anything to do with the school, football, etc., etc. I found that as much as the summaries could be helpful, just as much if not more work was created to double check much of what the AI produced for me.
We have to keep in mind that someone is writing the code which tells the AI how to interpret the data it is using to create the responses to the prompts it receives, and then how it will present the response, which can add even more bias. Katie references how each AI may prioritize the news sources it has access to, however, this is only one part of the prioritizations which are added to the system. The opportunity for AI to be very biased, however, in a very hidden way, is extensive, which is exactly the opposite of what I think society should want from a general resource utility. At its worst, AI can give BIG BROTHER even more opportunity to control what information we might see and what we do not.
I have found that AI does well with facts - in other words, it can look up information that is available and give me insight on where to go to get those facts. It is OK when it is asked questions about scientific facts (unarguable pieces of base information). As I move further into the meaning of those facts and that insight, things get sketchier and sketchier. So to me, it seems that as you increase the modeling between the information facts, AI currently goes further and further off of the deep end.
I don’t rely on AI as an authority on anything factual. I always cross check any references AI provides. A simple example, I asked Gemini if how many of the original cast members of an old TV program were still alive. The AI response was directly contradicted by the Google search results that appeared immediately below. Somehow it managed not even to use its own resources to generate a response. Weird.
I tried it looking for lines in plays I remembered but whose source or character was fuzzy. Each time I got a wrong answer and so I knew AI wasn't for me. Also, as a widely-published author for forty years, I don't need any help with ideas, outlines, dialogue, descriptions, anything at all like that. The deepest I go is spell check on Word.
Speaking of fact checking, it turns out that if you ask ChatGPT or Claude where you can stream a popular TV show or movie, the answer you receive is likely to be wrong 50% of the time
https://www.forbes.com/sites/rickellis/2026/06/04/chatgpt-and-claude-score-below-51-accuracy-on-streaming-availability/
I've had great success with AI when it is focused on accomplishing tasks, like computer coding. I had one agent write me a program in about 30 min that would've taken me months. But when it comes to facts, AI seems to miss quite frequently and requires secondary check. For instance, I work in aerospace and needed the takeoff Mach number for a 737. Google AI came back with Mach 0.25 which is roughly accurate, but then tacked on the "fact" that most 737s cruise at a Mach around 0.25. Folks, if you are on a 737 cruising at Mach 0.25, you are falling out of the sky.
Super helpful Chris - thanks!
Chris - I didn’t use AI for fact checking. If you’d asked I would have told you that. That’s a fact.
Such an important topic - thank you for providing to all of us!
When I wrote my book on my high school's football team from 1974, I was able to interview a number of the players. I would use AI to upload an interview, and ask it to summarize and pull out specific quotes to use as notes. HOWEVER, when I would upload one interview with one person, the AI would sometimes provide quotes and information from people and topics which had nothing to do with topic or interview. Sometimes, the quotes were from people who did not even exist, and/or never had anything to do with the school, football, etc., etc. I found that as much as the summaries could be helpful, just as much if not more work was created to double check much of what the AI produced for me.
We have to keep in mind that someone is writing the code which tells the AI how to interpret the data it is using to create the responses to the prompts it receives, and then how it will present the response, which can add even more bias. Katie references how each AI may prioritize the news sources it has access to, however, this is only one part of the prioritizations which are added to the system. The opportunity for AI to be very biased, however, in a very hidden way, is extensive, which is exactly the opposite of what I think society should want from a general resource utility. At its worst, AI can give BIG BROTHER even more opportunity to control what information we might see and what we do not.
Thank you, again!!!!!
AI is wrong all the time, it's stunning that people actually use it for anything. And a little scary.
Just ask AI how many mistakes it makes - the last time i looked it said 45%!
Was it even being "honest" - it does NOT know what honesty really is... ask AI!
We might ask the same question of Chris!!
I am sure he does better...
I have found that AI does well with facts - in other words, it can look up information that is available and give me insight on where to go to get those facts. It is OK when it is asked questions about scientific facts (unarguable pieces of base information). As I move further into the meaning of those facts and that insight, things get sketchier and sketchier. So to me, it seems that as you increase the modeling between the information facts, AI currently goes further and further off of the deep end.
Just my observation
I don’t rely on AI as an authority on anything factual. I always cross check any references AI provides. A simple example, I asked Gemini if how many of the original cast members of an old TV program were still alive. The AI response was directly contradicted by the Google search results that appeared immediately below. Somehow it managed not even to use its own resources to generate a response. Weird.
I tried it looking for lines in plays I remembered but whose source or character was fuzzy. Each time I got a wrong answer and so I knew AI wasn't for me. Also, as a widely-published author for forty years, I don't need any help with ideas, outlines, dialogue, descriptions, anything at all like that. The deepest I go is spell check on Word.