Tracking down reliable information is as tricky as ever. ChatGPT and Google can be excellent tools, but keep in mind that they are far from perfect. Let’s see why you should have a healthy dose of skepticism when doing research.
ChatGPT Confidence
Much has been said about ChatGPT not always being trustworthy. That shouldn’t come as a surprise, considering it’s not going to be 100% up-to-date. It’s more problematic when it sometimes decides that it’s going to make up answers.
Take this article as an example. When ChatGPT was fed 517 user-written software engineering questions, the result was incorrect more than half the time. I also liked how it mentioned that “more than 3 in 5 ChatGPT responses were more verbose than human-written answers.” It makes me think of the Abraham Lincoln quote, “Better to remain silent and be thought a fool than to speak out and remove all doubt.”
Pick and Choose
Along with ChatGPT, I wanted to take a moment and remind you that Google results aren’t always what they seem to be either. Google has been known to pick and choose facts for their featured snippet results that can steer you in the opposite direction.
For instance, imagine you’re searching for a top 5 list of what to do in a particular situation. In some cases, Google may find an article with a top 5 to-do list as well as a top 5 NOT to-do list. Google may include a what not to do in the featured snippet and give you the opposite of what you’re looking for.
I encountered this type of scenario when writing last week’s blog post. I know sp_ is not how you should start a stored procedure name, but I was curious what results Google would return. Google said this:
If I click into that article, I find that it also recommends against using sp_ as a stored procedure prefix. In fact, when searching for that same phase today, the snippet is a bit more complete:
From my experience, Google gets these featured snippets right most of the time. As you can see based on the example above, it’s always improving.
Read!
ChatGPT and Google are helpful for finding information, but remember not to always take the featured snippet, headline, etc. as gospel. Actually read the articles and check your sources!
Thanks for reading!
Chad Callihan 