December 2, 2025

Beware the Hallucinations When Coding with AI

Make sure to take what AI gives you with a grain of salt. It can help point you in the right direction as a first pass, but be wary. It might not really understand your question or the material.

When writing a UI automated test to confirm that a user could not continue using our React Native application without first accepting the Terms & Conditions, in order to see if a button was initially enabled or not, GitHub CoPilot suggested to use a Detox method ".toBeDisabled()". The problem? After delving into the Detox documentation and source code I found out this method doesn't actually exist.

When I told the AI it did not exist, it apologized profusely, then suggested I erase that bit of code it wrote, along with the code of other unrelated tests I was working on. When I finally found a solution, and ran it by GitHub CoPilot, after reviewing it, the AI wanted to replace the code with the non-existent method again.

... Eh, it's slightly better than frantically searching through Stack Overflow's archive and interpreting the solutions you find there.

Happy Testing!

-T.J. Maher
Software Engineer in Test

BlueSky | YouTubeLinkedIn | Articles

1 comment:

WhiteScholars said...

Very well written . Thanks for sharing this

If you are someone who is interested in Data and want to learn it with AI , then look no further Whitescholars provides the best Data Analytics training in Hyderabad. Various institutes provides Data Analytics coaching in Hyderabad but we offer it with Gen AI and that too with placement opportunites


data analytics course in hyderabad