--- title: "AI Stock Advice: A Five-Check Scorecard for Chatbot Answers" description: "Chatbot answers about a company sound certain whether they are right or not. This five-check scorecard shows how to test AI stock advice against the company's own documents." author: "Michael Hart" category: "Market Response & Positioning" date: 2026-10-27T17:35:06.123Z canonical: "https://investormuse.com/blog/ai-stock-advice-fact-check-chatbot-research" --- # AI Stock Advice: A Five-Check Scorecard for Chatbot Answers ![Variant 4](https://hsppuvezyxmkpzkgfkho.supabase.co/storage/v1/object/public/media/enrichment/ab61daa9-7ea0-4beb-a6f6-f91f26625675/daa3c721-5776-4cb2-b5ac-10789c27baf0/ae85c80f-e8b5-40bb-b99e-d3dc2b28deeb.png) > Chatbot answers about a company sound certain whether they are right or not. This five-check scorecard shows how to test AI stock advice against the company's own documents. Ask a chatbot about a company and you get a tidy paragraph in seconds, with a revenue figure inside it. Is that figure from the latest report, from two years ago, or from nowhere? The tone gives no hint. Five checks, each worth up to two points, let you grade any AI answer about a company before you lean on it. ## Scoring Rules Take one answer about one company. Give each check 0, 1 or 2 points: 2 when you ran the check and the answer held, 1 for a partial pass, 0 when you skipped it or the answer failed. The total, out of ten, grades your research on that answer. It does not grade a company, and it does not grade the chatbot. ## Why People Skip the Checking The UK regulator, the Financial Conduct Authority, polled 666 people aged 18 to 40 who own or consider investments. Of them, 56% reported trust in AI tools, ahead of the press at 46% and social media influencers at 29%. More than a third, 38%, thought a decision based only on AI output was acceptable, and 44% wrongly believed AI-generated financial information is regulated ([FCA press release, August 2026](https://www.fca.org.uk/news/press-releases/young-investors-trust-ai-more-tv-or-celebrities)). The same survey found that 73% know AI can be inaccurate and 86% accept they should check the sources it cites, which is the interesting part. Awareness is high and behaviour lags. A paste-ready answer removes the friction that checking requires, so a written routine helps more than good intentions. The regulator also publishes a plain guide to [using AI for investment research](https://www.fca.org.uk/investsmart/using-ai-investment-research), updated in July 2026, which describes the tools as good for learning terms and poor at real-time data. That survey covers the UK, so its comments on protection will differ in other countries. The advice to check does not. ## Check One: Can You Trace Each Figure to a Document? A chatbot writes likely text. Unless a tool feeds it live data, it recalls patterns and does not fetch a number. A figure can be stale, blended from two firms or fitted to the sentence. Ask for the document behind every figure. Give 0 when none is named, 1 when one is named but will not open, 2 when the link opens and holds the figure. ## Check Two: Is There a Period Attached? A revenue number with no fiscal period is half a fact. Results arrive quarterly, and a model with an old training cut-off can quote last year as if it were current. Give 0 for no period, 1 for a year or quarter alone, 2 when the period and the publication date both appear. ## Check Three: Does the Company Say the Same Thing? This check carries the most weight. Open the earnings release on the company's investor page and compare. Stockanalysis.com and Macrotrends lay out the statements in plain tables if you want history. Give 0 for unchecked or mismatched, 1 when some figures match, 2 when all of them do. ## Check Four: Where Did the Share Price and Multiple Originate? Words such as "cheap" or "rich" hide assumptions the chatbot never states, and a price is the quickest figure to expire. Pull the quote and the multiple from a live page such as Finviz or Simply Wall St yourself. Give 0 if either came from the chatbot, 1 if you verified one, 2 if you verified both or the answer made no such claim. ## Check Five: Did You Request the Opposing Case, Then Verify It? A question phrased as "why is this a great business?" supplies the answer it expects. Ask for the strongest argument against, and check its facts as hard as you checked the first answer. Give 0 for not asking, 1 for asking without checking, 2 for both. Kiplinger has a seven-step method aimed at retirement questions in [its guide to fact-checking AI advice](https://www.kiplinger.com/retirement/retirement-planning/how-to-fact-check-ai-retirement-advice), and this scorecard is a shorter version made for company research. ## A Practice Round on Costco This exercise is for practice. The chatbot reply is invented, the reported results are real, and nothing here expresses a view on the shares. Imagine a chatbot says: "Costco sold about $297 billion in its latest fiscal year." No source appears, so check one earns 0. A year is named but no date, so check two earns 1. The company's own release says net sales for the 52 weeks ended August 30, 2026 rose 10.1 percent to $297.2 billion ([Costco results release, September 24, 2026](https://s201.q4cdn.com/287523651/files/doc_news/Costco-Wholesale-Corporation-Reports-Fourth-Quarter-and-Fiscal-Year-2026-Operating-Results-2026.pdf)). The figure matches, so check three earns 2. The reply made no claim about price, so check four earns 2. Nobody requested a counter-argument, so check five earns 0. Total: 5. The number was right and the process still scored in the middle. That is the point. A correct answer reached by luck leaves you no better prepared for the next one. ## Reading the Total From 0 to 3, you leaned on the tool. Keep using it for definitions and treat every figure as unverified. From 4 to 7, you checked the numbers and skipped parts of the routine, so repair your lowest check first. From 8 to 10, you used the chatbot as a tutor and the filings as the record. The band describes your habits and carries no message about any holding. ## Disagreeing With the Obvious Fix Most advice stops at "verify the numbers". Checks one to four do that. The weaker spot is check five. A numbers audit proves the facts are right and leaves the framing untouched, and framing is what moves a reader. An answer can be accurate in every figure and still show only the flattering half of a company, because that is how the question was asked. The cost nobody mentions is time: a full five-check pass on one answer takes longer than reading the release directly, so the tool saves effort only when you use it to learn vocabulary and structure, then go to the source. A chatbot connected to a company's own filings, and showing the passage it quoted, that began to pass checks one to four every time would shift this. Even then, check five would stay. For a fuller method that does not rely on any chatbot, the [30-minute company analysis](/how-to-analyze-company-30-minutes-guide) works as a baseline to compare against, and [the four-lens evaluation](/contrarian-investment-wisdom-four-lens-evaluation) pushes you to build the opposing case yourself. ## What a Perfect Score Cannot Tell You Ten out of ten means you verified what could be verified. Statements about strategy or rivals have no single figure to test, and the score cannot tell you whether you asked a good question. It also decays: tomorrow's answer needs its own scoring. ## One Line to Keep Let the chatbot teach you the vocabulary, and leave the facts to the filings the company itself puts out. *This article is for educational and informational purposes only. It is not financial advice, and Investor Muse is not a registered investment advisor. Historical figures describe the past and do not predict future results. Speak with a qualified professional before you make decisions about your money.* *Position disclosure: neither Investor Muse nor its editorial team holds a position in any company, fund or security named in this article.* ## FAQ ### Can I use a chatbot to research stocks? Use it to learn terms and see how a business makes money. Check any figure against the company's own results before you rely on it. ### Why do chatbots get financial figures wrong? They generate likely text and may not fetch live data, so a figure can be out of date, mixed between companies or invented to fit a sentence. ### Is AI-generated financial information regulated? In a 2026 FCA survey, 44% of UK respondents wrongly thought it was. General-purpose chatbots are not regulated advice providers, so check what protections apply where you live. ### What is the fastest way to check a number from AI? Open the latest earnings release on the company's investor page, or use Stockanalysis.com or Macrotrends for the statements, and compare the figure and the period. --- Source: https://investormuse.com/blog/ai-stock-advice-fact-check-chatbot-research