‘Computer says no’: I asked AI the pitfalls of asking AI for money advice

Managing money well can be, well, confusing. It can even be confronting. And how are many of us are solving such issues these days with the help of artificial intelligence “buddies”.

Heck, I even have friends who use AI to draft replies to tricky text messages from their mum.

So is it any surprise that more than three million Australians are already using AI to help manage their financial activity, with the highest number – 25 per cent – among Gen Z Australians, says a new Understanding Modern Australia report.

Not that we necessarily – thank goodness – trust it. The report, by life insurer TAL with McCrindle, found only 22 per cent of us feel comfortable using AI to help make major financial decisions and only 20 per cent trust AI to recommend a financial product suited to their needs.

So, I thought I’d try an experiment: I asked AI the pitfalls of asking AI for money advice – and whoa.

In an emphatic “computer-says-no”, here are the 10 reasons the AI gave me not to use it – and how I might. (I’ve grouped them where AI doubled and even tripled down.)

‘An LLM’s training data has a cut-off, and even with web search, it can miss nuance, fail to update thresholds, blend outdated and current rules without flagging the difference.’

An AI’s response

1. No accountability or duty of care

Yep, good point Claude! No care and no responsibility does not make for the ideal economic confidante. The rules and requirements to give financial advice in Australia are strict. AI is not bound by any of them.

The education needed is also now extensive (part of why the profession is shrinking). But, mainly, a licensed financial adviser is bound by law to act in your best interest. In the candid confession, AI said: “An LLM has no such obligation and no licence that can be revoked if it gets things wrong.”

2. Regulatory grey zone

You need an Australian Financial Services Licence to give personal financial advice – it’s illegal otherwise.

Reputable AI tools build in disclaimers and stick to general information rather than personal advice. But how do you even tell the difference when it’s – you know – you asking?

3, 4 and 5. Lack of personalisation depth. Generic, one-size-fits-all framing. Data privacy

Variables that factor into smart financial decisions include your age, income, debts, risk tolerance, tax situation, existing assets, super balance, insurance and goals.

Does your AI know all of this about you? And, crucially, do you want it to? Where does that data go and how is it stored? “Even if they did, an LLM isn’t verifying any of it or cross-checking for inconsistencies the way a human adviser would,” confides the LLM itself.

The “cookie cutter” alternative isn’t great, either. The advice might just be versions of old financial adages like diversify, pay off debt and invest for the long term. They may not fit with your financial affairs at that moment.

6. Currency and accuracy problems

This is a massive one. Legislation changes regularly. Even in the past year, we have seen changes to the first home buyer 5 per cent deposit scheme, an overhaul of capital gains and a ban on negative gearing. Um, I won’t be trusting AI with any related financial decisions thank you!

Then there are just the regular updates and indexation to things like tax brackets, super contribution caps, income thresholds and interest rates. Old information could easily sink your future.

Need proof? “An LLM’s training data has a cut-off, and even with web search, it can miss nuance, fail to update thresholds, blend outdated and current rules without flagging the difference.” Thanks, Claude.

7 and 8. A false sense of diligence and confident-sounding errors (aka “hallucinations”)

We’ve all encountered this – AI intel that’s slightly insane! And the danger it poses in complex financial information or advice is: would you even recognise it? AI can look well-reasoned and comprehensive, giving you a sense of security when you should have nothing of the sort.

How’s this for candour: “LLMs can state incorrect figures, invent products or rules that don’t exist, or misapply a rule from one country’s system (e.g., US 401k rules) to Australia’s superannuation system, all while sounding authoritative.”

9. No understanding of behavioural or emotional context

Factor in your state of mind? That you might be responding based on your childhood “conditioning” rather than actual conditions? Or that you might have a different risk tolerance to the questions you are asking?

An AI model can’t possibly do this. Indeed: “An LLM responds to the words in the prompt, not the psychology behind them.”

10. No ongoing relationship or follow-through

A financial plan needs to evolve with you. But we’ve come full-circle to the point above about how much your LLM remembers, and whether you actually want it to.

Finally, I loved AI’s very watery disclaimer at the end of the list, compared with the one we are required to use (see below!). “AI can be a decent starting point for financial literacy and generating questions to ask a professional, but it’s a poor substitute for licensed, personalised advice.”

So, would I use AI for financial information? Only while good primary sources of that information still exist and are disclosed so I can check the pedigree and recency. (And those days are numbered as AI drives such traffic-based sites out of business.)

But AI for financial advice? Not unless you could interrogate just the one trusted financial philosophy, teaching or expert. Why would you put your money where AI’s possibly hallucinating mouth is?

Nicole Pedersen-McKinnon is author of How to Get Mortgage-Free Like Me, available at nicolessmartmoney.com. Follow her on Facebook, X and Instagram.

• Advice given in this article is general in nature and is not intended to influence readers’ decisions about investing or financial products. They should always seek their own professional advice that takes into account their own personal circumstances before making any financial decisions.

Expert tips on how to save, invest and make the most of your money delivered to your inbox every Sunday. Sign up for our Real Money newsletter.