Putting Ai US Tax Advice to The Test

Keeping up with the breakneck speed of Ai development can seem almost impossible, and its advancement in particular areas has been very noticeable.

This can leave many of us wondering how effective ChatGPT (or its rivals) can be for tax accounting, advice or research.

To begin with, of course my view on Ai tax advice is biased, given that our firm provides tax consulting services. However, we tend to provide advice to those with more specific or important questions (ie those that absolutely must be right), whereby the client needs trusted US advice and is unwilling to trust Ai. As a result, we’ve not seen much change in our booking demand as a result of Ai.

For those broader questions however, maybe those that don’t warrant professional tax advice, Ai can sometimes be more suitable. My personal opinion is that Ai can be an excellent tool to begin research, however answers can be incomplete or vague. For taxpayers looking to understand basic information on taxation, it can be a useful tool to save on accountancy and US tax advisory fees.

However, one key part of obtaining US (or NZ) tax advice, is knowing what questions to ask.

Professional US tax specialists are able to understand a situation, and discuss items or issues which a client may not have thought of, something that Ai tools are not usually able to replicate.

We recently consulted with a client whom had followed advice obtained from an online Ai tool. This information turned out to be incomplete at best, but could be considered incorrect. This resulted in an amended return needing to be filed in the US, and a voluntary disclosure in New Zealand.

This unfortunate event did prompt me to do a little further research on the impact of Ai tools, on tax advice.

Using Ai for US Tax Planning

Obtaining tax advice from Ai tools can certainly seem appealing, given its ease of use and availability. But how accurate is it?

Well, we’ve put it to the test with some basic questions, and the results were not great. Interestingly, we noticed that when we put the same question to the Ai tools multiple times, the answers were different each time, sometimes mostly correct, but sometimes not.

Whilst I was impressed with the answers it gave correctly, a success rate of 50% left me with some concern.

As mentioned above, the Ai gave generally correct very broad answers, which can be useful for those seeking basic guidance. But on the more specific questions, it struggled.

We spread the questions over 3 different platforms, with mixed results. Overall, we tested the Ai platforms with 6 different questions (two to each platform) of varying complexity, of which three it gave mostly correct answers (although vague), but the other three not so correct.

To note, the Ai tools gave very long answers to our questions below, so we have significantly shortened them.

Lets delve in:

Question: How are jointly reported capital losses carried forward treated when switched to separate returns for taxpayers?”

Ai response: Jointly reported capital losses carried forward are separated on a 50% basis for each taxpayer on their respective separate returns.

This is not correct, and upon prompting the Ai to re-answer the question:

Oh, I’m sorry, I made a mistake there, thank you for letting me know. Jointly reported capital losses carried forward are split based on which taxpayer originally incurred the losses

Not a great start…

Question: Is the government contribution to KiwiSaver considered to be general limitation income for purposes of US income tax reporting, or passive income

Ai response: The IRS categorises income from a KiwiSaver account in New Zealand in various and sometimes complex ways. A government contribution is usually treated as general limitation income.

Again, incorrect. Bizarrely, the answer was followed up by a large general description of how KiwiSaver is reported for US tax purposes, and I did notice that the writing looked like my own style. Upon checking the sources, I noticed it was indeed pulling information from The US Tax Team website.

However, on our website, we do not reference general limitation income at all, and I would suggest that this is one of the major risks of Ai tax research. It tends to pull correct information from reputable sources, understand it, and then apply it incorrectly to other information, ultimately resulting in the wrong answer.

Question: Can I switch from the accrued method for foreign tax credit (form 1116) to the paid method?

Ai Response: Yes, certainly! Changing from the accrued method to paid method for foreign tax credit can be done by making a request to the IRS commissioner.

Technically possible, depending on whom you speak to, but not at all common. Most other professionals I’ve discussed this with in the industry, are of the opinion that this is in essence an impossible process, without obtaining a private letter ruling (potentially $20-30k USD and still discretionary).

Sources of Ai information

Of course, we know that Ai tools scour the entire internet to develop responses, however what is important to remember is that it can only search publicly available information.

However, understanding the US tax code and the IRS interpretation and imposition of it is key to tax compliance and tax efficiency. Much of this interpretation, analysis and tax court or ruling history is behind paywalls. Simply, unavailable for Ai tools to access.

As mentioned above, amongst the sources that the Ai tool referenced, our website (usatax.nz) was referenced multiple times. Each time, it took generalised correct information, and attempted to apply it to the US tax code. Whilst this is in part flattering, it is also a cause for concern.

On our website, we publish a large number of articles and posts, covering various different tax topics that affect US citizens. However, our articles are all generalised, and not intended to provide conclusive tax advice to the reader. We make this clear in all of our articles, and ask readers not to rely on generalised, broad information.

When we’re dealing with generalised information, of course it may well apply to 50% of people, however the other 50% will likely need specific, personalised tax advice.

Concerningly, the US tax code as a source of information was not referenced once in any of the questions we posed to the Ai tool.

Despite Ai software having well posted warnings that it makes mistakes, it can be understandably be appealing to the user to assume that the information presented is correct.

Ai Tax Articles Published as Original Material

We’ve discussed above the risks of taxpayers using Ai as bona-fide correct information, but what about when tax professionals are doing so?

We’ve noticed an increasing number of articles online, published by in some cases by reputable tax firms, which are fully Ai generated material (and posing it as original content).

In at least one article, I’ve seen a clear error, however the article was not tagged as Ai generated.

This does substantially raise the risk of a taxpayer assuming the information published is correct, when it comes from an ordinarily reputable source.

Benefits of Ai Research

One key area that Ai tax research can be beneficial, is to set the stage for further tax advice. We’ve noticed over the last year, an increasing number of our tax advice clients are arriving for our initial meeting with a better understanding of the issues we’ll discuss.

By obtaining initial broad information through Ai, it can help make tax advice appointments more efficient, and to help make better use of the time (or even to use less time and thus lower fees).

We do pride ourselves on making tax advice understandable for clients, and walking them through the advice in a human manner. We don’t need or expect our clients to understand different financial terminology, and are happy to explain detail on any level.

However, since the advent of Ai tools, we do see clients coming to see us having already obtained their first tranche of knowledge through Ai, and then enabling us to fill in the gaps. It certainly does help reduce the time the clients need.

Summary

Whilst the advancement of free online Ai tools has been remarkable, users should still be wary about trusting any information produced by them as bona-fide fact. For self-research on the US tax system, be sure to obtain information from a number of sources before arriving at a conclusion, and where this is insufficient, specialist US tax advice from a trusted professional should be sought.

At The US Tax Team New Zealand, we provide specialist US tax advice in a manner that clients can understand. Learn more at http://usatax.nz, info@usatax.nz or 09-242-3445