AI (ChatGPT): Financial Wizard or Wishful Thinking?
At WESCAP, we have been using and testing ChatGPT 4.0 for some time.
Chat GPT 4 is fast and has vast amounts of data and information available to it. It can write spreadsheet formulas, do various math calculations. It can understand conversations and create well-written responses. It has aced legal bar exams and similar difficult tests that far exceed the average person’s ability to do so.
It and other similar large language models use a deep-learning neural network model. When asked about its own drawbacks and limitations, here is how ChatGPT responded:
- Understanding Limits: ChatGPT generates text based on statistical learning and lacks genuine comprehension, leading to potential errors or irrelevance.
- Data Bias: Training on internet data can embed existing biases into its responses.
- Static Knowledge: It doesn’t update its knowledge post-training, limiting its responses to the data it was last trained on.
- Resource Intensity: Requires significant computational resources for training.
- Misuse Potential: Its convincing text generation can be misused, such as in creating misleading information.
- No Emotional Intelligence: Lacks true emotional understanding, affecting its efficacy in sensitive interactions.
- Opacity: Often considered “black boxes,” neural networks provide little insight into their internal decision-making processes, posing challenges for interpretation.
- Overfitting: Without careful application of regularization techniques, neural networks risk fitting too closely to the training data, thus performing poorly on new datasets.
When asked how it compares to a human brain, here is how ChatGPT responded:
“The complexity, dynamism, and functionality of connections in the human brain are far more intricate and capable of more complex processing than the static parameters of a neural network. The human brain’s ability to learn, adapt, and interact with the world is profoundly more sophisticated and efficient than any current artificial system.”
So ChatGPT appears to be honest in its self-assessment in that it “lacks genuine comprehension, leading to potential errors or irrelevance.” Its “black box” nature makes it impossible for anyone, including its makers, to truly understand why it gives the answers it does.
At WESCAP, we have used it to find information faster than doing web browser searches. It can write custom reports and summaries on many topics very quickly as well. However, it often gets important facts wrong, such as confusing capital gain income with ordinary income when reviewing various tax sensitive topics. It failed to differentiate between open-end and closed-end mutual funds when generating a list for us for future research. After pointing out its errors, it often said “I apologize for the error” and then it would often, but not always, get it right on the next iteration. Its lack of true comprehension showed up in many responses on technical details. When asking ChatGPT to do an investment strategy simulation, it sounded very convincing, but when prodded to show the details, the results were clearly nonsensical, and after more prodding it could not come up with why it responded wrongly and why it could not get it right.
Therefore, when it comes to using any large language, neural network artificial intelligence model, it pays to already have a good understanding of the topics and expected output, so that human actual intelligence with real critical thinking comprehension can supervise the artificial intelligence.
An apt analogy for ChatGPT and similar AI systems is having an assistant who is extremely fast, has a photographic memory of everything written, is honest, and never gets tired or becomes irritable. This assistant has good writing and excellent math skills. However, the assistant is inexperienced, glib, has an attention deficit disorder, is over-confident, lacks critical thinking skills, can be gullible, and picks the first response that comes to mind, right or wrong. Nevertheless, errors will be corrected immediately when pointed out, if it is not beyond the computational ability to do so. In other words, ChatGPT can be a good assistant for some tasks, but one that needs well-thought-out directions, constant supervision, and independent fact-checking.
Therefore, it is up to human actual intelligence to double check all responses for accuracy and reasonableness. Prompt any AI system for more details. Make multiple requests to it in somewhat different ways to see if the response is roughly the same. Make sure it gives examples and illustrations if the topics have any complexity or calculations. Ask for additional details to get a better understanding of its response. You can always ask it to summarize or reduce its response later if it is too long or too detailed. Do independent research before and after using any AI so that errors can be identified and then make sure it agrees with the error correction. Edit any written responses that are too long, too short, or do not emphasize the critical points and facts. Delete any irrelevant or off-point facts.
This whole report was submitted to ChatGPT for grammar and comprehension check. It came up with a few improvements that did not change anything of substance, so we included most of the recommended changes, including a snappier title. Therefore, it was a good assistant in this case.
Undoubtedly ChatGPT and other similar tools will improve with time. At this point, WESCAP is not currently concerned about being replaced by ChatGPT or any AI.