Investment Insights
July 10 2024

Challenges and Risks in Generative AI: Considerations for Investors

10 min read

Since OpenAI's ChatGPT went viral in late 2022 for its unprecedented ability to engage in human-like conversations and provide articulate responses in wide-ranging domains of knowledge, several competitors have begun introducing their own iterations of the technology. This type of AI technology, known as generative AI, is based on large language models that are trained on massive amounts of data, which could include text, images or other media. The models identify the patterns and structures of the training data and then generate new content that has similar characteristics based on user prompts.

There are various benefits to incorporating generative AI in a business – process improvements, cost reduction and value creation, to name a few. Leveraging these opportunities, companies across different sectors have already begun testing and implementing generative AI tools. Examples range from financial institutions deploying chatbots trained on internal databases to provide financial advice to customers to healthcare institutions automating the generation of medical documentation based on conversations between patients and physicians. Across industries, companies are also incorporating generative AI tools in marketing, customer service and product development.

As such, investors have to pay attention not only to large tech companies that are building the foundational models, but also to companies that are starting to incorporate generative AI tools into their business. As with most new technologies, there are potential risks that should be adequately considered and safeguarded against before widespread deployment. Regulation will be important in helping reduce these risks. But because the development of regulation occurs at a much slower pace than the development and application of AI, investors should actively consider its risks and seek stewardship opportunities in companies involved in generative AI to address these risks.

Challenges and Risks in Generative AI

Generative AI models have various known issues. These models have the tendency to "hallucinate," generating false outputs that are not justified by the training data and presenting them as a fact. These errors can be caused by various factors, such as improper model architecture or noise and divergences in the training data. Opaqueness about how model outcomes are generated is also an issue. With billions to trillions of model parameters that determine the probabilities of each part of its response, it is exceedingly difficult to map model outputs to the source data, including in cases of hallucination.

In addition, if the training data contains societal prejudices or if the algorithm design is influenced by human biases, the model may learn and propagate these biases in its outputs. Enterprise applications could also be vulnerable to data privacy issues and cybersecurity threats. This includes leakage of sensitive information within the training data if the model is customer-/public-facing, usage of personal or sensitive data in model training that may have needed explicit consent to use, as well as malicious attacks from hackers that aim to manipulate model outputs.

These issues give rise to various legal and reputational risks, the scale of which depends on the criticality of the use case and the company's industry. For example, the financial and healthcare industries may be subject to severe consequences if problems do arise, due to the high-stakes nature of these industries.

Sample Use Cases in the Financial Industry

In financial advisory use cases, model hallucinations could give inappropriate advice or offer the wrong product to undiscerning clients, which could undermine public trust in AI systems and the financial institutions using them. Opaqueness about how model outcomes are generated is also a key issue for financial institutions, as these institutions are required to be able to explain their decisions internally and to external stakeholders. Considering all this, it is best practice to implement a degree of separation between direct model outputs and the customer, where internal staff could be trained to recognize potential errors and inconsistencies in model outputs and assume ultimate responsibility for the decision-making process.

Generative AI could also offer a quick and low-cost way for financial institutions to profile their clients for marketing campaigns, risk management and identification of suspicious transactions. However, overreliance on generative AI profiling could violate anti-discrimination laws due to potential bias embedded within the models. Appropriate human judgment will need to complement generative AI models that perform client profiling. Financial institutions will also need to have strong data privacy policies and robust cybersecurity measures to address generative AI's risks to their sensitive client information and proprietary data.

Questions for Investors to Consider

In view of all these issues and risks, below are questions investors should consider when assessing companies employing generative AI tools:

  • What are the risk-mitigating mechanisms and/or circumstances? Solutions include having trained internal staff act as an intermediary between direct model outputs and the customer; working to understand potential biases in the training data and address them in model design; regular and proactive monitoring of model output to promptly identify and address any signs of hallucinations; implementing robust cybersecurity measures; etc.
  • What is being done to enhance model performance? Solutions include ensuring that training data is high quality, accurate and up to date; implementing iterative feedback loops to refine and improve model performance; etc.
  • Are there any transparency and oversight on ethical AI principles? This pertains to providing transparency on data sourcing and data privacy concerns; defining clear policies and procedures to ensure compliance with ethical standards and emerging regulations; outlining the roles and responsibilities of individuals involved in the development, operation and oversight of the generative AI model; etc.

The information contained herein is for information purposes only. The information has been drawn from sources believed to be reliable. Graphs and charts are used for illustrative purposes only and do not reflect future values or future performance of any investment. The information does not provide financial, legal, tax, accounting or investment advice. Particular investment, tax or trading strategies should be evaluated relative to each individual's objectives and risk tolerance.

This material is not an offer to any person in any jurisdiction where unlawful or unauthorized. These materials have not been reviewed by and are not registered with any securities or other regulatory authority in jurisdictions where we operate.

Any general discussion or opinions contained within these materials regarding securities, real assets or market conditions represent our view or the view of the source cited. Unless otherwise indicated, such view is as of the date noted and is subject to change. Information about the portfolio holdings, asset allocation or diversification is historical and is subject to change.

Certain statements in this document may contain forward-looking statements (“FLS”) that are predictive in nature and may include words such as “expects”, “anticipates”, “intends”, “believes”, “estimates” and similar forward-looking expressions or negative versions thereof. FLS are based on current expectations and projections about future general economic, political and relevant market factors, such as interest and foreign exchange rates, equity and capital markets, the general business environment, assuming no changes to tax or other laws or government regulation or catastrophic events. Expectations and projections about future events are inherently subject to risks and uncertainties, which may be unforeseeable. Such expectations and projections may be incorrect in the future. FLS are not guarantees of future performance. Actual events could differ materially from those expressed or implied in any FLS. A number of important factors including those factors set out above can contribute to these digressions. Reliance on FLS should be avoided.

No investment strategy or risk management technique can guarantee returns or eliminate risk in any market environment. Any characteristics, guidelines, constraints or other information provided for this material is representative of the investment strategy and is provided for illustrative purposes only. Any information about the holdings, asset allocation, or sector diversification is historical and is not an indication of future performance or any future portfolio composition, which will vary. Portfolio holdings are representative of the strategy, are subject to change at any time and are not a recommendation to buy or sell a security. The securities identified and described do not represent all of the securities purchased, sold or recommended for the portfolio. It should not be assumed that an investment in these securities or sectors was or will be profitable.

If derivatives are employed, note that investing in derivative instruments involves risks different from, or possibly greater than, the risks associated with investing directly in securities and other traditional investments and, in a down market, could become harder to value or sell at a fair price.

The investments, commodities, derivatives, and other financial instruments disclosed herein may or may not be current holdings or positions in any strategy. Any reference to a particular commodity, commodity derivative, company, issuer, market, contract, or financial instrument is provided for illustrative purposes only and does not constitute a recommendation to buy, sell, hold, or otherwise transact in any such investment.

TD Asset Management represents TD Asset Management Inc. which operates in Canada and TD Asset Management (US), Inc. which operates in the U.S. Both entities are affiliates and wholly-owned subsidiaries of The Toronto-Dominion Bank.TD Asset Management products are also available through a network of affiliated and unaffiliated distributors.  Please contact our distribution partners to find out more.

®The TD logo and other TD trademarks are the property of The Toronto-Dominion Bank or its subsidiaries.

For Jurisdictions Outside of Canada and the United States: 

Australia: TD Asset Management (US), Inc. (ARBN: 636409320) holds an Australian Financial Services License (AFS License No: 5308587) and is incorporated in Delaware, USA (liability of members is limited). To the extent any statements contained in this document constitute financial product advice, those statements are issued by TD Asset Management (US), Inc. The information contained herein is intended for wholesale clients and investors only as defined in the Corporations Act of 2001.

New Zealand: This document is not an offer of financial products requiring disclosure under the Financial Markets Conduct Act 2013 (New Zealand). This document is only for “wholesale investors” within the meaning of clause 3(2) of Schedule 1 of the Act and any offer of financial products will only be made to “wholesale investors”. Neither TD Asset Management Inc., TD Asset Management (US), Inc., nor The Toronto-Dominion Bank are a registered bank in New Zealand.