INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent — part 5/6

Cleaned-text code points [54066, 69865). Bibliography omitted here; source.md includes it.

Short-term Memory
1 Sentiment: Negative: The key insights from
the news regarding Tesla Inc (NASDAQ:
TSLA) losing market share to Ford Motor
Company’s (NYSE: F) Mustang Mach-E in
the United States are as follows:1. **Mar-
ket Competition**: Tesla is facing increased
competition in the electric vehicle (EV) mar-
ket, particularly from established automakers
like Ford. The Mustang Mach-E’s success in-
dicates that other companies are effectively
entering the EV space, which could impact
Tesla’s dominance.2. **Market Share Im-
pact**: The loss of market share to Ford sug-
gests that Tesla’s growth trajectory in the U.S.
may be challenged. Investors should consider
the potential implications for Tesla’s revenue
and proﬁtability if this trend continues...
2 Sentiment: Positive : The news about a
Banksy artwork being burned and sold as a
non-fungible token (NFT) highlights a few
key insights relevant to investment decisions,
particularly concerning innovative and dis-
ruptive technologies: 1. **Emergence of
NFTs**: The transformation of physical art
into digital assets through NFTs signiﬁes a
growing trend in the digital economy. This
trend could inﬂuence sectors beyond art, in-
cluding technology and ﬁnance, as more in-
dustries explore blockchain applications ...
3 Sentiment: Neutral : To provide a summary
of key insights relevant to making investment
decisions about Tesla (TSLA) from the pre-
open movers news, I would focus on the fol-
lowing aspects: 1. **Stock Performance**:
Look for any signiﬁcant pre-market price
movements for TSLA. If the stock is show-
ing a notable increase or decrease, it could
indicate investor sentiment or reaction to re-
cent news.2. **News Catalysts**: Identify
any speciﬁc news items or announcements re-
lated to Tesla that might be inﬂuencing its
stock price. This could include earnings re-
ports, product launches, regulatory news, or
changes in leadership...
4 Sentiment: positive : The key insights from
the news about Tesla’s vehicle registrations
in Germany are as follows: 1. **Signiﬁcant
Growth in Registrations**: Tesla experienced
a 78% year-over-year increase in vehicle reg-
istrations in Germany as of January. This sub-
stantial growth indicates a strong demand for
Tesla vehicles in one of Europe’s largest au-
tomotive markets.2. **Market Penetration**:
The surge in registrations suggests that Tesla
is successfully penetrating the German mar-
ket, which is traditionally dominated by local
automakers. This could imply a growing ac-
ceptance and preference for electric vehicles
(EVs) in Germany, beneﬁting Tesla as a lead-
ing EV manufacturer...
...
Mid-term Memory
1 Sentiment: Negative : The news about U.S.
Senator Elizabeth Warren’s proposal for a
Democratic wealth tax could have several im-
plications for investment decisions regarding
Tesla (TSLA):1. **Impact on Wealthy In-
vestors**: The proposed wealth tax targets the
wealthiest Americans, which could include
major shareholders and investors in Tesla. If
implemented, this tax could lead to changes in
investment strategies among these individuals,
potentially affecting Tesla’s stock price due
to shifts in large-scale holdings....5. **Long-
term Implications**: While the immediate
impact might be uncertain, the long-term im-
plications of a wealth tax could lead to a more
equitable distribution of wealth, potentially
increasing consumer spending power. This
could beneﬁt Tesla if it results in a larger mar-
ket for its products. Investors in Tesla should
closely monitor the progress of this proposal
and consider its potential effects on market
dynamics and investor behavior.
2 Sentiment: positive : The key insights from
the news information regarding consumer
preferences in China for electric cars, partic-
ularly focusing on price and driving range,
can be relevant for making investment deci-
sions about Tesla (TSLA) in the following
ways: 1. **Market Demand Alignment**:
Tesla’s ability to align its product offerings
with the primary concerns of Chinese con-
sumers 2014price and driving range˘2014could
signiﬁcantly impact its market share and sales
growth in China. Investors should monitor
how Tesla’s models compare to competitors
in these aspects... Overall, Tesla’s success in
2521

addressing the key consumer preferences in
China 2014price and driving range 2014will
be critical for its growth prospects in the re-
gion, making these factors important consid-
erations for investors.
...
Long-term Memory
1 Sentiment: Negative: The key insights from
the news about Tesla Inc. (NASDAQ: TSLA)
facing competition in China are as follows: 1.
**Increased Competition**: Tesla’s Model 3
is experiencing signiﬁcant competition from
a budget electric vehicle produced through
a joint venture involving General Motors
(NYSE: GM). This indicates a more compet-
itive landscape in the Chinese electric vehi-
cle (EV) market, which could impact Tesla’s
market share and sales growth in the region...
For investors, these insights suggest a need
to closely watch Tesla’s strategic responses
to competition in China, its pricing strate-
gies, and any potential impacts on its ﬁnan-
cial performance. Additionally, understanding
the broader competitive landscape and Tesla’s
ability to sustain its growth trajectory will be
crucial for making informed investment deci-
sions.
2 Sentiment: positive : The news about Bill
Gates’ concerns regarding Bitcoin’s impact
on climate change highlights a broader issue
of environmental sustainability in the tech
and ﬁnancial sectors. Here are the key in-
sights relevant to making investment decisions
about Tesla (TSLA): 1. **Environmental Im-
pact Awareness**: Bill Gates’ concerns un-
derscore the growing awareness and scrutiny
of the environmental impact of technology
and ﬁnancial products. This is relevant for
Tesla, as the company positions itself as a
leader in sustainable energy and electric ve-
hicles (EVs)... Overall, the emphasis on en-
vironmental impact and sustainability in the
tech sector could reinforce Tesla’s strategic ad-
vantages and appeal to investors prioritizing
green investments.
...
D Extended Task: Multi-Asset Portfolio
Management Task
Besides the single-asset trading tasks presented
in the paper, I NVESTOR BENCH is able to incor-
porate the multi-asset portfolio management task,
which further illustrates I NVESTOR BENCH s gener-
alization capabilities. This advanced task involves
more sophisticated trading strategies and mathemat-
ical inference, allowing the agent to dynamically
allocate asset weights at each decision step and
rebalance the portfolio based on current market
conditions.
The experimental results for a compact portfo-
lio consisting of TSLA, JNJ, and UVV are shown
in Table 5, using three LLMs as agent backbones
(one from each size category). The I NVESTOR -
BENCH agent’s performance on this portfolio man-
agement task across LLMs aligns with the overall
trend with the single-asset trading task presented in
Figure 3b in our paper, the agent with the large-size
LLM delivers the best performance, while that with
small-size and mid-size LLMs performs lower and
closely in terms of primary evaluation metrics, Cu-
mulative Returns (CR) and Sharpe Ratio (SR). This
illustrates the consistency of our decision-making
agent performance across distinct types of LLMs
for various task types and trading strategies.
Table 5: Comparison of performance across LLMs of
varying sizes in multi-asset trading tasks
Model Type CR SR A V MDD
Llama-3.1-70B-Instruct 31.473 1.482 17.100 11.205
Yi-1.5-34B-Chat 15.458 0.903 13.769 9.883
Llama-3.1-8B-Instruct 20.255 0.953 17.106 11.219
E Case Study on I NVESTOR BENCH in
Extreme Market Conditions
Table 6: Comparison of performance across LLMs of
varying sizes in the extreme market conditions
Model Type CR SR A V MDD
Buy & Hold -56.738 -0.936 53.911 52.078
GPT-4 10.163 0.433 20.866 12.672
Llama-3.1-70B-Instruct -27.596 -0.633 38.708 34.797
Yi-1.5-34B-Chat -32.723 -0.606 48.012 41.197
Llama-3.1-8B-Instruct -37.56 -0.736 45.350 47.748
Here, we use a case study to demonstrate I N-
VESTOR BENCH ’s robust performance during peri-
ods of extreme market featured by signiﬁcant mar-
ket volatility.
2522

Table 7: Performance Comparison of LLMs in Multi-Asset Trading Tasks under Different Market Conditions
Model Type Bullish Bearish Mixed Signal
CR SR A V MDD CR SR A V MDD CR SR A V MDD
Buy and hold 51.987 3.654 36.322 12.551 -96.218 -2.884 60.645 65.24 39.244 0.600 52.339 37.975
GPT-4 27.219 2.095 33.163 12.551 61.236 1.781 62.513 18.759 45.246 0.821 44.088 25.031
Llama-3.1-70B-Instruct 43.926 3.144 36.349 12.551 20.819 0595 63.612 40.686 37.545 0.615 48.862 29.813
Yi-1.5-34B-Chat 17.571 1.790 25.062 5.617 10.424 0.317 59.668 41.988 35.364 0.558 50.757 35.49
Llama-3.1-8B-Instruct 18.685 1.417 33.655 9.778 -39.44 -1.386 51.737 44.274 35.622 0.574 49.636 36.383
We evaluated the asset, TSLA, by examining Cu-
mulative Returns (CRs) and Sharpe Ratios (SRs),
Annualized V olatility (A V) and Maximum Draw-
down(MDD). Our dataset spans a training period
from 2022-01-17 to 2022-03-31 and a test period
from 2022-04-01 to 2022-10-15. These periods
are selected because the VIX (CBOE V olatility
Index) remained elevatedaveraging above 20indi-
cating heightened market volatility.
We present the I NVESTOR BENCH agent’s per-
formance using four representative LLMs as back-
bonesone proprietary model (GPT-4) and three
open-source models of varying sizes (Llama-3.1-
70B-Instruct, Yi-1.5-34B-Chat, and Llama-3.1-8B-
Instruct)chosen for their relatively stable perfor-
mance (See Table 6 below). Overall, the agents
performance with each LLM aligns with the re-
sults in Table 2 as well as Figure 3a and 3b in
our paper. Speciﬁcally, 1) The proprietary LLM
outperforms the open-source models. 2) Larger
model parameter sizes consistently lead to higher
decision-making quality and robustness.
F Case Study on I NVESTOR BENCH
across Different Market Scenarios
We introduced an additional set of case studies
in bullish, bearish, and mixed-signal market envi-
ronments to highlight the robustness of the agents
performance in I NVESTOR BENCH .
We assess the I NVESTORBENCH agent’s perfor-
mance robustness using three separate time periods
with distinct market trends on the same assets, task
type, and LLMs. For the bullish market, the train-
ing period spanned from 2023-03-06 to 2023-05-
01, followed by a test period from 2023-05-02 to
2023-07-10. For the bearish market, the training
period ran from 2022-08-08 to 2022-09-26, and the
test period from 2022-09-27 to 2023-01-02. We
present Bullish and Bearish results below Table 7.
For the mixed market condition, we employed the
same dates speciﬁed in Section 4.1 of the paper.
In the mixed-signal and bearish market condi-
tions, the agents performance trends across various
LLMs remain closely aligned with the TSLA trad-
ing case shown in Table 2 as well as Figure 3a and
3b of our paper. In the bullish market scenario,
while larger LLMs generally maintain stronger per-
formance, the simple Buy-and-Hold strategysup-
ported by a continuously rising asset priceoutper-
forms strategies provided by the agent that con-
sider multiple investment perspectives. The latter
approach can be inﬂuenced by noisy market sig-
nals, such as occasional neutral or negative news.
The proprietary LLM GPT-4, characterized by its
comprehensive reasoning, exhibits more caution
and thus lower returns compared to the large-scale
open-source model Llama-3.1-70B-Instruct. How-
ever, both still surpass other LLMs, consistent with
several asset trading cases reported in Table 2.
Our case study proves that the I NVEST BENCH
agent’s performance stays robust and consistent
with the analysis outcomes in our paper in general.
G Case Study on Single Stock Trading:
Forecast for TSLA on 2022-10-25 to
Predict Trading Decision on 2022-10-26
2523

Initialize Proﬁle
1. Operations:
- Provide a performance overview of the trading stock based on available data.
- Set up the risk inclination as the key character of the trading agent.
2. Range: Financial information such as the ﬁnancial sectors, historical performance, and previous stock trends of the
trading stock.
3. Prompts: Y ou are an experienced trading manager and investment ﬁrm. Y our task is to make informed decisions on
the given stock based on the provided information.
Under Self-Adaptive Risk Character Setting: When historical momentum is positive, you are a risk-seeking investor.
But when historical momentum is negative, you are a risk-averse investor.
4. General background setting:
Y ou have accumulated a lot of information about the following sectors, so you are especially good at trading them:
1)Electric V ehicles (Automotive Sector). 2) Energy Generation and Storage...From year 2021 to 2022 September,
Tesla’s continued growth and solid ﬁnancial performance over the deﬁned period ...
Summarize
1. Operations:
- Summarize different types of input information.
- Distribute them to corresponding layers of the long-term memory database.
2. Range: Daily market news, Long Documents such as company 10-K and 10-Q reports
3. Prompts:
- (1). Summarize the contents: Summarize the following documents into 1000 words.
- (2). Comprehend the investment sentiment of news insights: The positive, neutral and negative scores are for
understanding the investment sentiments, opinions, or emotions. For example, positive news about a company can lift
investor sentiment, encouraging more buying activity, which in turn can push stock prices higher...
4. Outputs:
(1). To Shallow Memory Layer:
- [News (ID: 261)] Here’s How Much Y ou Would Have Made Owning Tesla Stock In The Last 10 Y ears Tesla
(NASDAQ:TSLA) has outperformed the market over the past 10 years by 50.69% on an annualized basis producing an
average annual return of 60.76%. Currently, Tesla has a market capitalization of $683.54 billion.... The sentiment is
{positive}.
- [News (ID: 278)] Tesla Q3 Earnings Are Imminent. Can Nio Foreshadow What’s To Come? What To Know Before
The Print Tesla Inc (NASDAQ: TSLA) shares were trading down slightly Wednesday afternoon ahead of the automaker ´s
third-quarter report, but the stock is up 6% over the last ﬁve sessions... The sentiment is {positive}.
- ...
(2). To Intermediate Memory Layer:
- [Form 10-Q (ID: 222)] Tesla Q3 2022 revenues were $21.5 billion, up 56% year-over-year. Automotive sales revenue
grew 56% to $17.8 billion driven by higher Model 3/Y and Model S/X deliveries. Gross automotive margin declined to
27.9% due to cost inﬂation and factory ramps. Net income was $3.3 billion, up 102% year-over-year. Positive free cash
ﬂow was $6.1 billion...
- [News (ID: 275)] Tesla Q3 Earnings Highlights: Record Revenue, Operating Margin And Free Cash Flow, Tesla Semi
Deliveries Coming In December Electric vehicle leader Tesla Inc (NASDAQ: TSLA) reported third-quarter ﬁnancial
results after market close Wednesday...The sentiment is {neutral}.
- [News (ID: 274)] Tesla Preps For 2023 Cybertruck Launch, Will Make Battery Packs In California The Cybertruck is
one of Tesla Inc. (NASDAQ: TSLA) most hotly anticipated, but also most delayed, products. - ...The sentiment is
{negative}.
(3). To Deep Memory Layer:
- [News (ID: 161)] Tesla Whale Trades Spotted A whale with a lot of money to spend has taken a noticeably bearish
stance on Tesla. Looking at the options history for Tesla (NASDAQ:TSLA) we detected 477 strange trades. The
sentiment is {positive}.
- [Self-reﬂection (ID: 226)] Given the short-term positive news score in the market for TSLA and a positive cumulative
return, there is a high probability of continued growth in the short term. However, investor should be aware of potential
threats in the mid-term market with competitors like General Motors, and Nio...
2524