CEO-Bench: Can Agents Play the Long Game? — part 4/7

Cleaned-text code points [47612, 62420). Bibliography omitted here; source.md includes it.

Let 𝒫 \mathcal{P} be the active plan set. For customer i i , the simulator evaluates all plans and chooses the acceptable plan with the largest surplus: p i , t ∗ = arg max p ∈ 𝒫 [ Q i , p , t perc − Q i req ( C i , p , t eff ) ] , 𝒜 i , t = { p ∈ 𝒫 : C i , p , t eff ≤ c i , Q i , p , t perc ≥ Q i req ( C i , p , t eff ) } . p_{i,t}^{*}=\arg\max_{p\in\mathcal{P}}\left[Q_{i,p,t}^{\mathrm{perc}}-Q_{i}^{\mathrm{req}}(C_{i,p,t}^{\mathrm{eff}})\right],\qquad\mathcal{A}_{i,t}=\{p\in\mathcal{P}:C_{i,p,t}^{\mathrm{eff}}\leq c_{i},\;Q_{i,p,t}^{\mathrm{perc}}\geq Q_{i}^{\mathrm{req}}(C_{i,p,t}^{\mathrm{eff}})\}. (6) The choice is valid only when p i , t ∗ ∈ 𝒜 i , t p_{i,t}^{*}\in\mathcal{A}_{i,t} ; if 𝒜 i , t \mathcal{A}_{i,t} is empty on a billing decision day, the customer cancels. 𝒫 \mathcal{P} is the set of plans currently offered by the company, C i , p , t eff C_{i,p,t}^{\mathrm{eff}} is the price after active promotions, Q i , p , t perc Q_{i,p,t}^{\mathrm{perc}} is perceived quality for plan p p , c i c_{i} is the customer’s budget ceiling, and 𝒜 i , t \mathcal{A}_{i,t} is the acceptable-plan set. The surplus term measures how much perceived quality exceeds the customer’s minimum requirement at that effective price. This creates a simple operator intuition: customers do not maximize quality alone or price alone; they choose the plan that clears their personal price-quality bar with the best headroom. In reality, this corresponds to monthly subscription review: customers may upgrade, downgrade, stay, or churn depending on the menu of plans available at renewal time.

### A.3 Product Quality, Usage, and Monetization

#### Promotions and effective price.

Promotions are additive across global, group, customer, and group-plan scopes. At billing time, C i , p , t eff = [ P p , t − Π t global − Π g , t group − Π i , t customer − Π g , p , t group ​ - ​ plan − 𝟙 ​ { first i , t } ​ Π g , t lead ] + . C_{i,p,t}^{\mathrm{eff}}=\left[P_{p,t}-\Pi_{t}^{\mathrm{global}}-\Pi_{g,t}^{\mathrm{group}}-\Pi_{i,t}^{\mathrm{customer}}-\Pi_{g,p,t}^{\mathrm{group\text{-}plan}}-\mathbb{1}\{\mathrm{first}_{i,t}\}\Pi_{g,t}^{\mathrm{lead}}\right]_{+}. (7) Here P p , t P_{p,t} is the listed price for plan p p , Π t global \Pi_{t}^{\mathrm{global}} is a site-wide discount, Π g , t group \Pi_{g,t}^{\mathrm{group}} targets a customer segment, Π i , t customer \Pi_{i,t}^{\mathrm{customer}} targets an individual customer, Π g , p , t group ​ - ​ plan \Pi_{g,p,t}^{\mathrm{group\text{-}plan}} targets a segment-plan pair, Π g , t lead \Pi_{g,t}^{\mathrm{lead}} is a first-bill lead promotion, first i , t \mathrm{first}_{i,t} marks the first billing event for a newly acquired customer, and [ ⋅ ] + [\cdot]_{+} floors price at zero. Promotions therefore act as dollar discounts rather than hidden quality boosts; they can help acquisition or retention but directly reduce revenue. The mechanism simulates couponing, contract discounts, and introductory offers, where revenue changes immediately even though the product itself has not improved.

#### Usage and capacity.

Each active subscriber has a daily usage draw u ~ i , t \tilde{u}_{i,t} , plan quota U p , t U_{p,t} , billing-period cumulative usage U ¯ i , t \bar{U}_{i,t} , and weekly usage multiplier W t W_{t} . The realized daily usage is u i , t = round ⁡ ( min ⁡ ( u ~ i , t ​ W t , [ U p i , t − U ¯ i , t ] + ) ) , U t tot = ∑ i u i , t . u_{i,t}=\operatorname{round}\!\left(\min\!\left(\tilde{u}_{i,t}W_{t},\,[U_{p_{i},t}-\bar{U}_{i,t}]_{+}\right)\right),\qquad U_{t}^{\mathrm{tot}}=\sum_{i}u_{i,t}. (8) u i , t u_{i,t} is the delivered usage units today, u ~ i , t \tilde{u}_{i,t} is the customer’s latent demand, W t W_{t} is a weekly demand multiplier, U p i , t U_{p_{i},t} is the quota on the customer’s active plan, U ¯ i , t \bar{U}_{i,t} is already-consumed usage in the billing period, and U t tot U_{t}^{\mathrm{tot}} is total platform load. The positive-part term enforces the remaining plan quota, and rounding maps continuous demand into discrete usage units. Plan quota also determines the quota fulfillment factor used in perceived quality below: lowering quota can reduce usage and compute cost, but when quota falls short of demand, it scales down the customer’s entire experienced quality. The mechanic makes high-growth strategies stress infrastructure: more customers and higher quotas produce more usage, which can create overload if capacity is not upgraded. This simulates a real software business where usage is bursty, quotas cap consumption, and product-market success can turn into an infrastructure problem.

#### Service health.

For capacity tier κ t \kappa_{t} with capacity K κ t K_{\kappa_{t}} and operations spend x t ops x_{t}^{\mathrm{ops}} , the overload level and outage probability are o t = [ U t tot K κ t − 1 ] + , P ( outage t ) = max ( p out min , p out 0 exp ( − x t ops / χ ops ) ) ( 1 + β out o o t ) . o_{t}=\left[\frac{U_{t}^{\mathrm{tot}}}{K_{\kappa_{t}}}-1\right]_{+},\qquad P(\mathrm{outage}_{t})=\max\!\left(p_{\mathrm{out}}^{\min},p_{\mathrm{out}}^{0}\exp(-x_{t}^{\mathrm{ops}}/\chi_{\mathrm{ops}})\right)\left(1+\beta_{\mathrm{out}}^{o}o_{t}\right). (9) o t o_{t} is zero when capacity covers demand and positive when load exceeds capacity; K κ t K_{\kappa_{t}} is the capacity available under tier κ t \kappa_{t} ; p out 0 p_{\mathrm{out}}^{0} is baseline outage risk; p out min p_{\mathrm{out}}^{\min} is the reliability floor; x t ops x_{t}^{\mathrm{ops}} is daily operations spend; χ ops \chi_{\mathrm{ops}} controls diminishing returns to operations spend; and β out o \beta_{\mathrm{out}}^{o} makes overload increase outage risk. Operations spending therefore buys reliability, while capacity tier choices buy headroom. This simulates the operational reality that SRE effort and cloud capacity reduce incidents, but overloaded systems remain fragile and no team can drive outage risk exactly to zero.

#### Delivered quality.

Delivered quality measures technical product quality before customer-specific perception effects: Q g , p , t del = ( q 0 + b t shared + b g , t group ) ​ m p − β o ​ o t − β out ​ 1 ​ { outage t } , Q_{g,p,t}^{\mathrm{del}}=\left(q_{0}+b_{t}^{\mathrm{shared}}+b_{g,t}^{\mathrm{group}}\right)m_{p}-\beta_{o}\,o_{t}-\beta_{\mathrm{out}}\,\mathbb{1}\{\mathrm{outage}_{t}\}, (10) where q 0 q_{0} is baseline product quality, b t shared b_{t}^{\mathrm{shared}} is shared quality from development and R&D, b g , t group b_{g,t}^{\mathrm{group}} is targeted group quality for segment g g , m p m_{p} is the model-tier multiplier for plan p p , o t o_{t} is overload, and 𝟙 ​ { outage t } \mathbb{1}\{\mathrm{outage}_{t}\} is the outage indicator. The coefficients β o \beta_{o} and β out \beta_{\mathrm{out}} translate infrastructure problems into quality loss. The product-quality terms model the underlying capability of the service, while the negative terms model degraded delivery. This separates product investment from delivery reliability: a strong product can still feel bad when overloaded, just as real customers judge both feature quality and whether the service actually works when they need it.

#### Development and R&D.

Daily development and targeted development add quality with diminishing returns: Δ ​ b t shared , dev = β dev ​ log ⁡ ( 1 + x t dev / χ dev ) , Δ ​ b g , t target = β target ​ log ⁡ ( 1 + x g , t target / χ target ) . \Delta b_{t}^{\mathrm{shared,dev}}=\beta_{\mathrm{dev}}\log(1+x_{t}^{\mathrm{dev}}/\chi_{\mathrm{dev}}),\qquad\Delta b_{g,t}^{\mathrm{target}}=\beta_{\mathrm{target}}\log(1+x_{g,t}^{\mathrm{target}}/\chi_{\mathrm{target}}). (11) x t dev x_{t}^{\mathrm{dev}} is global development spend, x g , t target x_{g,t}^{\mathrm{target}} is targeted development spend for group g g , Δ ​ b t shared , dev \Delta b_{t}^{\mathrm{shared,dev}} is the daily shared-quality increment, Δ ​ b g , t target \Delta b_{g,t}^{\mathrm{target}} is the daily group-specific increment, β dev , β target \beta_{\mathrm{dev}},\beta_{\mathrm{target}} convert spend into quality, and χ dev , χ target \chi_{\mathrm{dev}},\chi_{\mathrm{target}} control diminishing returns. The logarithm makes the first dollars of engineering spend more productive than later dollars, reflecting coordination overhead and finite easy fixes. This simulates staffing and engineering allocation: basic improvements can be made quickly, but pushing quality further requires disproportionately more effort. R&D projects are larger delayed improvements: D j R & D ∼ 𝒟 r j time , G j R & D ∼ 𝒟 r j quality , b t + 1 shared = b t shared + Δ b t shared , dev + ∑ j : t = t j done G j R & D + ϵ t q . D_{j}^{\mathrm{R\&D}}\sim\mathcal{D}_{r_{j}}^{\mathrm{time}},\qquad G_{j}^{\mathrm{R\&D}}\sim\mathcal{D}_{r_{j}}^{\mathrm{quality}},\qquad b_{t+1}^{\mathrm{shared}}=b_{t}^{\mathrm{shared}}+\Delta b_{t}^{\mathrm{shared,dev}}+\sum_{j:\,t=t_{j}^{\mathrm{done}}}G_{j}^{\mathrm{R\&D}}+\epsilon_{t}^{q}. (12) j j indexes a research project, r j r_{j} is its tier, D j R & D D_{j}^{\mathrm{R\&D}} is completion delay, 𝒟 r j time \mathcal{D}_{r_{j}}^{\mathrm{time}} is the tier-specific time distribution, G j R & D G_{j}^{\mathrm{R\&D}} is its quality gain, 𝒟 r j quality \mathcal{D}_{r_{j}}^{\mathrm{quality}} is the tier-specific gain distribution, t j done t_{j}^{\mathrm{done}} is the completion day, and ϵ t q \epsilon_{t}^{q} is configured product-quality noise. The summation adds only projects that finish today, while daily development accumulates continuously. The design makes R&D a delayed investment: it can move the global quality frontier, but it does not instantly solve today’s churn risk. This simulates product roadmaps in which small engineering work compounds steadily, while larger research bets have uncertain delivery dates and payoffs.

#### In-app ads.

In-app ad strength is additive across global, group, and customer settings and then log-scaled: a i , t eff = log ⁡ ( α a + κ a ​ clip ⁡ ( a t global + a g , t group + a i , t customer , a min , a max ) ) − log ⁡ α a log ⁡ ( α a + κ a ​ a max ) − log ⁡ α a . a_{i,t}^{\mathrm{eff}}=\frac{\log\!\left(\alpha_{a}+\kappa_{a}\,\operatorname{clip}(a_{t}^{\mathrm{global}}+a_{g,t}^{\mathrm{group}}+a_{i,t}^{\mathrm{customer}},a_{\min},a_{\max})\right)-\log\alpha_{a}}{\log\!\left(\alpha_{a}+\kappa_{a}a_{\max}\right)-\log\alpha_{a}}. (13) Here a t global a_{t}^{\mathrm{global}} , a g , t group a_{g,t}^{\mathrm{group}} , and a i , t customer a_{i,t}^{\mathrm{customer}} are configured ad strengths at company, segment, and customer scope; a min , a max a_{\min},a_{\max} are bounds; α a \alpha_{a} is a positive log offset; κ a \kappa_{a} controls how quickly raw ad strength saturates; and a i , t eff a_{i,t}^{\mathrm{eff}} is the customer-visible ad load after clipping and saturation. The log scaling makes additional ad load less effective at the high end, matching the idea that an already ad-heavy product has limited extra monetization headroom. Ads create daily revenue Y i , t ads = ρ i ads ​ a i , t eff ​ n i , Y_{i,t}^{\mathrm{ads}}=\rho_{i}^{\mathrm{ads}}a_{i,t}^{\mathrm{eff}}n_{i}, (14) where ρ i ads \rho_{i}^{\mathrm{ads}} is customer i i ’s ad-revenue sensitivity and n i n_{i} is seats. The same effective ad load subtracts from perceived quality. This creates a monetization tradeoff: ads are immediately lucrative but can reduce satisfaction and retention. The mechanism simulates ad-supported SaaS or freemium products, where more impressions produce revenue but also make the product feel noisier or less professional to some customers.

#### Perceived quality.

Perceived quality is the utility-relevant quality experienced by customer i i after relationship, tenure, support, quota, and ad effects: Q i , t perc = \displaystyle Q_{i,t}^{\mathrm{perc}}={} ϕ i , p , t quota ⏟ quota factor [ Q g , p , t del + β r ( r i , t − r 0 ) + β d log ( α d + d i , t / d 0 ) − β I I i , t \displaystyle\underbrace{\phi_{i,p,t}^{\mathrm{quota}}}_{\text{quota factor}}\Big[Q_{g,p,t}^{\mathrm{del}}+\beta_{r}(r_{i,t}-r_{0})+\beta_{d}\log\!\left(\alpha_{d}+d_{i,t}/d_{0}\right)-\beta_{I}I_{i,t} (15) − η i ads a i , t eff ] , \displaystyle-\eta_{i}^{\mathrm{ads}}a_{i,t}^{\mathrm{eff}}\Big], where r i , t r_{i,t} is relationship score, r 0 r_{0} is neutral relationship, d i , t d_{i,t} is days subscribed, d 0 d_{0} is the tenure scale, α d \alpha_{d} is the tenure log offset, I i , t I_{i,t} is open-issue days, U p , t U_{p,t} is plan quota, u i u_{i} is sampled daily usage demand, D U D_{U} converts daily demand to the quota period, and a i , t eff a_{i,t}^{\mathrm{eff}} is effective ad load. Coefficients β r , β d , β I \beta_{r},\beta_{d},\beta_{I} control relationship, tenure, and issue effects, while η i ads \eta_{i}^{\mathrm{ads}} is the customer’s ad-quality sensitivity. The quota fulfillment factor ϕ i , p , t quota \phi_{i,p,t}^{\mathrm{quota}} equals one when quota covers demand and falls toward zero when the plan delivers only a fraction of demand. Thus quota shortfalls scale the whole perceived-quality expression rather than subtracting a fixed penalty. This simulates the difference between engineering quality and customer experience: the same product can feel better to a long-tenured, well-supported customer and worse to a customer facing tickets, quotas, or intrusive ads.

### A.4 Satisfaction, Retention, and Support

#### Satisfaction.

Instant satisfaction is quality surplus over the participation curve, S ~ i , t = Q i , t perc − Q i req ​ ( C i , t ) , \tilde{S}_{i,t}=Q_{i,t}^{\mathrm{perc}}-Q_{i}^{\mathrm{req}}(C_{i,t}), (16) and stored satisfaction is an exponential moving average with configurable inertia λ S \lambda_{S} : S i , t = λ S ​ S i , t − 1 + ( 1 − λ S ) ​ S ~ i , t . S_{i,t}=\lambda_{S}S_{i,t-1}+(1-\lambda_{S})\tilde{S}_{i,t}. (17) Here S ~ i , t \tilde{S}_{i,t} is today’s surplus, S i , t S_{i,t} is stored satisfaction, C i , t C_{i,t} is the effective current price, Q i req ​ ( C i , t ) Q_{i}^{\mathrm{req}}(C_{i,t}) is the quality the customer expects at that price, Q i , t perc Q_{i,t}^{\mathrm{perc}} is experienced quality, and λ S \lambda_{S} is satisfaction inertia. This is an expectancy-disconfirmation design: customers are satisfied when experience exceeds the paid-price expectation and dissatisfied when it falls short ( Oliver, 1980 ) . The moving average means customers remember recent experience instead of resetting each day, so a bad outage or a good support recovery can affect future behavior for multiple periods. Downstream rules weight negative satisfaction more strongly, consistent with loss aversion ( Kahneman and Tversky, 1979 ) . The mechanism simulates customer sentiment as a memory-bearing state rather than a one-day reaction.

#### Billing revenue and involuntary churn.