Perspective

Anthropic’s Holdback Test Draws Criticism

By Claire Schultz

Updated

April 25, 2026

Reading Time

1 min

A tweet claiming that Anthropic was no longer offering Claude Code access to Pro subscribers paying $20 per month made the rounds on social media this week. This was seen as an indication that the company would need to take drastic measures to maintain service for customers amidst an ongoing compute shortage. Anthropic has not said how many Claude Code users it has currently, or how many of those users are Pro or Max subscribers.

As it turns out, the situation was overstated. Anthropic’s head of growth, Amol Avasare, responded directly to the post, saying that “we’re running a small test on ~2% of new prosumer signups. Existing Pro and Max subscribers aren’t affected.” In AB testing, this is called a “holdback test,” in which the value of a feature is measured by excluding a small subset of users from accessing it. Nevertheless, many questioned the wisdom of running a test like this on a tool as widely used as Claude, and competitors were quick to pile on. OpenAI directly implied it was a violation of customer trust, and Sam Altman mockingly replied “ok boomer” to Amol’s post.

Important Disclosures

This material has been distributed solely for informational and educational purposes only and is not a solicitation or an offer to buy any security or to participate in any trading strategy. All material presented is compiled from sources believed to be reliable, but accuracy, adequacy, or completeness cannot be guaranteed, and Contrary LLC (Contrary LLC, together with its affiliates, “Contrary”) makes no representation as to its accuracy, adequacy, or completeness.

The information herein is based on Contrary beliefs, as well as certain assumptions regarding future events based on information available to Contrary on a formal and informal basis as of the date of this publication. The material may include projections or other forward-looking statements regarding future events, targets or expectations. Past performance of a company is no guarantee of future results. There is no guarantee that any opinions, forecasts, projections, risk assumptions, or commentary discussed herein will be realized. Actual experience may not reflect all of these opinions, forecasts, projections, risk assumptions, or commentary.

Contrary shall have no responsibility for: (i) determining that any opinions, forecasts, projections, risk assumptions, or commentary discussed herein is suitable for any particular reader; (ii) monitoring whether any opinions, forecasts, projections, risk assumptions, or commentary discussed herein continues to be suitable for any reader; or (iii) tailoring any opinions, forecasts, projections, risk assumptions, or commentary discussed herein to any particular reader’s objectives, guidelines, or restrictions. Receipt of this material does not, by itself, imply that Contrary has an advisory agreement, oral or otherwise, with any reader.

Contrary is registered with the Securities and Exchange Commission as an investment adviser under the Investment Advisers Act of 1940. The registration of Contrary in no way implies a certain level of skill or expertise or that the SEC has endorsed Contrary. Investment decisions for Contrary clients are made by Contrary. Please note that, although Contrary manages assets on behalf of Contrary clients, Contrary clients may take any position (whether positive or negative) with respect to the company described in this material. The information provided in this material does not represent any investment strategy that Contrary manages on behalf of, or recommends to, its clients.

Different types of investments involve varying degrees of risk, and there can be no assurance that the future performance of any specific investment, investment strategy, company or product made reference to directly or indirectly in this material, will be profitable, equal any corresponding indicated performance level(s), or be suitable for your portfolio. Due to rapidly changing market conditions and the complexity of investment decisions, supplemental information and other sources may be required to make informed investment decisions based on your individual investment objectives and suitability specifications. All expressions of opinions are subject to change without notice. Investors should seek financial advice regarding the appropriateness of investing in any security of the company discussed in this presentation.

Please see www.contrary.com/legal for additional important information.

Authors

Claire Schultz

Research Associate

Claire is a Research Associate at Contrary. Claire's academic research has centered primarily on particle physics, and has been published in Nature. Prior to Contrary, Claire worked on the investment team at Bridgewater Associates and studied mathematics and physics.

See articles

© 2026 Contrary Research · All rights reserved

Privacy Policy

By navigating this website you agree to our privacy policy.