No. This is opportunity cost.

A clear example is Google. They can easily give all compute to Deepmind, yet majority is allocated to be sold at market rate.

So, every compute Deepmind and Googlers get is lost revenue opportunity and must be measured as such.

No other internal tools or servers has this much external demand to be comparable.

Also, Anthropic is showing that to operate at Anthropic level, you need $300,000 compute spend per employee

> No. This is opportunity cost.

It’s not opportunity cost. It would be opportunity cost if their employees not using it would lead to additional revenue. However, if Inwas to go to Anthropic and double my token usage Anthropic wouldn’t decline my money. If Anthropic’s employees would stop using Claude, their revenues wouldn’t increase by an equivalent amount.

Their revenues are not currently supply constrained.

It’s just a cost cost. If their employees stopped using these services they would simply need less infrastructure, and therefore save on the cost to build and run that infrastructure. It wouldn’t translate into an increase in revenue.

> Also, Anthropic is showing that to operate at Anthropic level, you need $300,000 compute spend per employee

They have infinite money to burn so they aren't going to optimize.

I've seen that trivial changes to workflows can token usage by 30%. Repeat this a few times and my token usage has plummeted over the last few months.

Only outputting test failures instead of having agents read test run output in real time. Not having build logs get fed into context. Projects like UCL (unified context layer) so agents don't rediscover the same facts about a code base again and again.

Giving tools agent output modes so they only print out the minimal info needed for an agent to get work done.

When I hear a company has an absurd AI spend I assume they aren't even trying to reign it in.

I don't buy it. A company like Anthropic is not going to just walk away from revenue. If external demand were really that high, then they should take the money first. And not just for the revenue; customer demand being that high is also an indication that the product is already great as-is so they stand to improve their bottom line by easing off on the R&D spend.

I didn’t say stop, I said ease off.

Acting like they’re constantly racing to catch up is not the rift behavior for a company that’s trying to present itself as a relatively stable market leader. It’s behavior that’s more consistent with a company whose leadership believes, for whatever reason, that their current position is insecure.

re: AGI and the singularity, I don’t have anything really to say to that. Nobody even has a working operational definition of AGI yet, let alone a clear roadmap for what technical challenges need to be solved to get us there, let alone even the beginnings of an estimate of what it might cost or how long it might take. Nor is there a story for how that will lead immediately to Kurzweil’s technological singularity that amounts to much more than waving one’s hands and shouting, “But wouldn’t it be exciting if it were true?” Treating that as if it were even a sketch of an actual business plan isn’t rational due diligence; it’s just chasing buzzwords.

> If external demand were really that high, then they should take the money first

AI infrastructure is supply constrained, so you have to make a decision: how much do I allocate to training vs inference?

Do I use it all for inference and do no further training? If so, what risks does that expose me to?

Supply constrained companies don’t offer 50% discounts.

It’s supply constrained for now, the moment companies like Anthropic have to switch to profit-making-mode, demand will dip as the prices skyrocket imo.

Not if you’re desperately trying to stay ahead of AI frontier pack.

It depends on your assumptions. When Google is in "space crunch" mode, there are opportunity costs. Otherwise, there aren't, because you could do both.

You could also look at the opportunity cost of not letting Deepmind do quite as much research. How much is additional research worth on the margin? There will be diminishing returns.

So every business should measure any client fungible resource that way? Should a bank put a revenue number to any capital held in excess of regulatory requirements, for example?

They also offer $300,000 in compute spend (allegedly) for $200 a month to anyone who isn't employed by a large corporation so their pricing should be taken with a grain of salt.

>$300,000 in compute spend (allegedly)

source? The most optimistic calculations don't come anywhere close to that.

https://x.com/SemiAnalysis_/status/2064815044085318040

Usage allowances have decreased since that analysis. Also the 20X plan became the 10X plan (cut in half).

It was never 20x, that was just 5 hour usage limits. There's a class action on this...

Are you talking about their $200 consumer plan? If you use 100% of the quota every week it's closer to $2000 at API rates.

They have never claimed that the API rates are the "real cost" of the compute spend.

> If you use 100% of the quota every week it's closer to $2000 at API rates.

I don't know where you got that from but the only time I maxed my $20 plan for a week I made the calculation that I had spend close to $200 in a single week. So maxing a $200 plan for a month should give you $8000 worth of token at API price.

> but the only time I maxed my $20 plan for a week I made the calculation that...

> but the only time I maxed my $20 plan for a week I made the calculation that I had spend close to $200 in a single week. So maxing a $200 plan for a month should give you $8000 worth of token at API price.

The $200 plan gives 10X the usage of the $20 plan.

So if you used $200 of equivalent credits on the $20 plan, that would be the equivalent of $2000 on the $200 plan.

I don't know how you got to $8000 in your math.

Recent analysis by industry analysts like Semi Accurate have measured Sol 6.1 on the $200/month plan to be around $2000 of API equivalent use.

I get a LOT of mileage out of my $20 plan. It's at least $500+ a week at API rates. I sometimes wonder if there's something wrong with their billing system.

On the $200* consumer plan, according to "npx ccusage", I'm using $11,000 per month at API rates. Possibly significantly more, because I was previously using the $100 plan so that only reflects about 1 week of the $200 plan.

I was spending thousands of dollars a day in funny money (free promotional tokens from VCs) before switching to a paid subscription for convenience.

> So, every compute Deepmind and Googlers get is lost revenue opportunity and must be measured as such

Maybe, but it also reduces supply that can drive up prices

Not really- since they have an internal view of low and high usage times of the day, they can optimize differently.

Even within a single moment, if there is capacity for an internal request, just send it, if market traffic is currently at 100%, don't.

> Also, Anthropic is showing that to operate at Anthropic level, you need $300,000 compute spend per employee

There are a bunch of open source harnesses that are competitive with CC and they aren't using anywhere near that many tokens to do it.

It's not just the LLM costs, to keep your agents busy you need to move them of your workstation onto a server/cloud. I currently have about 16 claude code agents working on different bugs, uses so much resources that my macbook pro m5 max becomes slow to respond. Using cloud session would free up my macbook for other work.

Anthropic recently handed out free cloud resources (it's still free, guess they want you to get used to the fact that cloud is not free). I went through $250 cloud budget in less than 2 days.

Feels almost as expensive as LLM calls.

I'm impressed you have a workflow that can keep 16 separate agents fed without the whole thing collapsing into nonsense. Even running full agent teams I can't think of ways to do that without being intentionally inefficient

Technically it is... According the original vision for workstation, a computer with >1MB RAM, >1 megapixel of screen resolution, >1 MIPS and costing under 1 Mega cents (10K dollars).

However, so is my phone!

https://en.wikipedia.org/wiki/3M_computer

just get a mac mini headless

Wouldn't that depend on how much hardware you have to run them on, too?

If it were opportunity cost, that would imply that Anthropic can't sell as many tokens as people want to buy. But as far as I can tell, there aren't any "token shortages," at least at API prices. Or is there a maximum per-customer limit that I'm not aware of?

Plenty of companies use retail/wholesale pricing internally to make sure a business unit doesn’t appear profitable just because it’s being subsidized by the one providing the resources. Plenty also have to have arm’s length pricing for legal reasons.

Anthropic isn’t booking it as GAAP revenue, they’re just using the number for marketing.

In the US, the IRS doesn't see it that way, at least when it comes to fringe benefits as a part of employee compensation. https://www.irs.gov/businesses/small-businesses-self-employed/employee-benefits says, "In general, the amount the employer must include is the amount by which the fair market value of the benefits is more than the sum of what the employee paid for it plus any amount that the law excludes."

As an example, as I understand it, if a hotel chain allows an employee to stay in one of their hotels for free when they go on vacation, the tax value of that benefit is what they would have charged someone for that hotel room, not how much it increased the hotel's costs to have one more hotel room full — which is generally almost nothing.

Of course, the IRS does agree with you when it comes to expenditures that are for the employer's benefit, rather than the employee's. If it didn't, almost all companies would have negative profits from the IRS's point of view.

Not an apt comparison. You are conflating the opex of the company with taxable benefits.

If what you said was how taxation worked, a hand tool manufactuting company giving hammers they make for $1 and retailing for $2 to workers building a shed on their property would be increasing the workers' income tax bill by $2. Of course, in reality the workers get no income tax bill on the hammer given to them for work and the company can get a $1 tax writeoff from the opex of making the hammer.

That's not really true. Measuring the equivalent retail cost of the usage is the only way to put it in a relatable perspective.

If Anthropic had revealed their internal cost, there comments would be complaining that they were hiding something and giving useless, unrelatable data.

The expenditure is most interesting when compared to retail rates, because that's the rate the rest of us have to compare against.

Most "enterprise" companies are paying full API costs. The subscription deals are only available to consumers. If you want enterprise levels of control over how your staff are using the models you need to pay up.

Not even close. "enterprise" companies are negotiating contracts. Call it pre-provisioned capacity, wholesale pricing, etc. Same is true for AWS, Azure, Salesforce, or literally any SaaS provider ever. There is a reason literally every single pricing page for any SaaS provider has an "Enterprise? call us" column.

Big companies actually are paying published API rates, Anthropic and OpenAI models are desirable enough that there is very little wiggle room to negotiate. It's their way or the high way. Same with things like "safety" filters and (for Fable) mandatory data retention.

I’ve worked at and with large orgs - it’s absolutely standard practice for BUs to bill other BUs at “retail” or at the very least a commercial rate.

Microsoft basically does the same with internal Azure services? When I was there our budget/usage was based on and consumed at the consumer price for GPUs and other compute. What happened behind the scenes really I have no knowledge.

Not true. Azure has a very different internal pricing. You can often get a glimpse of it if you have a.. particularly enterprising TAM. What happens sometimes (Like now when compute/GPUs are in high demand) is finance department would calculate the opportunity cost on allocating those resources internally vs selling them externally. The calculation isn't even always straight forward. What is the opportunity cost of using compute for internal CI or company operations vs selling them externally? It's not a straightforward equation and you would have to adjust the scale often depending on how clogged your internal processes get.

The extremely discounted internal cost promote waste, so there is always a room for improving efficiency, but that too has a cost.

> You blow $50k a year on servers running your LLMs? It cost you $50k.

Plus electricity

In a lot of companies, you would get charges back on usage/license/cost and this can get even funnier in NGOs.

In a similar way, internal employees would have also a "standard daily rate" like an external employee.

Actually I think it's very informational. For every other company, working on the cutting edge of technology, paying for intelligence at retail prices, means you are now paying more for AI than you are employees. That $ could have been spent on 2x the employees, however that's not as useful as 50% AI / 50% humans, and indicative of where we are headed.

>means you are now paying more for AI than you are employees.

Yea, isn't the entire point of RSI to replace your employees altogether? So you'd spend an infinite amount more on AI than employees in that theoretical.

Like, if you were in an energy generation business spending way more on fuel and capital expenditures is way better than having zillions of employees earning checks , it would be a sign of efficiency.

Anthropic's use of AI in no way translates or represents any other companies' use of AI, so your apparent extrapolation makes no sense.

Anthropic is full of the smartest talent and a bellwether for how the rest of the service industry will work. Just as cutting edge companies of the past, like 2005-2020 Google's use of computers and software were indicative of the software every other service company was going to be using, lest they turn into an old fashioned uncompetitive dinosaur.

Do they actually book the revenue against themselves?

This is the world of spend more, get valued more; obviously the investment will be used to spend more and get more, so it must be good. This is investor-think. If you don't have spend, why do you need investment?

(Or it could be bad. Very very bad)

It's kind of expected, no? If I was an Anthropic employee expected to dog-food the product and be on the frontier of usage, I'd probably be using Mythos all the time at the highest thinking level. And if I am not being metered, I'd probably do more stuff that I know are likely low yield since I am not worried about a quota. This would probably take my "API-equivalent" usage beyond the average salary in far less than a month.

I'm personally really not impressed by their software quality, at least we know from their experience that infinite tokens at the best model available still doesn't replace good engineering.

> employee expected to dog-food

I guess kinda put you in a bubble a little, I wonder how the experience of Anthropic eng who left the company would look like, when such eng would go and spend few $K a day in non AI lab, would that be encourage/allowed/taken as example?

I'm not really sure I understand the excitement around this. It seems like the implication is that if retail price > employee salary, you are replacing employees with AI spend? Yet, anthropic's head count has nearly doubled since last year - so, in a company where nearly half of that head count is engineers, but claude is "writing 90% of the code," what is everyone doing then? You would think, if the hype feeling around this held true, that the output would be going up and costs remaining stable. Costs rising as you scale is not a new thing but that's not really what AI promises. It sounds to me like the retail AI spend per employee is very high. What is the appeal of that?

Imagine an employee is 1x productive without AI.

Now spend 0.5x their salary on AI, and they become 5x productive compared to no AI.

Now spend 1.1x their salary on AI, and they become 20x productive compared to no AI.

It's still obviously worth it to spend the 1.1x. But it is truly remarkable that the biggest cost of hiring a knowledge worker is no longer their salary, but their AI usage.

There is no implication here that employees are being replaced by AI spend, as long as all of the productivity is generating profit. The AI usage still needs a knowledge worker to direct it in ways that it can't direct itself, so you still need the employees.

> Now spend 0.5x their salary on AI, and they become 5x productive compared to no AI. > > Now spend 1.1x their salary on AI, and they become 20x productive compared to no AI.

This is what I am getting at. If this were true - and I've yet to see evidence it is, despite some anecdotal evidence (organizationally, IME, it's about 2-3x for power users, and probably a net loss of productivity in certain areas) - why would their headcount be growing so much still YoY, pretty much doubling every year. If the AI is 10-20xing why such a big jump in headcount? Yes, the numbers are shiny, the growth is apparent, I would just expect with this AI narrative they'd be "doing more with less" as this thing improves every year.

> The AI usage still needs a knowledge worker to direct it in ways that it can't direct itself, so you still need the employees.

This is the story AI sells though, and Anthropic has participated in it heavily. Yet, their interview process requires hard skills and doesn't let you use AI, so clearly, what you're saying is correct, it just doesn't really align in my head with the whole knowledge worker being replaced narrative.

I mean, just what I did this morning with Claude would have taken me 2 weeks on my own. So 20x is pretty reasonable for certain types of engineering.

But it's also true that when I'm working on 2 things at once, the bottleneck is me. I'm spending all my time giving Claude direction that it can't give itself.

So you basically have to separate out the tasks where AI can produce a 20x productivity boost, and the stuff where it can't help at all.

There's still so much stuff where it can't help at all, you need real human employees for that. That's why Anthropic is continuing to hire at breakneck pace.

And that's what I'm saying -- knowledge workers aren't being replaced. They're just accomplishing tons more work, and companies aren't running out of work to do.

Sorry, I know this is the story, but am I expected to believe that Anthropic would need 20,000+ engineers in their org without their use of AI? I do not believe that is close to true.

Everyone is saying it is expected, and I agree, but it’s also clear that employees of AI companies spending unlimited zero-cost tokens in their day-to-day work is resulting in products that aren’t usable by the vast majority of regular workers who don’t have this luxury. Dogfooding your product is good but you should also use it with the same constraints that you place on your users, otherwise what signals are you really getting?

They're dogfooding the way Xerox PARC was dogfooding in the 01970s.

By giving hundreds of workers each a powerful desk-sized Alto minicomputer for their own needs, with a high-end graphical display, they gained insight into what desktop personal computing would look like 15 years later, enough insight to shape it profoundly. It's been 55 years, and although multitouch screens are proliferating, I'm still typing this on a keyboard hooked up to a framebuffer-driven display with a mouse hooked up to it, displaying hypertext, festooned with windows, icons, and a mouse that moves an onscreen pointer at the same 135° angle they settled on 55 years ago, positioning a vertical line to indicate the insertion point, with scrollbars. The display has twice as many pixels, the pixels are color, the mouse pointer is white rather than black, the screen is a flat LCD, the scrollbars are on the right, and the cursor blinks, but it's very recognizably descended from https://smalltalkzoo.computerhistory.org/HOPL-St78.html rather than, for example, ITS or GENESYS.

Only people with the luxury of unlimited zero-cost CPU cycles had a chance of seeing what the future looked like, and that allowed them to find out what worked and what didn't work in that environment. Nowadays, the regular workers who don't have the luxury of unlimited zero-cost tokens are manually reviewing voluminous AI code, because, even though that's a lot of slow work, they're still much faster programming that way than they were without it. Anthropic, instead, is doing insane things like eliciting a billion lines of Lean proofs to show that existing open-source libraries can't fail their OSS-FUZZ tests — and fixing all the bugs that effort turned up, in the process.

They aren’t usable? I’ve been using them a bunch and have been having good results. Dogfooding is always good, but I don’t think there’s some kind of major problem here.

Until employees can use those AI tokens for personal projects, it's not compensation. Rather, it's just a cost of doing business, like laptops, office space, or testing servers.

> There's a lot to unpack about this statement.

Including the fact that it's probably not true.

If something is compensation, it can often be negotiated and traded off against other compensation components. How many people have successfully negotiated AI tokens away in favour of salary/bonus/stock? I'm going to guess "approximately zero".

Is that really true? At least at large companies, the components are not usually fungible in my experience. At startups or smaller companies I'm sure it's a lot more flexible.

I think the author used the wrong word.

I haven't seen anyone treating AI tokens as employee compensation. If anyone can show me an article where employees were paying taxes on their token use at work, please share it.

Companies have been treating AI tokens as part of the cost of having employees, like the cost of their office or travel expenses.

> I think the author used the wrong word.

True, though that phrase understates the author's problem a bit, as it is surrounded with more wrong-words, indicating it's a wrong-idea rather than a slip of the tongue.

As you note, it's closer to office/travel expenses. Employee computers, licenses for software on those computers, licenses for SaaS stuff, etc.

I can see where there might be a truth to it. My company offers a fairly generous monthly token budget. If I were evaluating an offer from another company, but they offered a lower monthly budget, it would definitely influence my decision, all else being equal. It's not compensation, but it does affect my day-to-day experience at work.

There are many internal developers at $dayjob (where core products are not based on AI) that spend over a thousand dollars every day on average, which is likely higher than their salary.

I doubt they are bringing that much value to the company for whatever work is done with that much token.

Dumb question, what most engs at Anthropic work on? It doesn't seem like release some large new products all the time while they have quite a few of engs. What are the major branches of work besides LLM research? I guess there is a lot of infra stuff to support the scale of operation, just supporting API and TUI shouldn't be that demanding I guess.

- Cross-platform desktop apps - Cross-platform mobile apps - Claude Code TUI - Product-specific verticals like Claude Design - Corporate marketing website - APIs - Writing technical documentation for APIs - Enterprise support - And what must be a massive infra team to run training and inference at scale in a heterogeneous cloud environment

And I’m sure more than that. But there’s obviously plenty to work on, and they’re not at the point they can just Claude their way through it.

I'm kinda curious about this too. But then I'm always amazed at the enormous workforces these tech companies have, and I'm sure they cut them as thin as they can. Like adobe has 30,000 employees & some blogger estimated that 1/3rd of their workers are actually working on their software. I use their software sometimes and I really haven't noticed many changes to it over the years, except recently with the ai stuff. Like from 2015-2020 I can't think of any major things they added to adobe illustrator. I guess making and maintaining software is hard.

I fully agree most big tech companies are pretty bloated. But FWIW a lot of my time at SaaS companies, especially in the B2B space, was spend working on features for larger customers that most others would never use or even see. Like, yeah, product X isn't that complex, but $BigCorp wants to use it in their private cloud, or needs it to be single tenant, or needs these audits done, or want their $Gov client to use it. Now you have a ton of work for zero visible changes.

Note, this doesn't include options or stock benefits. That's a huge part of actual compensation for engineers at this level even if it's on a different balance sheet.

> excluding stock awards

So it exceeds employee pay if you discount the vast majority of employee pay.

I’m not sure what this “stat” tells us other than Anthropic is dogfooding its own tech. Which is kinda expected.

The real question is what could they have done with open weight models on the same compute they’re renting. If the answer is “basically the same” then this flips the entire conversion back to what is Anthropic’s moat here.

I mean, they probably could not have done most of it with open models. Again, a lot of this isn't going to standard software development, at least I would assume but instead to SOTA model development and RSI.

Do the same math on AWS cloud use at AWS and you'll probably see proportionate massive costs.

This is the nature of managed anything. AI efficiency will improve but essentially this is what automation does.

What does this translate to in terms of number of 24/7 coding agents per engineering staff member?

Regardless of one's opinions about future AI usage, I'm not sure this metric says anything at all.

Would Anthropic themselves be paying this much if it was actually billed? It is not obvious.

Is Anthropic a good sample for AI use? Surely not, since they have special needs (dogfooding) and a vested interest in proving that AI usage is the future.

Can we compare their results pre and post AI usage? Close to definitionally impossible.

Are their employees good examples of AI-assisted engineers? Not really, as (one would hope) they're more well versed on the topic than an average eng.

Could it be because spending it internally means they can’t receive retail prices for the tokens?

ummm yes, they have to dogfood it... this is that lucky area of the venn diagram where you have to do it, measure it an act on it... because improves the bottom end and the product.

For every new hire, you have to ask yourself: "Will this employee be better than another $300k in tokens?"

to spell it out: the implicit claim is "ai companies think ai is a scam, as evidenced by them using humans instead of ai", and I believe the huge internal ai use is evidence to the contrary

AI, the technology, can be useful... And... AI the investment case can be a scam.

These are not mutually exclusive points, and I find it a common issue to get both sides to understand.

Anthropic/OpenAI should start doing mass layoffs if they want to convince other companies that this is really the human-labor killer.

they're using ai instead of employing more people, how is that not exactly the same minus a completely morale-destroying marketing stunt

both of those companies combined are smaller than the number of people microsoft is trying to buy out right now

Anthropic 10x'd its employees over the past 3 years. Shouldn't it be cutting employees if that's the case? Especially because they probably have access to more powerful models which aren't released to the public yet, probably with fewer restrictions, and don't need to pay the markup for using their own service?