Can someone explain the Pale Horse reference?
In the Bible’s book of revelations, John (the author) is witnessing the end of the world and sees four horsemen being unleashed upon the world to spread a curse/trial/whatever wherever they ride. Each horseman brings with them something different- famine, disease, war (or strife), and death. Death is the last, IIRC, and rides upon a pale horse. I think that’s what they’re referencing. This person is saying that openAI is going to die soon.
it won’t. its backed by microsoft. they can literally afford to burn the cash on this while it becomes profitable, and it will AI has so many low hanging fruits to optimize its insane.
They will only pay as long as it benefits them.
Okay, explain. What kinds of low hanging fruit?
quants are pretty basic. switching from floats to ints (faster instruction sets) are the well known issues. both those are related to information theory, but there are other things I legally can’t mention. shrug. suffice to say the model sizes are going to be decreasing dramatically.
edit: the first two points require reworking the base infrastructure to support which is why they havent hit widespread adoption. but the research showing that 3 bits is as good as 64 is intuitive once you tie the original inspiration for some of the AI designs. that reduction alone means you can get 21x reduction in model size is pretty solid.
both those are related to information theory, but there are other things I legally can’t mention. shrug.
hahahaha fuck off with this. no, the horseshit you’re fetishizing doesn’t fix LLMs. here’s what quantization gets you:
- the LLM runs on shittier hardware
- the LLM works worse too
- that last one’s kinda bad when the technology already works like shit
anyway speaking of basic information theory:
but the research showing that 3 bits is as good as 64 is intuitive once you tie the original inspiration for some of the AI designs.
lol
Honestly, the research showing that a schlong that’s 3mm wide is just as satisfying as one that’s 64 is intuitive once you tie the original inspiration for some of the sex positions.
It’s actually super easy to increase the accuracy of LLMs.
import pytorch # or ollama or however you fucking dorks use this nonsense from decimal import DecimalI left out all the other details because it’s pretty intuitive why it works if you understand why floats have precision issues.
decimal is a severely underappreciated library
I have seen these 3 bit ai papers on hacker news a few times. And the takeaway apparently is: the current models are being pretty shitty at what we want them to do, and we can reach a similar (but slightly worse) level of shittyness with 3 bits.
But that doesn’t say anything about how both technologies could progress in the future. I guess you can compensate for having only three bits to pass between nodes by just having more nodes. But that doesn’t really seem helpful, neither for storage nor compute.
Anyways yeah it always strikes me as a kind of trend that maybe has an application in a very specific niche but is likely bullshit if applied to the general case
If anything that sounds like an indictment? Like, the current models are so incredibly fucking bad that we could achieve the same with three bits and a ham sandwich
Far as I can tell, the only real benefit here is significant energy savings, which would take LLMs from “useless waste of a shitload of power” to “useless waste of power”.
the fruit can’t be rotten, you must be picking it wrong
“look, Mme Karen, this is definitely not a rotten tomato. it can’t be a rotten tomato, we don’t sell rotten tomatoes. you can see here on the menu that we don’t have rotten tomatoes on offer. and see here, on your receipt, where it says quinoa salad? absolutely not rotten tomatoes!” explains the manager fervently, avoiding a tableward glance at the pungent red blob with as much will as they can muster
So many low-hanging fruits. Unbelievable fruits. You wouldn’t believe how low they’re hanging.
They are tremendous fruits. Many people are saying this. Fruits.
and we have concepts for even lower hanging fruits. beautiful concepts.
this is correct as to the background of the term itself, the reason ed uses it here is because it is the term that he selected some months ago when he listed “some likely pale horses that signal the bubble popping”
Good. Burn that thing to the ground.
Likely they’ll try to sell it to governments, and with Elon Musk proposing goVeRNmeNt eFfIciEnCy, at least xAI can become somewhat profitable.
$200 a month for a user is losing money? There’s no way he’s just including model queries. An entire a6000 server is around $800 / month and you can fit a hell of lot more than 4 peoples worth of queries. He has to include training and or R&D.
Surely this includes staff, real estate, and other operating costs.
I’m honestly fairly surprised as well, but at the same time, they’re not serving a model that can run on an A6000, and the people paying for unlimited, would probably be the ones who setup bots and apps doing thousands of requests per hour.
And honestly? Those people are 100% right.
If they can’t deliver true “unlimited” for 200 bucks a month, they shouldn’t market it as such.grumble grumble unlimited mobile data grumble grumble
To be fair, unlimited is supposed to mean unlimited for a reasonable person. Like someone going to an “all you can eat buffet”. However those purchasing these would immediately set up proxy accounts and use them to serve all their communities, so that one unlimited account, becomes 100 or a 1000 actual users. So like someone going to an “all you can eat” and then sneaking in 5 other people under their trenchcoat.
If they actually do block this sort of account sharing, and it’s costing them money on just prolific single users, then I don’t know, their scaling is just shite. Like “unlimited” can’t ever be truly unlimited, as there should be a rate limit to prevent these sort of shenanigans. But if the account can’t make money with a reasonable rate limit (like 17280/day which would translate to 1 request per 5 sec) they are fuuuuuucked.
Yeah, poor wording on my part, proxy accounts being banned is totally fair, but a user using various apps and bots is the type of ‘Power User’ scenario I’d expect a unlimited plan to cover.
Agreed. Like how fucking difficult is it to see “It costs us X per query, what Y rate limit do we need to put on this account so that it doesn’t exceed 200$ per month?”. I bet the answer to is hilariously low rate limit that nobody would buy, so they decided to value below cost and pray people won’t actually use all those queries. Welp. And if they didn’t even put a rate limit, also lol. lmao.
It includes anything that will keep them from having to pay investors back. Classic tech start up bullshit.
Silicon valley brain rot formula:
Losing money, get billions every month
Making money pay billions back
Which one do you think they pick
really looking forward to how these multi-billion dollar AI datacenter investments will work out for big tech companies
that said I’m pretty sure most of that capacity is reserved for the surveillance state anyway
I’m excited for the used hardware dump
CEO personally chose a price too low for company to be profitable.
What a clown.
More like he misjudged subscriber numbers than price.
please explain to us how you think having less, or more, subscribers would make this profitable
LLM inference can be batched, reducing the cost per request. If you have too few customers, you can’t fill the optimal batch size.
That said, the optimal batch size on today’s hardware is not big (<20). I would be very very surprised if they couldn’t fill it for any few-seconds window.
i would swear that in an earlier version of this message the optimal batch size was estimated to be as large as twenty.
yep, original is still visible on mastodon
this sounds like an attempt to demand others disprove the assertion that they’re losing money, in a discussion of an article about Sam saying they’re losing money
What? I’m not doubting what he said. Just surprised. Look at this. I really hope Sam IPO his company so I can short it.
oh, so you’re that kind of fygm asshole
good to know
Can someone explain why I am being downvoted and attacked in this thread? I swear I am not sealioning. Genuinely confused.
@[email protected] asked how request frequency might impact cost per request. Batch inference is a reason (ask anyone in the self-hosted LLM community). I noted that this reason only applies at very small scale, probably much smaller than what
OpenAI is operating at.@[email protected] why did you say I am demanding someone disprove the assertion? Are you misunderstanding “I would be very very surprised if they couldn’t fill [the optimal batch size] for any few-seconds window” to mean “I would be very very surprised if they are not profitable”?
The tweet I linked shows that good LLMs can be much cheaper. I am saying that
OpenAI is very inefficient and thus economically “cooked”, as the post title will have it. How does this make me FYGM? @[email protected]
Yeah, the tweet clearly says that the subscribers they have are using it more than they expected, which is costing them more than $200 per month per subscriber just to run it.
I could see an argument for an economy of scales kind of situation where adding more users would offset the cost per user, but it seems like here that would just increase their overhead, making the problem worse.
Wait but he controls the price, not the subscriber number?
Like even if the issue was low subscriber number (which it isn’t since they’re losing money per subscriber, more subscribers just makes you lose money faster), that’s still the same category of mistake? You control the price and supply, not the demand, you can’t set a stupid price that loses you money and then be like “ah, not my fault, demand was too low” like bozo it’s your product and you set the price. That’s econ 101, you can move the price to a place where your business is profitable, and if such a price doesn’t exist then maybe your biz is stupid?
I believe our esteemed poster was referencing the oft-seen cloud dynamic of “making just enough in margin” where you can tolerate a handful of big users because you have enough lower-usage subscribers in aggregate to counter the heavies. which, y’know, still requires the margin to exist in the first place
alas, hard to have margins in Setting The Money On Fire business models
despite that one episode of Leverage where they did some laundering by way of gym memberships, not every shady bullshit business that burns way more than they make can just swizzle the numbers!
(also if you spend maybe half a second thinking about it you’d realize that economies of scale only apply when you can actually have economies of scale. which they can’t. which is why they’re constantly setting more money on fire the harder they try to make their bad product seem good)
They’re still in the first stage of enshittification: gaining market share. In fact, this is probably all just a marketing scheme. “Hi! I’m Crazy Sam Altman and my prices are SO LOW that I’m LOSING MONEY!! Tell your friends and subscribe now!”
I’m afraid it might be more like Uber, or Funko, apparently, as I just learned tonight.
Sustained somehow for decades before finally turning any profit. Pumped full of cash like it’s foie gras by Wall Street. Inorganic as fuck, promoted like hell by Wall Street, VC, and/or private equity.
Shoved down our throats in the end.
It was worth it to finally dethrone Big Taxi🙄
well, yes. But also this is an extremely difficult to price product. 200$/m is already insane, but now you’re suggesting they should’ve gone even more aggressive. It could turn out almost nobody would use it. An optimal price here is a tricky guess.
Although they probably should’ve sold a “limited subscription”. That does give you max break-even amount of queries per month, or 2x of that, but not 100x, or unlimited. Otherwise exactly what happened can happen.
The real problem is believing that you can run a profitable LLM company.
What the LLMs do, at the end of the day, is statistics. If you want a more precise model, you need to make it larger. Basically, exponentially scaling marginal costs meet exponentially decaying marginal utility.
Some LLM bros must have seen this comment and become offended.
guess again
what the locals are probably taking issue with is:
If you want a more precise model, you need to make it larger.
this shit doesn’t get more precise for its advertised purpose when you scale it up. LLMs are garbage technology that plateaued a long time ago and are extremely ill-suited for anything but generating spam; any claims of increased precision (like those that openai makes every time they need more money or attention) are marketing that falls apart the moment you dig deeper — unless you’re the kind of promptfondler who needs LLMs to be good and workable just because it’s technology
and because you’re all-in on the griftlook bro just 10 more
repsgpt3s bro itl’ll get you there bro I swear broWell, then let me clear it up. The statistics becomes more precise. As in, for a given prefix A, and token x, the difference between the calculated probability of x following A (P(x|A)) to the actual probability of P(x|A) becomes smaller. Obviously, if you are dealing with a novel problem, then the LLM can’t produce a meaningful answer. And if you’re working on a halfway ambitious project, then you’re virtually guaranteed to encounter a novel problem.
Obviously, if you are dealing with a novel problem, then the LLM can’t produce a meaningful answer.
it doesn’t produce any meaningful answers for non-novel problems either
“Our product that costs metric kilotons of money to produce but provides little-to-no value is extremely difficult to price” oh no, damn, ye, that’s a tricky one
I signed up for API access. I run all my queries through that. I pay per query. I’ve spent about $8.70 since 2021. This seems like a win-win model. I save hundreds of dollars and they make money on every query I run. I’m confused why there are subscriptions at all.
More power to the open source community
By “Sam Altman said” in a “series of posts”, this article means these two tweets from 10 hours ago: https://twitter.com/sama/status/1876104580070813976.
This is a screenshot of a tweet talking about an article written about two tweets by Sam Altman. Is this really the world we’re living in, now?Yeah, and I think you’re pretending it’s more ridiculous than it is
Welcome to the wonderful XXI century where our innovations in communication technology and financial instruments allow a hyperoptimised economy where two tweets are more than enough to cause billion-dollar shifts on the market. Completely organic and based on solid fundamentals I am assured by the same people that assured me of this in 2000 and 2008.
I’ve seen more written on one post. People will eat up ‘news’ if presented in the right way. There is a reason the stupid websites and advertisers use the click-bait titles.
According to the Lemmy comment I read about it that is exactly what people are wondering
sam altman proving once again that he is not only a tech genius but also a business genius. make sure to let him scan your eyeballs before it’s too late.
“I personally chose the price”
Is that how well-run companies operate? The CEO unilaterally decides the price rather than delegating that out to the numbers people they employ?
I’m guessing that means a team or someone presented their pricing analysis to him, and suggested a price range. And this is his way of taking responsibility for making the final judgment call.
(He’d get blamed either way, anyways)
$20/mo sounds like a reasonable subscription-ish price, so he picked that. That OpenAI loses money on every query, well, let’s build up volume!
While the words themselves near an apology, I didn’t read it as taking responsibility. I read it as:
Anyone could have made this same mistake. In fact, dumber people than I would surely have done worse.
far, far, far, far, far, far, far fewer business people than you’d expect/guess are data-driven decision makers
and then there’s the whole bayfucker ceo dynamic which adds a whole bunch of extra dumb shit
it’d be funnier if it weren’t for the tunguska-like effect it’s having on human society both at present and in the coming decades to follow :|
There’s a reason so many companies fail
there is, but this isn’t (the primary) it tbh
I endorse this as a data professional. It’s maddening.
I think I remember Jeff Bezos in “The Everything Store” book seeing a price they charged for AWS and went even lower for growth. So there could be some rationale for that? However, I think switching AI providers is easier than Cloud Providers? Not sure though.
I can imagine the highest users of this being scam artists and stuff though.
I want this AI hype train to die.
It works for ice tea and hotdogs, why not AI? (I jest)
A real ceo does everything. Delegation is for losers who can’t cope. Can’t move fast enough and break enough things if you’re constantly waiting for your lackeys to catch up.
If those numbers people were cleverer than the ceo, they’d be the ones in charge, and they aren’t. Checkmate. Do you even read Ayn Rand, bro?
Is that what Ayn Rand is about? All I really remember is that having a name you chose yourself is self-fulfilling.
Oh boy I got a fun video for you: https://youtu.be/GmJI6qIqURA @26:50
Atlas Shrugged is so bad that if you didn’t know anything about the author, it could be read as a decent satire.
A monologue that last SIXTY PAGES of dry exposition. Barely credible characterization from the protagonist and villains and extremely poor world building.
Anthem is her better book because it keeps to a simple short story format - but still has a very dull plot that shoehorns ideology throughout. There’s far better philosophical fiction writers out there like Camus, Vonnegut, or Koestler. Skip Rand altogether imo
Ayn Rand is about spending your whole life moralizing a social philosophy based on the impossibility of altruism, perfect meritocratic achievement perfectly distributing wealth, and hatred of government taxation, regulation, and social welfare programs…
… and then dying alone, almost totally broke, living off of social security and financial charity from your former secretary.
In tech? Kinda yeah. When a subscription is 14.99 $£€/month it’s a clear “we just think it’s what people think is a fair price for SaaS”.
The trick is that tech usually works on really weird economics where the fixed costs (R&D) are astonishingly high and the marginal costs (servers etc) are virtually nil. That’s how successful tech companies are so profitable, even more than oil companies, because once the R&D is paid off every additional user is free money. And this means that companies don’t have to be profitable any time in particular as long as they promise sufficient projected growth to make up for being a money pit until then. You can get away with anything when your investors believe you’ll eventually have a billion users.
… Of course that doesn’t work when every customer interaction actually costs a buck or two in GPU compute, but I’m sure after a lot of handwaving they were able to explain to their investors how this is totally fine and totally sustainable and they’ll totally make their money back a thousandfold.
Should have asked chatgpt to play the role of a CEO.
This answer would be much funnier if that wasn’t his fucking plan.
This is my first experience listening to this guy, and I’ll be darned, it’s a another idiot billionaire.
I’d like to think there are intelligent billionaires but honestly folks, if you win that big and haven’t cashed out to do something more meaningful with you’re life, you’re an idiot.
jesus fuck how did i never see this before
Worth the watch just to hear the genuine laughter
If they are losing money on $200/month, that does not necessarily mean they lose money on the $20/month.
One is unlimited, the other is not. You only have to use the $200 subscription more than 10 times the amount the $20 subscription allows for OpenAI to earn less on that subscription.
We already know they’re losing money on everything
The plagiarism power virus is too expensive to operate? I’m shocked I tell you
Now imagine if they actually paid for the training data as well.
When has “not profitable” ever stopped a tech startup lmao
Hmm, we should get together some funds to buy a single unlimited subscription, and then let it continuously generate as large and complex prompts as the rate limitting allows.
On one hand, heck yes. On the other, part of the reason its so expensive is because of the energy and water usage, so sticking it to the man in this way also is harmful to the environment :(
Normally the people talking about water use have no idea what they are talking about. Normally data center cooling is closed loop, much like a cars cooling system. So they don’t use significant amounts of water at all.
hey shithead, what’s evaporative cooling and why metric-chasing design (PUE in this case) likes it so much?
Buy two. Ask the other to generate expensive prompts.
So people are really believing Altman would publish these damning statements without ulterior motives? Are we seriously this gullible? Holy shit, we reached a critical mass of acephalous humans, no turning back now.
losing money because people are using it more than expected
“I personally chose the price and thought we would make some money.”
Big MoviePass energy
Ahahahaha I forgot about moviepass
