The Gentle Singularity

[Analytics]

On Tuesday, June 10, 2025, Sam Altman made a blog post called “The Gentle Singularity” where he claims that in terms of our progress towards AGI, "We are past the event horizon; the takeoff has started.” The blog post talks about how A.I. is at the point of creating multiple self-reinforcing loops--A.I. being used to create better and faster A.I., which causes the rate of technological and scientific advances to accelerate, which allows us to build more chips faster, which will power more A.I., etc.

https://blog.samaltman.com/the-gentle-singularity

It isn’t a coincidence that on the same day this blog post was written, OpenAI released their newest public model: “ChatGPT o3-pro”. But how good is this new model?

I asked several questions to o3-pro and to an older model, 4o. In general, the o3 answers were marginally better. But then I asked it:

Can we go meta? I participate in DiscussMormonism.com, and several people there are skeptical about A.I.. Your task is to come up with a prompt, in the form of a question, related to Mormonism, that is tricky and is something that you (ChatGPT o3-pro) could do clearly better than 4o. In this prompt, be sure to ask that the answer be brief and text only so that it can be pasted into the discussion board.


You can read its response and the specific results of the test here:

viewtopic.php?f=4&t=159903&start=160

What I’d like to point out is how genius its answer to this was. It came up with a concise, specific, objective, doctrinally relevant question that it knew it would answer correctly, but that 4o would answer incorrectly. And it was right. My mind is blown by the elegance of this proof that it knows its stuff.

Looking forward, this sounds hard to wrap our heads around. But probably living through it will feel impressive but manageable. From a relativistic perspective, the singularity happens bit by bit, and the merge happens slowly. We are climbing the long arc of exponential technological progress; it always looks vertical looking forward and flat going backwards, but it’s one smooth curve. (Think back to 2020, and what it would have sounded like to have something close to AGI by 2025, versus what the last 5 years have actually been like.)

[Marcus]

It was interesting to see responses to Altman's article. Here's excerpts from one report:

Jun 11, 4:36 PM EDT by Joe Wilkins
Sam Altman Goes Off at A.I. Skeptic

"In the artificial intelligence world, there are two streams. One is the cool, analytical current of A.I. scholarship, flowing with genuine curiosity and drive to verify. The other is the boiling-hot torrent of commercial A.I. — excited, frenetic, gushing with utopian promises.

As A.I. hype blasts off into the heavens, one notable tech critic asks an important question: which of these streams should drive A.I. development?

Yesterday, neural scientist, A.I. scholar and outspoken OpenAI critic Gary Marcus took to X-formerly-Twitter to blast OpenAI CEO Sam Altman's brand of incessant A.I. hype.

"Sam keeps doubling down on bigger and bigger promises that are harder to keep," Marcus wrote. "Did Elizabeth Holmes do the same?"

...Marcus' comment came just hours after Altman published a characteristically strident essay, "The Gentle Singularity," in which he claims that "humanity is close to building digital superintelligence" — something which a lot of experts in the space say is patently false.

...Though Altman projects nothing but confidence, the strategy's long-term prospects remain divisive....

"Can't tell if he is a troll or just extremely intellectually dishonest," Altman continued in his broadside against Marcus. "Hundreds of millions of happy users, 5th biggest website in the world, people talking about it being the biggest change to their productivity ever... we deliver, he keeps ordering us off his lawn."

For his part, Marcus says he has no problem with OpenAI's commercial achievements, per se.

"I would be singing OpenAI's praises," the critic replied, "if it weren’t for the hype, much of it fanned by you, that has massively overstated what the technology can do today and in the near future."

If one could summarize Marcus' contention, it might be said that one can either be a rigorous A.I. scholar, or a billionaire tech emperor — but not both.

"I think this hype is harming the world," he said.

Altman's deep desire to be an A.I. thought leader — complete with bizarre pseudo-manifestos — is constantly putting him at odds with his tech dynasty, as his endless promises of "superintelligence," "solving physics," and "superhuman reasoning" make clear. The drive to hype his brand, not to mention his personal image, is constantly leading him to fantastic conclusions that fewer serious researchers would indulge, like the playground fantasy that functional humanoid robots "aren't very far away."

...Which of the two streams ultimately wins out will be decided in the years to come. For now, it seems commercial A.I. — and all the hype that follows — is dominating the conversation, though whether it can keep its lead as the financial losses mount is anyone's guess.

https://futurism.com/sam-altman-A.I.-skeptic

[Kishkumen]

I wonder why Sam Altman would promise the moon in his A.I. pronouncements.

If there a profit motive?

[Analytics]

The fundamental nature of exponential growth is that it can’t go on forever--there is always going to be an outside constraint in terms of money, energy, space, minerals, or something. The question is when will those limits hit and force the growth to slow down? When Rodney Stark predicted the Church would grow exponentially until it hit 260,000,000 members in the year 2080, he expected us to wait 100 years to find out if he was right. Fortunately, Sam Altman’s predictions are what will happen over the next 5 years. Whether he is right or wrong is something we’ll all find out in a blink of an eye.

Altman said, "Advanced A.I. is interesting for many reasons, but perhaps nothing is quite as significant as the fact that we can use it to do faster A.I. research.” I think Altman is situated to know whether or not that is really happening. As I recall, Physics Guy has argued something to the effect that training these same models with more and more data will only provide marginally better models, and thus, we are already near the plateau of what these models will ever be able to do.

But from my seat, LLM is just one component of A.I., and A.I. systems are getting much, much better at creating tools on the fly to solve problems and to validate their own answers. If I’m right about that, then its possible that the exponential growth is just beginning.

Ultimately, I’m an empiricist and have little interest in arguing about these things in abstract. The fact remains that a couple of days ago, I asked o3-pro to come up with a problem related to Mormonism that it could correctly solve but that the prior model could not. It came up with a brilliant question that was concise, relevant, interesting, and had an objective answer. And o3-pro got the answer right, while 4o got it wrong.

That is impressive.