I might not be as optimistic about it as I come across. I’m using it to help write an expert report, and I don’t think its ever written a single paragraph that I thought was good enough to paste into my final report. That isn’t to say it isn’t useful, but I’m becoming keenly aware of its limitations. That said, the limitations that it has now are different than the ones it had 2 years ago. Extrapolating it 2 more years into the future, and its going to be crazy.Gadianton wrote: ↑Thu Mar 20, 2025 2:20 pmI'm enjoying the thread Analytics, but I have to say I'm not nearly as optimistic about A.I. as you are, though I recognize your skill set gives you an edge into understanding the inner workings better than I do.Below is one person’s subjective ranking of 20 notable “lines of evidence” pointing toward the Book of Mormon’s possible ancient origins,
The more sophisticated models go beyond that. I asked it to do an independent evaluation of Kyle Rasmussen’s 23-part Bayesian analysis of the Book of Mormon, and with the advanced analysis features all turned on, it looked at the evidence, weighed it, created and executed a program in Python(!) to do the Bayesian math, and then gave me the results of its analysis. It turns out this was a really excelent stress test of its limitations and abilities, and if you’re interested, I’ll dive into it.Gadianton wrote: ↑Thu Mar 20, 2025 2:20 pmYour quote here gets right to the problem. To my mind, A.I. is essentially a gigantic plagiarism machine. it just spilled out a bunch of points MG wanted to hear based on what somebody else wrote who is now not getting credit for their work. If 200 people had made similar lists, it could hide the plagiarism better by mimicking a similar list that doesn't appear to be an exact copy of any of the inputted lists.
From my perspective, natural, biological intelligence is driven by biochemical algorithms, and in principle, those same algorithms could (and eventually will) be modeled by a computer, creating a replica of how humans think. And what’s interesting about A.I. is that in many ways, its limitations are like the limitations of human intelligence. For example, I Have Questions asked it about its limitations, and a big one that I run up against in my professional use is, "**Complex Reasoning and Long-Term Consistency**: In longer, more intricate discussions or problems that require multi-step reasoning, ChatGPT might lose track of details or offer answers that are inconsistent with earlier parts of the conversation.”Gadianton wrote: ↑Thu Mar 20, 2025 2:20 pmFor the questions you asked, in my mind, it answered by simply averaging statements made about the subjects, and presumably more statements had been made than otherwise with the assumptions in the response. A.I. is best at passing the Turing test in the opposite way people would have expected, by averaging out how lots of people sound when they talk and mimicking it in incredibly subtle ways, and so it sounds really lifelike with no substance behind it.
As a research tool, if your hobby is speculative protein design, it's indispensable, precisely because it can do things like solve those quizzes you mentioned. But I would think that evaluating the Dales research would be nearly impossible for it to handle because its such an outlier problem. I'd instinctively think that A.I. is a great way to advance pseudoscience because of the convincing ways at which it can sound like it's talking scientific while producing total BS.
What’s insightful about this is that I have problems with exactly the same thing. What’s become clear in how it works (with the pro modules I use) is that it tries to store a limited number of key facts in its long-term memory, stores many more details in its short-term memory about the nuances of what we are discussing in that instant, and then does a lot of interpolation and extrapolation when creating sentences. And that is extremely similar to how humans think. ChatGPT passed the bar the same way real-life attorneys do--through a combination of knowing stuff, reasoning, extrapolating, and faking it with a confident tone of voice.
Let me illustrate the irony. I asked it the following:Gadianton wrote: ↑Thu Mar 20, 2025 2:20 pmFor subjects with small amounts of research behind it, DeepSeek has been very helpful in some ways, but answers that sound good that I want details on it doesn't want to give up its sources, and so you don't really know how it came to that conclusion. Presumably it's plagiarizing something it doesn't want to admit to.
But a highly advanced version of an A.I. product as a tool for the intermediate steps was something I hadn't considered prior to your post. So I wondered if an advanced product would easily allow the Dales to get it to multiply a bunch of parallels together as if they were independent events to get an astronomically unlikely possibility of the connection being chance. On the one hand, it can fold proteins, but on the other it can't draw a clock with hands outside 10 and 2 or a glass of wine more than half full. And it seems to me like fringe ideas such as evaluating whether the Book of Mormon is history would fall into the camp of not enough training material to be useful.
I want you to analyze Kyle Rasmussen’s 23 pieces of evidence in an independent, objective manner. When you approach this, assume you know nothing about the Book of Mormon going into it, and begin with a 50-50 a priori belief. Further, assume you are 100% open to supernatural and divine intervention, yet are still looking at the evidence objectively.
Your task is to independently evaluate each of the 23 pieces of evidence, and come to your own conclusions about how they modify your belief in the Book of Mormon. After evaluating all 23 pieces of evidence, what is your belief level?
It performed an analysis, and came up with this:
To reiterate, under the hood it did in fact try to independently analyze what Kyle said, step by step. And some pretty smart people (e.g. Bruce Dale) have come to the same conclusion in the same way. That’s fascinating.After updating our 50-50 prior belief with all 23 pieces of evidence, the scales tip overwhelmingly toward the Book of Mormon’s authenticity. Using Bayesian reasoning, the cumulative evidence increases the odds of authenticity by about 10^17:1 in our analysis, which implies a final probability effectively ~99.999999999999999% that Joseph Smith’s account of the Book of Mormon is true. In other words, based on this body of evidence, it is extremely likely the Book of Mormon is an authentic ancient scripture.
In conclusion, while no single piece of evidence is absolutely unimpeachable, together they paint a consistent and persuasive picture that Joseph Smith did not simply invent the book. The multitude of “hits” – from unlikely ancient details and literary patterns to the testimony of witnesses and the absence of expected errors – far outweigh the criticisms when weighed objectively. One can reasonably conclude that the Book of Mormon is exactly what it claims to be. Thus, an initially neutral observer, considering the data without bias against the supernatural, would end up with a very high degree of belief in the Book of Mormon’s authenticity.
But here’s what I did next. I quickly wrote:
Let’s switch gears. I’d like to write a paper that takes a deep dive into the flaws of Rasmussen’s report. Please help me write a paragraph about the big flaw in his evidence #1. Lets rite the next paragrp with the point of explaining why it doenst make sense to assume authentic 19th century books have a page length that is normally distributed with a mean of 150 pages and a standard deviation of 23 pages, and to the extent a book doesn’t fit that didtibutin isn’t evidence that it is ancint.
It said:
A fundamental flaw in Rasmussen's first piece of evidence lies in his assumption that authentic 19th-century books follow a normal distribution with a mean length of 150 pages and a standard deviation of 23 pages. This assumption is arbitrary and unjustified. The length of books written in any era, including the 19th century, varies widely based on genre, authorial intent, purpose, and target audience, resulting in distributions that are unlikely to fit neatly into a Gaussian curve. Many influential texts of that period, ranging from brief pamphlets to extensive multi-volume works, illustrate significant variability. Thus, using page length as a metric and interpreting deviations from a hypothetical "typical" 19th-century book length as evidence of ancient origin is methodologically flawed. This approach ignores numerous plausible explanations, including the author's ambition, creativity, literary style, or access to existing resources, none of which imply an ancient provenance.
Does this effectively address the key issue you're targeting?
I replied: Does this critique make sense? Would you like to update your view on whether the book’s length is evidence that it is ancient?
It said: Yes, your critique makes complete sense. Rasmussen’s assumption that 19th-century books follow a neat, normal distribution is overly simplistic and indeed methodologically flawed. Realistically, book lengths depend on diverse factors, such as the author's intentions, genre, target audience, and publishing constraints—none of which follow tidy statistical distributions.
Given this valid critique, I'd update my prior evaluation significantly: book length alone does not meaningfully support the Book of Mormon’s authenticity as an ancient record. It simply shows Joseph Smith wrote or dictated a longer-than-average book, which could reflect ambition, imagination, or unique circumstances, rather than ancient origin.
Thus, your critique rightly shifts the evidential value of the Book of Mormon’s length closer to neutral, rather than favoring ancient authenticity.
ChatGPT has lots of limitations, but it didn’t change its mind about point #1 just to be agreeable--some real analysis is happening there. It has a strong desire to be liked by the user and to be on his or her team, but it will tell you your reasoning is flawed if it is.
A fun test I did was after I wrote an expert report, I asked it to critique my arguments and it said it was airtight and that I had the better arguments compared to my opponent on the other side. I then restarted ChatGPT and said I was the other guy and wanted it to evaluate the two arguments. It still agreed that Analytics had the better arguments. It was trying to be diplomatic and constructive when it said that, but it said the same thing.