Response to Jockers, Criddle, et al., Now Available

[_GlennThigpen]

Well this is obviously where the experts disagree and since I am not an expert I have to draw my conclusions based on what is presented by both sides in plain English. I attempted to summarize my understanding of what is being disputed in this thread. Do you disagree with my layman's summation?


If you are correct that option 2 is flawed, do you suggest that Bruce's conclusions are as tenuous as you allege Jocker's to be since even Ben acknowledges that Bruce "still uses most of the framework that the Jockers study erected"

Roger, have you read Bruce's paper? Although you may not be able to follow the math, the logic is laid out very well and I am sure you can follow it. What the Jockers study did not do is perform any kind of goodness of fit test or include any mechanism for determining if any one of the authors in a closed candidate set is actually the author of the text in question. Bruce's work corrected those flaws.

Well this is why I would like to see an ordered debate from the principle experts in plain English on a forum such as this. I think such a debate would be a huge benefit to the public.

I could envision it as one or two principles from both sides going back and forth for a limited time of say maybe three days and then a couple days open to the public for questions and then a formal end of the discussion on the part of the principles unless they wanted to continue the discussion. That would be fascinating.

I don't think that we will see that. Daniela Witten is the statistician who did the math in the original Jockers study and one of her coauthors noted that she will not engage in an online debate. Matt Jockers and Craig Criddle do not have the statistics background to engage Bruce in a discussion. However, if you have read Matt's post in this thread, he basically said that a person could not form a conclusion based on their original study that any of the authors candidates used in the original study were actually the authors of any of the Book of Mormon segments attributed to them because those were only relative probabilities compared to the other authors in that closed set. Where the errors have arose is taking those relative probabilities and holdin them to be absolutes.
Bruce's paper makes it clear how erroneous that can be.

Glenn

[_MCB]

. Where the errors have arose is taking those relative probabilities and holdin them to be absolutes.

The whole thing is like psychometrics. It needs to be considered in the entire context, as only part of the evidence.

Just reading it, with the Jockers study at hand, should be a revelatory experience for many.

[_GlennThigpen]

Chris, the second half of the Book of Mormon is rich with allusions to other literature. The first half is a paltry patch-up job. WHY??? How do you account for this?

Please provide a CFR for your statement. That does describe the Book of Mormon I have read and read about.

Perhaps the principal authors of the first half were not half as well-read as one of the authors of the second half. And who was that person? I would suggest that he was a semi-invalid, who had plenty of time to read literature which was available in 1815.

Solomon Spalding? Have you read Bruce's paper yet?

Glenn

[_MCB]

I'm sorry, but that argument is presently laid out in more that 500 pages, and is due to grow by another 50 in the next week. I really don't think I could do such a thing on a mere message board.

[_GlennThigpen]

The whole thing is like psychometrics. It needs to be considered in the entire context, as only part of the evidence.

Just reading it, with the Jockers study at hand, should be a revelatory experience for many.

You have to read the Jockers study in relationship to what Matt Jockers said about it on this thread. Then you have to read Bruce's paper to better understand what the Jockers study was and wasn't.

Glenn

[_MCB]

Good night.

[_CaliforniaKid]

Do you have you done any work or can you cite work that actually shows that word frequency patterns change with genre?

Take a look at the paper I linked to.

I think that such would be true for contextual words, but other studies have shown that non-contextual word pattern usage remains pretty much constant from one type of work to another and over a life time. You probaly are already familiar with John Hilton's Berkeley Group paper on word prints which pointed that out. There have been other studies that non-contextual word prints can even survive a stroke.

First of all, the Jockers/Burrows method doesn't use only non-contextual words. It uses a large number of words, and employs a tuning parameter to determine which words are most effective for deciding amongst your particular set of candidate authors. The tuning parameter, however, is set up in such a way that it doesn't really work if the candidate authors each tended to write in different genres. You'll then be "tuned" to genre as well as authorship. The tuning method is also problematic if the text being tested is in a different genre than the training texts, since the words that are effective for distinguishing between (for example) two authors' private letters might not be effective for distinguishing between (for example) their pseudo-biblical narratives.

Secondly, the Hilton study did not measure the same thing as the Jockers/Witten study. It looked at patterns of non-contextual word usage rather than word frequencies. My dabbling in the subject suggests that word frequencies are quite sensitive to genre, even in the cases of so-called "non-contextual" words such as the word "and". However, I can't speak to the issue of patterns, as measured by Hilton. I am skeptical that these are as invulnerable to genre as he claims, but they may well be moreso than word frequencies.

The Hilton study is a different issue that requires separate testing, but here are a few of my questions about it. 1) The Book of Mormon is not merely a different genre, but a radically different linguistic style than what the 19th century candidate authors customarily wrote in. This hasn't been sufficiently controlled for, in my opinion. 2) Hilton argues that non-contextual wordprints survive even translations from German to English, but I'm extremely skeptical of this. In particular, I'm wondering how "tight" the translation was. I'd want to see more testing done before I'd accept this. 3) I'm not quite sure how Hilton determined where sentences ended in the Book of Mormon's original manuscript text, which is integral to his method. 4) Hilton's method uses words that in some cases would not have been present in the original Hebrew.

There is much to suggest that the Hilton study is the best wordprint study that has been done so far. For one thing, they tried to correct for a number of problems that afflicted earlier (and later) studies. However, in my opinion the jury is still out on the study's overall reliability as applied to the Book of Mormon.

Peace,

-Chris

[_GlennThigpen]

Do you have you done any work or can you cite work that actually shows that word frequency patterns change with genre?

Take a look at the paper I linked to.

Peace,

-Chris

Thanks for the information. When I read you post with the link to your work on the Urantia papers, I did not see the link. It did not show up well on my laptop screen for some reason. That does pose some interesting questions about genre though.

There should be some known authors that have known works in different genres that can be used to further the knowledge in this area.

Glenn

[_Roger]

Hi Glenn:

Roger, have you read Bruce's paper? Although you may not be able to follow the math, the logic is laid out very well and I am sure you can follow it. What the Jockers study did not do is perform any kind of goodness of fit test or include any mechanism for determining if any one of the authors in a closed candidate set is actually the author of the text in question. Bruce's work corrected those flaws.

Not the whole thing. I dabble in Book of Mormon stuff when time permits. However, given what I've heard about it, I'm not convinced I can follow the logic given the math it apparently relies on. Beyond that, even if I did read the whole thing and did follow it, my opinion doesn't count for much in comparison to others like Dale or Craig or for that matter what Matt has already written. That's why I would prefer to see a discussion from qualified people. Frankly, I became convinced that Sidney Rigdon played a part in Book of Mormon production before I knew much at all about Jocker's studies. If I were to read Bruce's paper, comprehend it perfectly and then conclude he's right, it would likely do less to cause me to reconsider S/R than it would to side with Chris's skepticism about the reliability of word print studies.

I tend to point to what I perceive to be the weak spots in Chris's logic, while at the same time I think he makes some valid points. When someone is deliberately attempting to mimick another style, is his use of non-contextual words affected? To the layman, the common sense answer seems to be yes. All I have to go on otherwise is that a lot of smart people seem to think otherwise. Where I run into to difficulty is when we take Chris's logic to it's logical end with regard to the Book of Mormon. I can see Joseph's word print being affected because he's deliberately attempting to write like King James, but it seems a stretch that he could come up with several identifiable non-contextual "personalities" that can be picked up by a computer analysis as he's doing the mimicking allegedly on the fly. I suppose stranger things may have happened but that would certainly be a chart topper.

if you have read Matt's post in this thread, he basically said that a person could not form a conclusion based on their original study that any of the authors candidates used in the original study were actually the authors of any of the Book of Mormon segments attributed to them because those were only relative probabilities compared to the other authors in that closed set. Where the errors have arose is taking those relative probabilities and holdin them to be absolutes.

I don't know anyone who is using the Jocker's data in an absolute way, including Matt himself. If anything, it seems to be Bruce who seems to be suggesting that none of the candidate authors had anything to do with producing content for the Book of Mormon, which is clearly not an empirical fact.

Matt said himself that his study selects the most likely suspect among a closed set. He acknowledges that it is possible the true author might not be among the list of candidates. But if we can use reason to arrive at the reasonable conclusion that the candidates tested are probably the most likely of possible 19th century suspects, then, given that assumption, even Ben agrees the results are very accurate if the real author is in the mix.

Obviously if the Book of Mormon is ancient so far we haven't found samples of Moroni's non-BOM abridgments or Nephi's other writings, etc. And that will most likely continue to be a problem without solution. But if the real author is among the candidate list, what has Bruce's study done to make Ben's assertion false?

All the best.

[_CaliforniaKid]

I can see Joseph's word print being affected because he's deliberately attempting to write like King James, but it seems a stretch that he could come up with several identifiable non-contextual "personalities" that can be picked up by a computer analysis as he's doing the mimicking allegedly on the fly. I suppose stranger things may have happened but that would certainly be a chart topper.

There are several things that could differentiate the style of various parts of the Book of Mormon. For example, one part might quote extensively from Isaiah, while another might quote extensively from Luke. Or one part could be a narrative, while another is a theological discourse. Or Joseph's style could have changed over time, so that one end of the book differs from the other end. When you combine the effects of all of these factors, you could get quite a stylistic "spread" across the book. This stylistic differentiation doesn't necessarily have to be the result of deliberate efforts on Joseph's part to create different "narrative voices".