There are several interesting aspects of this discussion, assuming one is interested in where the Book of Mormon came from, but the thing that is most striking to me is this:
Ben wrote:
And of course, one of the more interesting features is that they applied the closed and open methods to the Hamilton papers, using the proposed authors - Joseph Smith, early Sidney Rigdon, late Sidney Rigdon, Solomon Spalding, Oliver Cowdery, and Parley Pratt. The results? of the 51 Hamilton papers, the closed NSC method (Jockers method) assigned Rigdon as the author of 28 of them (with probabilities ranging as high as 0.9999), Pratt became the author of 12, and Cowdery got the other 11. When the open NSC method was used, 2 of them were assigned to Rigdon, and the other 49 were assigned to an unknown author.
No matter how you slice the bread, the Jockers method is fatally flawed. And no amount of adding other possibilities or better controls will somehow make the Book of Mormon text come closer to Spalding or Rigdon. Adding more authors won't somehow make the language more similar.
This appears to be the strawman argument referred to by Matt and taken to it's logical conclusion. As Matt points out, the Jockers method is only fatally flawed if one is attempting to read something into the premise that was never there. As Matt puts it, the test was to determine the most likely
suspect within a closed set:
Matt Jockers wrote:
Our objective was to test *existing* theories of authorship of the Book of Mormon. And that's exactly what we did. We took the list of suspects and tried to rank them in terms of their likelihood. We point out in the paper that it is possible that the *real* author is not in the closed set. The real author could, for example, be Moroni or Napoleon. The point of our work was to "reassess" prior theories of authorship using machine classification. Our result is only compelling if you first accept the historical evidence and then accept that the candidates we tested represent a good set of candidates. The Schaalje paper creates a straw man argument and then plays fast and lose with the facts of our paper, cherry picking little bits here and there to make it look like we did something other than what we did, or that we had a goal other than what we had. In my opinion the fundamental conclusion of our work is that it lends additional support to the spalding-rigdon theory of authorship.
This, of course, is quite reasonable, and I believe Matt is certainly correct to assert that the conclusion of their work lends additional support to the spalding-rigdon theory of authorship--nothing more, nothing less. It is certainly not "fatally flawed."
What makes this even more interesting is that Ben also seems to recognize this:
Ben wrote:
The approach that Jockers and company took was, I think, quite valuable. It was something new - used some good tools, and so on. It had some flaws. This new paper does not destroy the Jockers methodology. It corrects it. It still uses most of the framework that the Jockers study erected. What it does is show that the flaws in the methodology resulted in bad conclusions in this particular test case. When the Jockers study has the actual author in the mix, it does very, very well (just as the open NSC method proposed in the paper under the same circumstances). When the author is not in the mix, it does very poorly - and there was no mechanism to determine whether or not the author was in the mix. What this paper does is introduce a mechanism to do just that - and in doing so allows the process to spit back a result that says that all of the potential authors are dissimilar enough to the test material to suggest that none of them are good candidates for authorship.
It is clear from this, then, when coupled with Matt Jockers statements about the scope of his study, that it is ultimately not the methodology Ben (and I assume most LDS apologists) dispute but rather the underlying assumption that the real author
is among the candidate set.
It is noteworthy that Ben acknowledges that "When the Jockers study has the actual author in the mix, it does very, very well..." If Ben is correct in that observation, we can at least conclude that if any of the candidate authors contributed content to the Book of Mormon, then Jocker's results are meaningful, and, as Matt concludes, offer additional support to the Spalding/Rigdon theory of Book of Mormon authorship. After all, how likely is it that Joseph Smith (for example) contributed nothing to the Book of Mormon text? Even many LDS apologists have come to associate the errors with Joseph Smith rather than ancient Nephites.
It is also interesting to note that Chris Smith disagrees--or at least is tentative about agreeing--with Ben on that point:
I'm not yet persuaded of that. Cross-genre tests still seem to be an intractable problem for the method.
Chris also made the astute observation that:
All I was saying is that their assumptions explain why they approach the problem so differently.
Of course, this is correct and we all bring various assumptions to the discussion.
For the defenders of the Book of Mormon as an ancient collection of texts, the assumption is that the true author is not among the set tested by Jockers.
For the defenders of the S/R theory, the assumption is that Jocker's candidates are the most likely
possible candidates.
And for the defenders of Joseph Smith alone producing the Book of Mormon, the assumption is that any test without Smith is meaningless and even with Smith included in the mix, the results may be questionable due to genre factors and whether an author has the ability to alter his non-contextual wordprint either consciously or subconsciously when his goal is to imitate another style.
From the layman's perspective, it would sure be nice to take about a week where experts from both sides lay out their respective cases in plain English on this forum. It would appear that that is not going happen, however, beyond what has already been stated on this thread, which leaves the rest of us to our speculation.
I suppose that even if we had such a debate, the end result would be to come back around to those assumptions with everyone still lining up where they started out. Nevertheless, I'd still like to see it!
In any event, to get
my perspective in the mix....
Jocker's studies do what they were intended to do... identify the most likely suspect within a closed set of the most likely set of candidates. Obviously results with Smith included are more meaningful in that regard.
While Chris suggests Jockers results may not be reliable "from the get-go" it is noteworthy that Ben, on the other hand concedes that when the Jockers study has the actual author in the mix, "it does very, very well."
Chris may have a point, but it would seem there are a lot of experts who think otherwise. It would appear--again to the layman--that what Chris is suggesting is that someone who is attempting to imitate another style can effectively mislead a computer. Or stated another way, that Joseph Smith could effectively structure his use of non-contextual words in such a way as to appear to be multiple authors. To me, this seems unlikely.
If that is correct, then we are left with the question: Does a list that includes Joseph Smith, Oliver Cowdery, Parley Pratt, Sidney Rigdon and Solomon Spalding at least include the most likely author(s) of the Book of Mormon? LDS apologists will obviously assume the answer is no while I conclude it does.
Ultimately it comes down to which underlying assumption is the correct one.
All the best.
"...a pious lie, you know, has a great deal more influence with an ignorant people than a profane one."
- Sidney Rigdon, as quoted in the Quincy Whig, June 8, 1839, vol 2 #6.