Response to Jockers, Criddle, et al., Now Available

[_GlennThigpen]

Okay, let's get back on track here. The thread is about the Bruce Schaalje's response to the Jockers, Criddle, Witten NSC study on the Book of Mormon authorship published in 2008. Bruce made some extensions to the Jockers study which corrects for the possibility that the actual author is not in a tested candidate set. His results led to his conclusion that neither Solomon Spalding, Sidney Rigdon, Oliver Cowdery, Parley P. Pratt, or Joseph Smith are the authors of the Book of Mormon.

Here is a recent (February 18) comment he made on the Mormon Heretic Blog site.

A few random thoughts:

1. Criddle commented that “Schalje relies on a statistical tool to assign posterior probabilities for his Bayesian probabilities, then discounts to zero all of the historical evidence.” This is an odd statement, and it’s also completely backwards. A turned-around version of this statement is our main problem with the Criddle study, and is the very reason we developed the open-set extension.

By using closed set methods with only the S/R suspects as candidate authors, Criddle discounted to zero all authorship possibilities other than his S/R candidates. The open-set method — with nonzero prior probabilities for all candidate authors — by contrast, does not discount anyone or anything (including Criddle’s controversial historical evidence). In the paper, we formally assigned equal prior authorship probabilities to all of the candidate authors, including Spalding and Rigdon. For fun, I just reran the open-set NSC attributions with prior probabilities set as follows: early Rigdon .4, late Rigdon .1, Spalding .2, Cowdery .14, Smith .13, Pratt .01, Isaiah .01, someone else .01. The final attributions were almost identical to the equal prior situation. This is because the data so strongly contradicts the S/R prior.

Criddle’s closed set method is equivalent to open-set NSC with equal nonzero priors for his S/R candidates, but priors forced to zero for everyone else. Criddle and Jockers are right about their own study: if you force the S/R theory to be true (by using closed set NSC model), NSC classification will generate results consistent with the S/R theory. But the crippling circularity of this reasoning renders their results inconsequential (alliteration inspired by Neal A. Maxwell).

2. Criddle said: “there are serious problems with the method promoted by Schaalje et al. I can’t go into that now, but will likely do so when my work situation becomes more manageable.” This smacks of false bravado. I will wait with interest for his detailed comments on this topic, especially since the open-set method is simply a generalization of the closed-set NSC method used by Criddle himself. But seriously, if there is something wrong with our generalization that is obvious to Criddle, I would like to know. (It’s certainly possible that we have missed something. Academic discussions about these kinds of things are what the professional literature is all about.)

3. Criddle and Jockers keep saying that our argument was a straw man argument. I just don’t see it. We provided literature citations for the importance of allowing the candidate set to be open, we extended the Criddle method to allow an open set, we pointed out empirical problems with ignoring text size (a point that Criddle and Jockers admitted being worried about), and we constructed an artificial attribution example to demonstrate that nothing is lost but much is gained from properly allowing the candidate set to be open. The point of the artificial Hamilton-Rigdon example was only that absurd results could be avoided using the open-set technology. We included all of the Criddle candidate authors in our study with nonzero prior probabilities, and we used exactly the same marker words. I think that the cry of ‘straw man’ is too often used in place of well-considered counter-arguments.

4. Bishop Rick (by the way, our first counselor in a bishopric a few years ago was named Rick, and because the Bishop was gone so often, was referred to by most of the ward members as Bishop Rick): It seems to me as well that “Criddle was trying to prove the Book of Mormon was written by S/R . . . despite his claims.” We were trying to bring some sanity to the situation. We didn’t prove that the S/R theory is false – we simply argued that Criddle’s methods were flawed, so his results are meaningless. In doing so, we found that the NSC method could be made more useful and powerful in authorship attribution problems. We admitted in the Conclusion Section that more work is needed; this is not the final stylometric word in Book of Mormon research.

I found the entire project a fascinating application of statistical methods, independent of the Book of Mormon connection. So for me, the project was anything but a waste of time. Jockers has moved on to comparisons of other machine-learning tools for authorship attribution, so I don’t think he sees this project as a waste of time either.

5. We made a small effort at dealing with the archaic language issue. As mentioned in the paper, in one of our analyses we dropped “and it came to pass that” from the Book of Mormon chapters (we actually pretended that it was one word, effectively dropping it from the texts). The classification results were almost identical. But clearly, more needs to be done.

6. Keller: Thanks for your initial help on the project. As I remember, you pointed out the Koppel paper, a great and timely review of the authorship attribution literature.

The work of Wrencher, Larsen, and Layton identified twenty-four different author styles in the Book of Mormon, none of which came from nineteenth century authors.

The Berkeley Group Study also concluded that none of the Book of Mormon came from nineteenth century authors.

Bruce Schaalje's NSC extensions also provide the same conclusions. The S/R theorists need to deal with this.

Glenn

[_Dan Vogel]

Glenn,

For much the same reasons he does not accept the Hurlbut affidavits. Hurlbut was on a mission to discredit Joseph Smith.

The Conneaut witnesses made their claims before Hurlbut interviewed them, largely because they heard Mormon missionaries reading or explaining the Book of Mormon in public gatherings. Hurlbut was sent there to investigate these claims. Contrary to what you might assume, I do accept Hurlbut’s affidavits from the Smiths’ Palmyra/Manchester neighbors as primary historical sources, and I reject Mormon apologists’ attempts to dismiss them as Hurlbut fabrications. Sure, Hurbut was on a mission to discredit Joseph Smith, which made him chose people who had negative evidence, but that doesn’t mean the statements are fabrications or that their content is historically unreliable. Each statement is the signer’s responsibility, not Hurlbut’s, and none accused Hurlbut of manufacturing their statement.

[_Dan Vogel]

Thanks. In studies of false memory, those who had such memories implanted continued to believe they were real memories despite the researchers explaining what had actually happened. In other words, once the memory is formed, it’s real to that person.

Dan, you and I will probably disagree with one another than we will agree, but on this one, I am confident that history and science is in our favor. Roger has yet to come up with any scientific rebuttal on this. His example of his memories of the "Winds of War" is not really on target because there was no reason for a false memory to have been implanted.

Glenn

Rather than believe the possibility that the memories of the Conneaut witnesses were mistaken, which to seems like a textbook case of the phenomenon, Roger would rather cling to the unsupported and extremely unlikely speculations that Whitmer and other supporting witnesses either lied or didn’t observe Joseph Smith using a MS in the translating room. He cites his personal, and anecdotal evidence (which doesn’t at all fit the Spalding witnesses), to brush aside the false memory theory, and then offers his own unsupported assertion that the Mormon witnesses lied, which he tries to support by illogical and polemical arguments—and he can’t understand why we don’t take him seriously?

[_Dan Vogel]

Marg, we are not talking about false memory implantation, but memory substitution aided by suggestion. Although not entirely different from an implanted memory, it is much more prevalent, especially if there is some type of incentive.

Glenn please cite or quote what studies support "memory substitution" aided by suggestion which would warrant a conclusion the Conneaut witnesses were likely mistaken on their memory of the phrase "and it came to pass" , biblical language employed, and specific names used in Spalding's manuscript. I'm aware of a post by Dan which I'll address tomorrow in which he cites specific studies.

I want details, so they can be addressed...not simply a mentioning of a name such as E. Loftus.

I don’t use false memory theory to prove the Conneaut witnesses were mistaken, but to show that there is an explanation if we decide Joseph Smith was the sole author. What makes false memory theory “likely” in this instance is the historically stronger Mormon testimony.

[_Benjamin McGuire]

Glenn writes:

The work of Wrencher, Larsen, and Layton identified twenty-four different author styles in the Book of Mormon, none of which came from nineteenth century authors.

The Berkeley Group Study also concluded that none of the Book of Mormon came from nineteenth century authors.

Bruce Schaalje's NSC extensions also provide the same conclusions. The S/R theorists need to deal with this.

None of this is accurate, Glenn. All of these studies suggest that the styles in the Book of Mormon do not resemble specific 19th century authors. There may well be 19th century authors which these methods would identify with the Book of Mormon - we just haven't examined them.

[_GlennThigpen]

None of this is accurate, Glenn. All of these studies suggest that the styles in the Book of Mormon do not resemble specific 19th century authors. There may well be 19th century authors which these methods would identify with the Book of Mormon - we just haven't examined them.

I stand corrected on that. I needed to add that caveat, none of the 19th Century authors that were tested.

Glenn

[_GlennThigpen]

The Conneaut witnesses made their claims before Hurlbut interviewed them, largely because they heard Mormon missionaries reading or explaining the Book of Mormon in public gatherings. Hurlbut was sent there to investigate these claims. Contrary to what you might assume, I do accept Hurlbut’s affidavits from the Smiths’ Palmyra/Manchester neighbors as primary historical sources, and I reject Mormon apologists’ attempts to dismiss them as Hurlbut fabrications. Sure, Hurbut was on a mission to discredit Joseph Smith, which made him chose people who had negative evidence, but that doesn’t mean the statements are fabrications or that their content is historically unreliable. Each statement is the signer’s responsibility, not Hurlbut’s, and none accused Hurlbut of manufacturing their statement.

Dan, I have not found any documents written before Hurlbut came with his allegations that note any suspicions that the Book of Mormon was a rip off of Spaulding's romance. All of the allegations were written after the fact. That does not necessarily mean there were none, but it is surprising that there is no mention of it in the local papers considering the level of negative sentiment that the LDS missionaries met in many quarters.
I have never stated that I thought that Hurlbut had fabricated any of his affidavits. I do think he did more than a little judicious prompting and did not report any evidence that did not square with his designed mission. He also certainly was able to find quite a few people who were willing to sign affidavits against the Smiths' character. Those affidavits have to be evaluated along with the other historical evidence, as does the affidavits and statements of the Spaulding witnesses.

Glenn

[Bedlamite]

...

The Conneaut witnesses made their claims before Hurlbut interviewed them, largely because they heard Mormon missionaries reading or explaining the Book of Mormon in public gatherings. Hurlbut was sent there to investigate these claims.
...

From what I've studied, this is partly true. Claims were being
made in the adjacent counties of Ashtabula, Erie and Crawford
as early as 1832. Hurlbut was not sent to Ashtabula to check
on these claims, but rather, he served a Mormon mission in
Erie and Crawford counties, where he first learned of the claims

Hurlbut's first direct encounter with a "Conneaut witness" was
probably in "Jackson's Crossroads" -- but possibly also in
Springfield twp., Erie Co., in the spring of 1833. That year he
also served as a Mormon missionary in Crawford Co., where
John Spalding lived. Since John's statement was evidently the
first testimony taken by Hurlbut, it is possible that Hurlbut
began his "interviews" there in Crawford Co., PA in the spring
or summer of 1833.

A very late article in an Ohio newspaper claimed to document
John Spalding's testimony, written in the form of a letter, and
presumably sent to D.P.Hurlbut during the late summer of 1833.

If Hurlbut was "sent there to investigate" -- then it must have
been after he contracted with Kirtland area anti-Mormons, to
provide additional written testimony -- that is, more than just
those initial written statements from John and Martha Spalding.

Between the time that Hurlbut was taking down the testimony,
and the time Howe's book was published (late Nov., 1834)
there is little in the way of known sources regarding the witnesses
and their claims -- just a "press release" in the Palmyra paper,
a notice to the public in the Painesville newspaper, and some
local reporting of Hurlbut's pre-trial hearing and April trial.

Here is one source, not generally cited:

there are several persons in Salem, now Conneaut, who have seen it [the MS], and recollect distinctly the character of the fiction. They were intimate with Mr. Spalding, and used frequently to converse familiarly about his story, and to read it, or hear it [the MS] read to them. One man with whom I conversed, said he recollected the names of the characters and places, and that in general they remained the same in the Book of Mormon, as in the manuscript of Spaulding. He feels not the least hesitation in testifying to this, and to the identity of the works so far as relates to the narrative, or historical part. Indeed the resemblance is so striking, that it was detected the first evening the Mormonites preached in that place, merely from the passages read in the Book during service. There is all the certainty on the subject, in the minds of those who have seen the manuscript, that could be felt in any case, unless the two books could be laid side by side and compared. One man especially, by the name of Miller, who worked for Mr. Spalding several weeks, perhaps months, states that the manuscript lay on a shelf in the room where he slept, and that he spent many of his leisure hours in reading it, so long as he worked at the house.

http://www.sidneyrigdon.com/dbroadhu/OH ... htm#061234

I have reviewed thousands of pages of 19th century newspaper files
and microfilmed historical sources relating to the Ashtabula/Erie Co.
border region, and have found nothing that would gainsay the
character or public reputations of the witnesses. Most of them were
cited, quoted or otherwise documented in the region's newspapers
for many years after Howe's book was published.

When I left off researching in Ohio, there remained additional hundreds
(if not thousands) of pages of early 19th century newspaper articles
and local area historical items to be studied in the Western Reserve
Historical Association's library in Cleveland. It is possible that some
additional information on the Conneaut witnesses could be derived
from those files.

Possibly a researcher could locate some source material not previously
uncovered, relating to the witnesses, their statements, Hurlbut, Howe,
etc., -- but my guess is that anything truly wonderful would have
been communicated elsewhere, and that library research will not give
us much more than we already have available for investigation.

UD

[_marg]

Glen I asked you ..to please cite or quote what studies support "memory substitution" aided by suggestion which would warrant a conclusion the Conneaut witnesses were likely mistaken on their memory of the phrase "and it came to pass" , biblical language employed, and specific names used in Spalding's manuscript.

In this post - here - you provided a quote from E. Loftus to support "memory substitution" occurring for the Conneaut witnesses I presume by the sort of questioning by Hurlbut.

In the situations which Loftus is referring to the subject don't have a clear memory of an event. It might be a crime scene in which they don't remember a detail clearly such as the color of a car, the face or particular features of a criminal, whether a building existed or not, whether a stop sign or yield sign was at a corner, those sorts of things. And when asked questions they try to give an answer to the questioner. And it is not as if the subjects think they have a clear memory on the particular detail. In these sorts of situations memory substitution can occur easily whether intentional or not simply by questions which suggest a detail that a subject is not clear about. Obviously in criminal cases this could be extremely important.

But this is not the situation with the Conneaut witnesses. I won't quote them for brevity. But many looked at the Book of Mormon before giving their statement to Hurlbut. And what they said was that with the passage of time, they had forgotten some aspects of the Spalding story but that certain aspects they CLEARLY remembered. This is not unusual because many described frequent repeated exposure of material that interested them. In addition, though this is not something I've come across Loftus discussing, but some ideas, concepts, words can have what Malcolm Gladwell calls a "stickiness factor". That is all details one is exposed to would not necessarily be equivalent in remembering. Unusual unique names would be less likely to be confused with other names. If you had an old school friend but couldn't remember their name because of the passage of time but knew it was unusual..let's say it was Zoolu..you'd know that if someone tried to jog your memory and suggested Susan, Mary, Jane that you'd confidently be able to reject those common names. And if someone suggested Zoola who also happened to know this person your memory could be jogged and you could be confident that you now clearly remember the friends name as Zoola. This is not memory substitution which Loftus is talking about. It's a different situation mainly because the Conneaut witnesses said they clearly remembered certain aspects of the Spalding story. Some remembered Lehi and Nephi clearly. Being as they were uncommon names repeated frequently in the story...it would not be unusual for them to recall that those were the names in the story they had read frequently or heard read frequently, particularly if their memory had been jogged by the Book of Mormon. And that stickiness factor is applicable to remembering the writing being in the unique style of King James english and remembering the repeated phrase "and it came to pass" particularly is some remembered Spalding being called "Old Come to Pass"

So Glenn the citation you gave with the example for memory substitution is not the sort of situation applicable to the Conneaut witnesses. People generally do know when they remember something clearly or not even if there has been a long passage of time. Of course having one's memory jogged can be helpful as well.

[_GlennThigpen]

Glen I asked you ..to please cite or quote what studies support "memory substitution" aided by suggestion which would warrant a conclusion the Conneaut witnesses were likely mistaken on their memory of the phrase "and it came to pass" , biblical language employed, and specific names used in Spalding's manuscript.

In this post - here - you provided a quote from E. Loftus to support "memory substitution" occurring for the Conneaut witnesses I presume by the sort of questioning by Hurlbut.

In the situations which Loftus is referring to the subject don't have a clear memory of an event. It might be a crime scene in which they don't remember a detail clearly such as the color of a car, the face or particular features of a criminal, whether a building existed or not, whether a stop sign or yield sign was at a corner, those sorts of things. And when asked questions they try to give an answer to the questioner. And it is not as if the subjects think they have a clear memory on the particular detail. In these sorts of situations memory substitution can occur easily whether intentional or not simply by questions which suggest a detail that a subject is not clear about. Obviously in criminal cases this could be extremely important.

But this is not the situation with the Conneaut witnesses. I won't quote them for brevity. But many looked at the Book of Mormon before giving their statement to Hurlbut. And what they said was that with the passage of time, they had forgotten some aspects of the Spalding story but that certain aspects they CLEARLY remembered. This is not unusual because many described frequent repeated exposure of material that interested them. In addition, though this is not something I've come across Loftus discussing, but some ideas, concepts, words can have what Malcolm Gladwell calls a "stickiness factor". That is all details one is exposed to would not necessarily be equivalent in remembering. Unusual unique names would be less likely to be confused with other names. If you had an old school friend but couldn't remember their name because of the passage of time but knew it was unusual..let's say it was Zoolu..you'd know that if someone tried to jog your memory and suggested Susan, Mary, Jane that you'd confidently be able to reject those common names. And if someone suggested Zoola who also happened to know this person your memory could be jogged and you could be confident that you now clearly remember the friends name as Zoola. This is not memory substitution which Loftus is talking about. It's a different situation mainly because the Conneaut witnesses said they clearly remembered certain aspects of the Spalding story. Some remembered Lehi and Nephi clearly. Being as they were uncommon names repeated frequently in the story...it would not be unusual for them to recall that those were the names in the story they had read frequently or heard read frequently, particularly if their memory had been jogged by the Book of Mormon. And that stickiness factor is applicable to remembering the writing being in the unique style of King James english and remembering the repeated phrase "and it came to pass" particularly is some remembered Spalding being called "Old Come to Pass"

So Glenn the citation you gave with the example for memory substitution is not the sort of situation applicable to the Conneaut witnesses. People generally do know when they remember something clearly or not even if there has been a long passage of time. Of course having one's memory jogged can be helpful as well.

I disagree with you marge. The further away in time an event occurs, the easier for memory substitution to occur. We do not know what questions Hurlbut asked those witnesses, what information he fed them, how those questions were phrased. But the identical phrase "by land and sea" by four of those witnesses are indicative of prompting. I asked my wife how Nephi and Lehi got to the new world. Her answer, on the ship Nephi built. None of the Conneaut witnesses even mentioned a ship. When asked what events were most memorable to her, the story of Laban was the one that grabbed her the most. These are historical elements and several of the witnesses said that the Book of Mormon was the same as Spalding's romance, except for the religious matter. However, only one witness remarked on "the tragic figure of Laban" which is more consistent with Labansko in the Oberlin manuscript. He is not a tragic figure at all in the Book of Mormon, but rather a thief, drunkard, and would be murderer. None of the witnesses noted building a ship, which was integral to their journey by sea. Yet they "clearly" remembered the romance?
Memory substitution is very plausible phenomenon with those witnesses, especially since none of the non Hurlbut coached witnesses had those memories of by land and sea, Nephi, Lehi, Lamanites, and Nephites.

Glenn