Searching for God in Mac Bible Software
A comparison of speed and accuracy
by David Lang (Posted: 7/28/04)
It was the comment about Bible Reader Free having a "much faster" search engine than MacSword that particularly caught my attention, because every time I've tried to do a search in Bible Reader Free, I've found it to be painfully slow. So out of curiosity, I did some comparative searches using each of the currently developed Mac Bible Software applications. The results were somewhat surprising, not only because of the vast difference in search speed among the various Bible programs, but also because of the high degree of variance in what was actually found.
Not an Ideal Tester
When I started doing these comparative searches, I didn't really intend to publish my findings, because, as I've mentioned elsewhere, I'm not really qualified to write comparative reviews. This is because I work for the developers of one of these programs (Accordance Bible Software), and so cannot act as an impartial reviewer. But my intention here is not to give my opinions or recommendations. It is simply to test someone else's speed comparison and to report on the differences in the way each search engine performs. Hopefully I can report the results without bias, but in case I can't, you've been given fair warning of my potential "conflict of interests."
Searching for God
Differences in search speed are more evident when searching for common words than for uncommon ones, so I decided to use as my test case a simple search for the word "God" in the entire text of the King James Bible. I set my menu bar clock to display the seconds, and simply watched how many seconds ticked by from the time I clicked each program's search button to the time the search was completed. I used the most recent version of each program, and conducted this test on a 1.2 GHz G4 iBook with 768 MB of RAM. Obviously, these times will vary depending on the computer being used.
The following chart shows the time each program took to complete the search in seconds (shorter bars are better):
A few notes about the times listed here. Although I put a time of one second for Online Bible, the search was really finished almost instantaneously. Likewise, the search in Accordance was completed in just under two seconds, but since I have no way of measuring fractions of a second, I rounded up.
From this simple test, it appears that Charles Moore's assessment of Bible Reader Free's relative speed was grossly inaccurate. Could a guy who reviews software for a living really be that far off the mark? Well, yes and no.
I suspect that when Mr. Moore praises Bible Reader Free for its speed, he is referring not to the time it takes to complete a search, but to the time it takes to be able to interact with the results of a search. Bible Reader Free is the only one of these programs that displays partial search results before the search is fully completed. Although the search for "God" took 52 seconds to complete, the first fifty or so occurrences of the search word were displayed after about a second, and the user can then begin looking through the verses that were found while Bible Reader Free plods through the rest of the search. As long as Bible Reader Free can carry out the search faster than the user can look through the results, the user is unlikely to feel the wait too keenly. Thus, in terms of sheer searching speed, Bible Reader Free is by far the slowest, but in terms of the time it takes to begin working with the results of a search, Bible Reader Free is just a hair off the lead.
As you can see, the question of comparative searching speed was relatively easy to answer. The real challenge was in figuring out the remarkable variance in the results that were returned.
Where Can God Be Found?
Every one of these programs offered some rudimentary search statistics. Some listed the number of verses displayed, some gave the number of times the search word was found, and some did both. The problem was that whatever numbers were given, they were different for each program!
Accordance listed 4,444 occurrences of the word "God" across 3,876 verses. Bible Reader Free reported that the search had "returned 4,455 items. iBible said that there were 4,470 "matches" for God. MacSword found 4,117 "references" meaning the number of verses rather than the actual number of occurrences. When searched for the word "God," Online Bible returned 3,637 verses. But Online Bible's search dialog also has a count button. Clicking this button will not only perform a search for "God," it will also report the actual number of occurrences. When I did this, Online Bible reported the count of hit words as 4,164. The only problem was that the number of verses displayed went down to 3,611!
"Lies, Damned Lies, and Statistics?"
Besides reinforcing Mark Twain's famous adage that there are three kinds of lies"lies, damned lies, and statistics"all these disparate counts reveal a tremendous lack of consistency in the way Mac Bible programs perform searches and gather statistics. What can account for all these differences?
Let's start with a baseline. As a developer of Accordance, my first concern was to verify that Accordance's count was correct. I therefore exported the text of the KJV from Accordance and opened it in a text editor which supports GREP searching. When I did an exact word search for "God" when not followed by an apostrophe or hyphen, the text editor returned a count of 4,444the same number Accordance had given. Whew!
By comparing the verses found by the various programs, I soon discovered that some programs were also returning occurrences of "God" when it is followed by an apostrophe-s. There are twenty-six occurrences of the word "God's" in the KJV, and when added to our baseline of 4,444 you get 4,470: the exact number of matches iBible returned.
Bible Reader Free's count of 4,455 is a little harder to reconcile. Bible Reader Free, like iBible, included the twenty-six occurrences of "God's," yet its count comes out to be fifteen less than iBible's. I managed to discover one of these missing fifteen occurrences (more on that below), but I have yet to figure out where the other fourteen are.
MacSword did not give the number of occurrences, only the number of verses, but the number of verses was so much higher than any other program that it was difficult to understand where these results were coming from. An examination of the verses which were only found by MacSword soon revealed that MacSword was finding every word that contained the letters "god." Thus, our search for "God" also turned up words like "ungodly" and "Gudgodah." I then remembered that Rubén Gómez had detailed this problem of "double-wildcard searches" in his review of MacSword.
To verify MacSword's count, I did a search for "*god*" in Accordance, but to my surprise, Accordance turned up 4,118 verses, one more than MacSword! The extra occurrence turned out to be the superscription to Psalm 90, which reads "A Prayer of Moses the man of God." MacSword's KJV text does not include the superscriptions to the various psalms. Another problem with MacSword's KJV is that it appears to be missing the "s" in many words ending with apostrophe-s. For example, the "s" in "Jacob's" and "God's" is missing in Genesis 30:2.
Bible Reader Free's KJV text likewise is missing the psalm superscriptions, which explains one of the missing fifteen occurrences of "God."
Online Bible's counts were the hardest to figure out. First, if you just do a regular search, Online Bible displays 3,637 verses, but if you do a "Count" search, Online Bible returns 3,611 verses. At first glance, these extra 26 verses appear to be false hits. For example, Genesis 28:19 is included when you do a regular search, even though nowhere in that verse does the word "God" appear. The answer is that Genesis 28:19 contains a footnote explaining that the name "Bethel" means "house of God." That verse is being returned because the word "God" appears in the footnote, but unless you have footnotes displayed, there is no way to see why this verse is being included. For whatever reason, the footnotes are ignored when you do a "Count" search, a fact which explains why this second method of searching only returns 3,611 verses.
A careful examination of these verses revealed several things. First, Online Bible's search for "God" also turns up all twenty-six occurrences of "God's" as well as the three occurrences of the compound word "God-ward." However, Online Bible misses all but one occurrence of "GOD" when it appears in all-caps. I'm not sure why it finds Acts 17:23 "TO THE UNKNOWN GOD" when it misses the other 309 occurrences of "GOD", but there you have it. If we add the 26 occurrences of "God's" and the three occurrences of "God-ward" to our baseline count of 4,444, then subtract the 309 missed occurrences of "GOD," we arrive at Online Bible's word count of 4,164.
Our little search for "God" in the various Mac Bible Study programs turned out to be more than just a good test of search speed. It also helped to reveal differences in search methods, problems with some versions of the KJV text, and a few apparent bugs which need to be fixed.
With respect to search method, a search for "God" in Accordance finds only that exact form regardless of case. Thus, "God," "god," and "GOD" are found but the possessive "God's" and the compound word "God-ward" are excluded. A search for "God" in iBible and Bible Reader Free will also return the possessive form "God's," but not the compound word "God-ward." MacSword always does a double-wildcard search, so that a search for "God" finds any word that contains "god," regardless of whether those three letters appear at the beginning, the middle, or the end. Online Bible finds both "God's" and "God-ward."
With respect to textual problems, both Bible Reader Free and MacSword omit the superscriptions to the psalms in the KJV, a problem which should be corrected. MacSword's KJV also has a problem with missing the "s" after an apostrophe. Again, this is an error which needs to be fixed.
With respect to bugs, Bible Reader Free appears to be missing fourteen valid occurrences of "God," but because I can't export Bible Reader Free's search results to compare them with those of other programs, finding those fourteen occurrences is like finding a needle in a haystack. Though not necessarily a bug, Online Bible is confusingly inconsistent in the way it handles the searching of footnotes. Online Bible also has a problem with finding words in all-caps, but since few words actually appear in all-caps, this latter problem is likely to affect only a handful of searches. One other minor problem which I failed to mention above is that MacSword only highlights the first occurrence of a word in each verse. For example, the word "God" appears twice in Genesis 1:10, but only the first occurrence is highlighted. I have little doubt that these minor issues will be addressed in future updates.
All in all, this has been an interesting exercise, even if it has required me to do more math than I'm typically wont to do! When it comes to searching for God in Mac Bible Software, may every developer help people find Him as quickly and as accurately as possible.
[Update: Charles Moore makes some clarifications and I make some corrections in this followup article: Searching for Jesus in Mac Bible Software]
Recent Bully Pulpit Features: