Chapter 2: Variable Coding and Data Quality (pp. 12-18), this is p. 15
(you can navigate to other pages by clicking on page numbers below)
 p. 12 p. 13 p. 14 ---- p. 16 p. 17 p. 18

operationalization of the adequacy-confidence scale. But when he felt that the credibility of the sources was such that straightforward application of the operational definition resulted in a confidence code value that did not reflect his own belief in the truth value or accuracy of the variable code, then he was free to revise his adequacy-confidence score accordingly.

The researcher was guided in interpreting the gradations in the adequacy-confidence scale by the conceptual definition of each scale category in Table 2.1 and the accompanying operationalizations of the coding categories.

The purpose of tagging each variable code with an adequacy-confidence score--henceforth referred to as the AC code--is twofold: (1) it provides a summary measure of the data quality for each variable in the study, and (2) it facilitates the selection of data for statistical analysis according to their rated quality. Thus, the summary statistics for each basic variable, reported in subsequent chapters, are accompanied by the mean of the AC values assigned by the analysts in coding the variable. A high mean AC score indicates adequate literature and confidence in coding; a low AC score suggests little information and much coding by inference. Those who might want to use only data above a certain quality level in their statistical analysis need only test for the accompanying AC code to filter out data below their selected level.

If the assignment of AC codes reflects random measurement error in the coding of variables, then the AC codes should themselves tend to be uncorrelated with their corresponding basic variable codes--henceforth referred to as BV codes. In fact, there are some pronounced cases of high-quality information being correlated with certain variable codes. High-quality electoral data, for example, tends to be associated with stability in a party's electoral fortunes. By and large, however, the correlations between AC and BV codes tend toward 0. Whenever the correlation between the AC and BV codes for a given variable is statistically significant at the .05 level, the correlation is noted. Those who might want to select data for analysis on the basis of some AC level should then be aware that they may be attenuating the variance in the BV codes because of the relationship between the values assigned to a variable and data quality.

The quality of any data set, of course, should also be assessed through formal tests of reliability. Reliability assessments of our coding in general were performed by pairs of researchers independently coding the same party variable combinations in a total of 557 instances, which involved nearly all the variables in the file. The mean product-moment correlation between these pairs of BV codes calculated separately by variable sets to control for differences in variance was a healthy .79.

After all the basic research was completed for each country and the information about its parties recorded to, the best of our analysts' abilities, we turned to outside experts to obtain critical evaluations of our product. Letters were sent to country and area specialists asking for their cooperation in reviewing the material we had generated on parties within their spheres of knowledge. About 40 specialists eventually agreed to review our material, with some gallant scholars consenting to handle two or more countries within their fields. The list of countries and cooperating reviewers is given in Table 2.2.

The outside consultants were provided with all the material pertaining to their countries (i.e., the material in Part Two of this volume) and were asked to examine each component according to a specific set of instructions, including these guidelines:

1. Essay on Party Politics in the Country during Our Time Period. Have I made any errors in spelling, dates, interpretations, facts, and so on? Insofar as possible, note corrections directly on the copy. Did I neglect to mention important developments? If so, please state what you think should be added. Of course, any additional comments are welcome.

2. Legislative Seats and Electoral Data. These are the best data that I could find for the parties in our study during our period of interest. Perhaps you disagree with our percentage entries. If so, please enter your dissent directly on the printout and, if possible, tell me your own sources of information.

3. Attraction, Concentration, and Reflection Data. For about half the countries in the study, I have been lucky enough to obtain sample survey data on which to base our percentages. For reviewers of these countries, perhaps you might check to see if we have done any violence to categorizing the "cultural differentiators." If you feel that other surveys would be better for our purposes, please identify them and tell me of their availability. Of course, if anything seems amiss on the percentages, please let me know. For reviewers of countries for which we lack sample survey data, do our impressionistic judgments seem reasonable to you? If not, please note your disagreements on printout and refer us to citations if possible. Our own data sources are described on the printout, with the three-digit codes for citations referring to identification numbers of documents in our bibliography. Of course, if you can direct us to sample survey data where we have none, that would be most welcome.

4. Variables and Comments on Individual Parties. For most of you, this section will be the most demanding for knowledge of specific parties. This section tells how we have coded a variable and why we have coded it that way. Please read through the printout for errors in our verbal statements, although I would also appreciate your criticisms and comments on our codes as well. If you have information on variables we were unable to code, your information would be greatly appreciated. If something seems strangely amiss to you, please consult the appropriate dis-