Independent Third Party Validation Study

A series of third party studies and validations were conducted on the Print^TM Survey over the period 2001–2003 by Mark R. Testa, PH.D of San Diego State University and came up with the following conclusions:

1. Over 3,000 respondents participated in various studies conducted to investigate the validity and reliability of the Print Survey.

Conclusions:

The Print Survey is a valid and reliable measure of personality as indicated by separate tests of the following:

face validity
100% of the respondents agreed that the Print identified by the Print Survey described and fit them either Totally Accurately or Very Accurately. Given the overwhelming accuracy of the survey, we can conclude that the Print Survey maintains face validity.
content validity
The inter-rater coefficient was .98 or 98%. Given the overwhelming accuracy of the statements used to represent each Print, it can be concluded that the Print Survey maintains content validity.
construct validity (convergent validity and discriminate validity)
Since the Chi Square statistic ranged from 132.8 to 404.5 with a confidence level of 99.99% accuracy, this suggests we would see a Chi Square this high only one-tenth of one percent of the time by random chance alone. Therefore, it can be concluded that the Print Survey has construct validity.
criterion-related validity (predictive validity)
Results indicate that all but 1 of the 1006 participants (99.9%) found a match between the Print category determined by the Print Survey and described by the workshop and their own perception of self. This suggests continuity between the Print Survey and the independent criterion (i.e., the workshop) which provides evidence of validity.
reliability (test-retest reliability)
The result of a .86 reliability coefficient is considered strong evidence of Reliability. This suggests that the Print Survey is a reliable measure of Prints.

The results suggest that:

The Print Survey in fact measures what it purports to measure and is a valid tool in identifying Prints.
This research study found that the Paul Hertz Group’s Print Survey is a valid and reliable measure of personality as indicated by face, content, construct and criterion-related validity tests, as well as a measure of test-retest reliability.
The Print Survey is an accepted form of measurement to be used in various training, development and performance diagnosis initiatives.

“The purpose of this research study was to assess the validity and reliability of the Paul Hertz Print Survey. To assess validity, face, content, construct and criterion-related validity procedures were conducted. In each case, support of the validity of the instrument was demonstrated. This suggests that the Print Survey in fact measures what it purports to measure and is a valid tool in identifying Prints. In summary, the Paul Hertz Print Survey evidences both validity and reliability.”

2. Prints are gender neutral

Title VII of the Civil Rights Act (1964) was designed to ensure that protected groups were not systematically discriminated against in personnel decisions. It is important, therefore to illustrate that tests used by organizations be blind to gender. One method of assessing the impact of gender is to analyze the distribution of the results between male and female workers. If the test discriminates on gender, the results for each group should be different.
Three hundred and fifty nine employees from 6 large organizations from eighteen different departments were given the Print Survey and the results were analyzed to see if gender systematically impacted the distribution of the results
An analysis of the responses using the Chi Square statistic shows that the Print Survey does not systematically discriminate between males and females. Equivalent numbers of each Print were found in each gender group.

3. Prints are age neutral

The Age Discrimination in Employment Act of 1967 provides that age may not be used as a consideration in personnel decisions. The ADEA is specifically designed to protect workers over 40. It is important, therefore to illustrate that tests used by organizations be blind to age. One method of assessing the impact of age is to analyze the distribution of the results between younger (under 40) and older (40 and over) workers. If the test discriminates on age, the results for each group should be different.
Two hundred and sixteen employees of varying ages from a large organization were given the Print Survey and the results were analyzed to see if age systematically impacted the distribution of the results.
An analysis of the responses using the Chi Square statistic shows that the Print Survey does not systematically discriminate between younger (under 40) and older (40 and over) workers. An equivalent distribution of each Print was found in both age groups.

4. Prints are ethnicity neutral

Title VII of the Civil Rights Act (1964) provides that ethnicity may not be used as a consideration in personnel decisions. Therefore, it is important to illustrate that tests used by organizations be blind to ethnicity. One method of assessing the impact of ethnicity is to analyze the distribution of the results between workers of various ethnic groups. If the test discriminates based on ethnicity, the results for each group should be different.
Two hundred and ninety one employees of a large organization were given the Print Survey and the results were analyzed to see if ethnicity systematically impacted the distribution of the results.
An analysis of the responses using the Chi Square statistic shows that the Print Survey does not systematically discriminate between employees of various ethnicities (i.e. Asian American, Hispanic American, African American and Caucasian). An equivalent distribution of each Print was found in each of the ethnic groups.

5. A highly statistically significant relationship (99.4%) exists between Shadow behavior of supervisors and the level of their job performance

NOTE: Researchers typically consider any statistical relationship equal to or greater than 95 % to be statistically significant.

A research study of one hundred and eight supervisors of a multi-billion dollar organization found that an extremely strong significant relationship exists between Shadow Behavior as measured by scores on the Paul Hertz Group’s “Shadow Behavior Index” and supervisor job performance.
The Shadow Behavior Index is a composite score of Print-specific Shadow behaviors, which are derived from a multi-rater assessment. A 360º assessment was conducted to evaluate the frequency of 20 Print-specific Shadow behaviors as observed by the supervisor’s peers, direct reports and manager. The assessment uses a scale of 1-5 with higher scores indicating more frequent Shadow Behavior and lower scores indicating less frequent Shadow Behavior.
For each assessment, an average score of the Shadow Behaviors – the Shadow Behavior Index (SBI) – was calculated.
Independently and without knowledge of the supervisor’s Shadow Behavior Index, managers of these supervisors rated each person according to the performance categories “Star” or “Non-Star”.
The Star group had a statistically significantly lower average Shadow Behavior Index score than the Non-Star group.
The results conclude that supervisors who exhibit more frequent Shadow behaviors tend to be poorer performers on the job than those who demonstrate less frequent Shadow behaviors.

6. A highly statistically significant relationship (99.0%) exists between Best Self Behavior of supervisors and the level of their job performance