The mini project task instructions: read about validity and

The Mini Project Task

Instructions: Read about validity and reliability starting on page 324 of the textbook.

Your assignment is to create a 5-page paper addressing the following questions:

a. What is the difference between reliability and validity? Which is more important? Why?

b. What are the different ways of assessing reliability?

c. What are the different ways of assessing validity?

d. What are the different ways of obtaining validity evidence?

 

The analysis requires the additional components:

 APA formatted paper including:

o Font: Times New Roman, 12 point, and double spaced.

o Margins: One inch margins, all around.

o Indents: One-half inch indent as to begin a paragraph.

o Proper APA citations and references.

o Proper use of Level 1 headings as to label the introduction, main body, and conclusions segments.

o Proper use of Level 2 headings as to label the sections within the main body and conclusions.

o A proper title page.

o A reference page utilizing hanging indents and alphabetized by the last name of the first author.

 Free of spelling errors and minimal use of passive voice.

 

Page 324

In general, reliabilities less than 0.60 are considered to be poor, those in the 0.70 range, acceptable, and those over 0.80 good. Thus, the internal consistency reliability of the measures used in this study can be considered to be acceptable for the job enrichment measure and good for the other measures.

It is important to note that all the negatively worded items in the questionnaire should first be reversed before the items are submitted for reliability tests. Unless all the items measuring a variable are in the same direction, the reliabilities obtained will be incorrect.

A sample of the result obtained for the Cronbach’s alpha test for job enrichment, together with instructions on how it is obtained, is shown in Output 11.3.

The reliability of the job enrichment measure is presented in the first table in Output 11.3. The second table provides an overview of the alphas if we take one of the items out of the measure. For instance, it is shown that if the first item (Jobchar1) is taken out, Cronbach’s alpha of the new three-item measure will be 0.577. This means that the alpha will go down if we take item 1 out of our measure. On the other hand, if we take out item 3, our alpha will go up and become 0.851. Note that, in this case, we would not take out item 3 for two reasons. First, our alpha is above 0.7 so we do not have to take any remedial actions. Second, if we took item 3 out, the validity of our measure would probably decrease. We did not include item 3 for nothing in the original measure!

If, however, our Cronbach’s alpha was too low (under 0.60) then we could use this table to find out which of the items would have to be removed from our measure to increase the interitem consistency. Note that, usually, taking out an item, although improving the reliability of our measure, affects the validity of our measure in a negative way.

Variable	Number of items	Cronbach’s alpha
Distributive justice	5	0.862
Job enrichment	4	0.715
Burnout	10	0.806
Intention to leave	2	0.866

Now that we have established that the interitem consistency is satisfactory for perceived equity, job enrichment, burnout, and intention to leave, the scores on the original questions can be combined into a single score. For instance, a new “perceived equity” score can be calculated from the scores on the five individual “perceived equity” items (but only after items 1, 2, and 4 have been reverse coded). Likewise, a new “job enrichment” score can be calculated from the scores on the four individual “job enrichment” items, and so on. We have already explained that this involves calculating the summed score (per case/participant) and then dividing it by the number of items.

325326

Output 11.3 Reliability analysis

From the menus, choose:

·         Analyze

o    Scale

§  Reliability Analysis …

§  Select the variables constituting the scale.

§  Choose Model Alpha (this is the default option).

§  Click on Statistics.

§  Select Scale if item deleted under Descriptives

·        

·         326327

Item-total statistics
	Scale mean if item deleted	Scale variance if item deleted	Corrected item-total variation	Cronbach’ls alpha if item deleted
Jobchar1	10.0706	6.480	0.624	0.577
Jobchar2	10.1471	6.552	0.646	0.568
Jobchar3	10.4176	8.481	0.171	0.851
Jobchar4	10.1118	6.325	0.664	0.552

 

Factorial validity can be established by submitting the data for factor analysis. The results of factor analysis (a multivariate technique) will confirm whether or not the theorized dimensions emerge. Recall from Chapter 6 that measures are developed by first delineating the dimensions so as to operationalize the concept. Factor analysis reveals whether the dimensions are indeed tapped by the items in the measure, as theorized. Criterion-related validity can be established by testing for the power of the measure to differentiate individuals who are known to be different (refer to discussions regarding concurrent and predictive validity in Chapter 7). Convergent validity can be established when there is a high degree of correlation between two different sources responding to the same measure (e.g., both supervisors and subordinates respond similarly to a perceived reward system measure administered to them). Discriminant validity can be established when two distinctly different concepts are not correlated to each other (for example, courage and honesty; leadership and motivation; attitudes and behavior). Convergent and discriminant validity can be established through the multitrait multimethod matrix, a full discussion of which is beyond the scope of this book. The student interested in knowing more about factor analysis and the multitrait multimethod matrix can refer to books on those subjects. When well-validated measures are used, there is no need, of course, to establish their validity again for each study. The reliability of the items can, however, be tested.

Once the new scores for perceived equity, job enrichment, burnout, and intention to leave have been calculated, we are ready to further analyze the data. Descriptive statistics such as 327328maximum, minimum, means, standard deviations, and variance can now be obtained for the multi-item, interval-scaled independent and dependent variables. What’s more, a correlation matrix can also be obtained to examine how the variables in our model are related to each other.

This will help us to answer questions like:

·         • What are the employees’ perceptions on job enrichment?

·         • How many employees have which degrees of burnout?

·         • Are the employees satisfied with their jobs?

·         • Is there much variance in the extent to which employees perceive the relationship with the company as equitable?

·         • What percentage of employees is inclined to leave the organization?

·         • What are the relationships between perceived equity, burnout, job enrichment, job satisfaction, and intention to leave?

 

Descriptive statistics such as maximum, minimum, means, standard deviations, and variance were obtained for the interval-scaled independent and dependent variables in the Excelsior Enterprises study. The results are shown in Table 11.4. It may be mentioned that all variables except ITL were tapped on a five-point scale. ITL was measured on a four-point scale.

TABLE 11.4 Descriptive statistics for independent and dependent variables.

	N	Minimum	Maximum	Mean	Std deviation	Variance
ITL	174	1.00	4.00	2.19	0.97	0.94
Job satisfaction	173	1.00	5.00	3.24	1.32	1.74
Perceived equity	171	1.00	5.00	2.32	0.97	0.94
Burnout	171	1.00	5.00	2.55	0.66	0.43
Jobchar	170	1.50	5.00	3.40	0.84	0.706

From the results, it may be seen that the mean on perceived equity is rather low (2.32 on a five-point scale), as is the mean on experienced burnout (2.55). Job satisfaction is about average (3.22 on a five-point scale), and the job is perceived as somewhat enriched (3.40). The mean of 2.21 on a four-point scale for ITL indicates that most of the respondents are neither bent on leaving nor staying. The minimum of 1 indicates that there are some who do not intend to leave at all, and the maximum of 4 indicates that some are seriously considering leaving. Table 11.5provides a more detailed account of employees’ intentions to leave. This table shows that a large group of 328329employees seriously considers leaving Excelsior Enterprises! Testing our hypotheses will improve our understanding of why employees consider leaving Excelsior Enterprises and will provide us with useful tools to reduce employees’ intentions to leave the company.

 

	Frequency	Percentage	Valid percentage	Cumulative percentage
1.00	39	22.4	22.4	22.4
1.50	23	13.2	13.2	35.6
2.00	49	28.2	28.2	63.8
2.50	7	4.0	4.0	67.8
3.00	21	12.1	12.1	79.9
3.50	22	12.6	12.6	92.5
4.00	13	7.5	7.5	100.0
Total	174	100.0	100.0

In sum, the perceived equity is rather low, not much burnout is experienced, the job is perceived to be fairly enriched, there is average job satisfaction, and there is neither a strong intention to stay with the organization nor to leave it. The variance for all the variables is rather high, indicating that participants’ answers are not always very close to the mean on all the variables.

The Pearson correlation matrix obtained for the five interval-scaled variables is shown in Table 11.6.

From the results, we see that the intention to leave is, as would be expected, significantly negatively correlated to job satisfaction, perceived equity, and job enrichment. That is, the intention to leave is low if job satisfaction and equitable treatment are experienced, and the job is enriched. However, when individuals experience burnout (physical and emotional exhaustion), their intention to leave also increases (positive correlation of 0.531). Job satisfaction is also positively correlated to perceived equity, and an enriched job. It is negatively correlated to burnout and ITL. The correlations are all in the expected direction.