TheProMESsenger
Professor Robert D. Pritchard
University of Central Florida

• ProMES Homepage
• Values Guiding
  the Research Program
• Major Research Findings
• Description of ProMES
• Advantages of ProMES
• ProMES Meta-Analysis
• History of ProMES
• Computer Program
  for ProMES
• Technical Questions
• Publications on ProMES
• ProMES Researchers'
  Contact Information

• Robert D Pritchard

Description of ProMES

Improving organizational productivity has received a great deal of attention because of its ties to important issues such as international competitive position, the success of industries, the survival of specific organizations, and the quality of life of individuals (Campbell & Campbell, 1988; Kendrick, 1984; Kopelman, 1986; Pritchard, 1992).

Conceptual Background
The work is based on the theory of motivation presented by Naylor, Pritchard, and Ilgen (1980). In this theory, motivation is maximized when people see clear connections between their efforts and the behavioral "products" or results of these efforts, there are clear perceived connections between a person's products and their evaluations, and there are clear connections between these evaluations and valued outcomes. When these conditions are met, motivation is high. In addition, motivation is maximized when the different evaluators and controllers of rewards in the person's environment such as the person himself/herself, peers, different supervisors, top management, and union personnel agree as much as possible on what should be done in the work and how it should be evaluated. When such agreement exists, the efforts of the person are more clearly directed and the same amount of effort results in greater productivity. In addition, stress and wasted effort are reduced.

Based on this theory, the research program has developed a formal method to measure productivity and use these measurements as feedback to people doing the work to help them increase their productivity through maximizing motivation. The idea is to maximize the variables indicated in the theory so that motivation will also be maximized. The approach is called the Productivity Measurement and Enhancement System or ProMES. It was developed by the author and his students (Pritchard, Jones, Roth, Stuebing, & Ekeberg, 1988, 1989; Pritchard, 1990). The idea is to give people the tools to do the work better while at the same time help them feel a sense of ownership in the resulting system and empowerment in determining important aspects of their work.

One of the key elements in ProMES is feedback. People doing the work get regularly occurring, high quality feedback about how the work unit is doing. The personnel in the work unit then use this feedback to develop plans for improving productivity. Feedback after this time tells them how well the plans they developed have actually improved productivity. Furthermore, since they are heavily involved in the design of the measurement system and resulting feedback system, they have more confidence in its validity and accept it more than systems imposed from above.

How ProMES is Done
ProMES is described most fully in a book on the system (Pritchard, 1990 and a book in German on ProMES: Pritchard, Kleinbeck, and Schmidt, 1993). Examples of using it are described in books of cases, Pritchard, 1995; Pritchard, Holling, Lammers, and Clark, 2002. The approach is a formal, step-by-step process that starts by identifying organizational objectives and developing a measurement system to assess how well the unit is meeting those objectives. A feedback system is then developed which gives personnel in the unit and managers information on how well the unit is performing on the measures. This feedback is designed to lead to improvements in productivity, and this leads to better meeting the organizational objectives.

In essence, ProMES is a formal, step-by-step process that identifies organizational objectives, develops a measurement system to assess how well the unit is meeting those objectives, and develops a feedback system which gives unit personnel and managers information on how well the unit is performing. The feedback system is designed to lead to improvements in productivity that should lead to improvements in meeting the organizations' objectives. The goal of ProMES is similar to other methods of performance management such as the Oregon Matrix (Felix & Riggs, 1983); MGEEM (Tuttle & Weaver, 1986); and the Balanced Scorecard (Kaplan & Norton, 1993; 1996). However, the theory behind ProMES is quite different from these approaches, as are many of the steps in its use.

Developing ProMES is done in seven specific steps that are described below. ProMES is typically done with groups that are entire work units; however, it can also be used with individuals (Miedema & Thierry, 1995; Watson, Hedley, Clark, Paquin, Gottesfeld, & Pritchard, 1995). Sometimes a combination of individual and group level measures has been used (Kleingeld & Van Tuijl, 1995). In the discussion and example to follow, we shall assume the system is being developed for a group.

Step 1. Forming the Design Team

The "design team" is the group of people who will be primarily responsible for developing the measurement and feedback system. It is composed of people actually doing the work, one or two supervisors above them, and one or two facilitators familiar with ProMES to guide the design team through the process. If the group doing the work is small (e.g., less than 7-8 people) the entire work group is used. If the work group is larger, a subset of the work group is in the design team. An important feature of ProMES is that the people actually doing the work are directly involved in its development.

Step 2. Identifying Objectives

The design team first identifies and comes to agreement on the overall Objectives of the unit. The mechanism used here, and in the entire ProMES process, is group discussion to consensus. The design team is asked to define exactly what that organizational unit is trying to accomplish. It is assumed that there will be some differences in opinion on the nature of these objectives. The role of the facilitator is to guide the process and make the discussion as constructive as possible. It is important that all aspects of the work are included in the objectives.

An example of an actual unit where the system was implemented will be used to illustrate the steps in the development of ProMES. The example is a manufacturing setting where various kinds of printed circuit boards were made. These boards were assembled and tested by a series of team-based work groups. One group could not finish more boards than the previous group had ready for them. Technicians in the unit were responsible for the fi­nal steps in the production of these circuit boards. Af­ter this unit finished the boards, the boards were in­spected. The unit was also re­sponsible for maintenance and clean-up proce­dures that were checked on a reg­ular basis. In this example, the design team came up with the following objectives.

1. Maintain High Production.
2. Make Optimal Quality Boards.
3. Maintain High Attendance.
4. Follow Clean-up and Maintenance Procedures Correctly.

Step 3. Identifying Indicators

After agreement on the objectives, the design team develops quantitative measures of how well these objectives are being met. These measures are called Indicators. One or more indicators are developed for each objective. The identification and refinement of indicators is also done through group discussion and consensus. An important criterion of indicators is that they be largely under the control of the people being measured. ProMES is designed to produce a measurement and feedback system that will help to improve productivity. Measuring people and expecting improvements on measures they cannot control produces a system that is not accepted, is seen as invalid, and tends to reduce motivation (Algera, 1990; Frese & Zapf, 1994; Muckler, 1982; Pritchard, 1992; Tuttle, 1981; Wall, Corbett, Martin, Clegg, & Jackson, 1990).

The indicators for the unit’s objectives might look like the following. (In the case we are using for this example, the unit actually developed 5 objectives and 9 indicators, but to keep the example manageable, only a subset will be used.)

» Objective 1: Maintain High Production.
» Indicator 1: Percent of Boards Completed. Number of boards completed, divided by the established goal for number completed.
» Objective 2: Make Highest Quality Boards Possible.
»Indicator 2: Inspections Passed. Percentage of boards passing inspec­tion.
» Objective 3: Maintain High Attendance.
» Indicator 3: Percent Attendance. Total hours worked divided by the maximum hours possible.
» Objective 4: Follow Cleanup and Maintenance Proce­dures Correctly.
» Indicator 4: Audit Violations. Number of violations on a regularly occurring audit of cleanup and maintenance proce­dures.

Once the design team finalizes objectives and indicators, they are presented to management. These are reviewed for consistency with the overall objectives of the broader organization. Any disagreements are discussed and a final set is agreed upon. This step demonstrates another important feature of ProMES: it is a bottom-up strategy rather than a top-down strategy. Measurement and feedback systems imposed from above are sometimes resisted, are seen as inaccurate, and it is difficult to get such systems accepted by unit personnel.

To give the flavor for the types of objectives and indicators that are developed in settings other than manufacturing, Table 1 shows objectives and indicators for a professional group and a service unit.

Step 4. Defining Contingencies

Once management and the work group finalize objectives and indicators, the next step is to define Contingencies. A ProMES contingency is the operationalization of the product-to-evaluation contingencies in NPI. As such, they are a type of utility function that relates variation in the amount of the indicator (the product) to variation in the effectiveness of the unit (evaluation). In other words, it is a function that defines how much of an indicator is how good for the organization.

Figure 1 shows four ProMES contingencies for the indicators in our ongoing circuit board manufacturing example. The upper left quadrant of figure 1 is the contingency for the first indicator, Percent Boards Completed. Varying amounts of this indicator are shown on the horizontal axis, ranging from a low of 85% of the goal to a high of 115% of the goal. The vertical axis is the Effectiveness Score. Effectiveness is defined as the amount of contribution that is being made to the organization. It ranges from -100, through 0 to +100. The zero point is defined as the amount of the indicator that just meets expectations. It is the level of output on the indicator that would neither be praised nor criticized; it is simply meeting organizational expectations. Indicator amounts that are above this expected level get a positive effectiveness score. The higher the unit is above this expected level, the more positive the effectiveness score. Indicator amounts below the expected level receive a negative effectiveness score.

The contingency relates indicator amounts to the effectiveness scores. For example, the Percent Attendance contingency in the lower left of the figure shows that the expected level is 96% of scheduled hours. It is not expected that personnel will work 100% because of illness, family emergencies, etc. The contingency indicates that dropping to 94% attendance is very negative, but below that, further decreases do not generate as large a drop in effectiveness. This was because at 94% attendance, they had to find replacements for missing group members. It was a difficult task to get a replacement, but it was not appreciably more difficult to get more than one replacement. Above 96% is above expectation and going from 96% to 98% produces a large increase in effectiveness, but going above 98% produces a much smaller improvement in effectiveness. This is because while attendance above 98% has positive consequences, having people come in when they were sick also has negative consequences such as spreading the illness. One contingency is done for each indicator, so in our example with its four indicators, there are four contingencies.

A formal step-by-step process is done to de­velop the contingencies. This procedure is described in Pritchard (1990) and in Pritchard et al. (1993). It essentially consists of group dis­cussion to consensus where contingency development is broken down into discrete steps done by the design team. Each of the different parts of the contingencies are agreed upon, and then put together into a whole. Once the design team has come to agreement on the contingencies, they are presented to higher management for re­view and approval. This is similar to the step done for objectives and indicators. The contingencies are dis­cussed and any disagree­ments are resolved. When this ap­proval process is complete, the measurement system is considered complete.

Three things are particularly noteworthy about the contingencies. First, the contingencies essentially scale how much output is done (the indicator level) relative to how good that is (the effectiveness score). In doing this, they formally define what is expected by the unit and by management which helps personnel do the work (Mali, 1978; Muckler, 1982). In ProMES, the expected level of output on each indicator is defined by the zero effectiveness point. This scaling of how much to how good also indicates the evaluation to be expected for all levels of output. It is valuable to get descriptive feedback on what was done, such as how many clients were seen or the number of errors made. However, it is also valuable to know how each level of output is evaluated by the rest of the work unit and by management, i.e., evaluative feedback. With the contingencies agreed upon, the individuals in the unit and their management know in advance how good or bad each level of output is considered. If a unit gets an effectiveness score above zero, the unit has exceeded expectations. The higher the score, the more they have exceeded expectations. Negative effectiveness scores mean the unit is below expectations.

ProMES Contingencies

Figure 1. Examples of ProMES Contingencies

A second feature of the contingencies is that they capture differential importance. Not every indicator is equally important, and the overall slope of the contingency captures this differential importance. For example, Figure 1 shows that Percent Boards Completed is the most important indicator because it has the steepest slope. Number of Cleanup and Maintenance Violations is the least important indicator.

The third noteworthy feature of the contingencies is that they capture non-linearity. The re­lationship between how much an organizational unit does on an indicator and the amount of contribution (effectiveness) that level of the indicator makes to the overall functioning of the organization is frequently not linear (Campbell, 1977; Campbell & Campbell, 1988b; Kahn, 1977; Pritchard, et al., 1989; Pritchard, 1992; Seashore, 1972). It is common, for example, that once the unit's level of quality reaches a point that satisfies the customer, further improvements in quality are not especially valuable. That is, a point of diminishing returns is reached. The contingencies in ProMES capture this non-linearity. For example, the contin­gency in Figure 1 for Percent Passing Inspection shows that there is a point of diminish­ing returns after 99.8%, a clear non-linearity. The contingency for Percent Attendance shows a different type of non-linearity. For this contingency there is little gain in effectiveness for going from 88% attendance to 94%, but big gains between 94% and 98%. It is also important to note that these non-linearities are very common. The vast majority of the contingencies developed in ProMES have some degree of non-linearity.

Step 5. Designing the Feedback System

Once the measure­ment system is finished, the feedback system is designed and implemented. Feed­back is then given to unit personnel and managers in the form of a regular writ­ten report. To prepare this report, the data for each indicator are collected for the work period. This period has most frequently been a month, but it has ranged from a week to a year. The time interval depends on the nature of the work being done. The Effec­tiveness Score for each indi­cator value is calculated from the contingency. Each of the objectives and their indicators for the period are shown along with the indicator value and the corresponding effectiveness score. In addition, an Overall Effectiveness Score is calculated which is the sum of the effectiveness scores across all indicators. It represents the overall index of the unit’s productivity for the time period. This index represents an overall score in that relative importance and non-linearity have already been included in the effectiveness scores because of the contingencies. An example of this part of the feedback report for our abbreviated example is shown in the top section of Table 2.

Table 2. Example Feedback Report

The feedback report also contains historical data. The change in each indicator from the previous month to the current month is shown as indicated in the middle section of Table 2. The indicators are shown along with the indicator value from the last month, the value for the current month and the change in effectiveness score from last month to the current month. This allows unit personnel and management to see where things are improving and declining.

Another feature of the feedback report is the identification of priorities for improvement. It is frequently difficult for the unit to know where to focus their efforts for improvement (e.g., Earley, Connolly & Ekegren, 1989). The contingencies are an approach to doing this. They identify the effectiveness score associated with each value of an indicator. As can be seen from the contingencies in figure 1, it would be quite easy to note the effectiveness score for the current level of an indicator and then calculate the change in effectiveness that would occur if the unit improved on that indicator by any specified amount. For example, suppose the Percent Boards Completed for a given work period was 95%. If the unit increased their percentage of boards completed by one interval of the indicator, they would go from 95% to 100%. The change in effectiveness from 95% with its effectiveness score of -10, to a value of 100% with its effectiveness score of +80 means a gain in effectiveness of +90.

This possible gain in effectiveness can be calculated for an increase on each of the indicators. An example is shown in the bottom section of Table 2, which is also part of the typical ProMES feedback report. For each indicator, the value for the current period is shown in the first data column, the indicator value that represents one interval of improvement is shown in the next, and the gain in the effectiveness score is shown in the last column. These effectiveness gain scores identify the impact of making improvements on each of the indicators on the unit’s overall effectiveness. This indicates what the priorities should be for improving overall productivity. Because it is based on the contingencies, it is based on organizational policy that was agreed to by all. In this example, the table indicates that the unit should first consider ways to increase the Percent Boards Completed since that is where the most gain in overall effectiveness (+84) could be made. The next priority would be attendance with its potential effectiveness gain of +30. It is clear that very little can be gained from trying to improve Percent Boards Passing Inspection (+4) or from trying to improve Violations (+5).

Also included in the report is a plot of the overall effectiveness score over time. In addition, graphs of changes in the effectiveness score for each indicator are sometimes included or at least posted somewhere in the work area.

Step 6. Giving and Responding to Feedback.

The feedback report is prepared as soon after the reporting period as possible, ideally within a few days. A meeting is then held with all the unit personnel and the supervisor. They talk about how well they did overall and then focus on the individual indicators. For those that improved, they discuss what they did to cause the improvements and how those improvements can be sustained. For indicators that decreased, they discuss what caused this decrease and how things could be changed to improve things. The feedback report is prepared for each feedback period and a meeting is held to review it. This process goes on permanently. Thus, the feedback phase provides a mechanism for continuous improvement.

Step 7. Monitoring the Project Over Time

Once the feedback system is in place, follow-up should be done on a regular basis. After a few feedback sessions there are sometimes aspects of the measurement system that need to be adjusted. This is especially likely if some of the ProMES measures were new to the unit personnel. In addition, it is desirable from time to time for the facilitator to determine whether changes in the work or changes in policy make a review of the measurement system necessary.

References

Annett, J. (1969). Feedback and human behavior. Baltimore, MD: Penguin.

Campbell, J. P., & Campbell, R. J. (Eds.) (1988). Productivity in organizations. San Francisco: Jossey-Bass.

Dockstader, S. L., Nebeker, D. M., & Shumate, E. C. (1977). The effects of feedback and an implied standard on work performance (NPRDC TR 77-45). San Diego, CA: Navy Personnel Research and Development Center.

Feather, N. T. (1968). Change in confidence following success or failure as a predictor of subsequent performance. Journal of Personality and Social Psychology, 9, 38-46.

Guzzo, R. A., Jette, R. D., & Katzell, R. A. (1985). The effects of psychologically based intervention programs on worker productivity: A meta-analysis. Personnel Psychology, 38, 275-291.

Hammond, K. R., & Summers, D. A. (1972). Cognitive control. Psychological Review, 79, 58-67.

Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behavior in organizations. Journal of Applied Psychology, 64, 349-371.

Ilgen, D. R., & Hamstra, B. W. (1972). Performance satisfaction as a function of the difference between expected and reported performance at five levels of reported performance. Organizational Behavior and Human Performance, 7, 359-370.

Ivancevich, J. M., Donnelly, J. N., & Lyon, J. L. (1970). A study of the impact of management by objectives on perceived need satisfaction. Personnel Psychology, 23, 139-151.

Kendrick, J. W. (1984). Improving company productivity. Baltimore: Johns Hopkins University Press.

Kluger, A. N. & DeNisi, A. (1996). The effects of feedback interventions on Performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 2, 254-284.

Kopelman, R. E. (1986). Managing productivity in organizations: A practical, people-oriented perspective. New York: McGraw-Hill.

Latham, G. P., & Yukl, G. A. (1975). A review of research on the application of goal setting in organizations. Academy of Management Journal, 18, 824-845.

Locke, E. A., Shaw, K. N., Saari, L. M., & Latham, G. P. (1981). Goal setting and task performance: 1969-1980. Psychological Bulletin, 90(1), 125-152.

Mali, P. (1978). Improving total productivity. New York: Wiley and Sons.

Muckler, F. A. (1982). Evaluating productivity. In M. D. Dunnette & E. A. Fleishman (Eds.), Human performance and productivity: Vol. 1. Human capability assessment (pp. 13-47). Hillsdale, NJ: Erlbaum Associates.

Naylor, J. C., Pritchard, R. D., & Ilgen, D. R. (1980). A theory of behavior in organizations. New York: Academic Press.

Pritchard, R. D. (1990). Measuring and improving organizational productivity: A practical guide. New York: Praeger, pp. 248.

Pritchard, R. D. (1992). Organizational productivity. Handbook of Industrial/ Organizational Psychology (2nd Edition). Volume 3. M. D. Dunnette and L. M. Hough, Editors. Palo Alto, CA: Consulting Psychologists Press, pp. 443-471.

Pritchard, R. D., Editor (1995). Productivity measurement and improvement: Organizational case studies. New York: Praeger, pp. 380.

Pritchard, R. D., Jones, S. D., Roth, P. L., Stuebing, K. K., & Ekeberg, S. E. (1988). The effects of feedback, goal setting, and incentives on organizational productivity. Journal of Applied Psychology Monograph Series, 73(2), 337-358.

Pritchard, R. D., Holling, H., Lammers, F., & Clark, B. D., Eds. (2002). Improving organizational performance with the Productivity Measurement and Enhancement System: An international collaboration. Huntington, New York, Nova Science, pp. 297.

Pritchard, R. D., Jones, S. D., Roth, P. L., Stuebing, K. K., & Ekeberg, S. E. (1989). The evaluation of an integrated approach to measuring organizational productivity. Personnel Psychology, 42(1), 69-115.

Pritchard, R. D., Kleinbeck, U.E., and Schmidt, K. H. (1993). Das Management-system PPM: Durch Mitarbeiterbeteiligung zu höherer Produktivität. (The PPM Management System: Employee participation for improved productivity.) Munich, Germany: Verlag C. H. Beck, pp. 255.

Pritchard, R. D., Montagno, R. V. & Moore, J. R. (1978). Enhancing productivity through feedback and job design. Air Force Human Resources Laboratory Technical Report, AFHRL-TR-78-44.

Pritchard, R. D., Paquin, A. R., DeCuir, A. D., McCormick, M. J., & Bly, P. R. (1997). Meas-uring and improving productivity: 15 years of research on the Productivity Measure-ment and Enhancement System (ProMES). Unpublished manuscript.

Pritchard, R.D., Watson, M. D., Kelley, K., & Paquin, A. Helping teachers teach well: A new system for measuring and improving teaching effectiveness in higher education. San Francisco; New Lexington Press, 1997.

Rizzo, J. R., House, R. J., & Lirtzman, S. I. (1970). Role conflict and ambiguity in complex organizations. Administrative Science Quarterly, 15, 150-163.

Tubbs, M. E. (1986). Goal setting: A meta-analytic examination of the empirical evidence. Journal of Applied Psychology, 71, 474-483.

Tuttle, T. C. (1981). Productivity measurement methods: Classification, critique, and implications for the Air Force (AFHRL-TR-81-9). Brooks AFB, TX: Manpower and Personnel Division, Air Force Human Resources Laboratory.