Opinions about member ratings
posted on
Jul 12, 2009 02:37PM
Member Inquiries, Feedback and Testimonials
I would very much appreciate hearing your opinions concerning the "member contribution to hub rating system being currently used by Aogaracom. I raise this issue with profound apologies to the fine management and staff of our Aoracom team whom I promised I would not raise this issue with again. However since they have recently appointed me as Hub Leader on the JAGH Hub I started I would like to raise this issue with my fellow Hub Leaders. While teaching both graduate and undergraduate courses in statistics and experimental design as part of my teaching load in the Department of Psychology I had an opportunity to extensively review the psychometric literature in the area of rating scales, particularly as they apply to peer to peer, student to faculty and employee to management performance assessments. The literature is replete with examples of just how problematic these ratings can be due to subjective rating bias and participation ratios. In the case of the Agoracom system the ratings often involve only a few of the board participants and therefore can be greatly influenced by very few members who may disagree with the opinions of another board member. Generally speaking it is the unhappy campers who tend to provide these ratings in greater numbers than the more satisfied ones. I initiated the JAGH board at the end of May, 2009. Since then a number of other Raging Bull members have joined us. However, few ratings of members have occurred with most members being rated by less than three of their fellow board members. Moreover, the new rating system makes these ratings a necessary but not sufficient requirement for advancement to the ranks of Vice President and President. Finally, this new system is retroactive thereby representing an imposition of an ex post facto rule to members whose original ratings were made under the old system. I personally would rather see more emphasis given to the message recommendations made by members of the actual messages being posted than to the ratings themselves. If the ratings are to be the prime criteria for advancement then I would recommend that Agoracom post messages emphasizing the importance of and necessity for members to rate fellow hub members. I wonder if either the general membership or the Hub Leaders alone were ever consulted concerning these changes? I would think that this should have been done at a minimum since the changes affect all participants on all of the hubs.
Thanks for your input.
Doc