The Problem of Evaluating Public School Teachers - 101

Today in Memphis the local newspaper reported that impending state law will penalize the bottom 20% of teachers each year by not offering them job tenure. Because teacher performance depends only on student performance (according to lawmakers), there will definitely be teacher lawsuits because of this ignorant – and single criterion – evaluation stance on teacher performance.

1. Let us consider a possible picture of the perfect public school teacher:

2. All his/her students perform acceptably on their performance tests,

3. Other teachers enjoy working with him/her,

4. No complaints from parents

5. The teacher takes professional education courses to maintain and improve performance for him/her – and the students

WHAT MORE COULD ANYONE EXPECT? WHAT INDEED?

First, let us look at a “test” in any subject area.

One problem I have found in college courses, for example, involves the instructor material – a teachers book, perhaps a DVD with lecture powerpoints – and a test manual with questions and answers. As a subject matter expert in test construction, I have been very disappointed in the testing material. I have worked for professors who required me to prepare, administer, and grade the tests. There were test questions for nearly every paragraph in the book. There were so many questions it was obvious the author’s graduate student(s) created the questions and the answers - without regard to item construction quality. Test items were, at best, minimally competent.

Public school teachers are not test construction SMEs, and I sometimes wonder if the end-of-the-year-test constructors are verifiable test constructors. I am certain that they are verifiable subject matter experts in their subject area.

Public school teachers are not SMEs in test construction, and if they were, they do not have the time to construct tests which can validly measure student knowledge and performance.

Further, notice that I mentioned “student knowledge and performance.” These variables overlap in the average student, but the fear of tests and the testing experience, can affect test performance. In Japan, national tests are required for students to go to college. Here better student performance means getting into a better, more prestigious school. The pressure on such students is enormous.

Teachers here in Tennessee are often accused of teaching to the test to improve scores. In Atlanta this year much turmoil occurred over reports of pervasive teacher and administrative cheating on standardized student tests.

Now, let us recall my earlier comment on eliminating the teachers who perform in the bottom 20% from acquiring tenure. If I am a teacher, do I feel comfortable being evaluated? Am I being evaluated fairly? What if I teach a difficult course – math or chemistry or physics? Am I really being evaluated in just the way history, English, or phys ed/health instructors are?

In the next post I will propose some possible solution to this problem.

Organizational Data and Management Assumptions-102

I was hired by an international manufacturer (>$15 billion in yearly sales) to examine computer data and its Oracle data warehouse. It seemed that the company and its many manufacturing plants was adopting SAP for its accounting system. During the long switchover to the new system, Oracle programmers would receive sales figures at the end of every sales month (at midnight). The data would then be massaged into the data warehouse so it would be available to manufacturing managers worldwide. Of course, the accountants would continue to keep their books. The problem was those books were not identical to the data warehouse totals. The data warehouse was always less.

I examined the data downloading procedure, the program (PL-SQL) which pulled the data from each manufacturing facility into the data warehouse. During this 3 month period, all the people who had been operating this system inaccurately were terminated – so I was the entire department. I was told not to bother the plant managers because they had too much work to do. – but I found nothing wrong.

Finally, I asked one of the manufacturing VPs who the smartest, most experienced plant manager was. I was referred to a female plant manager overseas. I called her and asked her when the end of the business month was. She responded: “at midnight on the last working day of each month.”

Since that was when the data was downloaded, I had no clue what I was doing wrong - so I asked “and when do you close the books for each month?”

She responded “on the fifth working day after the day previously mentioned.”

What a bombshell!!!

So, when the programmer asked the questions to write the Oracle program, he misunderstood what he was told.

At midnight of the last business day of the month, no more sales were made –

But bookkeeping additions continued for five more days. That is why the monthly download was always less than the accounting department totals!

I would have found all this out if I had been permitted to re-engineer the software requirements from scratch. Incorrect programmer assumptions caused much turmoil in the management ranks.

Engaging Employees 101

An important skill all managers must develop is the ability to engage employees – engage them to perform better, to be better focused, and to not impede other workers from performing better. A previous analogy I have used refers to engaging all hands to work together, with the goal of getting the organization’s ox cart out of the ditch and placing it back on the road to recovery. At the simplest level consider the goal of management: to align all decision making outcomes with the goals of the organization. With this kind of thinking, reword the statement for the management of workers.

“Engage all workers to perform in a manner which will produce outcomes that are aligned with the goals of the organization.”

When considering just one worker, you must understand what will motivate him. At a minimum, a manager can discuss this topic during the employee’s performance reviews, but this minimal effort is unlikely to be very productive. MBWA (in an earlier post) is helpful here, but what if the large number of workers prohibits this method? In an earlier post I discussed the importance of the ‘linking pin’ and management of the ‘informal’ social groups which exist in all organizations. Using MBWA (Managing By Walking Around) and/or meetings with identified linking pins is an excellent place to begin. But how will you identify enough different informal group leaders? You do this by:

Using your own trusted social contacts,

Asking your newer contacts developed during your ongoing MBWA efforts,

Talking to all available workers, supervisors, and managers to develop this information, and

Attempting to communicate effectively with the identified informal group leaders.

Unfortunately, if you – as a manager – are not an adept social animal, attempting any of these social interactions will be an iffy proposition at best. It might be useful to identify a management truism at this time: Since the organization lives through the support of informal groups, it will perform poorly unless it can align the goals of those groups with the goals of the organization. Of course, good communication between all groups is of great importance, and what is communicated is a rationale for the informal groups to support management goals.

Organization Data and Management Assumptions-101

Once upon a time I presented myself as a consultant to a large company which had nearly 300 stores spread around the SE states. I was told they needed a mathematical model which could more accurately predict sales, by store and by all stores. I asked how accurate their current model. It was ±15%, not very accurate. I asked the following questions:

How many stores, and which ones, were ‘county seat’ stores? (about 260)

2. Do these stores all carry the same inventory? (yes)

3. I noticed a flashy store of yours just down the street. How many of these stores are there?

(7) – Do they all have the same inventory – (Yes, but the store in Nashville does not)

At this point I could identify three different types of stores. (County seat, city, and stand-alone in Nashville)

The company produced the previous 4 years of sales data. I was to use the first three years to predict last year’s sales. The company wanted ±3% accuracy.

I divided all stores into the three groups. I achieved the desired accuracy on the first pass. Intrigued, I looked for the outliers (poorly predicted sales). I made a fourth group out of these. My prediction on all other stores was slightly better than ±1.75% accuracy!

I presented the results to my executive contact and he was ecstatic! I explained the outlier group by store name, and he responded to each one with a reason for its failure: “We have never understood this store,” “The manager was fired for excessive shrinkage,” and so on.

So you can see that assuming all stores perform like all others is counter productive – and not a good way to look at sales data. Even a 'simple' assumption with a rationale (like mine) is far better than not thinking about your problem.

What is diversity?

Where do I stand on the subject of Diversity?

Everyone talks about diversity, but what does the term mean? I waited outside a classroom in which diversity was being taught (my class was in the same classroom during the following period). When the class ended I entered and passed pleasantries with the African-american instructor. I smiled and indicated I was a member of a minority. She stifled what appeared to be an offensive look, but she did ask in a confronting tone when she asked “And what would that minority be?” I responded, “I am an older person.” She could not disagree, but she was not convinced.

As a business consultant most of my working life, I was constantly faced with problems of diversity: some clients were classic rednecks, some were quite religious in their life’s considerations, some were greedy and self-absorbed, and some were complete jerks.

People have always referred to the “richness” of individual differences found in the USA. I agree, but it was only recently that I understood how individual differences, diversity considerations, and purpose – business, personal, team, etc. – were interrelated. As a subject matter expert in developing and building competent and productive teams, I should have had a proper view on the subject of diversity. I believe I did, but expressing the topic of diversity is now quite clear to me. Let me share it with you.

Diversity should not be limited to groups, rather diversity, as represented by the richness of the myriad individual differences around us, should be more reflective:

“The wide diversity of our many personal differences ought to be considered a rich source for leveraging those differences into solving many or most problems - personal, organizational, team or group, business, political, and problems in general.” (The parentheses are mine).