Category Archives: accountability

Teaching What to Learn and Learning How to Teach

thumbby Hap Aziz

In his article “The Top 5 Faculty Morale Killers” published in The Chronicle of Higher Education online (April 25th, 2016), Rob Jenkins discusses several of the ways in which middle managers at academic institutions might influence faculty members’ experiences, for good or bad. Considering full-time faculty (rather than adjuncts), he discusses topics of micromanagement, trust issues, hogging the spotlight, the blame game, and blatant careerism; and for the most part, I find myself in agreement with his management observations and commentary. However, there is one area on which Jenkins touches that is problematic and often a subject of (sometimes heated) discussion at many of the institutions I’ve encountered over the past couple of decades. Under the heading of “micromanagement,” Jenkins writes,

“If, as an academic middle manager, you wish to destroy morale in your department, you can start by dictating to your faculty members exactly what to teach, how to teach it, which materials to use, and how to evaluate students.”

In this sentence, Jenkins links four related yet separate points, which he earlier categorized as being issues of academic freedom. I don’t believe the blanket application of the concept of academic freedom applies equally to all of these points, specifically as a protection against the potential administrative requirement to meet a certain standard of professional competency regarding learning outcomes. This discussion has only broadened as faculty and students both have become more involved with online and technology-mediated learning models, and some of those online learning concerns and considerations may be instructive in this context. Let’s examine Jenkins’ statement point by point.

  • what to teach

When it comes to making decisions regarding the subject matter being taught, there has been little disagreement with the idea that the full-time faculty member is the ultimate decision-making authority; that is, within generally accepted content parameters established largely through professional consensus, and as agreed upon by academic departments as to what content should be covered within courses. There are some dissenting viewpoints, often related to more politicized or controversial content as highlighted in this Huffington Post article. However, there is not enough cause to argue this point with Jenkins, and I see little downside in letting the subject matter expert (especially in contrast with the opposite approach) determine the subject matter being taught.

  • which materials to use

As with the point of what to teach, the selection of materials may largely be left to the faculty member. Certain decisions regarding text-book adoption, inclusion of supplementary materials, etc. may be subject to moderation by the appropriate academic department, but even so, the departments themselves include the teaching faculty. The remaining two points are where the conversation may be considered contentious.

  • how to teach it

When online courses and programs began to gain traction and popularity as an option for students in the late 1990s and early 2000s,  student outcomes lagged comparatively for the online alternatives. Eventually, it became obvious to institutions that basic faculty teaching and technology skills were not enough to replicate the on-ground classroom experience. In the 2004 study, “Online, On-Ground: What’s the Difference,” Ury and Ury found that “the online  student mean grade (80%) what significantly lower than the mean grade of the students enrolled in traditional sections of the same course (85%).” Drop-out rates continue to be problematic for online programs due to a number of variables, many of which are differentiators between online and on-ground instruction, as observed by Keith Tyler-Smith in his 2006 Journal of Online learning and Teaching article, “Early Attrition among First Time eLearners: A Review of Factors that Contribute to Drop-out, Withdrawal and Non-completion Rates of Adult Learners undertaking eLearning Programmes.”

The preponderance of research has demonstrated that building a successful online course is not simply a matter of selecting the appropriate content (or translating and transferring content from an on-ground format to an online format–whatever that might be). As the pressure for accountability grew (for a number of reasons), the notion also grew that faculty, by virtue of their subject matter expertise were not also necessarily well-qualified to develop effective online courses. Interestingly, this was by no means a new assessment or understanding. The instructional design community has understood this for quite some time, but without the mechanism for providing a comparative illustration–which online courses provided–faculty design of courses and how to teach them–was standard practice.

It does not necessarily follow that having subject matter expertise means that faculty also have teaching methods expertise. This is true for online courses, certainly, but it is also true for on-ground courses. Institutions serious about service to their learning populations must decide how they will equip their faculty for success, whether that is through ongoing professional development, the provision of support resources such as instructional design staff, or any combination of methods. But that will mean some form of “micromanagement” as institutions get a handle on assessing the performance of their academic programs and measuring the success of their students.

I remember reading an interview with Isaac Asimov in which he talked about his writing. In his life, he authored over 500 books along with countless essays, short stories, and articles. He was asked how he did what he did, and what advice he might give to aspiring authors. With perhaps uncharacteristic humility, Asimov admitted that as much as he wrote, he really had no idea how to explain how to do it. Writing was something he did prolifically, yet that did not qualify him to teach writing to others. Not coincidentally, he also expressed that he would make a poor editor, which brings me to the final point.

  • how to evaluate students

In the past decade, institutions have become quite serious about measuring student success, expending significant resources to determine what is affecting student engagement, retention, and persistence. The Spellings report (2006) emphasized accountability as one of the four key areas requiring attention in U. S. higher education. There are now, at many institutions, a variety of data-mining tools that allow academic leadership as well as faculty to assess student performance across a wide range of metrics. While a faculty member may be the best person to determine the quality of a student essay based on an articulated mastery of the content area, there are a host of other reporting metrics that address student performance issues and success that are not directly related to content mastery. Today’s reality is that student evaluation is most effective as a collaborative activity in which faculty play a key but partial role along with others in the institution.

So, yes, Rob Jenkins has identified several potential morale killers that institutional management might inflict upon teaching faculty. But to no small degree, some of what Jenkins identifies as morale killers is what I’d identify as entrenched attitudes that will lead to pain if they are not willingly let go. Of course I’m not saying that all faculty are in this situation, and I’m not even saying that there are no faculty at all that are able to teach well or effectively evaluate student performance. However, these two points are tied to an older way of thinking of the teaching and learning enterprise, in which the faculty member is the sole connection point to the student learning experience. With all the tools and resources available to faculty members in the technology-mediated classroom environment, it’s that older way of thinking that’s the true morale killer.


Filed under accountability, education, education technology, face-to-face instruction, faculty, higher education, instructional design, online education, teaching

Society or Student: What Should Education Serve?

Hap Azizby Hap Aziz

As educators, politicians, employers, technology futurists, and others debate the challenges facing education in the United States, the very basic question of what an education should provide is not often a key component of that debate. When the discussion turns to “common core” or “competency-based learning,” the terminology exposes the bias that there are subject areas or skill sets that are important for our students to master… and that, of course, implies that there are facets of human endeavor that are less important, at least from a public policy and funding standpoint.

At the Learning Impact 2013 conference in San Diego, this was one of the themes woven throughout Dr. Yong Zhao’s keynote address. His comments were provocative but very compelling along this line of reasoning: The greater specificity in education content (exercised through design control from some central, external entity accountable to societal demands), the less likely that students will be able to navigate a creative, entrepreneurial path in life. It is this premise that Dr. Zhao used to buttress his premise that the United States, despite having students that often score near the bottom in world-wide academic performance, produces inventors and innovators and entrepreneurs in much greater proportion than do countries with top test-performing students such as China, for example. While many people in the U.S. have high regard for the Chinese education system, it is instructive to know how non-Americans assess China:

“China must have entrepreneurs like Steve Jobs.”
– Wen Jiabao, Former State Premier


“The next Apple or Google will appear, but not in China unless it abolishes its education.”
– Kai-fu Lee, Founding President of Google China

Part of this is a cultural mindset, and in October of 2010, a Gallup poll found the entrepreneurial mindset to be much more prevalent in the U.S. than in China (or even the European Union).

The question on entrepreneurship and culture, Zhao argues, is very much related to the success-or failure-of an education system to squash creativity and independent thought. The reason our workforce is more entrepreneurial is due, at least in part, to the fact that the American education system does such a poor job of educating students in those categories that our society most values.

This is what Steve Wozniak comments about the top-ranked Singapore education system:

“Apple couldn’t emerge in societies like Singapore where ‘bad behavior is not tolerated’ and people are not taught to think for themselves.”

Author and CNN Travel contributor Alexis Ong remarks:

“Wozniak’s comments are really a scathing indictment of the Singapore education system, its strictly regimented curriculum and by-rote study techniques that sustain the city’s “formal culture.”

Consider that Steve Jobs, Steve Wozniak, Bill Gates, Mark Zuckerberg, Michael Dell, and Larry Ellison all dropped out of college. If we accept the metric that college completion equals education success, then these tech giants are failures by the established education standard. Certainly I’m not arguing that students cast off the repressive chains of education to have a successful and fulfilling life. However, it is extremely important that we as a society understand what we want our education system to accomplish, and if we consider the system to be broken that we understand the actual problem in order to fix the system rather than further remove the ability of creative thought from our students.

I’m not confident that we are paying adequate attention to actual challenge in our seemingly singular pursuit to improve learning outcomes at all levels of the education process. In an article titled “Laptop U” published in The New Yorker, Nathan Heller writes extensively on the topic of MOOCs (massive open online courses), and how many educators as well as legislators see MOOCs as a solution to several types of education challenges. While he acknowledges there is controversy surrounding the use of MOOCs, Heller provides the reasoning of supporters that MOOCs “are designed to insure that students are keeping up, by peppering them with comprehension and discussion tasks,” and they will have high production values (apparently to better engage students).

Yet there is discouraging data. A study cited by Inside Higher Ed concludes that the “average completion rate for massive open online courses is less than seven percent” (strongly suggesting that students are not, in fact keeping up). Early data from Coursera indicates an overall completion rate of seven to nine percent (although Coursera co-founder Daphne Koller argues that this is misleading, as most students enrolled in MOOCs have no intent to complete). Regardless of statistics, it appears that the MOOC strategy is to funnel more students through massively standardized model (whether through implementing common core curriculum or creating large-scale technology-mediated courses). Voices for customizing the education experience to fit individual students and cultivate unique talents and characteristics is a very faint part of the discussion.

The current “crisis” in American education shouldn’t come as a surprise, as Zhao points out that students in the U.S. have scored below the students of other countries over decades. This is not a new phenomenon. However, government spending on education has increased dramatically year over year since the 1960s (some data charts here), and people are demanding accountability for these expenditures. He who pays the piper calls the tune, after all. It is likely that as long as funding dollars continue to be poured into education with little evident or immediate improvement, those in charge of administering the funds will determine what the funds will buy in terms of technology, policy, and curriculum design.

Leave a comment

Filed under accountability, common core, cost of education, education, education course content, education technology, government funding, Hap Aziz, MOOCs, online education, standards

Michelle Rhee, and Superman’s Long Fall from the Clouds

by Hap Aziz

Given her role as he chancellor of schools in Washington, DC, from 2007 to 2010, I am somewhat disappointed in this article on Michelle Rhee in the New York Times. This, especially after having seen and appreciated the documentary Waiting for Superman. It is important to know that the Inspector General in the Department of Education under Arne Duncan’s has been investigating whether Washington school officials cheated to raise test scores during Ms. Rhee’s tenure, but that does not mean that A) any cheating had actually occurred, or B) that Ms. Rhee was involved in the cheating. Still, it leaves a bad taste in one’s mouth for a number of reasons.

Two things trouble me from a fundamental perspective about our education system personnel infrastructure. The first is the whole concept of integrity (or lack thereof) when it comes to reporting actual performance measures for student outcomes. Is the thinking among the cheating educators so skewed that they don’t see that inflating test scores hurts the students moving forward? Surely, that must be clear. The other troublesome thought is related to compensation: are our educators concerned about pay increases so much that they are willing to commit wholesale fraud for it? Even being charitable and admitting to the possibility that cheating is done primarily to preserve their own jobs in a more “competitive” environment, that simply leads me to question the educators’ faith in their own ability to do good work.

One of the underlying themes to the whole issue is the idea that teacher evaluation either should or should not be in some way tied to student performance. Is it possible to evaluate teachers in some objectively fair manner, or should seniority be the sole (or primary) driver for security of employment? That question is certainly worth a deeper discussion, and perhaps we’ll approach the topic here at a later date. Let me know what you think!


Filed under accountability, education, Hap Aziz, high school students, learning outcomes, standards

Teaching With or Without Technology?

by Hap Aziz

Recently The Chronicle of Higher Education online ran an article titled, “A Tech-Happy Professor Reboots After Hearing His Teaching Advice Isn’t Working,” a tale of two professors with two seemingly widely divergent instructional methods for connecting with their students. Michael Wesch, associate professor of cultural anthropology at Kansas State University, is the “tech-happy” professor, utilizing all manner of technology tools from Twitter to YouTube videos to collaborative Google Docs in the active process of engaging his students. The article begins by describing Mr. Wesch’s teaching-with-technology evangelism, and how some encounters with other instructors that have tried his methods unsuccessfully set him on a path of rethinking those methods.

Enter Christopher Sorensen, who also teaches at Kansas State University as a professor of physics. Mr. Sorensen applies a decidedly low-tech approach in his classroom interaction, avoiding tools such as clickers and even PowerPoint–which he feels would get in the way of his teaching. From the article:

“Exactly how he connects with a roomful of students is unclear to him, but he senses that it happens. ‘I walk into the classroom, and I get into a fifth gear, you might say. My voice goes up and down. It’s almost like being an actor. But don’t get me wrong, I’ve never been an actor or anything.'”

Elsewhere in the article it is mentioned that Mr. Sorensen has seen research that indicates students retain perhaps 20 percent of the material they are exposed to through the lecture format, and that he is still a strong proponent for lecturing as a method of classroom engagement. Of course, I’m curious as to what his thoughts are on that research, but there was nothing in the article to give an indication. This point raises another question that was not answered (or asked) regarding both professors: what are their students’ outcomes? It’s difficult to assess the effectiveness of either approach without some data (and while Mr. Sorensen was shown research regarding his method of engagement, there was no information regarding his particular case).

The question of presentation style in the classroom does not have a one-size-fits-all answer, and much depends on the level of comfort an instructor has with the particular methodology he or she utilizes. Mr. Wesch encountered other instructors that tried incorporating some of his techniques only to find that the results were not as expected (or desired). That isn’t necessarily surprising, given that the other instructors may have been unfamiliar or uncomfortable with making the approach actually meaningful for their students. On the flip side, were there any instructors that used Mr. Wesch’s techniques to great success? The article does not state so (although it does point out that Mr. Wesch has rethought at least a portion of his message).

This article reminds me of an anecdote I like to share when I make presentations regarding the role of technology in offering solutions to new challenges: the story of NASA and the Space Pen. In the 1960s, when NASA sent our astronauts into space with the intent of conducting experiments, there were no devices like tablets or laptops, so the way they recorded the experimental results was through pen and paper. However, pens did not function well in the low-pressure, micro-gravity environment within the space capsules. So NASA spent several years and millions of dollars developing the Space Pen; a gas-pressurized writing instrument that can write in zero gravity, upside down, or even under water.

The Soviets, on the other hand, sent their cosmonauts up with pencils.

The point being that the proper technology is the one that works, and often there are “low-tech” solutions that will fit the bill just fine, while certainly in other cases, more technologically complex solutions might be required. What happens in the teaching and learning environment is dependent upon many factors, including students and their learning styles, instructors and their level of comfort with different tools, and the resources and support available to facilitate learner success. And if we’re going to discuss the use (or non-use) of technology in the classroom, we really need to include student outcomes as an essential part of the conversation. If the outcomes aren’t satisfactory by reasonable criteria, then whatever we are doing needs to be carefully reexamined.


Filed under accountability, colleges and universities, education, education technology, effective practices, face-to-face instruction, Hap Aziz, higher education, learning outcomes, technology

Considering the Long-term Future When Selecting a Program of Study

by Hap Aziz

A couple of days ago, I wrote on the topic of the “Education Bubble,” and how student choices in their chosen programs of study can effect their financial future as well as the overall debt load on society. There’s a legitimate connection to be made between public financing of education and the latitude within which students are able (or should be allowed) to select their pathways to potential careers. I would argue that if a student decides to pursue a program of study for personal enrichment, entertainment value, or any other non-career-related focus, the student should have that freedom, provided that he or she is able to bear the burden of the cost and bear the responsibility of the long term consequences.

Over at The American Interest site, Walter Russell Mead writes in his Via Meadia blog much the same thing, but from a slightly different perspective–and his assessment of these “offbeat” programs is more critical (and more humorous). His piece is worth a read as a reality check on value of educational content being offered to students in the marketplace today.


Filed under accountability, colleges and universities, education, employment

Higher Education: All Bubbles Are Not Created Equal

by Hap Aziz

In his article “The Dwindling Power of a College Education” (NY Times Magazine, November 23, 2011), Adam Davidson posits, “One of the greatest changes is that a college degree is no longer the guarantor of a middle-class existence.”He supports his statement with the data points that up to the early 1970s, most college graduates–11 percent of the adult population–could find relatively decent jobs, while now nearly a third of the adult population possess college degrees, and the employment picture is much less positive. Davidson goes on to reason that in addition to the degree, graduates need to have some sort of “special skill” that employers value. There may be some hair-splitting required to support that line of thought, however. Employers have always desired particular skill sets depending on the type of job, and certainly many of us remember the age-old chicken-and-hen conundrum for newly-minted college graduates: “I have a degree, but I can’t get a job without any real work experience. So how can I get work experience if no one will hire me for a job?”

The point not to be lost in this view is that there is a non-insignificant cost to education, and this cost is financed in large part by borrowing. According to the U.S. Department of Education, National Center for Education Statistics (2009), roughly two-thirds of all undergraduate students needed some sort of financial aid for the 2007-2008 school year. Factor in rising levels of unemployment among college graduates, and we see that repayment of loans becomes a hardship for steadily increasing numbers of students. Market forces converge, resulting in the decline of undergraduate degree value, and in effect the degree drops underwater; the perceived value of the degree falling below the actual debt burden.

Comparisons to the housing market abound. Peter Thiel speaks extensively on the Education Bubble in a National Review Online interview back in January of this year. Thiel’s commentary is sobering:

“Probably the only candidate left for a bubble — at least in the developed world (maybe emerging markets are a bubble) — is education. It’s basically extremely overpriced. People are not getting their money’s worth, objectively, when you do the math. And at the same time it is something that is incredibly intensively believed; there’s this sort of psycho-social component to people taking on these enormous debts when they go to college simply because that’s what everybody’s doing.

“It is, to my mind, in some ways worse than the housing bubble. There are a few things that make it worse. One is that when people make a mistake in taking on an education loan, they’re legally much more difficult to get out of than housing loans. With housing, typically they’re non-recourse — you can just walk out of the house. With education, they’re recourse, and they typically survive bankruptcy. If you borrowed money and went to a college where the education didn’t create any value, that is potentially a really big mistake.”

The last part of this quote is worth dissecting further: “a college where the education didn’t create any value.” Thiel does point out that a college education can often be a consumption decision rather than an investment decision, and therein lies the much of the valuation measure. The following list is a snapshot of some of the more unusual offerings available to students. As you go through the list, keep a mental checklist of those you would classify as consumption versus investment offerings.

  • Bowling Industry Management at Vincennes University
  • EcoGastronomy at the University of New Hampshire
  • Floral Management at Mississippi State University
  • Puppetry at the University of Connecticut
  • Adventure Recreation at Green Mountain College
  • Family Resource Management Studies at Ohio State University

While it can be argued that these degrees have at least some value in a specialized marketplace, it becomes a greater stretch to justify the cost of the degree given the salary earned through work in the field. I’m not going to consider attending Mississippi State University to pay out-of-state tuition for the Floral Management program if I’m looking at the average starting pay at the florist in my home town. Indeed, a wiser course might be for me to take part-time employment at a florist when I’m in high school and learn the trade as an apprentice instead.

The Governor of Florida, Rick Scott, got himself into a bit of hot water when he made the following statement:

“We don’t need a lot more anthropologists in the state. It’s a great degree if people want to get it, but we don’t need them here. I want to spend our dollars giving people science, technology, engineering, and math degrees. That’s what our kids need to focus all their time and attention on, those types of degrees, so when they get out of school, they can get a job” (October 10, 2011).

While Governor Scott’s phrasing was questionable and certainly elicited quite a bit of negative reaction from anthropologists as well as educators in general, it’s worth examining the statement to see if there is any validity. The U.S. Bureau of Labor Statistics is anticipating a 28 percent growth in Anthropology jobs from 2008 to 2018, and the BLS calls this “much faster than the average” for all occupations. But examination of the actual numbers show national growth of 1600 jobs in the field over the 10 year period, and that is across the entire country (with the main employer being the Federal Government). The calculus regarding the investment value of this degree is unclear on quick inspection. If you were making this decision for yourself or advising your son or daughter, what would your reasoning be?

Still, the issue goes deeper than whether or not the purpose of college degrees is purely or at least primarily gainful employment. When we consider the question regarding the purpose of education, it is important to recall the core ideas underlying the concept that liberal arts education has value beyond the proposition of facilitating employability. In discussing the societal value for liberal arts, Gregory Dunn looks to C. S. Lewis for clarity on the topic:

“Lewis contrasts liberal arts education with what he calls ‘vocational training,’ the sort that prepares one for employment. Such training, he writes, ‘aims at making not a good man but a good banker, a good electrician, . . . or a good surgeon.’ Lewis does admit the importance of such training–for we cannot do without bankers and electricians and surgeons–but the danger, as he sees it, is the pursuit of training at the expense of education. ‘If education is beaten by training, civilization dies,’ he writes, for ‘the lesson of history‘ is that ‘civilization is a rarity, attained with difficulty and easily lost.’ It is the liberal arts, not vocational training, that preserves civilization by producing reasonable men and responsible citizens.”

What we see happening now in terms of an “education bubble” is the clash between the ideal of liberal arts and the reality of cost-benefit ratios that point to unsustainable debt on both an individual as well as societal level. This, by the way, is one of the dangers of the State taking on the role of “provider” in society, whether it involves healthcare, education, or any other area of commerce. He who pays the Piper calls the tune, after all, so along with any funding a student receives from public sources will undoubtedly come greater and greater restrictions and requirements on how that funding may be allocated. The bottom line is that if I want to major in Puppetry, and I can absorb the consequences of my own decision, then I have every right to do so… and hopefully I’ll become a much more responsible citizen in the process. However, if my degree decision results in the accumulation of a tremendous amount of debt along with a lack of job prospects, I’ve merely set myself up for some difficult and frustrating times in life. At what point will the government step in and simply dictate what types of degrees are eligible for financing through student loans?

1 Comment

Filed under accountability, colleges and universities, education

Direct Evidence of Learning

by Brooks Doherty

Just a few days after my last post, which advocates for greater attention to instructional quality in higher education along with greater learning outcomes transparency, The National Institute for Learning Outcomes Assessment released a report detailing the state of such transparency at our colleges and universities. While the overall findings are rather promising, I want to focus on one tiny but critical moment tucked quietly into the middle of the report.

In 2010, the NILOA spent many hours combing through the websites of hundreds of colleges and universities, analyzing the level and nature of learning outcomes transparency. Which colleges publish what their students were learning? Were these colleges providing data that could be directly linked to teaching and learning, or simply indirectly? In short, are colleges providing prospective students, families, elected officials, and the public-at-large information that helps us understand what a degree from your institution means?

The nugget on which I would like to focus: This 2010 study showed the 200 institutions assessed provided “more information with indirect evidence of student learning than was found in 2009 but less evidence of capstones and portfolios.” In other words, colleges and universities were more likely to provide (indirect) evidence surrounding graduation and persistence rates, or student and alumni surveys versus (direct) evidence of student learning such as results from internships, capstone projects, portfolios, state or local standardized tests.

I am encouraged by the movement toward transparency since the Spellings Commission pointed out its scarcity six years ago. Building upon the publicizing of these data loosely connected to the classroom, it’s imperative now that colleges and universities begin to better understand how to measure and show the public what learning with them looks like: tell them what your degree means. This need not necessitate widespread standardized testing at the post-secondary level, but will need to focus equally as closely on the learning process as it does currently on the learning product.

Leave a comment

Filed under accountability, education, learning outcomes