Category Archives: standards

The Quality of Learning

thumbby Hap Aziz

I find that in the never-ceasing stream-of-consciousness that represents my current and evolving thoughts on technology-enabled education, the theme of quality is a constant. In all sectors of the education enterprise, there seems to be a consensus that quality (whatever that might represent) must be an essential component of learning content and experiential process. Even before I thought to quantify the characteristics of quality in education, I had a strong sense that there were indeed characteristics to be measured. But as Hamlet might say, “aye, there’s the rub!” The challenge is in determining what those characteristics are before we can begin to consider how to measure them.

Which brings me to my second Shakespearean reference in as many paragraphs. In Act IV, Scene 1 of his play The Merchant of Venice, Shakespeare wrote the following lines:

“The quality of mercy is not strained;
It droppeth as the gentle rain from heaven
Upon the place beneath. It is twice blest;
It blesseth him that gives and him that takes”

Yes, there are two definitions to the term quality. The first, which I used in the context of learning, is the idea that quality is a measure of how good or bad something is. The second definition as used in The Merchant of Venice is that quality is an attribute of something, and in this case, Bill is describing an attribute of mercy. Reread the passage above, substituting the one instance of the word “mercy” with the word “learning.” Now consider the line “it blesseth him that gives and him that takes.” It doesn’t take a great shift in mental perspective to think of quality not as a measure of the learning experience, but rather as an intrinsic attribute that blesses both the teacher and student alike. All we need to do is optimize conditions for this attribute to be revealed.

There are several components to learning that may function as a blessing–or as a curse if poorly executed, and the following are just a few:

  • The facilitation of the relationship between teacher and student
  • The manner in which content is organized and made available
  • The kind of support provided to teacher or student when technical difficulties arise
  • The ability to leverage additional tools that may enhance the learning experience

What level of resources or commitment of effort does it take to optimize these conditions in any particular learning environment? I probably needn’t point out that there has been much relevant research performed. But it is important to remember that we can lose sight of the big picture when we dive into the weeds of data, and that it is always a good idea to revisit key principles on a regular basis. Probably the biggest of the big picture views is the concept that the entire institution must be aligned from top to bottom and side to side on the core mission of learning. In fact, the institution should commit itself to the ideal of being learning-centered. (While I won’t explore the implications of terminology here, I will point out that there is a significant different between being learning centered as opposed to being learner or student centered. See the work of Terry O’Banion with the League for Innovation in the Community College.)

Quality as an attribute provides a basis for agreement on a common philosophy regarding the learning experience; “it blesseth him that gives and him that takes.” Once this is understood and adopted as a foundation construct, then we may begin to articulate the idea of quality as a measure of the learning experience. This is where we enter the world or metrics and assessments with the intent to execute an effective feedback and improvement cycle. Fortunately there are tools that may assist us in this process:

While these tools are extremely valuable on their own, I would never recommend adoption as an excuse to breathe a sigh of relief as though the quality question has been answered. These tools may be integrated in whole or in part into the overall governance and strategic planning process that subsequently drives day-to-day decisions regarding how learning activities are conducted. Human intelligence in the learning enterprise is still the prerequisite to data-driven decision making. Or at least it should be.

One of the reasons that it’s difficult to answer the “quality question” is that quality can be categorized in multiple ways, each with multiple considerations. The following diagram depicts a possible model.

Quality of Online (or Technology-mediated) Learning

diagram Copyright (c) 2016 by Hap Aziz

The four columns represent the categories in which we might assess quality attributes.

  • Framework – Here we consider the quality of technology infrastructure and support across an institution. How well equipped, for example, is the academic technology group in order to provide exemplary levels of service to the various end users?
  • Content – The quality of course design process has a direct impact on the actual materials and media that both educators and learners will interact with during the duration of a particular course. You might think of the difference between a well-curated academic journal and a tabloid pseudo-news publication.
  • Experience – When we think of the quality of faculty and student end-user experience, we need to consider both the end-to-end experience as a service as well as a product. What will students say after they have taken the course? The answer often comes back to how they felt about what they experienced throughout.
  • Design – Program design quality includes components of the three other quality measures, but it is also an overarching theme that spans an entire program of study rather than individual courses. This means that individual course quality measures “interact” in the learner’s mind–so a single poor experience might negatively impact the whole program experience.

The horizontal themes are representative of characteristics that are common across all the quality attributes.

  • Ethics involves topics from intellectual property policies and considerations to online harassment and bullying.
  • Resources addresses the way in which institutions provision their online operations, hopefully positioning themselves for success.
  • Constituents is all about audience: who is participating, and what is important to them.
  • Measurement is the ever-present need to understand how well we are executing to our goals at every level of the institution from leadership to department to individual instructor.

It’s at the intersection of each column and row that we might explore some questions regarding quality, such as what the ethical issues around the use of particular course content might be, or how we might go about measuring the user experience. Some of the questions might point to best practices that could be applied to most institutions under most circumstances, while others might be very specific to individual institutions, programs, or courses. I’ll be facilitating this discussion, in fact, at the Online Learning Consortium Collaborate regional conference in Las Vegas on June 10th this year, and the result should be a list of questions and considerations around those points of intersection in the diagram. I’ll follow up with a subsequent blog entry, so watch this space!

Advertisements

1 Comment

Filed under effective practices, Hap Aziz, higher education, instructional design, learning, online education, Online Learning Consortium, online quality, standards, technology

Society or Student: What Should Education Serve?

Hap Azizby Hap Aziz

As educators, politicians, employers, technology futurists, and others debate the challenges facing education in the United States, the very basic question of what an education should provide is not often a key component of that debate. When the discussion turns to “common core” or “competency-based learning,” the terminology exposes the bias that there are subject areas or skill sets that are important for our students to master… and that, of course, implies that there are facets of human endeavor that are less important, at least from a public policy and funding standpoint.

At the Learning Impact 2013 conference in San Diego, this was one of the themes woven throughout Dr. Yong Zhao’s keynote address. His comments were provocative but very compelling along this line of reasoning: The greater specificity in education content (exercised through design control from some central, external entity accountable to societal demands), the less likely that students will be able to navigate a creative, entrepreneurial path in life. It is this premise that Dr. Zhao used to buttress his premise that the United States, despite having students that often score near the bottom in world-wide academic performance, produces inventors and innovators and entrepreneurs in much greater proportion than do countries with top test-performing students such as China, for example. While many people in the U.S. have high regard for the Chinese education system, it is instructive to know how non-Americans assess China:

“China must have entrepreneurs like Steve Jobs.”
– Wen Jiabao, Former State Premier

and

“The next Apple or Google will appear, but not in China unless it abolishes its education.”
– Kai-fu Lee, Founding President of Google China

Part of this is a cultural mindset, and in October of 2010, a Gallup poll found the entrepreneurial mindset to be much more prevalent in the U.S. than in China (or even the European Union).

The question on entrepreneurship and culture, Zhao argues, is very much related to the success-or failure-of an education system to squash creativity and independent thought. The reason our workforce is more entrepreneurial is due, at least in part, to the fact that the American education system does such a poor job of educating students in those categories that our society most values.

This is what Steve Wozniak comments about the top-ranked Singapore education system:

“Apple couldn’t emerge in societies like Singapore where ‘bad behavior is not tolerated’ and people are not taught to think for themselves.”

Author and CNN Travel contributor Alexis Ong remarks:

“Wozniak’s comments are really a scathing indictment of the Singapore education system, its strictly regimented curriculum and by-rote study techniques that sustain the city’s “formal culture.”

Consider that Steve Jobs, Steve Wozniak, Bill Gates, Mark Zuckerberg, Michael Dell, and Larry Ellison all dropped out of college. If we accept the metric that college completion equals education success, then these tech giants are failures by the established education standard. Certainly I’m not arguing that students cast off the repressive chains of education to have a successful and fulfilling life. However, it is extremely important that we as a society understand what we want our education system to accomplish, and if we consider the system to be broken that we understand the actual problem in order to fix the system rather than further remove the ability of creative thought from our students.

I’m not confident that we are paying adequate attention to actual challenge in our seemingly singular pursuit to improve learning outcomes at all levels of the education process. In an article titled “Laptop U” published in The New Yorker, Nathan Heller writes extensively on the topic of MOOCs (massive open online courses), and how many educators as well as legislators see MOOCs as a solution to several types of education challenges. While he acknowledges there is controversy surrounding the use of MOOCs, Heller provides the reasoning of supporters that MOOCs “are designed to insure that students are keeping up, by peppering them with comprehension and discussion tasks,” and they will have high production values (apparently to better engage students).

Yet there is discouraging data. A study cited by Inside Higher Ed concludes that the “average completion rate for massive open online courses is less than seven percent” (strongly suggesting that students are not, in fact keeping up). Early data from Coursera indicates an overall completion rate of seven to nine percent (although Coursera co-founder Daphne Koller argues that this is misleading, as most students enrolled in MOOCs have no intent to complete). Regardless of statistics, it appears that the MOOC strategy is to funnel more students through massively standardized model (whether through implementing common core curriculum or creating large-scale technology-mediated courses). Voices for customizing the education experience to fit individual students and cultivate unique talents and characteristics is a very faint part of the discussion.

The current “crisis” in American education shouldn’t come as a surprise, as Zhao points out that students in the U.S. have scored below the students of other countries over decades. This is not a new phenomenon. However, government spending on education has increased dramatically year over year since the 1960s (some data charts here), and people are demanding accountability for these expenditures. He who pays the piper calls the tune, after all. It is likely that as long as funding dollars continue to be poured into education with little evident or immediate improvement, those in charge of administering the funds will determine what the funds will buy in terms of technology, policy, and curriculum design.

Leave a comment

Filed under accountability, common core, cost of education, education, education course content, education technology, government funding, Hap Aziz, MOOCs, online education, standards

Michelle Rhee, and Superman’s Long Fall from the Clouds

by Hap Aziz

Given her role as he chancellor of schools in Washington, DC, from 2007 to 2010, I am somewhat disappointed in this article on Michelle Rhee in the New York Times. This, especially after having seen and appreciated the documentary Waiting for Superman. It is important to know that the Inspector General in the Department of Education under Arne Duncan’s has been investigating whether Washington school officials cheated to raise test scores during Ms. Rhee’s tenure, but that does not mean that A) any cheating had actually occurred, or B) that Ms. Rhee was involved in the cheating. Still, it leaves a bad taste in one’s mouth for a number of reasons.

Two things trouble me from a fundamental perspective about our education system personnel infrastructure. The first is the whole concept of integrity (or lack thereof) when it comes to reporting actual performance measures for student outcomes. Is the thinking among the cheating educators so skewed that they don’t see that inflating test scores hurts the students moving forward? Surely, that must be clear. The other troublesome thought is related to compensation: are our educators concerned about pay increases so much that they are willing to commit wholesale fraud for it? Even being charitable and admitting to the possibility that cheating is done primarily to preserve their own jobs in a more “competitive” environment, that simply leads me to question the educators’ faith in their own ability to do good work.

One of the underlying themes to the whole issue is the idea that teacher evaluation either should or should not be in some way tied to student performance. Is it possible to evaluate teachers in some objectively fair manner, or should seniority be the sole (or primary) driver for security of employment? That question is certainly worth a deeper discussion, and perhaps we’ll approach the topic here at a later date. Let me know what you think!

4 Comments

Filed under accountability, education, Hap Aziz, high school students, learning outcomes, standards

IMS Announces Educational Positioning System Pilot

by Hap Aziz

If you have been following this blog, you may be aware that I have been involved in the development of a concept known as the “Educational Positioning System,” or the EPS. You can read some of my past blog entries on the topic here, here, here, and here. The EPS has gotten quite a bit of attention as a framework that can potentially transform the the level of engagement and control that students have regarding their own education. This represents a very disruptive level of technology that could flip the entire ownership conversation of academic data. Aneesh Chopra, the current Chief Technology Officer for the United States recognized this in bringing the concept back to the Obama Administration. Further, the IMS Global Learning Consortium (an organization dedicated to the advancement of education through the implementation of standards and use of effective practices) has taken on the EPS concept. I facilitated a workshop on the EPS in November of last year at the IMS Global Quarterly Meeting, and today the Consortium has issued the following press release:

Dear Friend of IMS Global,

Today, at the EDUCAUSE Learning Initiative (ELI) meeting in Austin, TX, USA, there will be a presentation at 3 PM announcing a new IMS project. The presentation is entitled:  The Educational Positioning System: Guiding Learners Along Their Academic Path.

The EPS has emerged as a topic of interest in the U.S. in recent months, receiving some attention after it was brought up in a panel discussion as the EDUCAUSE annual conference in October:
http://chronicle.com/blogs/wiredcampus/start-up-companies-tell-white-house-tech-chief-of-struggles-with-colleges/33872

On January 19th the White House announced several initiatives that are complementary to the EPS concept:
http://www.whitehouse.gov/sites/default/files/microsites/ostp/ed_data_commitments_1-19-12.pdf

IMS applauds the effective use of data.  Our focus is the use of data and interoperability to help individual students succeed.

Today, IMS is announcing a new project to work with IMS member organizations to implement EPS pilots. See the Call for Participation here: http://www.imsglobal.org/news.html

Currently we believe that the ideal initial focus for EPS pilots are systems of institutions. We are very pleased that the Lone Star College System has stepped up to lead the first pilot. In coming months IMS will be working with our members to pull this pilot together and hopefully initiate additional pilots.

We will also be covering this topic in depth at the annual IMS Learning Impact conference, May 14-17 in Toronto. Details for the conference are here: http://www.imsglobal.org/learningimpact2012/

Tune in to IMS for future announcements,

IMS Global

Right now I’m at the EDUCAUSE Learning Initiative 2012 Annual Meeting in Austin, Texas, where I’ll be co-presenting “The Educational Positioning System: Guiding Learners Along Their Academic Path.” It will be during this presentation when we make the EPS announcement officially to the public. But if you are reading this blog entry before 3 pm Central time, remember, you heard the news here first!

2 Comments

Filed under announcement, education, education technology, Educational Positioning System, EDUCAUSE, effective practices, Hap Aziz, higher education, Open Government Platform, standards, technology, U.S. government