What lousy public policy. What we have developing now is a whole myriad of systems that may or may not communicate with one another, and that frustrate any serious attempt to understand what happens to students who may start at one college and end up at others.  What types of students might do that? Well, the other 85% for starters.  

So, who opposed having the feds develop a single, unified federal system?  Let me quote Peter Ewell from the article:  “It is clear that this agenda is moving forward, despite opposition from the private colleges. The accountability push is such that these numbers are just simply going to be produced whether anyone likes it or not.”  But, doing it state by state is certainly not efficient.

Ewell goes on to say that “private colleges are all in favor of data. They just don’t want anybody but them to know.”

I have blogged many times about Transparency by Design and its Web site, College Choices for Adults, which is intended to provide useful information to adults seeking to enroll in a college. A national database that could track students who attend multiple schools would be very useful for our work. We would be able to indicate how many adults who end up attending multiple schools actually finish their desired degree.  Without this information, we simply don’t know. This is but one kind of information that could be gathered from such a database. And, those institutions in Transparency by Design not only want the data, but we want to share it, to make it transparent.

Why would any colleges oppose having data available?  Because in a world devoid of data, the reliance on reputation (whether deserved or not) rules.  There are many schools that rely on reputations that may or may not have been earned, but that drive their institutional revenues. Any change to rely on something that is based on data might threaten their reputations … and their revenues.

Mr. Ewell is correct when he says that numbers are going to be produced whether anyone likes it or not. Too bad that the numbers could not be produced in the most efficient way, and in a system that considers the entire country.  But, we have public policy to prevent that … because there are some who might lose.

Please feel free to leave a comment and let me know what you think.

Mike

But, I wonder how seriously.  For an organization that has “learning outcomes assessment” in its name, why only look at core learning such as writing, critical thinking and analytical reasoning. What about learning outcomes in the student’s major? At the program level? The report states that the most common approach is to use something like the Collegiate Learning Assessment (CLA), and to focus on the core areas. Of course, if you aren’t going to pay attention to whatever the assessments have to tell you, maybe it is just too much trouble to think about what ought to be measured.

As I have stated many times in this blog, the basic focus of Transparency by Design and its Web site http://www.collegechoicesforadults.org has been on program-level learning outcomes. Some have criticized us because the outcomes are not directly comparable—that is because every school claims unique program outcomes, and that those outcomes differentiate their school and programs. Regardless, why isn’t any other initiative looking at that level of learning? Why isn’t the National Institute for Learning Outcomes Assessment encouraging that level of assessment?  If we are all going to get real about assessing learning outcomes, let’s get down to the important stuff:  what is it that we intend to have students learn in order to earn a degree?  How do we know if our graduates are actually learning what they are supposed to be?

I would hope the courage is out there for others to begin to do what the Transparency by Design institutions have started. But, the outlook is not promising. Just recently I was warned by a person from a traditional school that we should all be careful what we measure, because if you measure it, you may not be pleased with what you find.  You certainly would not want what you find to get out in the open!

The approach of the institutions in Transparency by Design is the exact opposite. Let’s measure what is important and, if we are not doing well today, let’s get about fixing it.  It is unfortunate that much of higher education goes through the motions of assessment, avoids assessing the really important things, and then ignore what they find.  I would suggest that it is precisely because of this culture that we are under such pressure to measure more, be transparent about what we find, and use assessment to drive improvement.

Please feel free to leave a comment. Thank you.

Mike

Martin and Gillen call for breaking the spiral by reducing uncertainty about quality. That could be done by reporting outputs such as learning and research outcomes. This shift would move colleges to a “more beneficial type of competition based on who provides the greatest value (who can produce the most value-added education per dollar).” They imply that research outputs are measured and available while teaching outputs and productivity are neither measured nor available. They say that there is a need for “serious studies to measure teaching productivity; once this is accomplished, professors can be rewarded for the quality of their teaching instead of just the quality of their research.” And they conclude by calling for “improved transparency.”

I think they are on the right track and it is transparency about learning outcomes that has been at the core of an effort that I have written about before: Transparency by Design. While the primary purpose of this initiative is to provide consumer information for adult students, this initiative provides visibility into learning outcomes. For this initiative, quality is defined as consisting of the experience a student has and the learning she is able to demonstrate. The experience is measured through feedback from current students on external instruments asking about their satisfaction and sense of engagement, and from alumni asking about how satisfied they remain after graduation. Learning outcomes are measured and reported at both core (writing, critical thinking, etc.) and program levels.

While I had not thought about the cost implications of failing to share outputs, those of us who are working on Transparency by Design and College Choices for Adults are very aware that quality needs to be defined by outcomes. And we are presenting our outcomes so that the world may see the quality we deliver. We want to build reputation based on that demonstrated quality. And if that helps to stop the cost spiral, all the better.

Please feel free to leave a comment.

Mike

There are a number of factors that are considered to increase attrition rates. These include being older, coming from a low-income background, being the first generation to attend college, studying part-time, working while a student, and more.

In addition to the for-profit career colleges, at-risk students are more likely to be enrolled in substantial numbers at community colleges and open access public colleges and universities.

The problem that the reported research is attempting to address is that the public and many policy-makers tend to take a rather simplistic approach to assessing completion rates, assuming that those institutions that report high completion rates are good and those that have lower rates are not so good. But, it is not that simple. As this article makes clear, there are differences in how institutions operate that complicate the matter. The easiest way to assure good completion rates is to just admit students who are highly unlikely to fail. That is the traditional, elitist approach. But that approach does not hold up when we try to create equity in access and in a time of economic crisis when one pathway to better economic security has been identified as increasing higher education attainment rates. So, institutions that admit those qualified students who have one or more of the risk factors for attrition can be expected to produce lower completion rates, on average, than institutions that have students without the risk factors.

The complexity goes beyond student attributes. Institutional models also impact completion rates. Many institutions allow students to only transfer in a limited number of credits and require the student to take the vast majority of credits for a degree at that institution. But there are other institutional models where emphasis is placed on helping students aggregate credits earned primarily at other institutions and to complete only a limited number of credits from that institution to qualify for a degree. Obviously, the completion rate is going to increase if students take a relatively limited number of credits rather than 90 or 100 that may be required by some institutions to qualify for a baccalaureate degree. The latter type of institution is usually one that serves adult students and we encountered this model variation with Transparency by Design.

There needs to be a way to better understand completion rate variances and to allow for more informed comparisons than simple completion rates across different types of institutions. The career colleges study offers one possible way to address these differences. The researchers state that when the percentage of a college’s students eligible for Pell Grants is at 75%, the career college students graduate at higher rates than at four-year private, four-year public, and community colleges.

Again, this may be a way that we can better compare completion rates across the different sectors and institutional models. At minimum, it will be most interesting to see the final report and to be able to examine the data when it is released later this summer. If nothing else, this study tries to tackle a rather vexing problem with the comparison of completion data.

Please feel free to leave a comment.

Mike

While the Education Sector reports deals with state colleges and universities, and how effectively each state assesses learning outcomes, all colleges and universities need to assess learning outcomes. Some time ago, I reported on the efforts of colleges and universities that serve adults at a distance to assess learning outcomes, and use these and other data to better inform adults seeking a college to attend. The institutions engaged in this effort have worked, in partnership with WCET, a part of the Western Interstate Commission on Higher Education (WICHE), to develop a Web site to deliver information on learning outcomes and other consumer information. Since the initial post about Transparency by Design, the original 12 institutions have been joined by American InterContinental University and Colorado Technical University. And, we partnered with WCET, which serves as the aggregator and publisher of data from the participating institutions on a Web site – www.collegechoicesforadults.org – that is not intended for marketing, but rather for consumer information.

The type of information on the site parallels the information that Education Sector recommends that the states use. Some of the information is not be included yet because institutions are still collecting the data, which will be added to the site as it becomes available. For example, the site will eventually include National Survey of Student Engagement (NSSE) data, Measurement of Academic Progress and Proficiency (MAPP) data (this is similar to the CLA data that Education Sector mentions), and other data similar to the Voluntary System of Accountability, that is a system used only by public colleges and universities.

In addition, the site has been designed to include institution-specific information on intended program learning outcomes, how these are measured, and how well graduates demonstrate that they have achieved these intended outcomes. This is the major innovation on this site. We feel it is critical to report how well adult students have learned their major area of study. We will also include common questions each institution has asked alumni and current students’ satisfaction with their school and program.

Transparency by Design is different from what the Education Sector calls for because we are going to report our information and outcomes directly to our consumer, the adult student. However, the spirit is the same. We seek to better inform the consumer and let the consumer make judgments about how well we are performing. We also intend to use what we learn to improve both our institutional performance and to continuously improve our efforts with Transparency by Design.

Please feel free to leave a comment.

Mike

I have posted in the past about Capella’s involvement with Transparency by Design, an effort to create a trust-worthy, non-marketing site for adults trying to find a college to attend. This initiative involves a number of regionally accredited institutions that specialize in serving adults at a distance. Because many of the universities involved are not household names, the adult seeking a place to attend college can wonder about their legitimacy. The accreditation process is one means to assure that an institution is viable and has been objectively reviewed by knowledgeable educators.

However, the adult seeking a college can be bombarded with options and some of those options are not always legitimate. It is a harm to the consumer and to the higher education sector to have diploma mills out there. Worse yet, there are also accreditation mills. We all need to work to identify these mills and eliminate their ability to prey on the unsuspecting, or to accomodate those who seek to simply add a degree to their resume without doing any work to earn it.

CHEA has done good work to support legitimate accreditation and create resources for those who want to verify the validity of an accreditation agency. All 50 states need to prohibit the use of bogus degrees. This would be a good step toward undercutting the business of running a diploma or accreditation mill.

Please feel free to share your comments.

Mike

http://www.usnews.com/articles/education/2009/08/04/how-much-do-colleges-really-teach-students.html

http://www.insidehighered.com/news/2009/08/04/transparency

http://chronicle.com/article/New-Web-Site-Compares-Student/47924/

Capella also just launched our own learning outcomes and accountability Web site, which provides even more in-depth information about the programs we offer and what Capella learners will actually take away from their education. You can view it at CapellaResults.com.

Mike

I really like the quote from Alexander McCormick, who is the NSSE director and professor of education at Indiana University:  “Critics of distance education assume that face-to-face classes have inherent advantages as learning environments.  But these results indicate that those who teach classes online may be making special efforts to engage their students.”  Absolutely correct.  McCormick goes on to say “It may also be the case that online classes appeal to students who are more academically motivated and self-directed.”   Consider that comment in combination with a quote from Bob Gonyea of the Center for Postsecondary Research, “I believe one part of the explanation is that online learners tend to be older students who are somewhat more motivated and responsible in getting things done. . .I also think that people who teach online classes don’t take engagement for granted.  They have to structure assignments that get students connected.”

Wow!  This is why I really love being at an adult online university.  Now I don’t want to get too big a head and start feeling sorry for my colleagues who work at face-to-face, campus-based universities.  But, it certainly is a joy to work in an environment where we truly have faculty who love to teach adults; adult students who are motivated, responsible and desire to be engaged; and where there are conscious efforts made to engage those students. 

I recall a couple of years ago as we were planning Transparency by Design, and we decided to include NSSE data in our reports.  NSSE had not previously reported online versus face-to-face data and there was some nervousness.  But, when the data came out there was elation.  This inspires me that the more we can learn about what results we are actually producing, the better we can make higher education and the higher education experience for learners of all ages.  While NSSE data is favorable to online learning, there may well be metrics that will not be so favorable.  But we need to identify, measure, and be transparent about those metrics if we are serious educators.  If we find we are not doing well, we need to figure out how to fix what we are doing.  Too much time is spent fearing what data might tell us and how it might be used.  More time needs to be spent in sorting out what is essential to successful higher education, how to measure it, actually measuring it, and then sharing what we learn.

I recognize that not everyone may agree with me.  Please see http://www.insidehighered.com/news/2008/12/04/humanities or http://www.insidehighered.com/news/2008/12/04/assessment to see some alternative perspectives.

Your thoughts? Feel free to leave a comment.

Mike

This will allow the institutions involved in Transparency by Design the opportunity to expand our work. The initial public reports will be published in early 2009.

Mike

Jones notes that voluntary systems are creating more openness and helping institutions identify improvements but is concerned that the data sets don’t provide many states with complete information.  Transparency by Design is one of these voluntary efforts.  It is being lead by Capella but involves a number of adult-serving, cross-border, online programs and universities.  We have a deep interest in working with the states because we have much to offer and would like to see the states recognize the work we are doing and the opportunities we offer.  And, Transparency by Design is working with NCHEMS to help assure that the data and information we provide to prospective adult students is the right data. 

The work that the National Center for Public Policy and Higher Education (the Measuring Up folks) and NCHEMS are doing is important.  The Transparency by Design institutions stand ready to help make that work even better because we are dedicated to adult students, transparency, and improved understanding of the complex contemporary world of higher education.

I think that is important. What do you think? I look forward to reading your comments.

Mike