Quality in Online Learning – Gates Foundation

Goal of the Day

What actions should the foundation take to dramatically increase low-income young adult online completion, learning outcome attainment, and progression to credential – both financial and non-financial actions?

· Focus on high school reform

· New focus – increase higher education completion – especially low income and marginal groups

· Made an effort on entrance but the challenge is on completion of post secondary institutions

· Widely varied educational experiences – lift up the floor (minimum standard) and how do we lift the ceiling (raise the bar)

· Bill Gates Letter regarding online learning
http://www.gatesfoundation.org/annual-letter/2010/Pages/education-learning-online.aspx

· http://www.gatesfoundation.org/postsecondaryeducation/Pages/default.aspx

Wiki Notes

http://qualityonline.pbworks.com/

Hilary Pennington, Director of Education, Postsecondary Success, and Special Initiative

http://www.gatesfoundation.org/leadership/Pages/hilary-pennington.aspx

Institutional Context

· Pedagogy/architecture

· Content

· Faculty

· Peers

· Supports

Student Learning Experience

Student Learning

Outcomes Measures

· Assessments

· Completion

· Progression

· Success in Future Courses/Credentials

· Learning Efficiency (e.g. learning per minute)

· Citizenry

Notes:

Quality Matters – process – not just a rubric

http://www.qualitymatters.org/

Largest drop-out rate is in the first couple of courses – confidence issue, not a technology issue

More than an instructional design issue

The Quality Matters Rubric

The eight broad standards include:

1. Course Overview and Introduction
2. Learning Objectives
3. Assessment and Measurement
4. Resources and Materials
5. Learner Engagement
6. Course Technology
7. Learner Support
8. Accessibility

Learning Outcomes Assessment

http://www.apus.edu/community-scholars/learning-outcomes-assessment/

Focus on preparing faculty

· $$$

· Time

· Quality

EDUCAUSE

· Students have a direct voice in the conversation

· Focus on “what you do”

· Focus on purpose and values of higher education – more than just “going online”

· Large classrooms – distance education begins after the fourth row (Dan Greenstein, University of California)

· Moving beyond a single course focus

Next Generation Modeling Team – Gates Foundation

· Gates Foundation - All lives have equal value

· Post-secondary – social justice agenda – give everyone an equal opportunity

· Students make an intentional choice to attend post-secondary education – balancing live/work/school balance

· Higher completion without compromising quality

· Core focus – break inter-generational poverty cycle

· Focus on “big problem” areas

· Catalyze conversation on student completion – move beyond access issues

· Options for completing courses and programs

Focus on open educational resources – share

Clayton Christensen – Disruptive Technologies

http://www.claytonchristensen.com/

Cable Green – Open Educational Resources

http://globaleducation.ning.com/profile/CableGreen

Rio Salado College

http://www.riosalado.edu/online/Pages/default.aspx

Predictive analytics – model student success in courses (e.g. logon access)

Predicting student behavior, showing the learning, where is the quality “centered”

Vernon Smith

K to 20 California system – collaborate

Changes at the “edge”

Improve graduate rates, decrease completion times, and improve quality experience

How do you maintain quality with increasing demand

SUNY Online – started in 1994 – focus on the student experience

Survey of community college entrance

Visible Teaching and Learning – teachers seeing learning through the eyes of our students, students seeing themselves as their own teachers

Community focus – share what you know – participants in understanding “what works” – focus on improving quality for all of us!!

Professional development – experience from the student perspective

Community College Survey of Student Engagement

http://www.ccsse.org/

Interesting results to look at

1995 – was a big year for online learning in the United States

Standardization – leading to quality online learning – quality assurance – faithful to the process

Question – how do we sustain growth with increasing demand?

Academic integrity issue – especially in community colleges – student fraud

University of Texas online courses – 88 to 96% completion rates

CAROT – assess course quality

http://www.telecampus.utsystem.edu/

Importance of the student perspective

WCET

http://www.wcet.info/2.0/

Ellen Wagner’s Focus

· Focus on solutions

· Accountability

· Execution

Transparency by Design - WCET

http://www.wcet.info/2.0/index.php?q=TransparencyByDesign

Focus not just on the quality of individual learning but on organizational learning

Making the teaching and learning process in higher education in “visible”

Agenda

Where are we with “quality” in online learning?

Gates Keepers

http://gateskeepers.civiblog.org/

Quality Measures in Online Learning

USDLA – John & Nish

http://www.usdla.org/

Link between opportunities and challenges

Ten criteria

· Mission

· Standards

· Integrity

· Student enrollment and admissions

· Learning environment (LMS)

· Student support services

· Evaluations

Education – change agent of society

· 42 states have K to 12 online accreditation programs

Rubrics and Measurements

· Grew out of the K to 12 sector

· Psychometric testing, personality testing, formative and summative assessment, authentic assessment, standardized testing

· Instruments – embedded in the course – diagnostic assessment – classroom assessment techniques

· Using LMS data – analytics – inferential statistics – predictive behavior of students in an online environment

· E.g. system biology – how everything in the system is connected

· Systematic learning – how the system impacts student learning in higher education – move away from social science research

Quality Measures in Online – Broad Discussion

Strengths

· General agreement of broad buckets of best practice

· Lots of student data

· Data explosion

· Most evaluation tools have been validated

Weaknesses

· Student instrument response bias

· Definition of quality

· Do we know enough about how people learn?

· Validation to completion

Opportunities

· Imbedded assessment

· Systematic analysis

· Listening to the student voice (MRU Assessment Seminar)

· Merging data

· Better dissemination

· National normative data base – be able to used in different formats

Threats

· Highest quality versus forced curve

· Wedded to different instruments – different underlying learning theories (e.g. behaviourism vs constructivism)

· Standards vs craft

· We will never get “perfection”

· Selecting for easy completion versus “quality” courses

· Power structures – faculty associations

· Defensive – quality of online learning

Agreement

· Student tenacity can override quality issues

· Faculty can override quality issues

Debate

· Ability to map approach to learning strategies

· Definitions of quality – based on inputs or outcomes

Unknown

Other Notes

Edupunk conversation

http://en.wikipedia.org/wiki/Edupunk

Proficiency and basic competency – goal is to have every student get a C in a course

Define the “problem” – clear outcome and measures and process – focus on the “ultimate” prize

Current focus on “learning outcomes” - accreditation

Issues – student authentication in online learning courses and programs

Focus on progression – open up world of prior-learning assessment – do it online – shorten the time line to degree completion

Goals

Low-income completion rates

Report Back from Groups – What needs to change?

· Need to get data on low-income students and their online course experiences

· Induction – use the data to find the key questions

· Create a data consortium from existing online programs – aggregate and create a meta data-set

· Semantic analysis – identify pathways of success – individualized pathways

· Issues of relevancy

· Gates Foundation – catalyze data collection, provide seed funding for initial analysis of the data

· Identify attributes of successful online learning experiences for students

· Get Gates Foundation to underwrite an existing foundation – EDUCAUSE, Sloan, WCET

· Gates Network – set of rules with rewards (e.g. scholarships)

· Universal university transcript – portable, portfolio, work experience and products – Bologna Accord of transferability

· Academic credit bureau

· No need to re-invent the wheel

· Need to implement and scale what we already know

· Continually evaluate

· Financial – improve affordability of higher education – open educational resources – people contributing

· Student support – monitor progress, early warning system/progress, readiness for college, work of HS, early college – send the data directly to the student – data that shows they need to improve

· Pedagogical – rewards, financial incentives, personal learning systems, adaptive systems, course collections

· Systems – reward outcomes, funding for progress and/or completion, data-driven systems – campus, state, national

· College completion service – overcome road blocks such as loans, support systems

· College services clearinghouse – meta-university services – e.g. advising, registration, enrollments – scale services

· Laddered – work/learning system

· Cooperative education system for the 21^st Century

· Antioch example - http://www.antioch.edu/

· Online courses – be very visible about the cost of higher education – cost model

· Delta cost model - http://www.deltacostproject.org/

· Gates Foundation – focus on disruption – preserve the problem that they are part of the solution

· Acceleration of the learning process

· Incentives – change faculty, student, and institutional behavior

· Faculty – reward and tenure process

· Institutional – focus on collaboration

· Student – change expectations – college completion should be an expectation

· Technology – needs to be ubiquitous

· Public policy – public good such as health care, kindergarten to higher education public policy

· Create a culture of expectation for higher education – bridges between community college and universities – degree bridging

· Gates University – best faculty of America, best of the for-profit institutions (business acumen, ethos of a public institution) – modeling appropriate behaviours, bottom up from the faculty (faculty own curriculum), force a culture change

· Future is in blended rather than online education

Existing Evaluation Tools

NSSE

http://nsse.iub.edu/

Quality Matters

http://www.qualitymatters.org/

Community of Inquiry

http://communitiesofinquiry.com/methodology

Transparency by Design

http://presidentsforum.excelsior.edu/projects/transparency.html

Resources

Gapminder

http://www.gapminder.org/

Next Steps for the Gates Foundation

· Post-secondary success focus

· Catalyze action – higher education completion agenda

· Digital learning fund – set of challenges, problems that maybe digital technologies can solve

· Might be “prize” funded

· Shared course library around clear learning outcomes – aligned with assessment

· New forms of engagement with low-income students – e.g., gaming

· Blended networks – on the ground, local support with online learning

· Data analytics – empowering students, faculty, and advisors

· Align with the college-ready programs – integrate the K to 12 and post-secondary education systems – clear and seamless pathways

· Community of learners – teaching presence

· Prizes for innovation

· Thought partnerships

· Competency-based education – 15 week semesters are a limiting structure – developmental courses, prior learning assessment

Other Notes

· More than academic content – social context for learning, “hands-on” experiential learning component

· Personalization of learning – need to have a local mentors

· National Repository of Online Courses
http://www.montereyinstitute.org/nroc/

· President’s Forum
http://presidentsforum.excelsior.edu/

· Clear outcomes – but many paths to achieve the outcomes

· Student conversations

· Iron Triangle – access, quality, cost

· Who defines quality?

· Engagement for different people in different contexts

· Student empowerment

· Allow for evolution – do not restrict innovation

· Network – not what you know but who you know

· Question about the impact of technology in higher education

· Disruptive education

· Online education – still is at the tipping point – has not crossed the “sound barrier” – technology can be a catalyst for this process!!