How well does Khan Academy teach?

By Valerie Strauss

Here is a new critique of the Khan Academy, the subject of a widely read post I published Monday about the hype and reality of the academy. You can find that post here. And you can find a response to that post from the founder of the Khan Academy, Sal Khan, by clicking here.

The following was written by Christopher Danielson and Michael Paul GHoldenberg. Danielson holds a Ph.D. in mathematics education from Michigan State University. He teaches math at Normandale Community College in Bloomington, MN. He maintains the blog “Overthinking My Teaching” and has written for Connected Mathematics. As of this writing, he has three badges and 11,041 energy points on Khan Academy. Goldenberg holds a master’s degree in mathematics education from the University of Michigan, as well as master’s degrees in English and psychological foundations of education from the University of Florida. He writes the blog “Rational Mathematics Education” and was a co-founder of the group Mathematically Sane. He currently coaches high school mathematics teachers in Detroit.

By Christopher Danielson and Michael Paul Goldenberg

Nearly everyone believes that K-12 mathematics education in the United States is in desperate need of improvement. One person whom many feel has built a definitively better math teaching mousetrap is Salman Khan, whose free on-line library of short instructional videos has had millions of hits (170,000,000 as of this writing) and drawn heaps of praise and capital from such luminaries as Bill Gates. Gates has called Khan, "the best teacher I've ever seen." But we contend that, rather than revolutionizing mathematics teaching and learning, Khan’s work adds a technological patina to a moribund notion of teaching and learning mathematics. What is more, his videos reveal an ignorance of how we know students learn mathematics.

So pervasive and fawning is the current media rhetoric surrounding Khan Academy that when Newsweek ran a cover story recently about the top 100 digital innovators, the question was not whether Sal Khan would be on the list, but whether revolutionary would be used to describe him. Sure enough Newsweek gushed, “Khan Academy offers an innovative portal that could revolutionize the American educational system.”

We have great respect for the stated goal of Khan Academy — “A free, world-class education for anyone, anywhere.” Yet, we have some serious concerns about the quality of the instruction providing this education. Indeed, if either of us were supervising Khan’s instruction, we would point to some concrete and important gaps in his practice.

What do you need to know to teach?

Our view is that content knowledge alone is inadequate for quality instruction. While knowing mathematics is of course necessary and contributes to making good decisions in a math lesson, it is not sufficient. Many mathematics educators stress another kind of knowledge necessary to design and deliver quality instruction: pedagogical content knowledge (PCK). PCK refers to knowledge of content as it relates to teaching.

A teacher with strong PCK knows from both educational research and professional experience which mathematical ideas and tasks individual students are likely to find easy, what they are likely to find challenging, which examples, models and metaphors will most likely be helpful for explaining a particular idea, and what misconceptions students will bring to tasks that will lead them to make further errors within given mathematical topics.

Pedagogical content knowledge is an important foundation for planning lessons and for decision making during a lesson. Recent research has highlighted the difference between content knowledge and pedagogical content knowledge in the domain of elementary mathematics. In particular, researchers are finding evidence that particular kinds of PCK are associated with greater gains in student learning in elementary mathematics.

Does Khan know what he needs to know?

We contend that the videos at Khan Academy are alarmingly devoid of PCK. We claim further that (1) the examples Khan chooses appear selected at random and thus are, perhaps unsurprisingly, often quite poor. They are prone either to create further confusion, or to fail to address fundamental questions students are likely to have; and (2) Khan’s explanations are frequently off target in addressing likely student questions that experienced teachers would anticipate and elicit.

We offer two examples. It is important to note that we did not choose these examples after watching hundreds of Khan’s videos. Instead, we came across them in the process of evaluating the usefulness of Khan’s video instruction for courses we teach, and for content in which we are interested. We contend that these examples are typical and that our critique applies broadly.

Comparing decimals. Decimal fractions (decimals for short — the numbers to the right of the decimal point) are a notoriously challenging topic in the elementary math curriculum. This difficulty is due at least in part to the fact that many rules that apply to whole numbers do not apply to decimals. Perhaps the most famous of these involves comparing numbers. When two whole numbers have different numbers of digits (e.g., 435 and 76), the one with more digits is greater (and usually by a lot!) This is not true of decimals: 0.435 is less than 0.76, despite having more digits.

Therefore, any attempt to help students improve their understanding of comparing decimals needs to deal with the case of decimals having different numbers of digits to the right of the decimal point. The standard American curricular treatment, in which students are instructed to append zeroes to the shorter decimal to equalize the number of digits helps students to perform this task correctly, but fosters its own misconceptions.

Khan Academy avoids this standard curricular approach. Instead, Khan has two videos involving comparing decimals. One is titled Comparing decimals; the other is titled, Decimals on a Number Line. In the first, Khan uses exactly one example, comparing 45.675 to 45.645. In the second, there is also exactly one example, comparing 11.5 to 11.7. In both cases, a student can get a correct answer by applying an incorrect rule (such as ignore the decimal point and treat what remains as a whole number). What is more, the accompanying explanation does nothing to address common misconceptions.

An important feature of Khan Academy is its online exercises, and those who defend Khan Academy against critiques of its videos often point to the exercises as making up for any deficiencies in the videos.

The first of the decimal comparison videos we discuss has no exercises. The second has exercises that ask students to locate numbers on a number line, but these numbers all have one decimal place and are to be placed on a number line that is already subdivided into tenths. In short, the exercises offer no intellectual rigor and do not address our central concern.

A student who thinks that 0.435 > 0.76 is offered nothing in the way of correction on Khan Academy. In fact, one of the top questions on the page for this video (as of July 18, 2012) is “So is .02009 greater than .0207?” This is exactly the sort of question that a competent teacher of arithmetic needs to anticipate and to answer. Khan fails to pose it.

Equality. The equal sign (=) is used in arithmetic and algebra to indicate that two things have the same value. Students often misinterpret the equal sign to mean and now find the answer. In order to learn algebra meaningfully, students need to interpret the equal sign correctly. When fourth graders were asked to write a number in the box to make the number sentence below true, the most common answer was 12.

8 + 4 = ☐ + 5

This represents an incorrect view of the equal sign. A student thinks, 8 plus 4 is 12, write that, then plus 5… This also explains another common answer, 17, in which the student adds all numbers in the expression and writes the answer in the empty box.

Ideally, a high-quality math teacher would: (1) discuss the meaning of the equal sign frequently and explicitly, and (2) model correct use of the equal sign.

Yet when Khan is teaching students to multiply in the Multiplication 4: 2-digit times 1-digit number video, he multiplies 16 x 9, which leads to the computation 9 x 1 + 5 (the 5 is the ‘carry’ from 6 x 9). He notates that as follows:

9 x 1 = 9 + 5

In a later video, Khan notates something similar as follows:

3 x 3 + 1 = 10

In both cases, the notation goes unexplained. He does not correct or discuss the previous notation; he just starts writing it differently. This seems to be because he does not know that the first notation perpetuates important student misconceptions. He seems to lack the knowledge of how students learn (and fail to learn) the idea of equality. He knows that the first notation is incorrect, but he doesn’t know (or else doesn’t care) that it will reinforce students’ wrong ideas. And his preparation methods or lack thereof don’t lead him to anticipate such student thinking and/or misunderstanding.

Views of research. Some readers will surely perceive our critique as nitpicking Khan’s errors. All teachers make mistakes, after all, and Khan should not, the reasoning goes, be called out for making the same mistakes we all make. There is truth in this argument to be sure. But competent teachers are interested in correcting those mistakes and in understanding their importance.

In particular, competent teachers understand that learning research offers insights for their own teaching. Sal Khan is on record as dismissing this research. In a recent interview he did with Harvard’s EdCast, he said the following:

I think frankly, the best way to do it is you put stuff out there and you see how people react to it; and we have exercises on our site too, so we see whether they’re able to see how they react to it anecdotally.

Khan will put the video out there and see how people react to it. He perceives this to be a better approach than incorporating results of quality research projects into his instructional decisions. In the age of No Child Left Behind and its mandate for “scientifically based research” as the foundation for classroom instruction, this seems lazy.

Furthermore, as noted above, many of Khan Academy’s exercises do not reveal the misconceptions that many students have about the topic at hand, e.g. decimals.

Summary

Khan has many fans. We are aware that our critiques will not please those fans. And at the same time, it seems clear that we are not alone in our concerns. When we read student comments on the site, we get the overall impression that many students — particularly those who may have grasped the procedure in class but wish to see the teacher's lecture reduced to a short, reviewable form — think that Khan is a dream come true.

But we also see from comments that the student who comes to these videos really lost is likely to leave them just as lost if not worse off, particularly if there's anything conceptually or procedurally challenging that they hadn’t previously mastered or understood.

If Khan’s videos occasionally popped up in a Google search, we would be content to have him carry on. There is lots of worse information available on the web. But Khan is hailed as “unbelievable” ( Bill Gates) and his work as “sparking a revolution in education” ( USA Today) In the description of his forthcoming book, he is described as having “established himself as an outsider, with no teaching background to tie him to broken models.”There certainly are broken models in education, but there is absolutely no evidence that competent knowledge of student learning and thinking is one that teachers can afford to jettison.

We do not suggest that Khan should enroll in a teacher education program (although perhaps that is not such a bad idea). Instead, we suggest that Khan Academy desperately needs voices of teaching experience. Khan could tap into any number of existing networks of exemplary teaching, perhaps from recipients of the Presidential Award for Excellence in Mathematics and Science Teaching or the National Board of Professional Teaching Standards. He could seek these teachers’ advice and collaboration on issues large and small. A modest but productive proposal would be to have experienced teachers prepare the examples he uses in his videos. In this way, he would not need to lose his trademark unrehearsed style.

Whether small steps or large, we urge Sal Khan and his funders to put their time, effort and dollars to the best possible ends, particularly when it comes to making decisions grounded in accurate, carefully considered pedagogical content knowledge . The Khan Academy videos as they presently exist fall far short of this goal.