WEBVTT - Auto-Grading Systems Are Getting It Wrong and Causing Remote Learning Headaches

0:00:00.120 --> 0:00:04.080
<v Speaker 1>It's Monday, November. I'm Oscar Emeras from the Daily Dive

0:00:04.160 --> 0:00:08.280
<v Speaker 1>podcast in Los Angeles, and this is reopening America for

0:00:08.320 --> 0:00:10.639
<v Speaker 1>all those parents with kids doing remote learning this year,

0:00:10.840 --> 0:00:13.760
<v Speaker 1>pay attention to your children's grades. In some cases, the

0:00:13.800 --> 0:00:16.960
<v Speaker 1>autograding software used to grade quizzes and schoolwork are getting

0:00:16.960 --> 0:00:20.000
<v Speaker 1>it wrong. If a student's answers don't match exactly what's

0:00:20.000 --> 0:00:22.320
<v Speaker 1>in the teacher's answer guide, it can be marked wrong,

0:00:22.400 --> 0:00:27.040
<v Speaker 1>even if it's just a capitalization discrepancy. Julie Jargon, family

0:00:27.120 --> 0:00:29.920
<v Speaker 1>and tech columnists at The Wall Street Journal joins us prout.

0:00:30.000 --> 0:00:32.920
<v Speaker 1>These auto grading systems are causing some headaches. Thanks for

0:00:33.000 --> 0:00:36.680
<v Speaker 1>joining us, Julie, Thanks for having me. I wanted to

0:00:36.720 --> 0:00:38.839
<v Speaker 1>check in on how everything is going with all this

0:00:39.040 --> 0:00:42.800
<v Speaker 1>remote learning that's going on right now because of that pandemic. Julie,

0:00:42.800 --> 0:00:45.080
<v Speaker 1>you actually wrote an article about how in a lot

0:00:45.159 --> 0:00:49.680
<v Speaker 1>of cases, bots are grading the schoolwork and quizzes for

0:00:49.760 --> 0:00:52.640
<v Speaker 1>a lot of students in some of these online classrooms,

0:00:52.960 --> 0:00:55.400
<v Speaker 1>and a lot of times they're actually getting it wrong.

0:00:55.480 --> 0:00:59.520
<v Speaker 1>They're marking answers that are right incorrectly, and it has

0:00:59.560 --> 0:01:03.120
<v Speaker 1>to do with very simple type of corrections. But either way,

0:01:03.360 --> 0:01:05.399
<v Speaker 1>kids are coming home with lower grades things like that,

0:01:05.440 --> 0:01:07.760
<v Speaker 1>and then parents on the flip side have to contact

0:01:07.800 --> 0:01:10.760
<v Speaker 1>teachers and try to get everything squared away. Julie, help

0:01:10.840 --> 0:01:14.559
<v Speaker 1>us walk through some of what's happening. I only discovered

0:01:14.600 --> 0:01:18.720
<v Speaker 1>this because my son, who's doing remote pole, he studied

0:01:18.720 --> 0:01:21.600
<v Speaker 1>really hard for a science exam. He expected to do

0:01:21.680 --> 0:01:23.880
<v Speaker 1>well I thought you would do on it, and then

0:01:23.880 --> 0:01:26.000
<v Speaker 1>we found out he had a very low grade on it.

0:01:26.120 --> 0:01:29.520
<v Speaker 1>So I just opened into the test online to see

0:01:29.840 --> 0:01:32.720
<v Speaker 1>the questions he'd missed, and I found several questions where

0:01:32.720 --> 0:01:36.080
<v Speaker 1>he did answer the question correctly, but he was marked

0:01:36.120 --> 0:01:41.559
<v Speaker 1>wrong because his answer that they typed in contained capital letters,

0:01:41.600 --> 0:01:43.880
<v Speaker 1>where the way the teacher input it into the answer

0:01:43.959 --> 0:01:47.680
<v Speaker 1>key it was lower case. So there were several things

0:01:47.760 --> 0:01:50.920
<v Speaker 1>like that that I discovered between him and my other

0:01:51.080 --> 0:01:52.760
<v Speaker 1>kids when I went back to kind of look at

0:01:52.800 --> 0:01:56.360
<v Speaker 1>what was going on here, and then I also create

0:01:56.400 --> 0:01:58.720
<v Speaker 1>a bunch of other parents to see if they were

0:01:58.720 --> 0:02:02.600
<v Speaker 1>discovering similar issues, and they were, Yeah. It's like when

0:02:02.640 --> 0:02:05.840
<v Speaker 1>you're entering a password for your email logging or something,

0:02:05.840 --> 0:02:09.200
<v Speaker 1>and it always says don't forget the password is case sensitive,

0:02:09.440 --> 0:02:11.359
<v Speaker 1>so you have to type in the exact upper case

0:02:11.400 --> 0:02:14.080
<v Speaker 1>and lower case combination the right way, and that's exactly

0:02:14.120 --> 0:02:16.800
<v Speaker 1>what was happening. You even had a few screenshots in

0:02:16.840 --> 0:02:18.920
<v Speaker 1>your article about what was going on, and one of

0:02:18.919 --> 0:02:22.200
<v Speaker 1>the questions was something about certain plants have special tissues

0:02:22.240 --> 0:02:25.640
<v Speaker 1>that move sugar, water and minerals between plant parts. And

0:02:25.680 --> 0:02:29.280
<v Speaker 1>the answer is vascular plants. But when the teacher inputs

0:02:29.360 --> 0:02:32.640
<v Speaker 1>the answer, you know, she had a capital V for

0:02:32.760 --> 0:02:35.560
<v Speaker 1>vascular and then a lower case P for plants, and

0:02:35.600 --> 0:02:37.760
<v Speaker 1>if you put it, I think in the example here,

0:02:38.000 --> 0:02:40.840
<v Speaker 1>both the V and the P were capitalized and because

0:02:40.880 --> 0:02:43.440
<v Speaker 1>of that they got the answer wrong. So tell us

0:02:43.440 --> 0:02:46.920
<v Speaker 1>about what online system that they were using there, and

0:02:46.960 --> 0:02:51.040
<v Speaker 1>a little bit more about how these answers are inputed. Yeah,

0:02:51.040 --> 0:02:53.040
<v Speaker 1>and that was actually an example from my son's tested

0:02:53.280 --> 0:02:56.240
<v Speaker 1>the screenshot of that. Menis teacher went back in what

0:02:56.320 --> 0:02:58.799
<v Speaker 1>they told him and gave the point for it. So

0:02:59.200 --> 0:03:01.920
<v Speaker 1>he and my other kids are using their school stricts

0:03:01.919 --> 0:03:04.640
<v Speaker 1>to using a system called Canvas, which is pretty one

0:03:04.680 --> 0:03:08.040
<v Speaker 1>of the big online school platforms a lot of districts

0:03:08.120 --> 0:03:11.839
<v Speaker 1>and colleges are using. There's another educational software company called

0:03:11.919 --> 0:03:15.760
<v Speaker 1>Otis that also has auto graded as a feature in

0:03:15.919 --> 0:03:18.760
<v Speaker 1>online classroom So what they are doing is they are

0:03:18.840 --> 0:03:23.040
<v Speaker 1>relying on these kind of autograding systems, which were I

0:03:23.080 --> 0:03:25.680
<v Speaker 1>think build as a way to save teachers time. But

0:03:25.800 --> 0:03:29.800
<v Speaker 1>what it's doing instead in many cases is requiring teachers

0:03:29.840 --> 0:03:32.280
<v Speaker 1>to then go back manually through all of these tests

0:03:32.320 --> 0:03:36.200
<v Speaker 1>for all of their students and make sure that they're

0:03:36.240 --> 0:03:40.240
<v Speaker 1>getting credit for the answers that they do are down correctly.

0:03:40.720 --> 0:03:42.480
<v Speaker 1>I want to give one more example, because these are

0:03:42.560 --> 0:03:45.480
<v Speaker 1>very simplistic. I mean, if somebody was grading this manually,

0:03:45.640 --> 0:03:47.640
<v Speaker 1>you can see where they would get it right. What

0:03:47.760 --> 0:03:51.720
<v Speaker 1>happens when magnets come in contact with other magnets. So

0:03:51.840 --> 0:03:54.520
<v Speaker 1>the incorrect answer that was put in there was they

0:03:54.560 --> 0:03:57.560
<v Speaker 1>pulled together or they repel, But the correct answer, as

0:03:57.600 --> 0:04:00.680
<v Speaker 1>the teacher would have inputed it, was the attract or

0:04:00.720 --> 0:04:04.160
<v Speaker 1>repel each other. So just these very little, minor, minor

0:04:04.360 --> 0:04:07.360
<v Speaker 1>differences are causing kids to get the answers wrong on

0:04:07.400 --> 0:04:10.040
<v Speaker 1>these quizzes and other schoolwork like that. So what has

0:04:10.080 --> 0:04:13.360
<v Speaker 1>been the response by teachers and the company as well?

0:04:13.400 --> 0:04:17.560
<v Speaker 1>Because you spoke to representatives from Canvas, the Canvas acknowledge

0:04:17.600 --> 0:04:19.440
<v Speaker 1>that this is an issue. They said that they are

0:04:19.560 --> 0:04:22.560
<v Speaker 1>aware of it and that in fact they frequently received

0:04:22.720 --> 0:04:26.120
<v Speaker 1>questions about this and you know, complaints about it from teachers,

0:04:26.240 --> 0:04:29.440
<v Speaker 1>and they're aware of it. They have a quiz system

0:04:29.520 --> 0:04:32.440
<v Speaker 1>that kind of their older version of their quiz system

0:04:32.520 --> 0:04:35.719
<v Speaker 1>that seventy of their customers are still using, and about

0:04:35.760 --> 0:04:39.640
<v Speaker 1>eighteen months ago they began rolling out an upgraded quiz

0:04:39.680 --> 0:04:44.120
<v Speaker 1>feature that will recognize answers when they are close enough

0:04:44.480 --> 0:04:48.120
<v Speaker 1>to the answer that the teacher put in the quiz.

0:04:48.240 --> 0:04:51.400
<v Speaker 1>So they are trying to make improvements overall to their

0:04:51.440 --> 0:04:54.680
<v Speaker 1>online quizzing, but the majority of their of the customers

0:04:54.680 --> 0:04:57.159
<v Speaker 1>are still using them the older version with these mistakes

0:04:57.320 --> 0:05:00.960
<v Speaker 1>happening and its exact matchesn't And part of what they

0:05:01.000 --> 0:05:03.120
<v Speaker 1>said too was that in the rush to do all

0:05:03.120 --> 0:05:06.200
<v Speaker 1>the remote learning, teachers didn't have enough training also on

0:05:06.240 --> 0:05:07.600
<v Speaker 1>how to do some of this, So they were also

0:05:07.640 --> 0:05:11.320
<v Speaker 1>putting it on lack of training for teachers too. Obviously,

0:05:11.400 --> 0:05:14.760
<v Speaker 1>school districts last spring and just were kind of caught

0:05:14.760 --> 0:05:18.120
<v Speaker 1>off guard by the sudden switch. And while they did

0:05:18.120 --> 0:05:21.159
<v Speaker 1>have more time over the summer to prepare for this fall,

0:05:21.279 --> 0:05:24.640
<v Speaker 1>who's still a big learning curves I would imagine for

0:05:24.720 --> 0:05:28.679
<v Speaker 1>teachers to learn the new systems, you know, the online grading.

0:05:28.800 --> 0:05:31.400
<v Speaker 1>It's not just online grading, it's the whole platform of

0:05:31.440 --> 0:05:34.240
<v Speaker 1>where you submit, where you post your assignments, where you

0:05:34.600 --> 0:05:37.400
<v Speaker 1>have your zoom links for the kids classes. All of

0:05:37.400 --> 0:05:40.160
<v Speaker 1>those things are kind of in one place, and that's

0:05:40.200 --> 0:05:43.040
<v Speaker 1>a lot of to learn how to navigate. So it

0:05:43.120 --> 0:05:46.080
<v Speaker 1>could very well be that, you know, in many cases,

0:05:46.120 --> 0:05:49.479
<v Speaker 1>teachers didn't have enough time to fully understand that this

0:05:49.560 --> 0:05:51.920
<v Speaker 1>could be a problem. I mean, I know some teachers

0:05:51.920 --> 0:05:55.359
<v Speaker 1>are aware of it, others don't know about you know,

0:05:55.400 --> 0:05:58.159
<v Speaker 1>because it's it's easy to miss if your student has

0:05:58.400 --> 0:06:01.000
<v Speaker 1>one or two answers wrong. I mean, if the most

0:06:01.040 --> 0:06:03.599
<v Speaker 1>of the quiz is multip choice and just a couple

0:06:03.640 --> 0:06:05.839
<v Speaker 1>of writing answers, and they get those wrong, they're also

0:06:06.000 --> 0:06:07.560
<v Speaker 1>going to be or maybe a CE on a test.

0:06:07.600 --> 0:06:10.520
<v Speaker 1>You may not think that's anything to worry about, but

0:06:10.760 --> 0:06:13.800
<v Speaker 1>if enough of those kind of add up, the student

0:06:13.839 --> 0:06:16.120
<v Speaker 1>might get a family in grade versus a passing grade,

0:06:16.240 --> 0:06:17.920
<v Speaker 1>or a C in a class instead of a B

0:06:18.120 --> 0:06:20.400
<v Speaker 1>when they deserve that B or a day. And even

0:06:20.400 --> 0:06:21.919
<v Speaker 1>some of the teachers you spoke you said, you know,

0:06:21.920 --> 0:06:25.040
<v Speaker 1>it's taken them hours to grade these automatically graded quizzes,

0:06:25.040 --> 0:06:27.120
<v Speaker 1>so you know, it just creates a big headache. So

0:06:27.320 --> 0:06:28.960
<v Speaker 1>I'm glad to see that they're at least rolling out

0:06:29.000 --> 0:06:31.560
<v Speaker 1>improvements to try to correct some of this stuff. But yeah,

0:06:31.600 --> 0:06:33.680
<v Speaker 1>as a parent, you know you gotta watch out for

0:06:33.720 --> 0:06:36.039
<v Speaker 1>this and bring it to the teacher's attention if something

0:06:36.480 --> 0:06:39.200
<v Speaker 1>is a miss like that. Julie Jargon, family and tech

0:06:39.240 --> 0:06:41.080
<v Speaker 1>columnist at The Wall Street Journal, thank you very much

0:06:41.080 --> 0:06:46.159
<v Speaker 1>for joining us. Thank you for having me. I'm oscar

0:06:46.200 --> 0:06:49.280
<v Speaker 1>A Mirrors and this has been reopening America. Don't forget

0:06:49.480 --> 0:06:51.480
<v Speaker 1>effort today's big news stories. You can check me out

0:06:51.480 --> 0:06:54.279
<v Speaker 1>on the Daily Dive podcast every Monday through Friday. So

0:06:54.360 --> 0:06:56.680
<v Speaker 1>follow us and I Heart Radio or wherever you get

0:06:56.720 --> 0:07:00.560
<v Speaker 1>your podcast. FA