Immediate feedback from a good autograder provides a much more interactive learning experience for students. They are able to face and correct their mistakes in real time until they arrive at a correct solution. That's a real learning opportunity.
The value of educational feedback drops rapidly as time passes. If a student receives immediate feedback and the opportunity to try again, they are much more likely to continue attempting to solve the problem. Autograders can support both; humans, neither. It typically takes hours or days to manually grade code just once. By that point students are unlikely to pay much attention to the feedback, and the considerable expense of human grading makes it unlikely that they are able to try again. That's just evaluation.
And the idea that instructors of computer science courses are in a position to provide "expert feedback" is very questionable. Most CS faculty don't create or maintain software. Grading is usually done by either research-focused Ph.D. students or undergraduates with barely more experience than the students they are evaluating.
We've been doing this at Illinois for 10 years now. Here's the website with a description of the facility: https://cbtf.illinois.edu/. My colleagues have also published multiple papers on the testing center—operations, policies, results, and so on.
It's a complete game changer for assessment—anything, really, but basic programming skills in particular. At this point I wouldn't teach without it.
And the proud Illinois tradition of some mission-critical service crashing on the first day of class continues.
In this case, it is an external service. However, I also suspect that the Duo outage is probably shielding other on-campus services from load surges that would probably be causing them to get crashy.
I guess I don't know how we could ever prevent such incidents. Given that the first day of classes is a well-kept secret /s.
Note that admissions rates for female applicants are higher across all categories—international, out-of-state, and in-state. Obviously you can't fully tell what's going on here without more of an understanding of the strengths of the different pools, but a 10–30% spread (for in-state) suggests that gender is being directly considered.
IANAL, but I'm also concerned about the degree to which this decision affects the use of other factors during college admissions. Fundamentally admissions is a complex balance between prior performance and future potential, and only admitting based on prior performance means that we're stuck perpetuating existing societal inequities.
I do know that 25 years ago or so there considerable weight given to gender in sciences and engineering. I do feel like all talk of it has disappeared, and wasn't sure if it was because it was no longer a factor or because race became the dominant talking point.
From the data you present I suspect that there is weight still given to gender. I wonder how much energy there would be to investigating this? I wonder how many guys who get rejected from MIT CS will now do Tik Toks about how a girl took his spot, since he can no longer say it was a black kid?
As a CS faculty member at Illinois (aka UIUC), I don't think that we fit this model.
At least according to my quick reading of the article, ASU has a significant focus on inclusion as a core value. Overall Illinois does admit a large percentage of applicants: about 50% over recent years. (The number dropped a bit after we began participating in the Common App, which makes it easier for students to increase the number of institutions they apply to.)
However, that number hides the fact that admission to top programs like computer science is extremely selective and exclusive. Admission rates to CS have been around 7% recently. And while we've made a CS minor somewhat more accessible, we've also closed down pathways that allowed students to start at Illinois and transfer into a computer science degree. (At this point that's pretty much impossible.) We do have blended CS+X degree programs that combine core studies in computer science with other areas, and those are less selective, but they have their own limitations—specifically, having to complete a lot of coursework in some other area that may not interest you.
I think what's fooling you about Illinois is the fairly odd combination of a highly-selective department (CS) embedded in a less-selective institution. I'm sure that there are other similar pairings, but overall this is somewhat unusual. If you think about other top-tier CS departments—Stanford, Berkeley, MIT, CMU—most are a part of an equally-selective institution.
So with Illinois you're getting the cache of an exclusive department combined with the high acceptance rate of an inclusive public land-grant university. But on some level this is a mirage created by colocated entities reflecting different value systems. And, unlike places like Berkeley and Virginia, which have been trying to admit more students into computing programs, no similar efforts are underway here at Illinois. (To my dismay.)
Overall, unfortunately it's still very obvious to me that exclusivity is part of what we're selling to students as a core value of our degree program. You're special if you got in—just because a lot of other people didn't. Kudos to anyone moving away from this kind of misguided thinking.
I've taught CS1 for going on 6 years now, to almost 10K students. I'll admit I found this post depressing to read. Because many of these pain points have obvious solutions that have been in use at many institutions for years. (Some of these solutions are in the paper, but are not novel.)
When / where are students struggling? Assess them frequently and you'll find out! We run weekly quizzes in my class. So we know exactly who's struggling with exactly what, and quickly. That allows us to do individual outreach, and for students to catch up before they get too far behind. We also use daily homework, for the same reasons. But a lot of CS1 courses are still using the outdated midterm and final model, maybe with some homework sprinkled in.
Frequently a glut of repetitive student questions points to bad course materials or poor course design. Make things more clear and make it easier for students to find information and at least some of the repetitive question asking will diminish.
Grading and TA support are related. Graduate TA quality does vary greatly, and you need to design around this. For example: Never put students in a position to suffer for an entire semester at the hands of a bad TA. (Many courses do.) Undergraduates are almost always better at assisting with early CS courses, and usually cheaper. We've been shifting gradually toward more undergraduate support for our CS1 course, and it has been working out well. They frequently outperform graduate staff.
But no amount course staff will be sufficient if you have them spend all of their time on tedious tasks that computers can do better: Like grading code! It's 2023. If you can't deploy your own autograder, buy one. Staff time grading code should be minimized or eliminated altogether. Freeing staff time for student support allows you to provide students with more practice, and accelerates the overall learning process. But many early CS courses are stuck in a situation where staff grading is bottlenecking how many problems they can assign. That's insane, when autograding is a well-established option. (Even if you want to devote some staff time to grading code quality, autograding should always be used to establish correctness. And you can automate many aspects of code quality as well.)
In my experience, what's at the root of a lot of these problems is simply that many people teaching introductory CS can't build things. Maybe they can implement Quicksort (again), but they can't create and deploy more complex user-facing systems. I mean, you can create an autograder using a shell script! Not a great one, but still far superior to manual human grading. Part of this is because these jobs pay poorly. Part is how we hire people for them, because the ability to build things isn't typical a criteria. Part of it is that there's little support for this in academia. It took me years of inane meetings to get a small cluster of machines to run courseware on for my 1000+ student class that generates millions of dollars in revenue.
But there's also a degree to which the CS educational community has started to stigmatize expert knowledge. If you do enjoy creating software and are good at it, you get a lot of side eye from certain people. "You know that students don't learn well from experts, right?" And so on. Yes, there is a degree to which knowing how to do something is not the same as being able to teach someone how to do it. But would you take music lessons from someone who was not only a mediocre player, but didn't seem to like music that much at all?
Absolutely. The article makes it sound like problems with poor advising are confined to community colleges. That's ludicrous.
And it's not just state schools. When my wife went back to RIT (Rochester Institute of Technology, an expensive private school) to study photography, nobody there could even answer questions like: What courses do I need to graduate? Knowing that might be useful!
A lot of the student management systems used at many schools are extremely antiquated. Degree requirements are expressed in ways that make it difficult or impossible to create tools to help students plan their schedules and stay on track to receive their degree on time. Registration can make it hard for students to enroll in courses they need to complete their program. Many advisers are doing their best, and come up with hacks and workaround to try and help individual students, but access to advising can be limited and advisers are also affected by the poor data and tooling.
And of course, it's worth pointing out that universities are positioned to benefit from student mistakes. Didn't realize that you needed that course to graduate? See you next semester! Make sure to bring your (or your family's) checkbook. I'm sorry you couldn't register for that popular course as an undergraduate! Have you considered our MS programs?
These problems are not limited to community colleges.
One of the biggest problems with overheads is that administrators are frequently simply unable to explain or justify them.
The vast majority of research-active universities in the US are non-profit, and so it doesn't really make sense to say that the universities is "profiting" off of overheads. Overheads are also negotiated with each funding agency, and my understanding is that the university has to prepare a fairly detailed case to support their particular number. They can't just make something up.
But overhead rates can vary fairly significantly between similar institutions. And when, as a faculty member, you start asking questions about why, you're frequently met with defensiveness, nonsense, or both. When I worked in the Northeast, I remember someone claiming that our overheads were 10+% higher than another institution due to... snow removal. (Which wasn't that effective either.)
Overheads also seem to only go up. Because of course.
Provide a reasonable justification for why you're charging me a certain amount, and how the money gets spent, and we're cool. I understand the university is a complex place and there's a lot involved in supporting what I'm doing. But when you can't provide a reasonable explanation, people start to wonder why not.
> are non-profit, and it doesn't really make sense to say that the universities is "profiting" off of overheads
IMO “non-profit” is a misnomer.
It only implies that the organisation’s owners (funders, equity holders, etc) do not take a profit.
It does /not/ mean that the people involved don’t make money - income, fees, admin overhead, building/contracts/employment favoritism, etc. There are infinite ways to leech money that don’t involve taking “profits”.
My partner works in grant management for a public r1.
I believe the rate the university gets is about .57.
That pays for facilities and education programs and funding the department itself. Bringing in money makes the university happy and your pay check bigger though the accounting on it sounds obnoxious and not always guaranteed (salary cuts to cover overruns, etc).
However if you think about it,
That's probably about how much private employees make for their employer anyways if not more. Annoying but hey that's the way the system is, you're getting paid to make someone else more money.
From the perspective of a grad student, you're getting paid $20k/year in a HCOL area, the grant is getting charged like $40k a year (somewhere around $20k of "tuition remission" charged on fake tuition that nobody actually pays), and then the government is actually paying $60k/year due to overhead. This is frustrating.
Where's that extra $40k going?
I had a small desk in a shared office and I guess used some electricity for my computer, and there was custodial staff who cleaned that building. That's about all I can think of, and I don't think it cost the university $40k/student/year.
Campus amenities were paid for by a $600 mandatory fee which came out of my paycheck separately.
It's not paying for compute -- the computing resources I used were bought separately through grant money. If it was paying administrators, then how come every time I needed to do something it had to go through the same handful of extremely overworked people?
You could see where people would start to get annoyed.
Agree, it's complete crap. Until COVID, my faculty used to charge monopoly money for grad student tuituion fees (which is to say, these costs were always waived provided the candidate had good grades and funds could be found for a living stipend). Since COVID, we now charge many tens of thousands per candidate per year. Why? grad students cost SFA in terms of additional overheads and they are the folks who do the actual "work" that brings grant ideas to fruition. Its not like (undergrad) student numbers are down vs pre-COVID times either nor is there some huge sudden influx of graduate student.
"That pays for facilities and education programs and funding the department itself."
Buildings, rooms costs rent from each department.
Departments have expenses. Salaries to pay. Equipment to upgrade. Sometimes costs go over or researchers suddenly leave for whatever reason. The department they worked in is holding the bag on that.
Literally all that .57 or whatever part of every of every dollar you bring is going to the university to do with it how they like it. You're always working to make someone else more; that's why they hired you.
Now why is it going through the same people all the time (like grant management I presume?), Because grants have limited amount of money and people that fund research like to know the money was spent the way it was supposed to and the university itself likes to know all the money its researchers are bringing in and avoid conflicts of interest.
Edit: I also forgot to include, don't forget that like part of your pay you don't directly see, it's a about a third more, which goes to your benefits.
It is worse than that. You get paid 20k/y. You pay 20k/y in tuition. But only 40% of the grant money ends up going to this 40k. So it costs the funding institution $100,000 to pay you 20k.
Wut? First, why the self contradiction. If it is annoying then there is a problem, that point of annoyance.
And keep in mind, as also pointed out elsewhere, this is not how it is everywhere.
You might be content with the situation (which is understandable, as your wife gets her pay by it), but that does not mean there are no issues are that's how it is.
That's how it is in the bigger world outside of academia... Why would it be any different?
If you want to make the full value of your work then become your own boss. Though as others in the parent thread pointed out businesses cost about .5 or so for every dollar you make.
So it's either suck it up and accept that's how our society is structured and it happens to be an annoying aspect of it. Or we try redesigning society and discover different annoying problems/issues.
Though researchers love to buy big expensive sample fridges or other equipment when they're ending a project and need to spend anything left over in the projects funds.
Either spending is wasteful or it isn't. People don't work for free and the money is accounted for.
For example the Federal government and public universities negotiate a rate at which F&A (overhead, facilities and administrations) can be at. Then the Federal government, since they're paying for it, ask for receipts. The money is being accounted for.
I mean, on a per grant basis, the direct costs (and thus indirects) for a non-modular NIH R01, which is probably the most common form of grant, haven't changed since the Clinton era.
This varies a lot from faculty to faculty and from position to position. Some will work nonstop regardless of how many grants they have. (If you have grants, use them; if not, write them.) For pre-tenure research faculty, behaving this way is pretty much expected.
Others who are more senior and less research active may shoo their students off to internships (particularly in CS) and spend the summer doing very little. A lot of departmental committee work is paused, and people tend to know who is and is not available over the summer and will make arrangements accordingly. (I.E., don't assign Professor Tee Time to a committee that needs to function over the summer.)
I find this unsurprising. But I think the problem is upstream.
Getting into a top-tier CS program has gotten incredibly competitive, and college has gotten incredibly expensive. At Illinois (where I teach), the university admits 40% of its applicants. Our CS program admits 7%, and a demographic profile that bears no relationship to the broader university population. Relevant to this article, far more out-of-state students, and therefore students from wealthier backgrounds. (Because our out-of-state tuition is insane.)
So if lower-income students can't get internships, it's probably because it's hard for them to get into a decent CS program in the first place.
And from where I sit, the situation is even more frustrating, because we have thousands of data points showing that students drawn from the general university population can and do succeed in our CS courses—at least those required for the minor, which has become increasingly popular. But a lot of CS departments have felt increasingly besieged over the past few decades, as we've been swamped with students and frequently not been provided with appropriate resources. (Although there's also plenty of programs out there insisting on doing things that don't scale well, which exacerbates the problem.) So increasingly the "answer" is to clamp down on admissions, in ways that usually disproportionately affect certain populations.
We have a lot of ongoing BPC and DEI efforts in my department. But there's very little if any focus on admissions. I asked recently, and apparently we don't even know the demographic breakdown of our applicant pool.
Regardless of what aspects of diversity you care about, one of the biggest sources of inequity in CS today is in university admissions.
> So if lower-income students can't get internships, it's probably because it's hard for them to get into a decent CS program in the first place.
Sure. But every big tech company has relationships with universities, and has billions of dollars to invest, and would easily recover in the form of talent if they invested in education, even before the undergraduate level. I've worked at small consulting firms that were able to provide a few scholarships on much smaller budgets.
You're right to hold academia responsible as well, but a big reason that systemic problems like this aren't fixed is that the responsibility for fixing them is spread out across a lot of different groups. This lets any individual group point the finger at another group not doing their part in fixing the problem and use that as an excuse for not doing their part in fixing the problem. People are doing that all over this thread.
Kudos to you for doing your part and acknowledging university admissions as part of the problem, but I'd be less eager to hold big tech blameless.
The value of educational feedback drops rapidly as time passes. If a student receives immediate feedback and the opportunity to try again, they are much more likely to continue attempting to solve the problem. Autograders can support both; humans, neither. It typically takes hours or days to manually grade code just once. By that point students are unlikely to pay much attention to the feedback, and the considerable expense of human grading makes it unlikely that they are able to try again. That's just evaluation.
And the idea that instructors of computer science courses are in a position to provide "expert feedback" is very questionable. Most CS faculty don't create or maintain software. Grading is usually done by either research-focused Ph.D. students or undergraduates with barely more experience than the students they are evaluating.