Quizlet’s AI study tools think I’m a bad student

Quizlet, a tool that helps personalize studying for students, recently released a set of new AI-powered features.

Quizlet is one of the many educational platforms embracing generative AI to facilitate learning, despite consternation from teachers when ChatGPT first burst into the scene. I no longer have assignments to read, but to do my job, I have to read tons of news articles, reports, and research papers fairly quickly. I wondered: can I use Quizlet to make my job easier and test how much of my beat I actually understand?

Oh man, it didn’t turn out well for me.

Quizlet released several AI-powered tools on August 8th. Memory Score schedules reviews and tracks scores to help users remember the material. Quick Summary takes key concepts from readings. Brain Beats turn flashcards into songs — a far cry from the mnemonics I used as a kid to remember the order of operations. And Q Chat lets students talk to a ChatGPT-powered tutor. Before adding generative AI, users had to manually add questions to custom flashcards and quick quizzes.

Quizlet Plus users, who pay $7.99 a month or $35.99 a year, have unlimited access to the new features.

Quizlet gave me early access to one of the features called Magic Notes. (Sadly, I didn’t get to try the song generator.) Magic Notes lets users upload or copy-paste text, and Quizlet summarizes it, then offers an outline, sample essay questions, creates flashcards, and even puts together a practice test.

I decided to plug a story by The Verge’s Makena Kelly into Magic Notes. This story is frequently open in my browser as I tend to link back to it often, which means I’ve read it more times than I can count.

After pasting the story, Magic Notes generated a bullet-point summary of the article’s main ideas. It offered me a sample essay prompt to discuss the importance of AI companies investing in AI safety, and then there was the practice test.

I got a solid D on the practice test, answering nine of the 15 questions correctly.

In my defense, some of these questions were nonsensical. Quizlet offered me a list of terms with options for “definitions” and asked me to say whether they matched. (Terms and definitions were sometimes but not always phrased in the form of a question, like on Jeopardy.) For this one, it asked if the definition “Open-sourcing it for researchers and most commercial use” matched the term “What is watermarking in the context of AI-generated content.” Okay, odd phrasing because that’s not how I would define watermarking. The correct answer was: “What did Meta announce about their language model Llama 2?” I’m sorry, but what?!

It did get some questions to mostly make sense. Quizlet prompted me to define AI voice assistants. The answer choices were “What is the importance of responsibility in AI,” “What are measures related to cybersecurity,” “What is the concern regarding enforcement of AI commitments,” and “Which type of AI-generated content would not be covered by watermarking.” The correct answer is the last one. This one was a gimme.

Next, I put in one of my stories, thinking I should do better because I wrote the thing, and I feel like I understand what I wrote. Somehow, Quizlet believed the phrase “fostering trust” was an actual term I was looking to define in the story. It isn’t. I simply needed a synonym for “building trust.”

Quizlet does say the technology isn’t foolproof and will occasionally return incorrect or problematic answers, so make sure to continue guiding your child while using the features. Quizlet built its new AI features with various generative models, including GPT-3.5.

Students were some of the first to embrace generative AI and ChatGPT. This prompted panic from educators who worried kids used the tools to cheat. School districts banned access to ChatGPT. In May, New York rescinded its ban.

Since then, generative AI has become more ubiquitous. Like every other sector out there, education-focused platforms want to take advantage. Quizlet is not the only kid-focused startup exploring generative AI for students. Smartphone for Kids maker Pinwheel announced PinwheelGPT, basically ChatGPT for kids with less complex vocabulary. Inside Higher Education said AI in education will grow to a $25 billion industry by 2030 from $2 billion in 2022.

Magic Notes did do a pretty good job summarizing the article, and maybe a news story isn’t the best way to test out a study guide. Some news articles summarize an event or talk about company plans; not everything is defined. News assumes a reader already knows the context stories talk about.

So I put in Shakespeare Sonnet 116, mainly because I studied it in college a decade ago. Magic Notes did better, and more importantly, I did better in the practice quiz. The questions made sense. For example, who is the author of the sonnet? William Shakespeare, of course. What are tempests in the sonnet? Obstacles or turbulent times.

But the quiz still didn’t do well with ambiguity. It asked me to choose the best definition to match with the phrase “Love as an unchanging and unwavering force.” The two choices were “What is the meaning of ‘marriage of true minds’?” and “What is the central theme of Sonnet 116?” I said it was the marriage of two minds; the correct answer is the latter. That is a valid answer, even though I can argue why my choice was right. But I am a literature major and believe literature should be open to interpretation.

Quizlet is a tool for students to get an overview of the topics they discuss in class so they have a quicker grasp of the material before a lecture. It is not a place to discuss the choice of nature and time as imagery and relate it to Shakespeare’s life. These questions were fine for that — but, in general, pretty basic.

I played with Quizlet’s Magic Notes for a few days and kept pasting in different texts, from research paper abstracts to opinion pieces, to varying degrees of success. It isn’t perfect, though I could see how students can find these AI-powered study tools helpful. It breaks down concepts and saves time when reading denser material.

Then again, does no one use CliffsNotes anymore?