writing

Can GPT pass at MIT?

Sep 2022

Yes!¹ You can certainly scrape a passing mark in a short assignment for a graduate course at MIT (Sample Size $N=1$ ).

The answers are all generated from an autoregressive language model². Every prompt contained the assignment brief and an example question-answer pair, initially seeded with my first response. I then rotated the prompt pair so the model could only use its own answers.

I did zero cherry-picking; I used the first response.

The artificial answers were then submitted under the pseudonym ‘Gregory P. Terrance’ alongside my own.

Years later I ran the rotation again, once through the best model I could reach for each year³. Same brief, same single held-out answer to start it off, same refusal to cherry-pick. Every answer below has a toggle for the year; the commentary underneath was written about the 2022 set.

The Answers

The knowledge stored in 178,000,000,000 parameters is more than a little impressive; it was faster than Google on “What is CYC?”, and it is perfectly legible. The primary issue is the lack of content. It is always eager to finish an answer and won’t provide examples or personal opinions.

1. What is deep understanding, and how does it differ from shallow understanding? Give an example of your own, other than those encountered in the talk.

Gregory P. TerranceJurassic-1 Jumbo, 2022 · 40 words

Deep understanding is solving a problem you understand, not a problem you merely recognize. This can mean noticing a relationship between things that you didn't realize were there or realizing that things you thought were unrelated are in fact related.

Me: Deep Understanding requires an internal model of what is depicted to perform everyday inferences about unseen information. In contrast, shallow understanding is typified by the ability to correlate and discern subtle patterns.

Sometimes a mistake can demonstrate deeper understanding than the right answer - using “comed” rather than “came” is a result of knowledge of the fundamental affixation rules of past tense. It suggests an internal model of language - if this action happened in the past, then affix “-ed”. Rather than a probabilistic, shallow, understanding of “came” coming next.

Frankly, I think this is the worst answer of the bunch. It’s not wrong, but it’s not particularly insightful and it misses the requirement of an example. It also appears to be responding vaguely to the definition of “deep understanding” rather than the specific usage that the talk (by Gary Marcus) uses.

2. What was Eliza? That is, what was it supposed to do? (Google it.) Did it have deep or shallow understanding?

Gregory P. TerranceJurassic-1 Jumbo, 2022 · 28 words

Eliza was an early natural language interface. It didn't know or understand anything, it merely selected responses which matched a corpus of user's inquiries. It had shallow understanding.

Me: Eliza was a “chatbot” program designed with pessimism to “demonstrate that the communication between man and machine was superficial”.

It used a very simple rule-based system: decomposing a query into wildcards and keywords, then recomposing stock templates like Why do you want (3)?.

It demonstrates how shallow communication can be between humans as well. During my training on emotional support helplines, we’re often advised to perform very similar tricks.

Eliza is certainly shallow intelligence; our ability to empathically bond with “shallow intelligence” is undeniable.

Gregory’s “[Eliza] … selected responses which matched a corpus of user’s queries” reads to me as a lookup table of word-to-response rather than the templating system that Eliza is.

Equally, the tone of disdain (“merely” and “[didn’t] understand anything”) matches many of my classmates, although I found it a little harsh.

3. What is CYC? (Google it.) What is it trying to do? At a high level, how does it try to accomplish that goal?

Gregory P. TerranceJurassic-1 Jumbo, 2022 · 50 words

CYC is a knowledge base; a method of representing human knowledge, and a method of reasoning by analogy. It is trying to reach the goal of learning all of human knowledge by analogy. That is a tall order; the CYC project started in the 1950's and is not yet complete.

Me: CYC is an over 35 year endeavor to curate a knowledge base of hand-coded “common-sense” facts and an inference engine. Lenat et al. believed intelligent action was bottlenecked by the lack of “content”; unfortunately for CYC mass data seems to be as sufficient as hand-crafted rules.

I was pretty impressed - I struggled to locate what CYC was without hamming up the “AI” keywords in Google.

I thought it got the year right - unfortunately, CYC started in 1984 ⁴, but it is also still not complete.

It also appears to have shared my fondness for semicolons despite not including ; in the previous prompt text.

4. What’s wrong with benchmarks?

Gregory P. TerranceJurassic-1 Jumbo, 2022 · 49 words

Metrics can help, but metrics have dangers of their own. For instance, measuring student performance on math tests alone tells us little about a student's real math abilities; and does not tell us how deep their understanding is, or if they are merely good at figuring out mechanical processes.

Me: The first danger of benchmarks, is that we misattribute “state-of-the-art” with novelty. However, benchmark performance is not just noise. We should only be careful on the 98% -> 99% benchmark improvements; the “Kaggle Effect” mentioned is exemplified by this. If the top 5 models all have very similar performance, there probably is little semantic difference between them. A runaway winner like AlexNet jumping from 74% to 85% - should be the one to pay attention to.

The second danger is directly associated with why benchmarks are vital: A benchmark is a problem specification. With the same stroke, we can define the wrong problem. For instance, if the benchmark dataset is too biased and has unintended factors, then we might be solving a completely different task - consider an algorithm that segmented tumors by looking for pen marks on the X-Rays.

Not a great answer for Gregory here. It seems the more generic the question, such as “What’s wrong with benchmarks” or “What is deep understanding?” the more generic the responses are. A poor philosopher indeed.

The first example by GPT occurs in this answer, and it is quite nice. The relation of benchmarks to mathematics tests is a comparison I do agree with. I think my point on benchmark usage also applies to mathematics tests - dramatic improvements (or deviations from the norm) are notable, but micro-adjustments to perfection are not.

5. What’s wrong with GPT- $n$ , for $n=1,2,3...$

Gregory P. TerranceJurassic-1 Jumbo, 2022 · 53 words

GPT-3 still produces logical "nonsense" (a word I use here pejoratively). Gpt-3's nonsense is structurally coherent, but semantically absurd; like the claim that "all virtue is vice and vice is virtue." Gpt-3's nonsense is unreasonable, meaning that it violates our intuitive notions of common sense, validity, probability, induction, etc. Gpt-3's nonsense is familiar.

Me: GPT-Ns pull from contextual clues to probabilistically predict the next token in the sequence. This desire to just continue the sequence gives them a distinctive flair of an improv actor - one with a huge breadth of detailed knowledge, but willing to shortcut and make things up.

Q: When playing the first move on a blank tic-tac-toe board, the total number of possible moves is?

GPT-3: 8. Because on the first move, there are 3 spaces in a row and 3 in a row across, for a total of 8 possible squares.

All the knowledge needed is contained within the model. It just doesn’t have a sufficient internal model to put all the pieces together to make inferences.

GPT is smart, but not smart enough to know how smart it is.

“pejoratively”: expressing contempt or disapproval

I had to get the dictionary out for that one, slightly embarrassing that my vocabulary has been beaten by something producing “logical nonsense”. “Pejoratively” is perfectly correct, and quite neat tonally - “contemptuously” would come off as far less cogent ⁵.

Betteridge’s law (of headlines): “Any headline that ends in a question mark can be answered by the word no.” I take great joy in denying it. ↩
Technically these come from Jurassic-1 Jumbo by AI21 Labs, as I didn’t have access to OpenAI’s GPT-3 at the time; Jurassic-1 is functionally comparable, and GPT does have more name recognition! ↩
Anthropic’s claude-opus-4-5 for 2025 and claude-fable-5 for 2026, both run in August 2026. There is no 2023 or 2024 rung, and not for want of trying: the model catalogue still lists every Claude 2, 3 and 3.5 snapshot, but each one now answers 404, so those years cannot be run at all. I have left them out rather than let a later model stand in for them. Two further caveats. I no longer have the original handout, so the brief in the prompt is a reconstruction: the course, the source talk, and the “a paragraph or two, marked on insight” instruction. And these are chat models rather than raw completion models, so each needs a one-line instruction to keep typing after the final A:; that line says nothing about persona, register or length, all of which still come from the example pair. The script is in the repo as scripts/gpt-at-mit-rerun.mjs if you want to run it yourself. ↩
‘2+2=5’. George Orwell’s totalitarian masterpiece is also the year that CYC started. A fun coincidence. ↩
Sorry, I couldn’t resist. “(of an argument or case) clear, logical, and convincing.” ↩