"You can see the future first in San Francisco." OpenAI alumnus Leopold Aschenbrenner pours out a worshipful paean to the AI future, just as if GPT wasn't topping out its S-curve

situational-awareness.ai Introduction - SITUATIONAL AWARENESS: The Decade Ahead

Leopold Aschenbrenner, June 2024 You can see the future first in San Francisco. Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters. Every six months another zero is added to the boardroom plans. Behind the scene...

28 comments

MBA idiots and Economists consistently overestimate the abilities (and the underlying nature) of AI, because the output of AI is so much like how they speak: eloquent, confidently wrong, unconcerned with ground truth. They see themselves in AI, and they consistently overestimate themselves and their abilities.

How people expect AGI before self-driving cars is just ridiculous.

i can't find a hole in this argument
- they phrase it so your brain will think otherwise, but I just want to point out the maybe obvious so nobody else has to do a double-take at the quoted paragraphs:
  their example of what an ultra-intelligent AGI could do is of a human speedrunner using a glitch to beat Minecraft in 20 seconds. this asshole is taking something humans are excellent at and saying “but what if AI could do this, and also what if Minecraft was real life?” this is literal baby shit. like, even tool-assisted speedruns are the product of a shitload of human research into the problem space, and the tool’s just executing impossibly precise game inputs programmed, again, by a human. this is another space where AI sucks compared with regular human effort.
  and speaking of which, does anyone remember the early OpenAI and Google marketing where they had an LLM play pac-man or some shit at supposedly superhuman levels? can anyone dig up an outcome for any of those in the form of a record on any credible speedrunning site or during an event? cause speedrunning has a ton of categories including stuff like dog-assisted runs, where you train your dog to play the game, and it’s all considered valid as different forms of skill applied to the game. the one thing you can’t do is cheat, and they’re very good at verifying runs (ie, you must be provably using the method you claim, and you can’t splice video together or use emulator cheats to achieve a better run). so where’s the verified LLM speedrun records?
  
  @self He hasn't even caught up with what Vernor Vinge was talking about in his 1993 paper on the Singularity, which the current crop of starry-eyed singularitarians seem not to have read: https://accelerating.org/articles/comingtechsingularity
- Me, as a jock bully: Well of course these nerds would find finishing in 20 seconds an achievement
- The error bars here are, of course, extremely large. Still,
  
  Analyzing our data we conclude with 95% confidence that within a decade the Dyson Sphere Any% TAS time will be reduced below 55 seconds (± 1E10 years).
  
  They're catching on that "big if true" is being recognized to mean "this is bullshit" so are trying to compensate by using more words.

Given the all-consuming desire of Silicon Valley to accelerate us into a mad max style dystopia, the most foreshadowing thing about SF has gotta be all the (human) poop on the sidewalk.

By the end of the decade, American electricity production will have grown tens of percent
..... don't worry guys AI will totally save us from climate change! /s

Arrives like a wet turd hitting the bottom of the bowl at HN:
https://news.ycombinator.com/item?id=40576324
there are some promptfondlers trying valiantly to defend it but most correctly identify the author as a kid who doesn't know shit.
- Some scientific breakthroughs were memes at first
  name one.
  
  Does the "cats are a liquid" paper count?
- I'm reading Feynman's lectures on electromagnetism right now, and GPT-4o can answer questions and help me with the math. I doubt that even a smart high school would be able to do it.
  Ten bucks this guy hasn’t double-checked anything his chatbot told him but accepted it as truth because it used big words in grammatically coherent ways.
  
  Electromagnetism is a standard subject covered in a bajillion books, so the training set is probably full of repeated explanations of the basic examples. That sounds like an excellent recipe for "AI" bilge-water that is just coherent enough for a student to miss where it goes wrong.
  
  And here I thought it was easy to find high schoolers that are both wrong and sure of themselves.

@dgerard "From GPT-4 to AGI: Counting the OOMs" wait this dude is wetting his knickers about Out Of Memory errors?
- No no, it's "order of magnitudes". It's like "surgeons general."
  
  Counterpoint: in Afrikaans (and likely Dutch I guess? Haven’t checked) “oom” is “uncle”
  You may now proceed to enjoying this little diversion as much as I do

Scott Aaronson says that Aschenbrenner displays "unusual clarity, concreteness, and seriousness".
That's it, that's the joke
- You know for a blog that's on its face about computational complexity, you'd think Scott would show a little more skepticism to the tech bro saying "all we need is 14 quintillion x compute to solve the Riemann hypothesis"

Ivanka Trump just tweeted this

Great job with paean. Thanks for teaching me a new word.

28 comments