orph
@orph_
human alignment // metacognition @mcognitivelabs // Eastern Roman ☧
2.5K Following 20.7K Followers
that's it I'm out
Paul Skallas, Landshark, Roon, The Last Psychiatrist, Sighswoon, Honor Levy, Tao Lin, Donald Boat, Bronze Age Pervert, Radfem Hitler, Rayne Fisher-Quann, Joe Weisenthal, Aella Girl
"it does not bode that well for the major model companies, oai included, that fundamental breakthroughs like this are coming from outside. we've become too much of a pressure cooker focusing on local incentives, studying narrow tracks under old paradigms, mostly doing engineering, rather than fundamental research, and implicitly hoping the next model will figure it out"
Show more
a few thoughts:
1. pretraining science is kind of like how people built quite good steam engines before they understood thermodynamics. “Science owes more to the steam engine than the steam engine owes to science.”-(L.J. Henderson). if nick is right, which i am pretty sure he is, progress points in the direction of more interpretability, and pretraining was "slouching towards monosemanticity". my guess would be that major breakthroughs in interp also advance capabilities
2. nick is a good friend of mine, safe to say my origin story at openai wouldn't have happened if i hadn't met him, and it's fascinating watching him operate and seeing the early drafts evolve into this. he's the most profoundly happy guy I know- not driven by guilt, activism, or religious terror, but by fascination. he of course understands the stakes of understanding neural networks, but it doesn't seem like the core reason he does it: he just loves the neural nets. afaict he's posting this from some beautiful hideaway in tokyo. there's a lesson in there for all the +EV maxers. i feel very grateful to have met him
3. i do think it's crazy how few people across the world were staring at neural nets growing: most have moved to a different level of behaviorist analysis, thinking about reward functions and test scores and whatnot. it does not bode that well for the major model companies, oai included, that fundamental breakthroughs like this are coming from outside. we've become too much of a pressure cooker focusing on local incentives, studying narrow tracks under old paradigms, mostly doing engineering, rather than fundamental research, and implicitly hoping the next model will figure it out
Show more
people who’ve never organized anything LOVE to misattribute conspiracy theories (overestimate coordination and underestimate incentives)
people who have never organised anything LOVE conspiracy theories
wow great paper
perseverative cognition (worry, rumination) may be protective in the short-term bc it generalizes & prevents from focusing on specific, emotionally-heavy episodes, but is costly long term bc it makes it harder to update from disconfirming & corrective experiences
Show more
the paper that singlehandedly 2× improved my understanding of neuroticism
oh sh*t of f*ck
BREAKING: A radioactive peptide that targets cancer cells and delivers radiation to destroy them has just succeeded in a Phase 3 trial.
that's also one of my best friends
Yale math post-doc, classically trained musician, could only ever study by/while smoking pot
recently learned one of the smartest most productive people i know smokes pot almost every morning before running 3-5 miles while blasting classical music
has had the same routine for a decade+. happily married, multilingual and has multiple kids too. never seen anything like it
Show more
rumination persists partly bcs ruminators hold positive beliefs about it ("this lets me be more prepared") so there's an epistemic distortion that reinforces the rumination beyond just the emotionally protective mechanism. this is true of many dysfunctional/obstructive behaviors
Show more
many early childhood adaptations/behavioral loops that end up obstructive in adulthood are directly downstream of overfit epistemics and/or carried over ontological distortions
but bcs ontology is pre-verbal & the very frame that colors your experience, you cannot observe it directly & have to seek it in the cracks
Show more
some of you may know that I lost my Dad to cancer when I was 7yo
when doing attachment repair exercises, where you are supposed to imagine a Perfect Attachment Figure, I usually go with God and my Dad
Show more
besides the cancer analogy, Tao's Math 2.0 slides are quite clear-headed & illustrate a path forward
Math 1.0 operated under the assumption of "proof scarcity", and most institutions and incentives were designed around this
Math 2.0, under proof abundance, has to expand the research frontier & decenter problem solving in favor of deeper human understanding, experimentation, and new principles and syntheses of research.
"The primary bottleneck to achieving “Math 2.0” is not technology, or even institutional culture. It is imagination."
Show more
was trying to grasp why seeing someone narrate/define the child's experience made me uncomfortable
this seems to be a relatively common experience with neurodivergent folks, where being observed/seen feels intrusive and/or impositional
when you are used to experiencing shame for behaviors that feel natural to you, or your actions are routinely ascribed false intent, you 1) grow distrustful of authority & attempts of authority to describe you 2) get used to masking to prevent judgment
so you end up experiencing loss of control when actively observed & described
a la Winnicott, you lose the capacity to be "alone"/your "true" self in the presence of another person, and learn to organize the self as an object of someone else's interepretation
(not saying this is what's happening here! just trying to understand my own emotional reaction)
Show more
Shocked at this video of Chris Barber with his kid because the kid goes through the emotion SO FAST. About as fast as I do since I started feeling lol. And it sparked this hypothesis: perhaps people are terrified of emotions because, somewhere in the dynasty, for some reason, someone didn't have the skills/context/privilege necessary to process something *fast*. So the idea was ingrained that when emotions come they stay a looooooooooong time. And if that is the case then any negative emotion is EXTREMELY BAD because it stays for soooooooo long. This would lead not just to you being avoidant of your own emotions but being avoidant of others' including your kids', hence replicating the pattern in a self-fulfilling prophecy sort of way! They don't get the necessary support to go through them quickly so they DO indeed take a long time to get over and thus having negative emotions IS indeed extremely bad!
Show more
have been offered this role a few times (or played this role as an advisor) but it only works out in very particular configurations
most companies are not structurally/culturally able to allow for the freedom aspect of this role (the other aspect being, ofc, reliable execution), so they end up either micromanaging or trying to box you in to feel secure
which is just frustrating for this particular type of person
Show more
literal hardest role to hire for rn and every ai startup wants someone
> chronically online (knows trends)
> has taste & can create (new media)
> understands ai & technical
> can execute with quality (operator)
such a small niche of folks who can do this
Show more
First they came for the software engineers and I said nothing for I was not a software engineer.
Then they came for the mathematicians and I said nothing for I was not a mathematician.
Then they came for the lawyers and I said "make sure you get all the lawyers"
Show more
v interesting that in the Expert Symposium that Stripe convened, the biggest hurdle to ending respiratory infections that people identified was a lack of funding & appropriate resourcing to solve the problem
technical feasibility was only second
Show more
