2023 in Reviewlife learning progress
03 Jan 2023
15 minute read
Table of Contents #
- General Goals
- Books read
Doing yearly recaps helps me observe my progression in many areas, and keeps me accountable for next year. I share things I’ve done, learned, read, and often sprinkle in some introspection on recurring motifs that have animated my year. I also included a best of twitter.
General Goals #
These are not directly learning/project-related, just random stuff I’d like to do/did.
Last year’s goals #
- get into a university I’m happy with (10/10)
- learn more piano (7/10, didn’t do enough but am doing a lot since 2 days ago)
- go to more museums and concerts, if COVID allows (8/10)
- general possibility expansion, as I discover new paradigms and ways of thinking or creating, as I have in the last two years. (10/10, maybe did do much of this as opposed to object level work)
2023 Goals #
- publish a paper on theory of deep learning / model interpretability (ambitious)
- get an interesting/meaningful internship this summer
- reach a new level of math ability (also ambitious, but there is lots of space to grow)
- listen to more music, have stalled in college, want to publish an inventory of songs that I resonate with
- publish at least one substantial coding project (wrote too much private code this year, or didn’t launch what I was building)
- write >= 5 blog posts
- learn to accept my decisions and their results
- get back on track with my personal projects
One of the main themes this year was college, applying, choosing where I want to go, and then actually doing my first semester. I decided to go to MIT, instead of my second choice which was a good preparatory school in Paris. I have trouble making decisions and honestly hesitated a lot on this one, both options having drawbacks and advantages. I think I need to learn to accept the decisions I make and become less anxious about optimization errors of past decisions, but I’m not sure how.
The first semester of college has been interesting. I’ve met many very cool people and have started doing some theory of deep learning research with a grad student. My courses had high variance in quality, but I think next semester will be more consistent. The transition to the US was hard in terms of moving to a new country far from friends/family. I had spent time in the US but it took getting used to.
Thread with first impressions:
Given that my first class starts tomorrow, here are some thoughts on my experiences at MIT so far, with orientation / dorm discovery and other various pre-actual work things.— Uzay (Paris) (@uzpg_) September 7, 2022
What I wanted to do #
From last year:
- launch Espial, build Roam and Archivy integrations (4/10): totally changed directions with the project and built something better, but didn’t launch.
- improve Archivy, democratize use to non technical users and investigate federation. (4/10): Improved archivy but not enough/not in these ways.
- improve music organization on spotify, ideas similar to this thread. (0/10)
- Abstract goal: find entirely new domains / directions I want to build towards (10/10): I am much more confident about building things with ML now and can do cool things here hopefully
What I did #
I did ML research this semester with Eric Michaud in the Max Tegmark lab! Based on hypotheses set forth in In-context learning and induction heads and A mechanistic interpretability analysis of grokking, we ran experiments to get a better grasp on the relationship between phase changes (rapid decreases in test loss) over time and the formation of algorithmic circuits in models.
Doing research in an official context for the first time was really interesting. It was cool to see my internal models of how these neural networks learn get more and more accurate (I think?) although there’s still a ways to go. I’m happy with the paper-reading/experimentation/ML engineering skills I’ve been building here.
I worked a lot on my project Espial but have yet to actually publish anything, sadly. I think this is because of a set of things, notably:
- college/research kind of took up a lot of time
- I have some mental block in terms of using money to experiment with technical projects, which isn’t really warranted, and it turns out this is bad when you’re building something that needs compute
- I had some acquaintances build similar things and got a bit demotivated with the project, even though I do have a working tool for myself
However, I started working on it again with a friend recently and am quite excited/hopeful that I can get something out there related to this in 2023. Time will tell!
- Various improvements and additions to Archivy
- Playing around with integrating ChatGPT into my knowledge process
- I’ve been working on a mentorship/wiki project about things high-schoolers who are curious and want to learn can do if they want to be more agentic, but it’s not out yet
What I want to do #
- publish a paper (or at least submit)
- actually publish something related to my work on Espial
- code a functional programming project, not sure about what, maybe a clean/intuitive math markup language using AI
- want to build a better interface to my music, AI could be powerful here
- publish agency project
I took 5 courses:
- Algebra I (18.701) (10/10): this course used Artin’s Algebra and covered linear algebra and abstract algebra, also making bridges between geometry and algebra. This course was extremely good. The lectures were captivating, the problem sets were very well thought out and the professor was very kind. Will be reviewing this in the next few weeks. To be honest, this class was intense. I spent lots of time on the weekly problem sets and am happy to have put in the effort. I think there’s still a lot for me to learn though, and I’m unsure whether the French preparatory school format might have prepared me better in terms of grinding math.
- Macroeconomics (14.02) (8.5/10): This course was quite interesting, the professor constantly made efforts to link ideas back to current events, and emphasized high-level ideas that are good to understand. Problem sets were often too computational however.
- Multivariable Calculus (18.022) (7/10): I already knew some multi so I felt like I could have maybe gained more from another math class, but lectures were good and also covered interesting ideas in analysis. Problem sets were average/not that interesting.
- Intro to C and Assembly (6.190) (7/10): I didn’t learn that much from the content I think, but the labs where we played with a micro-controller were fun. Weekly exercises weren’t very interesting until we got to assembly.
- Intro Chemistry (5.111) (5/10): My recitation leader was extremely kind and helpful but the teaching quality was bad from the professors and it was hard to build intuitions for things, which I hate. I basically spent the minimum effort on this class, didn’t go to lectures and spedran problem sets.
Overall, my takeaways here are to be very thoughtful about which classes I take, and to shop for classes at the beginning of the semester.
What I wanted to learn #
Didn’t do that well on the stuff I wanted to learn here, in part because I was quite busy with intense French graduation exams in philosophy/literature. More for next year!
From last year:
- NLP: transformers and text generation / question answering (10/10)
- statistics (3/10)
- linear algebra (9/10)
- cryptography (6/10)
- competitive math (7/10, practiced a bit, got HM in national olympiad but didn’t grind)
- game theory (6/10)
- turkish (0/10, didn’t get much better)
- physics: review notes on classical mechanics, and learn QM and quantum computation (read quantum country), (2/10)
- education: I need to learn by writing, and publish the drafts I have lying around on this. (0/10)
- Haskell / Rust: implement practical projects (0/10)
- cybersecurity (6/10, practiced a bit)
- economics (8/10)
What I learned #
- NLP: I did the coursera NLP specialization. [Notes]
- started doing research in deep learning theory and language models
- I worked a lot on Espial, my AI system for knowledge discovery, even if I haven’t published anything
- engaged a lot with literature on the alignment problem, learned about Eliciting Latent knowledge and model intepretability. Wrote up a roadmap of links
- read lots of papers, I feel much more comfortable when opening up a new paper now
- competitive math and physics: did some grinding for national olympiads, could have done more but it was a fun and worthwhile exercise
- cybersecurity: did some competitions and learned a bit here, including CSAW CTF finals
- philosophy: 4 hours / week in high-school, I appreciated the subject and read L’Étonnement philosophique (Philosophical astonishment) (notes
- I would like to keep condensing and reviewing my notes on that book. I’m still not sure what I have to show for this time, but I don’t regret it
- school stuff:
- history ~ interesting stuff about latter half of 20th century
- physics: more optics, electricity, mechanics
- graduation exams took up a LOT of time here
- Svelte/Web Extensions: played a lot with these for Espial, I really like Svelte, but web extensions are annoying to develop
- asking good questions about people/observing things: I think I got better at this during a camp I went to this summer, and am happy about it
- courses above
What I want to learn #
- I want to get back to coding in languages I really like, for example Crystal or Rust/Haskell. I want to get comfortable doing functional programming for practical projects
- get to a new level in terms of manipulating abstractions in math
- cool physics next semester
- reinforcement learning
- get better at cooking good meals quickly
- get a deeper understanding of linear algebra
- more philosophy, on ethics
- get to a next level in terms of asking good questions about people/observation
- ~concurrent programming, maybe
What I wanted to write #
- public book reviews: I need to figure out how to do these well. Currently have drafts for 3 book reviews in my files (The Selfish Gene, The Unconsoled, and GEB)… (0/10)
- an overview of the french education system: I’ll explain its structure & specifics, and tie it into a general comparison with other education systems. (0/10)
- My vision of how AI and web content can change knowledge management. I have a written draft, and am just waiting to launch my related project before I share (Current title: A New Approach to Knowledge Management, using Natural Language Processing and the Internet’s abundance of content) (10/10)
What I wrote #
I only wrote two posts on my blog this year, coming in at ~5.5k words:
- How I learn (math)
- Using AI and web content to improve organization and discovery of knowledge - Pretty happy with this blog post, I think I accomplished something cool here and enjoyed doing research on AI + knowledge management. Hopefully I can push this further this year.
- Last year review
I have too many drafts and struggle to make time for writing long-form posts I’m happy with. I probably need to be less perfectionist and just output stuff.
I’ve started a newsletter where I post semi-frequent updates (in theory it was weekly), and am quite happy about what I write up on there (link). I wrote 4k words on there. I really enjoy this format, where I kind of speak my mind and introspect, on a more regular basis than with what I do here.
Books read #
- L’Étonnement philosophique - Jeanne Hersch (9/10): Great summary of philosophy I started last year, finished in 2022. Notes
- The Precipice: Existential Risk and the Future of Humanity - Toby Ord (9/10): describes many potential futures of humanity, and risks that stand in our way. Recommended. Detailed thoughts here
- Why Nations Fail - Daron Acemoglu (8.5/10): I didn’t read it in its entirety, but the base idea was really interesting: world inequality can be in lots of cases traced down to institutional differences — differences between institutions that don’t give incentives/means for their citizens to grow and improve their lives, versus those that do. Comes with detailed historical analysis. I liked it but don’t think you need to read it all to get the gist.
- Mathematics: A very shot introduction - Timothy Gowers. Neat introduction of many principles, but quite basic and didn’t learn much. Would give a low rating but don’t think it’s bad.
- Letters to a young poet - Rainer Maria Rilke (8/10)
- La Promesse de l’Aube (The Promise of the Dawn) - Romain Gary (9/10): I really liked this weird mix where the author wrote a semi-fiction, semi-real autobiography. My favorite parts of the book were related to motivation and seeing the main character go down
- Lolita - Nabokov (9/10): this book was really weird. I had to put it down several times because I was disgusted by what was being described. But it’s well written and pretty captivating.
- Si c’est un homme (If it were a man) - Primo Levi (8.5/10): Stunning book about Primo Levi’s experience in the holocaust. Sometimes I didn’t love the writing style, but it was a striking book and reminded me of how easy it is to forget the pain the world has seen.
- Kafka on the Shore - Haruki Marukami (8.5/10): I really liked this book. It was a wild fantastic ride, but one that would be hard to explain properly. Recommended.
- Manon des Sources / Jean de Florette (7.5/10)
I’ve been much more social this year generally, and quite extraverted. My MBTI has gradually shifted from I to E, and I’m becoming much more proactive in terms of reaching out to people. I think this is good: people are super interesting and I’ve been making many good friends. I also believe it’s important to leave a big amount of time to actual object-level projects and deepening my understanding of the content I want to master. I want to find my balance regarding this in 2023.
I used Twitter a lot more this year, meeting and scheduling one-on-ones with many cool people, especially in the knowledge management space. Here are my most interesting tweets this year: (note: I’m mostly putting this here for myself because Twitter search is bad and I want to be able to compare my thoughts on the scale of years)
- Thoughts on ChatGPT
- Thread asking for advice to a younger self
- MIT experience
- Thinking about sad music
- Blog drafts
- Espial demo
- Howard’s End passages
- Thread about foundational authors
- Processing the idea of existential doom
- Stuff I want to read
Overall, 2022 has been a year of heavy change. I’ve kind of been all over the place (physically and in terms of my thoughts), and now am living in a new country. I worked a lot on my projects but didn’t publish much yet, however I look forward to making updates soon. I’d say it’s been a year of discovery, in terms of people and ideas. Many of those discoveries are already precious to me, and I hope the others flourish into more things I can be grateful for.
Have a happy new year!
Subscribe to the blog's Valuable Entropy newsletter: