Courage
GPT-3 was possible because Ilya had the courage of his convictions; many folks had the compute, data, and money to run it--the missing piece was the courage to put it all together
The 4-minute miler principle suggests that once people see that something is possible, they’re more likely to try doing it; i.e., after the first person ran a 4-min mile, hundreds did it
The secret to making the atomic bomb was the knowledge that it’s possible; Americans knew that
In some ways, beliefs are self-fulfilling because if you don’t believe in something, you won’t even bother trying
Enterprise software
Most software is built for internal use; i.e., Coca Cola employs thousands of software engineers but doesn’t have any external software out there
Bottoms-up go-to-market may work for a dev tool with a freemium model
An excellent question to ponder on is why Salesforce has such a massive monopoly on CRM systems?
Invention
At Xerox PARC, Butler proposed two rules: 1) to always use the stuff they were building, and 2) to engineer for 100 users
The page effect; the idea for GUI came from imitating physical office space to make it easier for people to understand what a computer can do--this had a downside that we got stuck “in the page” and don’t really appreciate the full capabilities of the medium
The invention of a new medium usually follows by years of insufficient adoption where people are trying to figure out its full capabilities; i.e., the first TV shows were just like radio shows but with the guy on them
Insight through tinkering is a powerful tool for invention; Picasso used to say that you can’t see the painting until you begin drawing
Scientists learn from error; the most important thing after your hypothesis fails is to ask, “what was wrong with my initial assumptions?”
Spatial thought is the foundation of abstract thought; we use the same areas of the brain (i.e., grid cells in the entorhinal cortex) to understand how connections between objects in space and between concepts and ideas -> by mapping our thinking spatially, we utilize our spatial cognition superpower, that’s why paper works so well for drafting ideas while Word doesn’t
Misc
When launching a product, it’s better to err on the side of speed than perfection; that’s because a bad product for a great market will do just fine, but a perfect product for a terrible market will fail
When there’s a waitlist, launch quickly; Pebble almost died when they delayed the watch launch for 8 months due to production difficulties; despite thousands of people requesting access to the new watch, nobody wanted one after eight months
If the scaling hypothesis works, we’ll see more and more advanced meta-learning applications in the next few years, i.e., what’s gonna happen if we increase the number of parameters for GPT-3 10x?
Spreadsheet creators are programmers. They just don’t yet know about it; they’ve been deceived into thinking that programming is typing green symbols into the black terminal box
Relational databases are harder to understand than spreadsheets because of invisible connections
Cults always start against something; there has to be a common enemy for people to unite