Rendered at 12:58:09 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
aitchnyu 1 hours ago [-]
Around 2011, my exposure to AI was like Drools planner, the predecessor of Timefold. It was to generate an ideal timetable. It used an n^2 memory heavy Rete algorithm, a scripting language in JVM to write rules with a terser syntax and simulated annealing to explore/choose states that obeyed constraints. In college AI was fancy algorithms and languages with terse syntax.
A few years later I saw
- neural network projects which can do basic image recognition
- a paid smartphone app that can count objects instantly
- determine the mood of an online comment after you trained on a sample
I never imagined a tool that packages all knowledge and requires no training on my part.
Panzerschrek 7 hours ago [-]
Running GPT-2 after it's trained wasn't as computationally-easy as nowadays. So, you not only need the best supercomputer in the world to train it, but also a datacenter with similar computational power to run enough instances to make it useful.
bearsyankees 17 hours ago [-]
also the internet wasn't as robust so less training data ofc
simianwords 3 hours ago [-]
My take was that we didn’t have these 4 things
1. Someone with enough conviction of end goal which is agi
2. That someone with enough capital or agency to pursue it
3. Enough capital to explore all options - at that time we didn’t know about transformers
4. Enough smartness to know what ideas to not purse and which ones to prioritise
As things got cheaper, it allowed more people to explore.
fuzzfactor 16 hours ago [-]
>There was no grand plan that led to LLMs being created. People just did a bunch of things step-by-step. With so much uncertainty, progress is more “little hops” rather than grand journeys—technology is more “evolved” and less “designed”.
This pretty much sums up the whole article, and it makes perfect sense.
So much was going to happen anyway that you really do have to look away from the technology to answer why.
I would say GPTs didn't come earlier because sama didn't have enough high-dollar contacts yet who would extend their confidence to him.
And then what you get is little hops done big because that's whats already familiar to the big money owners.
So LLMs are first out of the gate to reach impressive capabilities, but something completely unfamiliar that may be superior due to purposeful design including fundamental principles will have to lag behind until a very big change would become possible.
A few years later I saw - neural network projects which can do basic image recognition - a paid smartphone app that can count objects instantly - determine the mood of an online comment after you trained on a sample
I never imagined a tool that packages all knowledge and requires no training on my part.
1. Someone with enough conviction of end goal which is agi
2. That someone with enough capital or agency to pursue it
3. Enough capital to explore all options - at that time we didn’t know about transformers
4. Enough smartness to know what ideas to not purse and which ones to prioritise
As things got cheaper, it allowed more people to explore.
This pretty much sums up the whole article, and it makes perfect sense.
So much was going to happen anyway that you really do have to look away from the technology to answer why.
I would say GPTs didn't come earlier because sama didn't have enough high-dollar contacts yet who would extend their confidence to him.
And then what you get is little hops done big because that's whats already familiar to the big money owners.
So LLMs are first out of the gate to reach impressive capabilities, but something completely unfamiliar that may be superior due to purposeful design including fundamental principles will have to lag behind until a very big change would become possible.