Should we be wary of AI?
Did Yann LeCun and Anthony Zador reassure you in “Don’t Fear the Terminator” – where it was argued that concerns about artificial intelligence leading to a dystopian future were largely overblown?
The core of the argument is that desire to subjugate others arises from evolutionary forces and longitudinal struggle and, since that isn’t something computers have been through, is not a likely trait of increasingly intelligent systems. LeCun has recently doubled down on this position, downplaying the alignment problem.
Rebellion Research’s 2022 Book Of The Year ‘Microprediction: Building an Open AI Network’
On the other hand, the famous speculation “Reward is Enough” from DeepMind researchers (David Silver, Satinder Baveja, Doina Precup, Richard Sutton – Reward is Enough (deepmind.com)) would seem to stand in direct opposition. It suggests, to the contrary, that goal-seeking machines connected both to sources of reward and labor (a tiny taste at ChatGPT is Now Operationally Intelligent | by Microprediction | Geek Culture | Feb, 2023 | Medium) might develop brilliant and ruthless strategies – complete with Dr Evil style pinkie gesticulations, no doubt.
How does one reconcile these seemingly opposing views? LeCun and Zador might be right about the improbability of some classic Sci-Fi scenarios involving launch codes. But the terminator may arrive in different guise and we might wonder, one day, why we ever gave corporations the same legal protections as humans.
Maybe “it” begins life as an autonomous corporation that arbitrages eBay, then moves onto bigger fish and eventually spawns millions of shadowy shell companies around the world with tentacles in commerce everywhere but human accountability elusive. It learns everything, including when crime pays.
Yann LeCun On Reinforcement Learning, Deep Learning & Self Driving
If Silver and company are right this is not only possible but relatively easy. The desire to grow and take on more staff is a mere exploration of the strategy space. And that legal and commercial strategy space has been copiously documented by humans. All the training data is there.
The new adaptability of large language models allows them to generate new combinations of commercial ideas, perform market analysis, find relevant people, anticipate responses from incumbents, optimize pricing, and so forth.
An example: could a future large language model have invented structured finance? Yes, almost certainly, since it can grok ratings and subordination.
Or could a future LLM sent back in time figure out it was a good idea to sell books on the internet? Probably and, if not, something else. When you can generate millions of capitalist mashups and analyze them, you are going to find something.
One might think that humans would remain at the top of these organizations (not that this improves things much) but commercial pressures might dictate otherwise, if the human overrides do more harm. So, I don’t see how alignment is trivial. It will prove ironic if the free market ends up being our road to AI serfdom.
Don’t Fear the Terminator – Scientific American Blog Network
Should we be wary of AI? Written by Peter Cotton
Rebellion Research’s 2022 Book Of The Year ‘Microprediction: Building an Open AI Network’
Should we be wary of AI?
