Musings on whether the “AI Revolution” is more just like the printing press or crypto. (Spoiler: it’s neither.)

I’m not nearly the primary person to take a seat down and really take into consideration what the arrival of AI means for our world, nevertheless it’s a matter that I still find being asked and talked about. Nonetheless, I believe most of those conversations appear to miss key aspects.

Before I begin, let me provide you with three anecdotes that illustrate different facets of this issue which have shaped my considering these days.

1. I had a conversation with my financial advisor recently. He remarked that the executives at his institution were disseminating the recommendation that AI is a substantive change within the economic scene, and that investing strategies should regard it as revolutionary, not only a hype cycle or a flash within the pan. He desired to know what I believed, as a practitioner within the machine learning industry. I told him, as I’ve said before to friends and readers, that there’s plenty of overblown hype, and we’re still waiting to see what’s real under all of that. The hype cycle remains to be happening.
2. Also this week, I listened to the episode of Tech Won’t Save Us about tech journalism and Kara Swisher. Guest Edward Ongweso Jr. remarked that he thought Swisher has a pattern of being credulous about latest technologies within the moment and changing tune after those latest technologies prove to not be as impressive or revolutionary as they promised (see, self-driving cars and cryptocurrency). He thought that this phenomenon was happening together with her again, this time with AI.
3. My partner and I each work in tech, and commonly discuss tech news. He remarked once a couple of phenomenon where you’re thinking that that a selected pundit or tech thinker has very smart insights when the subject they’re discussing is one you don’t know quite a bit about, but after they start talking about something that’s in your area of experience, suddenly you realize that they’re very off base. You return in your mind and wonder, “I do know they’re flawed about this. Were in addition they flawed about those other things?” I’ve been experiencing this now and again recently with reference to machine learning.

It’s really hard to know the way latest technologies are going to settle and what their long run impact can be on our society. Historians will let you know that it’s easy to look back and assume “that is the one way that events could have panned out”, but in point of fact, within the moment nobody knew what was going to occur next, and there have been myriad possible turns of events that might have modified the entire consequence, equally or more likely than what finally happened.

AI will not be a complete scam. Machine learning really does give us opportunities to automate complex tasks and scale effectively. AI is also not going to vary every thing about our world and our economy. It’s a tool, nevertheless it’s not going to exchange human labor in our economy within the overwhelming majority of cases. And, AGI will not be a sensible prospect.

AI will not be a complete scam. … AI is also not going to vary every thing about our world and our economy.

Why do I say this? Let me explain.

First, I would like to say that machine learning is pretty great. I believe that teaching computers to parse the nuances of patterns which can be too complex for people to essentially grok themselves is fascinating, and that it creates a great deal of opportunities for computers to unravel problems. Machine learning is already influencing our lives in all types of the way, and has been doing so for years. Once I construct a model that may complete a task that may be tedious or nearly unattainable for an individual, and it’s deployed in order that an issue for my colleagues is solved, that’s very satisfying. It is a very small scale version of among the innovative things being done in generative AI space, nevertheless it’s in the identical broad umbrella.

Talking to laypeople and talking to machine learning practitioners will get you very different pictures of what AI is anticipated to mean. I’ve written about this before, nevertheless it bears some repeating. What can we expect AI to do for us? What can we mean after we use the term “artificial intelligence”?

To me, AI is largely “automating tasks using machine learning models”. That’s it. If the ML model could be very complex, it would enable us to automate some complicated tasks, but even little models that do relatively narrow tasks are still a part of the combo. I’ve written at length about what a machine learning model really does, but for shorthand: mathematically parse and replicate patterns from data. So which means we’re automating tasks using mathematical representations of patterns. AI is us selecting what to do next based on the patterns of events from recorded history, whether that’s the history of texts people have written, the history of house prices, or the rest.

AI is us selecting what to do next based on the patterns of events from recorded history, whether that’s the history of texts people have written, the history of house prices, or the rest.

Nonetheless, to many of us, AI means something much more complex, on the extent of being vaguely sci-fi. In some cases, they blur the road between AI and AGI, which is poorly defined in our discourse as well. Often I don’t think people themselves know what they mean by these terms, but I get the sense that they expect something much more sophisticated and universal than what reality has to supply.

For instance, LLMs understand the syntax and grammar of human language, but don’t have any inherent concept of the tangible meanings. All the things an LLM knows is internally referential — “king” to an LLM is defined only by its relationships to other words, like “queen” or “man”. So if we want a model to assist us with linguistic or semantic problems, that’s perfectly wonderful. Ask it for synonyms, and even to build up paragraphs filled with words related to a selected theme that sound very realistically human, and it’ll do great.

But there may be a stark difference between this and “knowledge”. Throw a rock and also you’ll discover a social media thread of individuals ridiculing how ChatGPT doesn’t get facts right, and hallucinates on a regular basis. ChatGPT will not be and won’t ever be a “facts producing robot”; it’s a big language model. It does language. Knowledge is even one step beyond facts, where the entity in query has understanding of what the facts mean and more. We usually are not at any risk of machine learning models getting thus far, what some people would call “AGI”, using the present methodologies and techniques available to us.

Knowledge is even one step beyond facts, where the entity in query has understanding of what the facts mean and more. We usually are not at any risk of machine learning models getting thus far using the present methodologies and techniques available to us.

If individuals are taking a look at ChatGPT and wanting AGI, some type of machine learning model that has understanding of knowledge or reality on par with or superior to people, that’s a totally unrealistic expectation. (Note: Some on this industry space will grandly tout the approaching arrival of AGI in PR, but when prodded, will back off their definitions of AGI to something far less sophisticated, with a purpose to avoid being held to account for their very own hype.)

As an aside, I’m not convinced that what machine learning does and what our models can do belongs on the identical spectrum as what human minds do. Arguing that today’s machine learning can result in AGI assumes that human intelligence is defined by increasing ability to detect and utilize patterns, and while this actually is certainly one of the things human intelligence can do, I don’t imagine that’s what defines us.

Within the face of my skepticism about AI being revolutionary, my financial advisor mentioned the instance of fast food restaurants switching to speech recognition AI on the drive-thru to scale back problems with human operators being unable to know what the shoppers are saying from their cars. This is likely to be interesting, but hardly an epiphany. It is a machine learning model as a tool to assist people do their jobs a bit higher. It allows us to automate small things and reduce human work a bit, as I’ve mentioned. This will not be unique to the generative AI world, nevertheless! We’ve been automating tasks and reducing human labor with machine learning for over a decade, and adding LLMs to the combo is a difference of degrees, not a seismic shift.

We’ve been automating tasks and reducing human labor with machine learning for over a decade, and adding LLMs to the combo is a difference of degrees, not a seismic shift.

I mean to say that using machine learning can and does definitely provide us incremental improvements within the speed and efficiency by which we are able to do plenty of things, but our expectations ought to be shaped by real comprehension of what these models are and what they usually are not.

Chances are you’ll be considering that my first argument relies on the present technological capabilities for training models, and the methods getting used today, and that’s a good point. What if we keep pushing training and technologies to supply increasingly more complex generative AI products? Will we reach some point where something totally latest is created, perhaps the much vaunted “AGI”? Isn’t the sky the limit?

The potential for machine learning to support solutions to problems could be very different from our ability to appreciate that potential. With infinite resources (money, electricity, rare earth metals for chips, human-generated content for training, etc), there’s one level of pattern representation that we could get from machine learning. Nonetheless, with the actual world by which we live, all of those resources are quite finite and we’re already coming up against a few of their limits.

The potential for machine learning to support solutions to problems could be very different from our ability to appreciate that potential.

We’ve known for years already that quality data to coach LLMs on is running low, and attempts to reuse generated data as training data prove very problematic. (h/t to Jathan Sadowski for inventing the term “Habsburg AI,” or “a system that’s so heavily trained on the outputs of other generative AIs that it becomes an inbred mutant, likely with exaggerated, grotesque features.”) I believe it’s also price mentioning that we’ve got poor capability to tell apart generated and organic data in lots of cases, so we may not even know we’re making a Habsburg AI because it’s happening, the degradation could creep up on us.

I’m going to skip discussing the cash/energy/metals limitations today because I even have one other piece planned concerning the natural resource and energy implications of AI, but jump over to the Verge for a great discussion of the electricity alone. I believe everyone knows that energy will not be an infinite resource, even renewables, and we’re committing the electrical consumption equivalent of small countries to training models already — models that don’t approach the touted guarantees of AI hucksters.

I also think that the regulatory and legal challenges to AI corporations have potential legs, as I’ve written before, and this must create limitations on what they’ll do. No institution ought to be above the law or without limitations, and wasting all of our earth’s natural resources in service of trying to supply AGI could be abhorrent.

My point is that what we are able to do theoretically, with infinite bank accounts, mineral mines, and data sources, will not be the identical as what we are able to actually do. I don’t imagine it’s likely machine learning could achieve AGI even without these constraints, partially resulting from the best way we perform training, but I do know we are able to’t achieve anything like that under real world conditions.

[W]hat we are able to do theoretically, with infinite bank accounts, mineral mines, and data sources, will not be the identical as what we are able to actually do.

Even when we don’t worry about AGI, and just focus our energies on the form of models we even have, resource allocation remains to be an actual concern. As I discussed, what the favored culture calls AI is admittedly just “automating tasks using machine learning models”, which doesn’t sound nearly as glamorous. Importantly, it reveals that this work will not be a monolith, as well. AI isn’t one thing, it’s 1,000,000 little models in every single place being slotted in to workflows and pipelines we use to finish tasks, all of which require resources to construct, integrate, and maintain. We’re adding LLMs as potential selections to fit in to those workflows, nevertheless it doesn’t make the method different.

As someone with experience doing the work to get business buy-in, resources, and time to construct those models, it will not be so simple as “can we do it?”. The true query is “is that this the best thing to do within the face of competing priorities and limited resources?” Often, constructing a model and implementing it to automate a task will not be the most precious method to spend company money and time, and projects can be sidelined.

Machine learning and its results are awesome, and so they offer great potential to unravel problems and improve human lives if used well. This will not be latest, nevertheless, and there’s no free lunch. Increasing the implementation of machine learning across sectors of our society might be going to proceed to occur, similar to it has been for the past decade or more. Adding generative AI to the toolbox is only a difference of degree.

AGI is a totally different and likewise entirely imaginary entity at this point. I haven’t even scratched the surface of whether we’d want AGI to exist, even when it could, but I believe that’s just an interesting philosophical topic, not an emergent threat. (A subject for an additional day.) But when someone tells me that they think AI goes to completely change our world, especially within the immediate future, because of this I’m skeptical. Machine learning may also help us a fantastic deal, and has been doing so for a few years. Recent techniques, equivalent to those used for developing generative AI, are interesting and useful in some cases, but not nearly as profound a change as we’re being led to imagine.