Most people don't realize it, but scientists started working on "artificial intelligence" (AI) as a scientific pursuit in 1956, when it was established as an academic discipline. A 65-year-old-joke is that artificial intelligence is anything that doesn't work. Japan focused research on AI in the 1980s with the Fifth Generation Computer Systems (FGCS), which was widely regarded as an expensive failure.
Scientists thought AI would work if they could just figure out how to configure it. But it wasn't until 2012 that things began looking promising -- and from a completely unexpected event: Nvidia's Graphical Processing Unit (GPU).
Computers of the day used Central Processing Units (CPUs) for processing computer tasks. CPUs were great for opening programs and saving files but for mathematically demanding tasks were woefully slow. For graphically demanding tasks motherboard designers added a socket for a graphic unit to speed up picture generation. It turned out that GPUs did double duty as mathematics processors.
Picture generation was dependent on generating large amounts of data in a short period of time. Given that computer screens in the day were made up of grids of about 1,024 horizontal data points by 768 vertical lines, if you wanted to draw a picture of a cat you had to compute which data points were the cat, what color to make the different parts of the cat, and what to do with the background -- a task far faster with GPUs than CPUs.
Essentially, drawing a picture of a cat was a mathematical exercise.
With GPUs, pictures of an AI cat suddenly turned from a barely recognizable maybe cat, dog or rabbit into a recognizable Persian cat.
In the period from 2012 to 2022, using GPUs, AI defeated Chess and Go champions, became the Bitcoin mining processor of choice and drew pictures of every protein on earth -- but didn't capture the attention of the public until OpenAI's ChatGPT. It took a month for ChatGPT to amass 100 million users -- the fastest growth in computer history.
The question is: How does it work?
No one knows. Scientists have experimentally figured out how to get information out of Large Language Models (LLMS are large AI's like ChatGPT) but no human on earth knows how they work.
LLMs are predictive mediums. Given what came before, what should come next? LLMs actually use tokens (units of syllables from one to three -- rarely more), but humans don't think in syllables. It's clearer to use words as examples.
[The following examples are taken from Stephen Wolfram's blog "What is ChatGPT doing . . . and why does it work?" His blog quickly evolved into heavy math. I've tried to make it more accessible, but you should read his blog.]
Suppose you input the following sequence of words into ChatGPT: "The best thing about AI is its ability to" and wanted to predict what might ChatGPT reply next? Here are ChatGPT 2's choices with the percentage of time the words were found in the training data:
All ChatGPT is doing is figuring out what the next word should be, given the text so far. Usually the choice will be the most common next word in sequence, in this case, "learn" used as the next word 4.5% of the time. It's predicting, not writing. It doesn't always insert the most popular word because when tried the text was too "flat" -- it read like an uninteresting encyclopedia. So, researchers started experimenting with how often ChatGPT should take a less predictable word -- with values called "temperature" or "heat" as sometimes called.
A temperature of 0 only uses the most likely word. A temperature of .5 will generate conversational output, .3 will generate storytelling. A related concept is top_p, how far the LLM will search for an alternate value. The bigger top_p is the more "creative" is the output -- good for storytelling, but not for essays in class.
We are at the very beginning of the LLM deployment. Technological innovations go through three general phases.
Streamline how we are currently doing business to take advantage of the innovation.
Change what we are doing to take advantage of the strengths of the innovation.
Develop a business model that has never been envisioned -- one that was not possible before the innovation.
With the cell phone, the first step was to replace land lines. Chat was an extension of email. The second step was to include the Internet and make the phone a substitute for a computer. The third step was to develop unexpected new ways to do business such as Uber and Airbnb.
We are still at the "Make AI write computer code (or any other existing task)". The second stage will be to replace or convert the cell phone with an operating system that asks, "What do you want to do?" and then guides you to solving your issue. No more having to know where Settings are hidden. It will be generated from your initial query. In the third stage someone will develop a new business model that had previously been impossible without AI.
The conceptual problem is that AI is a series of contradictions:
It is bad at everything that computers excel at.
Not predictable. It can be right one time and wrong the next.
Does not "understand" even though it looks as if it does.
For business, AI is currently writing computer code, business plans, etc. -- tasks that are well-defined. Humans need to review output, but AI does a good job at this kind of task. This is essentially what interns were doing.
We can envision a time when a business person could tell a local AI agent, "Develop a program that will create a balance sheet, compare it to industry averages and write a report highlighting where we are behind industry averages." The second stage will be to tell the AI what you need and let the computer figure out how to do it.
The third step will come out of the blue. Humans will figure out how to get AI to do business in a completely new, never previously envisioned way.
It took 15 years for the cell phone to take over our lives. AI won't take nearly as long
For investors, it will be important to track where we are on AI. We are currently in the first phase -- businesses are trying to implement AI into their existing business models. The company, Humane has developed a new cell phone with no screen that you pin to your shirt. It's not ready for investment, but is an example of an attempt at a new paradigm. Something will spark user's imaginations and become investable, as will the new businesses taking advantage of the second stage investment period.
Keep a sharp eye out for third stage inventions that develop new ways of doing old things, or a way to do completely new things. That's where the biggest rewards will be. We just don't know what they are of when they might come.
Sorry for the long delay. I had health issues. Musings will be published on Thursdays going forward.
Please forward to interested third-parties. If you wish to have the regular Wednesday Musings in your email box, but aren’t ready for a paid subscription, just click on “Subscribe” and select “none”. If you are one of the few who wish to sponsor my effort, bless you. But you get the same messages as everyone.
I try to answer all questions or comments.