The technology pace has been fast, but the AI pace in the last year since OpenAI launched GPT 3.5, has been ludicrous in Tesla's terms. If so, the last week has been warp speed. We now have a new public term: Q*, which may have reportedly been the reason why Sam Altman was fired.
So, what is Q*?
Q* is the closest we have come to Artificial General Intelligence. Q stands for Q-learning, which is a model-free reinforcement learning algorithm used to learn the value of an action in a particular state by trial and error and repeating the correct action to improve itself.
LLMs, Large Language Models, use the deep neural network to create generative AI, which creates original texts or images, and this is what GPT, Bard and Inflection are using now.
When you ask an LLM any question, the answers will be creative, however, each time you ask, there will be different answers. It can write creative and rich content blog posts or even poems, but it will not be able to create universal answers, such as answering to math problems.
This is where Q* differs from LLMs.
Q* is a model-free reinforcement learning algorithm that learns like a baby by trial and error and repeating while learning from its mistakes to create the right actions the next time.
What OpenAI employees achieved was creating the Q* that is now able to solve math problems and if the AI is able to solve math, then, that means we are on the path to reaching AGI and OpenAI is pretty aware of what’s coming.
So, let’s further define and describe how Q* works so that we can build a foundation of how it can be a roadmap to AGI.
There are basically 4 key terms in Q*, and they are Q-Learning, Q-function, A*, and Deep Q-Learning.
Q-Learning: The Path to AGI
Q-learning is a model-free reinforcement learning algorithm. In model-free reinforcement learning, the algorithm learns directly from interactions with the environment without requiring an explicit model of the environment. Q-learning, specifically, focuses on learning the values of actions in different states to make decisions that maximize cumulative rewards over time.
Q-Function
According to Perplexity.ai, the Q-function in Q-learning represents the optimal long-term value of action a in a state and plays a crucial role in determining the best action to take based on the current state. The Q-function is updated iteratively using the Bellman optimality equation, and the algorithm converges to the optimal Q-function, which in turn leads to the optimal policy for the reinforcement learning task.
A*
A* is a powerful algorithm for finding the shortest path between two points in a graph or a map while efficiently navigating the environment.
In the context of Q*, A* could be used to find the optimal path for the AI agent to navigate through the environment, ensuring efficient and effective exploration and action selection.
Advanced reasoning: The combination of Q-learning and A* algorithms in Q* might enable the AI model to perform advanced reasoning tasks, such as solving complex mathematical problems or optimizing resource allocation.
This could potentially lead to breakthroughs in various industries, including artificial intelligence and quantum computing.
Knowledge distillation: It is possible that the Q* project could use A* algorithms to learn from the experience of Q-learning and improve its performance in complex environments and tasks.
This could involve knowledge distillation, where the AI model learns from the expertise of another model (in this case, the Q-learning model) to enhance its own capabilities.
Deep Q-Learning
In essence, Q* orchestrates a symphony of algorithms — Q-learning, A*, and Deep Q-learning.
Q-learning, model-free and iterative, forms the basis.
A* contributes its path-finding prowess, enhancing logical and mathematical reasoning. Deep Q-Learning, with its neural networks and iterative update process, elevates Q* to handle the intricacies of AGI.
Together, these algorithms harmonize to create a model that excels in mathematical domains and promises revolutionary advancements.
Q*: Behind the Scenes of The Sam Altman Weekend
AI is such a force of nature that with deep neural networks and the incredible computing power of the supercomputers, not even the AI people who create them know what the AI is capable of and how close or far we are from AGI.
We are at an immense turning point in the history of humanity that for the very first time, a synthetic being made of GPUs, chips, silicon, and conductors could become more intelligent than us in the near future and the pace at which it develops is far greater than our understanding of how we can control and regulate it so it is aligned with our own values.
At this point, there is Sam Altman, who is the AI leader what Elon Musk is to electric vehicles. Sam Altman is a pioneer in Silicon Valley innovation and startups. He was the CEO of Y-Combinator, which has been the most important startup incubator in the world for a long time.
Basically, he has it all, skills, intelligence, network, funds, and more to create the most value that anyone can build in the world.
He is incredibly ambitious about AI as well.
Before he was fired, he was in discussions with companies to build AI hardware and high-performance AI chips to compete with NVIDIA and eliminate its monopoly.
So, Sam Altman is for ultra-rapid growth.
And this could have been exactly the reason behind the board’s decision following the email they received from the employees that they are on the verge of a groundbreaking invention, Q*, the first steps to AGI.
Interestingly, Sam Altman also had a hearing with the government asking for firm monitoring and regulation of AI development.
Therefore, he is not pro-capitalism, but more like pro-development.
He doesn’t even own a share in OpenAI or make any money from Microsoft investments or generated revenue.
He already has incredible wealth and power.
His purpose is to change the future for humanity and he is leading the AI movement, which is seen by many even more important than the internet and is as important as electricity.
All in all, we will likely hear more and more about Q* in the coming months and other companies will also use the same method to create AGI.
This reminds me of 2 things that created sudden action in industries.
One is the Pfizer—BionTech vaccine using the mRNA method. It was announced in late November 2020, and in a week, Moderna also announced its vaccine using the same technology. Then, the others followed.
The other is ChatGPT. As OpenAI launched ChatGPT on November 30th, 2022, other companies followed such as Google with Bard. Suddenly, AI became the next big thing that materialized with AI tools, and plug-ins and the market competition fed the industry in search for AGI.
We have seen the growth of the internet and are now, seeing the hypergrowth of AI taking long strides for AGI.
Sam Altman will be one of the key figures in shaping the AI future and we will witness the history in the making.
How lucky we are! After 150,000 years of being homo sapiens, the technological evolution of humanity is facing its epic growth in just one generation.
Thanks for reading. If you want to build or grow your digital business via writing, design, development or marketing, the Awaynear startup design agency is here to help.