Why ML Engineers Are the Bottleneck

Neo's founders count 600 Kaggle grandmasters in the world, and say every company needs one. Our vision with Neo is to provide an AI Kaggle grandmaster in every company on the planet.

Clips and outtakes on Instagram ›

In this episode · 18 chapters

  1. How many machine learning engineers there are
  2. CERN
  3. The core idea
  4. What do you tell people you do?
  5. Fooling people
  6. What Neo is
  7. Kaggle as an entrance exam
  8. Bronze, silver and gold
  9. The benchmark
  10. The grunt work
  11. The airplane analogy
  12. Multi-step reasoning
  13. Why agents hallucinate
  14. Michelangelo
  15. Moore's law, and parameters
  16. Quantum
  17. What did you not believe today?
  18. Neo are not about that
So the problem with the AI systems would always be about two things. One, it will always follow the rules within the boundaries of the entire training set that has been given. Second, it’s about taste.
Saurabh Vij, Neo

Saurabh and Gaurav Vij built their first startup on an idea from CERN: running AI models on the spare GPUs in crypto mining rigs. Now they are building Neo, an autonomous machine learning engineer meant to take over the grunt work of building models.

The brothers explain why Kaggle medals work as an objective test, how switching from one agent to a team of agents cut hallucinations, and why Michelangelo's study of cadavers shows where AI falls short on taste.

Saurabh and Gaurav Vij, the brothers behind Neo, tell the Thinking On Paper podcast that machine learning engineers are the bottleneck in AI. Saurabh calls Neo “the world's first autonomous machine learning engineer”, a multi-agent system that he says outscored OpenAI's best model on a benchmark built from Kaggle competitions. Their goal is “to provide an AI Kaggle grandmaster in every company on the planet”.

How many highly skilled machine learning engineers are there?

Saurabh says the world has about 2 million data scientists and machine learning engineers, but only 300,000 highly skilled ones, 5,000 Kaggle masters and 600 Kaggle grandmasters. Kaggle is a platform where ML engineers compete to build the most accurate model for a set problem, and Gaurav says the leaderboard score is “purely mathematical”.

How did Neo score against OpenAI on the Kaggle benchmark?

Saurabh says OpenAI built a benchmark for agentic systems from 75 of the toughest Kaggle competitions. He says OpenAI's best model at the time scored 16.9% and Neo scored 26%, and that the result drove Neo's early traction with heads of AI and data science.

Which parts of the machine learning workflow does Neo automate?

Neo automates the whole machine learning workflow, from building ETL pipelines (the steps that extract, transform and load data) and cleaning data to training models from scratch, while keeping humans in the loop. The grunt work it targets includes data processing, exploratory data analysis, iterative experimentation and sandbox setup.

How does Neo's multi-agent design reduce hallucinations?

Neo is a multi-agent system, meaning several AI agents that work like a team of data scientists, data engineers and ML researchers with a feedback loop between them. Saurabh says moving from a single agent to multiple agents cut error rates and hallucinations “drastically”.

What are the three bottlenecks in AI, according to Saurabh Vij?

Saurabh predicts “almost a million AI native companies” by 2030, each needing a team of three to four machine learning engineers, against 600 Kaggle grandmasters today. He names three bottlenecks in AI, data, compute and talent, and says Neo addresses talent.

Where they disagree

Jeremy asks how Neo earns the kind of trust a pilot gives an autopilot, and Saurabh answers that OpenAI's benchmark made Neo's performance objective. Afterwards Mark says he is unsure what separates a Kaggle grandmaster from the other 5,000 strong engineers, and suggests the million-company forecast could easily be tripled.

Questions

Who are Saurabh and Gaurav Vij?

Saurabh and Gaurav Vij, the brothers behind Neo, tell the Thinking On Paper podcast that machine learning engineers are the bottleneck in AI. Saurabh and Gaurav Vij built their first startup on an idea from CERN: running AI models on the spare GPUs in crypto mining rigs.

What is Neo?

Saurabh calls Neo “the world's first autonomous machine learning engineer”, a multi-agent system that he says outscored OpenAI's best model on a benchmark built from Kaggle competitions.

What is the goal of the team behind Neo?

Their goal is “to provide an AI Kaggle grandmaster in every company on the planet”.

Transcript

Jeremy Gilbertson Disruptors and curious minds, welcome to another episode of Thinking on Paper, your personal vehicle to understand the intersection of emerging technologies and culture, business, relationships, family, what it means to be human, what this technology is doing to affect us. My name is Jeremy. This is Mark. We're writers. We're curious minds. We're trying to figure all this stuff out. Here we are today, Mark.

It's a new year. We've got some great segments coming up. We've got three new segments to the show. We're trying to be a little more organized, a little more intentional. So stay tuned for these new segments. It's going to be a blast. I'm excited. Mark, what are we learning about today? I'm excited too. Yeah, new year, same awesome show. Artificial intelligence, AI. So every AI product you use, every service you use, it begins with machine learning engineers, but there's a problem.

How many machine learning engineers there are

Mark Fielding There are only about 300,000 of them worldwide, and maybe only a fraction of those are actually good, like 600 who can actually change and engineer meaningful products. Why is that? That's what we're asking today. Why are there so few engineers when all eyes are on AI? Is it because it's beyond most? Is it because it's too cumbersome, dare I say it, unenjoyable? Is there too much upfront grunt work?

Jeremy Gilbertson What is the problem? And today's guests are going to give us a solution to the challenge. So yeah. That sounds like a good focus. We'll take good notes and stay tuned for the very end of this show. We're actually going to unpack our immediate insights for what we learned today, because we're here to learn as well. So Mark, who do we have on today and where are they from and where are we headed?

Mark Fielding Well, this might be a first. Myself, I have two brothers. I'm a middle child and today's guests are brothers, Sourav and Gaurav Vij and I'll let them explain where they are. I think they're in different continents. Maybe that's a brother thing to keep working together. I don't know, but find out. But welcome to the show. Hi guys. Thanks. Thanks for having us. And dialing in from Canada, Toronto, we just had a snowstorm.

Jeremy Gilbertson So yeah. I'm calling in from New Delhi, India. Where you haven't had a snowstorm. No. No, absolutely not. It's pretty cold, but yeah, not a snowstorm. We have a pollution storm, by the way. Pollution storm. All hands are on deck in Atlanta, Georgia, in the States, where one snowflake shuts the city down. So I'll keep you guys posted on how that goes. Tell us a little bit about yourselves, guys. Give us a short little piece of your background.

Gaurav Vij And I've got a question. I always like to kick off these discussions. So give me one or two sentences about your background and how it applies to the conversation. Yeah. Gaurav, you can. Yeah. So just a quick brief background. Like eight years back, I was running my computer vision startup where I led the research and development on building computer vision models for background removal. In real-time streams.

This is something that I'm using even right now on this call, background removal. But at that time, it was not so exciting. After COVID, it became quite exciting for video solutions, video call solutions to have this capability. So, but at that time, I faced two biggest problems while building such a solution. The cost of building an ML model is very high due to the complexities involved, such as the cost of computing on cloud platforms and also hiring a good ML talent is scarce and costly.

So these were the two roadblockers that turned out to be the biggest challenges that I ultimately wanted to solve for me and later on realized that the entire machine learning community is dealing with that. So that's what inspired me to go on this path of solving these challenges. And you do it with your brother, which is really exciting. So what about you? That's the best part. I was a particle physicist.

CERN

Saurabh Vij Fortunately, I got a chance to work at CERN, where they found the Higgs boson, media called it with the weirdest name like God particle and many other things like that. But that gave me exposure to high performance computing systems and neural networks. And it was fascinating how, you know, using AI machine learning at a very basic level, even, you know, 10-15 years back, had a very significant impact on particle physics.

And that really, you know, inspired me to work in this field. Then I collaborated with Gaurav, you know, over a project which became our first startup. This is our second startup now. So the first startup, you know, was inspired by an idea that I got from my research days at CERN, which was about distributed computing. To give you a brief about that, what happened, generally what happens is when you want to run your AI models, you run this on public clouds like AWS or GCP.

The core idea

Saurabh Vij Our core idea was what if you can run your AI models on thousands of crypto mining rigs, because the same crypto rigs also have GPUs, which are incredibly powerful, but they were not designed for AI, they were designed for gaming or crypto. So we repurposed them, it kind of became an Airbnb for GPUs, but specifically for AI. So we had built the entire stack, solved for the computing layer. We ran that for around four years, served more than 1 million computing hours. This led to 80% lower cost. And that became the foundation for whatever, you know, we are doing right now.

Jeremy Gilbertson So you were taking available capacity in these crypto based GPUs. I love the Airbnb model for crypto GPUs. I think that's really smart. Compute in general, no matter what it was, you know, from way back to now, there was an inherent capacity gap in usage, right? Especially for enterprise IT systems, they build these big systems and they wouldn't use them to their capacity. And there's so much latent potential and capacity.

Saurabh Vij So that's super interesting. Is that a precursor to Deepin, to decentralised infrastructure networks where you're using the leftover, the residue GPU of these toe rigs? That was the, I would say, original vision where, you know, let's say you're sleeping and your laptop is computing for, let's say, a data scientist in other parts of the world, while you get paid for that. And he gets to leverage a much lower cost, you can say, computing system.

Jeremy Gilbertson It's kind of like SETI that NASA used, right? So you could hook up your computers and all of these astronomical models that, you know, need a compute power, you hook up your computer to use it. It's also like, Mark, your wish and your dream as an avid snowboarder to have someone just stretch you while you're sleeping instead of you having to work on your own flexibility. All right. So here's our question, guys.

What do you tell people you do?

Jeremy Gilbertson Here's our kickoff question. What do you tell your friends and your family about machine learning, about AI, to kind of pull them in to the inner circle, to give them a bit of a leg up, to help them understand it faster than traditionally? That's an interesting one. So let's say we talk to our parents, you know, and generally at this age, you know, most of the conversations, you know, would always converge around health.

Saurabh Vij And, you know, when it comes to health, you should never go to Google, you know, to get an answer, because it will directly lead to, you know, the most dangerous crap out of you. Yeah, exactly. If you have a cough, you probably have throat cancer, something like that. So because there is no brain, there's just, you know, enormous amounts of data. Now, what if there is there's a human being which can synthesize all of that data and give you a summary, and also filter out bad information.

Now, that human being is, is an AI, where you can talk to that, instead of, you know, just talking to a doctor, you can talk to this guy for the early symptoms. And it will give you, you know, very synthesized, but very high quality answer to some of your questions. And the same goes for your little kids who are working on math problem or someone who is trying to understand physics. So for just imagine having a PhD level, human being all the time with you to help you with any of your problems.

Gaurav Vij So that's what you know, AI is. And that's what, that's how we explain this to you know, our friends and family. I like it. Gaurav, you have any thoughts on that? Any additions? Yeah, like, like, just an example that comes into my mind, like, it's sort of mentioned about me talking to our parents, like, I used to tell them in a way that this is like a personal genie. Like, I'll give you a quick example. Like, we have all of our parents every morning, they send good morning text messages to their family members and friends.

So I asked her like, you can create a chat message with chat GPT that would be that would sound very professional and very friendly that you can share with your friends. So it became very easy for her to interact with such chat based bots.

That I think has been the precursor for enabling people who could not understand AI, these type of chat based interfaces have definitely enabled access to people who don't need to go about like, what is the transformer architecture, how GPT models work, but still, they are able to deliver value in form of meaningful outcome, for which they don't need to talk to another person, they can talk to a chat bot. So I think that is how I introduced AI to parents.

And similarly, friends have got to know from the sources, the blogs and all that. Definitely, they asked me like, where it is heading towards? What are we seeing in the next year? And definitely the speed at things are moving in this industry. It's very hard to even catch up at times and figure out what might happen in the next year or this year itself in the next three months. So yeah, I wish we'd done this conversation before Christmas and I went home at Christmas and I tried to have this conversation with my parents and my family and I didn't use those analogies and perhaps I should have but maybe next Christmas.

Mark Fielding Those are great. Everything will change by that time. Well, if everything, just before we get into Neo and that, if everything's going to change, people very often talk about education and educating people about emerging technologies. How important is it for me to explain to my family about AI? Is it? Should I be worried about explaining it to people or should I just let them, kind of, because it's going to be so pervasive in society anyway, they'll find their own way and they don't need me to tell them.

Fooling people

I think if you, let's say, if you're the expert among us, among them, you should definitely, you know, kind of give them a warning because I can bet on this, in the next few years, there will be a lot of misinformation in every single form, in text, audio, video. It's like getting closer to reality. You would have seen, you know, the latest video models by some of these companies. They're getting close to reality.

Jeremy Gilbertson So it's very, becoming easier to fool people. So misinformation is going to first, there will be volume of, more volume of information and then it will also spread much faster. And it still lives on all the same problem, problematic infrastructure that is social media algorithms, which tend to promote things that are, that are a little more jarring, a little more offensive, maybe, right? That's, that's the stuff that like scales the audience, that increases interaction, right?

So we're gas on the fire, man, you know. Exactly. So we need to warn people here. We need to, you need to, you know, help your family members understand to some extent how to differentiate between, you know, real information and AI generated information. It's difficult. Also, also way to draw the line while sharing personal information.

Mark Fielding Just a quick shout out to, so we have a book club as well as every week interviewing the leaders in tech. We have a book club and at the moment we're reading the Nexus and it's all about information networks, disinformation, misinformation, thinking on paper, X, Y, Z. Sorry. There you go. Well, hey, let's, so let's dive into Neo. So we teed up kind of the idea of what we believe you guys are trying to solve, but it's always great to hear in your own words.

What Neo is

Saurabh Vij So what is Neo and what problem are you trying to solve with it? So Neo is the world's first autonomous machine learning engineer. The demand for machine learning is increasing exponentially. Every company wants to build their models or want to, wants to, you know, fine tune on existing models. But the problem is it's getting complex day by day. There are hundreds of new research papers, hundreds of new architectures and new optimization frameworks.

Now to really integrate that and leverage them and use them, you need highly skilled machine learning engineers with years of experience. So the demand has completely outstripped the supply. There are only, you can say 2 million in total data scientists and machine learning engineers. But out of that only 300,000 highly skilled machine learning engineers that can really do the job and help you build very high quality models.

And the entire pipeline. And out of those 300,000 ML engineers, there are only 5,000 that ever became Kaggle masters and 600 Kaggle grandmasters. Our vision with Neo is to provide an AI Kaggle grandmaster in every company on the planet. Neo automates the entire machine learning workflow from building your ETL pipelines, cleaning data to building models from scratch. You can interact with Neo just through a chat and share your problem.

It could be that I want to build a credit risk model and here is my data sets. So it will just take that information and take over the entire workflow and build the model for you. But we have designed it in a way that you have humans in the loop. So you can always change its behavior, direct its behavior. And when you integrate that within your company, it's like inducting a new employee. Just like a new employee would learn your playbook, your processes, where you store your data, how you clean your data.

Gaurav Vij It will learn all of that. And that will make it more and more human-like. Can we unpack master and grandmaster real quick? Yeah, what's Kaggle? Gaurav, do you want to take that? Yeah, I can take that here. So Kaggle is a platform on which ML engineers in a way compete with each other. So you can think of it like there are benchmarks today in the industry for AI models. Where AI models are competing in terms of reasoning capabilities, mathematical capabilities.

Kaggle as an entrance exam

Gaurav Vij So just like an entrance exam, Kaggle is like a competitive platform where ML engineers participate in various different competitions. And the people that win, you can say, there is a leaderboard score.

And when you are basically performing or participating in a competition, the competition could state that you have to build. The task is to build a model for sentiment analysis, sentiment classification. Here is the data set. So you would have to come up with a unique approach to solve that problem. And there is an evaluation that happens along with the validation data set that is provided in the competition.

Now, with that validation comes a score. And that score is identified on the leaderboard that suggests that where do you stand with your approach. Out of thousands of participants, you might be in the top 100. You might be in the top 50. So in the top, you can say 100 scorers. Out of, let's say, thousands of participants, there is a potential of winning a bronze medal. So as, you can say, your score increases, there is a high likelihood that you can win a medal.

Bronze, silver and gold

Gaurav Vij There is a bronze medal, silver medal, and a gold medal. And as Saurav mentioned, the masters and grandmasters have won a medal in some of the competitions that they participated in. So there is a criteria for defining what a master and a grandmaster is based on the amount of medals that you have won on the platform. And is this a community-based award of recognition? Or is there a committee or people that are outside of the community that determine who gets the medal or not?

So for the competitive medals, so there are different types of, you can say, ways to win a medal. There is how you are contributing to the community, how much contributions you have made, data sets, medals, how many data sets that you have provided, quality of them. What I'm talking right now is about the competitive medals. So when you participate in a competition, community doesn't provide you a medal.

It's more about where do you stand in the competition based on your approach, what score you got after participating. And that score is not provided by any community member. It is purely mathematical and it comes from the basis of your approach. You have trained a model. There is an accuracy of that model. Now, the higher the accuracy, the higher will be your score. And based on that, you can win a medal.

Jeremy Gilbertson Ah, so push the subjectivity aside, purely objective, purely mathematical. Wow. Okay. All right. So listeners, this is super important, right? So if you're looking for resources, right? And correct me if I'm wrong, gentlemen, if you're looking for resources that machine learning engineers, people in this, that would be a pretty good spot to go and say, hey, you know, where could I find a good one?

Where could I validate a good one? Not by Mark's opinion, but by Matt's opinion. I think that's pretty cool. Just a quick Google search as well. And there are about 2000 chess grandmasters. So there are, in fact, chess grandmasters, machine learning gaggle grandmasters, which is pretty surprising in this world. How many do you think we need? I know it's a bit of a stupid question, but in the next five years, we have 600.

Saurabh Vij How many do we need to keep up with the demand for AI? Do you think ish? By 2030, there will be almost a million AI native companies. When I say AI native companies, what I mean is you would need a three to four machine learning engineer kind of team in your company, building your own models. Because now the trend is a lot of these companies have so much data that they can't leverage unless they fine tune these models or build their own models.

You can't just use open AI for all the different use cases. So there will be 1 million AI native companies, but there are only 600 Kaggle grandmasters today. So the vision with Neo is, can we provide a Kaggle grandmaster for all of these companies? So imagine you have not even a PhD, beyond a PhD level expert in your team that works for you 24 by seven for a fraction of the cost, which would have costed you millions of dollars earlier.

Now, that's one. Second is just to complete the loop on master and grandmaster. One definition of Kaggle grandmaster is you have to win five gold medals in five life competitions. Once you are able to do that, you become a Kaggle grandmaster. And that's why there are so few. And so open AI, recently what they did was, they created a benchmark for agentic systems like Neo. So Neo is a multi-agent system.

The benchmark

Saurabh Vij So they created a benchmark for these kinds of system. And the benchmark was again on Kaggle competitions. What they did was they ran their model along with different agents on 75 Kaggle competitions. These are like some of the toughest problems. It can take almost a month sometimes to solve one problem. They are so complex. Now, open AI with the best model they had at that time, it was in October 24, just three months back, with the best model Owen, they got a score of 16.9%.

Jeremy Gilbertson Neo got a score of 26%. So Neo was able to outperform open AI at a benchmark that they created. That's powerful. Congrats, guys. Wow, that's impressive. So let's talk about, and in my research and our pre-show discussion, the idea that what Neo can help engineers automate are some more of the basic tasks, the foundational tasks of building these super complex models. So let me throw an analogy at it.

The grunt work

Jeremy Gilbertson What's that? The grunt work, I believe it's on the website. Right, right, right. But there's value in the grunt work. So I don't like to use that. There's value in the little bits and pieces, right? So if we use an analogy, like I'm an airplane builder, right? And instead of me bending the metal to make the wings and riveting these things in place and screwing on bolts to make the jet engine work, I can put automated entities in place to help me with that while I figure out how to fly the plane faster, better, smoother, cheaper.

Is that reasonable? That's a good analogy, yes. If you're an airplane designer, you would rather focus on making a faster plane or a much sexier plane instead of spending your time on... Exactly, exactly. You can go for 10 times bigger goals in life because now you don't have to deal with bolts and screws or taking care of engine oil and all of that. So all that grunt work, which you would today give to interns or you can say a lot of junior level engineers can now be automated.

Yeah, there's a lot of grunt work in machine learning space also, like data processing, exploratory data analysis, wrangling with the data sets, iterative experimentation, sandbox environment setup management. A lot of these things can be highly automated while still creating a generalized system which can generalize on a lot of different problem statements. That's what Neo enables while being smart enough.

Like we discussed earlier, to deal with different clouds, different GPUs, fit your model inside those GPUs, containerize them, learn orchestration among all of that. So first, it's a lot of dirty work. It's a lot of grunt work that they are forced to do right now and that wastes a lot of time. And generally what happens is it's not a one-time thing. You have to keep on doing that. You have to scale up your systems and these things tend to come in your way when you really just want to focus on building better models and you want to be closer to your customers.

The airplane analogy

Jeremy Gilbertson So let's stay with our airplane analogy again. So in order for me as a pilot to trust autopilot, push the button and say, hey, this machine is going to fly the plane. How are you building trust with the people that are going to lean on your systems? Are we back to objective demonstrable math or are we trust a subjective? Yeah. Exactly. You know, that's a very good question and a perfect analogy here.

Saurabh Vij So to trust an autopilot, you would look at its history. Was it able to first, did they test it out, you let's say with 1 million simulation runs or they were able to test it out for let's say 10 years in real life scenarios. So what Kaggle has done for us is these are some of the toughest problems given by these big companies like Tesla, OpenAI, some of the hedge funds in financial markets.

They've thrown out these problems and because of this OpenAI's benchmark, it has become really objective and that's why this score of 26% really helped us in getting our initial traction. It really resonated with a lot of, you can say machine learning engineers and heads of AI data science heads. They, like some of the biggest brands, you know, on the planet, they signed up on our waitlist.

Gaurav Vij Purely, I would say because of this validation, this objective validation of the score. Along with that, like what happens like in case of autopilot, like Swarup mentioned, there are millions of hours of learnings that it can, that the system has. Like as a pilot myself, I might have 10,000 hours of flying experience. When I have this much experience, I learn on each hour of like flying the plane. How to land, when to launch the gears.

So these are small things which as a human, I learn. The same way Neo, being an autonomous engineer is capable of learning from the experiments that it performs. And with each experiment it is learning. It is storing that in its memory and that ultimately leads to a more powerful system as it progresses through these experiments and Kaggle challenges which we made it participate in. I'm gonna ask a silly question, please.

When it's learning from previous behaviors, how does it deal with mistakes that it's made? Definitely, like there are core level issues that it, that can debug. There are issues around how to wrangle and perform data exploration, how to analyze the data, how to do feature engineering. There are many steps in the ML engineering pipeline at which it can fall apart. Now the idea is this, and why it falls apart is because unlike an auto ML pipeline where you have a standardized workflow, where everything is templatized.

Multi-step reasoning

Gaurav Vij Here we have a system which is presented an unseen problem. Now it can, now it can, it might have to work on a data set that it has never seen before. It might have to work with an architect model architecture which it has never even thought about. Like what, how many parameters are there? How many dense layers are present in this neural network architecture? It may not have that information prior to getting started with the task.

So there are a lot of variables which can come, which can lead to breakages in the pipeline. Now the idea is this, with so much experimentation and the learnings that it has gained, it is able to generalize across different scenarios where even if I present a model architecture which it has never seen before, it still has an understanding of how to analyze the architecture and how to go by, go through the process in a step-by-step phased manner.

Saurabh Vij And that is what we call as multi-step reasoning that enables, that reasoning enables the system to perform exceptionally well at such complex challenges that we presented. I would also add one more thing to solve this problem. When we started out like six months back on this problem, we started with a singular agent flow. So what we did was we created an agent which was trying to mimic the behavior of an ML engineer.

Why agents hallucinate

Saurabh Vij And agents tend to hallucinate because LLMs behind them hallucinate. So one of the ways we solved this problem was, instead of a singular agent, we created multi-agent system. With that, we are trying to mimic an entire team. So what happens in reality is, whenever there's a problem, machine learning problem, a single engineer is not solving that problem. It's a team of data scientists, data engineer, ML engineers, ML researchers, all working together.

With a strong feedback loop among themselves to solve a problem. And that's what we are building with multi-agent system. And when we switch from single agent to multi-agent, you can see error rates reduced drastically. Hallucinations reduced drastically and we were able to get much more accurate results. So going back to just the idea of, this has been a thread that Mark and I have talked about for years.

Jeremy Gilbertson The idea of interdisciplinary teams bringing something really special to the table instead of having a single threaded entity trying to figure out a problem. It's this feedback loop between this, dare I say, nexus of resources, right? Gentlemen, it's time to stretch. It's time to get ready for your gold medal test. This is Mark's five questions in 30 seconds test. So guys, we're going to put 30 seconds on the clock.

Mark has some quick yes and no answers. Let's bounce back and forth. I don't know who feels spry. Gaurav, you want to handle the first one? And we'll bounce back and forth. All right, here we go. Are you ready? Never been attempted before on a podcast anywhere in the world. Let's go. Stay tuned. Here we go. Is the clock up? 29. Question number one. What is the best AI assistant for the general non-coding population?

Saurabh Vij Non-coding population. General AI assistant, I would say right now in my experience at this clock. Question number two. What is the best book on AI, fiction or non-fiction? Doesn't matter. OK, that's an interesting one. OK, question number three. What will AI agents use as currency? Dogecoin, if Elon Musk would promote it. Reasoning, I would say. Reasoning is the currency. But I think just on a serious note, I think there should be new currency backed by compute and contribution from developers.

So imagine a decentralized currency specifically built for AI, but it is backed by how much. So just like, you know, in the crypto, in the Bitcoin world, you know, the early developers who contributed got a share of those early Bitcoins because they contributed. who they're coding, you can say talent, then there are people who are contributing through their computing, through all the GPUs, crypto mining rigs.

Similarly, what if you can do something for AI? And I think eventually someone will build such a network where specifically for AI, and these agents then can use that currency. Just an idea. Okay, so just as a judge here, Mark, we failed tremendously delivering all the answers and the questions in 30 seconds. Just your interdisciplinary feedback loop at hand. Please continue. Well, I'm just going to ask you the two questions because I've got to close the loop.

You know, I won't sleep otherwise. Question number four, Hollywood, doomed by AI or reborn? I've been thinking about this. I would say reborn. I would say, I see on the positive side, I believe there's a whole lot of creativity that would be boosted with AI. The way, you know, I look at Hollywood and any other art form, I would say, it's a long answer.

Michelangelo

Saurabh Vij I'm not sure if I'll be able to finish this in 30 seconds, but my view is entirely driven by history. Where Michelangelo, they were, you know, artists, sculptors before Michelangelo. And if you were to build an AI, you know, based on all the training set, all the data that you would have collected from the previous generation of artists, you would have built something, something really beautiful.

But the thing with Michelangelo was he went far. He crossed boundaries. He went after cadavers, you know, at night to really understand deeper levels of anatomy. And that gave him so much insight that when he built his first, you know, at the age of 19, when he built Hercules and later when he built his David, you could actually see the expression of the nerves, you know, coming out of those marble pieces.

You can only do that when you go beyond and cross these boundaries. So the problem with the AI systems would always be about two things. One, it will always follow the rules within the boundaries of the entire training set that has been given. Second, it's about taste. So, you know, the greatest artists of our time and in the history, they had a very nuanced and a very different type of taste.

And I don't know how to create a mathematical equation to describe taste. So based on these two things, I would say people who are really great at their craft, now they will become superhuman. Instead of, you know, like Christopher Nolan, instead of releasing one movie in three years, probably he can now do more. And we would love to see that. And so it will increase volume, but people who are mediocre, their jobs can be taken away.

Jeremy Gilbertson You've just answered question number five as well as being, giving us an incredible answer. Talk about superhuman. Well, yeah, question. Are you using a model in the background? What's happening over there? Yeah. Well, hey, so thanks for participating in the hot buttons. You know what it's time for guys? You know what it's time for? It's time for TOP news. All right. So it's CES week, right?

It's CES, all kinds of stuff going on in Vegas. Specific to NVIDIA, we want to kind of focus on some announcements that NVIDIA has put out there. The first one that's super interesting guys that I would love to get your thoughts on, they announced a new personal AI supercomputer called Digits. Digits, right? So it can support 200 billion parameters, whatever the hell that means. Guys, what is Digits?

Saurabh Vij And is it going to do anything to influence the AI revolution? I couldn't personally spend a lot of time on Digits, but I spent just 10 minutes, you know, through the announcement and everything. My belief is it will first, you know, really, I mean, imagine, you know, an exa-flop supercomputer, which would have taken, you know, a house like this multiplied by three. Now in your hands, what are the things you can do? If you're brilliant, if you're great, if you have some really innovative ideas, there is no excuse now. For $3,000, you get a fucking supercomputer with one exa-flops. You can do so much.

And I'll tell you, you know, all the brilliant innovators in the world now get their shot. They can build things. So imagine, sorry, I'm getting very excited about this, but there are three bottlenecks, you know, always. Data, compute, and talent. We are solving for talent with Neo. Now, you know, if you are a good Python engineer, a regular software engineer, you don't have to, you know, go through the complexity of machine learning engineering.

Using Neo, you can bridge that gap. Second is about compute. So Digits is solving that. So with Neo and Digits, there are things you can do now is going to like blow up everything. Like in next three to five years, it's going to be the best time, you know, in the entire history of our planet. There will be scientific revolution. It is a new renaissance, I would say. Scientific revolution and then political chaos, potentially, with people with these powerful computers trying to re-engineer messaging.

Jeremy Gilbertson But we'll save that for another episode. We just need to buy Canada and Greenland. That's it. Ooh, okay. All right, so that's a great answer. I appreciate that. That's great context and almost inspiring context as well. But Gaurav, let's talk about this. So in 1965, there was a guy named Gordon Moore that decided to predict how quickly transistors would double over time, right?

Moore's law, and parameters

Gaurav Vij So Moore's law, two years doubling of transistors. The CEO of Nvidia now says Moore's law is bunk. It's, you know, but what could a common metric, a new common metric to define the progress of AI supercomputers moving forward? Right, I think, like, in fact, this was actually something that was coming in my mind when Saurabh was answering the previous question that just like Moore's law, there has to be another law which would definitely soon somebody would write about it that the number of parameters in the model triples every year.

So earlier we had 8 billion parameter models, then 32 billion, 64 billion, more than 100 billion, now 200 billion that could run on your desktop. The thing we would definitely see is, so number of parameters definitely doesn't justify the intelligence of a model. There has to be a lot of the things as have been shown by other researchers in the past also is that there has to be very high quality data which goes hand in hand with the number of parameters.

But definitely having more parameters gives the edge to a model for generalizing to a lot bigger dataset. So that definitely enables a model to act like a super intelligent system where it can focus on 10 different tasks instead of just being a narrow AI, it feels as if it's like a general intelligence. So bigger models definitely tend to offer this type of perspective and that could definitely be, in my personal opinion, number of parameters can definitely lead to a signal that how intelligent the model is and that could be considered as the unique value for just like we have had Moore's law.

Jeremy Gilbertson So as in the number of parameters, like we referenced in digits, supposedly 200 billion parameters, right? But it's also a garbage, like you said, it's a garbage in garbage out situation. So Mark is the premier snowboarding athlete that he is, he's got all of these pieces and parts that work very, very well. But if he goes down the mountain the wrong way and runs over rock and avoid snow, he's not gonna be a great snowboarder, right?

Gaurav Vij So you gotta have the right stuff that's using these things, right? Right, absolutely. Along with that, one more thing that could definitely act in pushing this, you can say stage of AI transformer based architecture models is memory. Like this is a concept which came in a year back only, I believe the idea is like how a model can memorize concepts beyond the training data on which it was trained.

Now that leads to learnings in real world scenarios, just like humans, like if as a baby we start to crawl and then step by step we start to walk. If you see reinforcement learning agents, there are millions of iterations after which they start to learn. But why are they taking millions of iterations? Why are they not able to walk in, let's say the next, the fifth iteration itself? Can I pause on that real quick?

Sorry to interrupt, can I pause on that? What do you mean by concept? Can you relate, like what is a concept that we would memorize or a machine would memorize? Like for example, in case of reinforcement learning, there is a concept called as reward functions. Like let's say I get hit by a car while walking on a road, I get negative reward. If I am able to save myself, I get a positive reward.

So the idea, the concept is then how do I save myself? So that is the way it memorizes concepts. And in a way, this type of system can learn in real world scenarios and definitely a very large model which already has been trained on very huge amount of generalized data, high quality data, along with the capability to memorize concepts while as it interacts with the world can potentially lead to a general intelligence type of system.

So how good, so a metric could be how good that model, that system takes in a concept and applies it to becoming more efficient at what its job is. Right, that's what actually made the core difference between the GPT-3 API, which was there before ChatGPT and then ChatGPT came up with RLHF, Reinforcement Learning with Human Feedback. So that provided a reward function, which enabled the model to learn what to do and what not to do while during the training phase.

But the idea is models cannot be, we cannot keep training because training is costly. There has to be a new way of making the models learn in real world scenarios when the training data has been exhausted. What to do now? Now let's put it in real world scenario and then it can go through certain experiments and come up with what it should not do. Could models be intrinsically motivated to get excited about being rewarded?

Yeah, I believe so. We can, that's more on the side of nature of a model. Yeah, like we as a human have nature. Every human has a distinct nature of where we get aligned to it. Like someone might have a sharp nature of understanding things quickly. Somebody might be a slow learner. But that's how our different brains work. Similarly, a model can be tuned to like focus on positive reward function.

That's actually, you can think of it like a, there was a concept called a chain of thought reasoning that was introduced a few years back and was implemented again in O1, which enabled it to reason very well compared to previous models. So chain of thought was implemented right in the dataset itself. The dataset was presented with examples to the model so that it can learn on these examples, like how to think in form of a chain of steps.

So just like that, we can potentially create a dataset that could make the model aligned towards focusing on positive reward. And that could lead to potentially better or quick learning for the model. It's super interesting. I wanna wrap this one thought up and then I think we wanna go to the carryover question for Mark, you could lead the charge on that. But it's almost like the relationship between coaches and athletes.

Jeremy Gilbertson I coach lacrosse at multiple levels and some players are motivated if you get in their face and get really, hey, what are you doing or whatever? And then there are other people that are motivated in different ways that that might not work for them. So similarly, what you're saying is these models can be set up to take in a particular reward mechanic better than another. Is that right? Am I grasping that?

Gaurav Vij Yes, we can potentially drive the nature with dataset designed that way. At least we have not tested that, but I believe with providing the right set of examples, with a lot of examples, we can potentially make a model that can be tuned towards this type of, we are basically changing the nature of the model. That's what we call today as thinking, like oneness thinking. So it's doing chain of thought reasoning and reinforcing itself on each iteration so that it can come up with better tokens.

Amazing. This is fascinating stuff. Holy cow, like, yeah. I'm sorry, but when you asked that question about could the AI model learn to intrinsically chase rewards, you had a little glint in your eye. And what was the book, the best book on AI that we should read? Superintelligence. I know where you were going with that question. Speaking of questions, we've got a can of worms today. Yeah, Pandora's box all over again?

Jeremy Gilbertson No, so, all right, speaking of questions, carry over question, let's hit it. Guys, this has been a fascinating discussion. Thanks for rolling with the punches of the new segments and everything. And I think there's a lot of great nuggets that our listeners are getting out of this. And yeah. Just to add a few words for that question, would there be a new law? I think for all the contribution that Jensen has done, I think we should have a new law with his name, Jensen's law.

Just for you. So we have an NVIDIA proponent on the interview session today. I like it. No, he's done with, yeah. I hope someone wants to get in a room with him. We have sent him some notes to be on the show.

Mark Fielding Maybe it's his time now. Perfect, perfect timing. Yeah, well, it'd be amazing. Tie it together, Mark. What do we got? Well, tie it together, I'm actually gonna throw a spanner in the works and change the question because I think with the hot buttons were so on point that they've answered the question because it was really about how this new generation of creatives will live with AI.

So I'm gonna change the question because I believe, physicists next week we have XPRIZE on, we're talking about quantum computers. In the past, one of the big take-homes for me from all the quantum conversations we've been having is there aren't any quantum developers. There aren't any quantum engineers. And there just seems to be this dearth of developers across emerging technology. And yeah, next week we have XPRIZE coming on who are talking about how they're gonna, they're trying to fix that in quantum.

Quantum

Gaurav Vij Do either of you two have any views, opinions on quantum and AI and how you see the two paths going ahead, merging, not merging in the imminent future and the more long-term future? Right, like just to start off on that, like I recently got a chance to learn about the development that Google presented in the keynote for quantum computing. And that definitely turned out to be pretty, pretty vital implementation in quantum physics, quantum computing.

The ability of a system to perform so many amounts of multiple simulations in parallel, iterations in parallel in a way to come up to a solution. That ultimately, I believe is the key to getting super intelligence. Having so much amount of compute available, a normal computer, we can increase flops, we can keep on adding transistors, but definitely there would be, unless we change the architecture, unless we go beyond transformer architectures, models would need, bigger models would need more compute.

That would be costly and beyond a certain limit, we would face issues with networking because we would have multiple GPUs connected to each other over network, latencies and all those things. So, and this I think also is a perspective that as a human, when we are tasked with a problem, if we have a capability of thinking about multiple solutions ahead of time, then we always come up with the best solution.

If we get enough time to think. So similarly, a quantum computer can basically compute multiple iterations in parallel. That gives it the opportunity of solving the solution in a very quick time. And a model which needs, which is just spitting tokens without any reasoning. If it is given the capability of thinking multiple iterations in parallel, then it can come up with a better solution compared to what it might have come up in the first scenario itself.

Saurabh Vij So quantum computing can, in my opinion, definitely help AI go beyond the phase of where it is as just as a- It's like having the superpowers of Dr. Strange. You can simulate all the possible worlds and find the one way you can live. So I think that's the exciting part, I would say about building a quantum computer. Like Gaurav rightly said, let's say you're solving a scientific problem where you have to build, let's say, a new form of, let's say, material or new alloy. Now, generally, it's a very experimental job in nature.

You have to go through so many combinations. What if you can simulate all those different combinations in like five minutes? That would, this impact it would have on this scientific exploration of new materials, new research. When you're solving, you can say, medical problems, you can simulate all those different paths. And this will accelerate a lot of things, not just, you can say, in the field of energy in material science, but also in the medical science.

But the cons, these are all the pros of, I would say, pros of quantum computing. But one of the problems, I think, which world is not ready, I would say yet, is when quantum computers are able to solve encryption. What happens to all the blockchains that we have built so far? What happens to the highest encryption standards that we have built if quantum computers can break that? So that's one area where I think, as researchers, everyone should invest a lot of time in building much more robust, you can say, encryption systems.

Mark Fielding Before we launch, we push the nuclear button of quantum computers onto the world. So just a quick shout back to previous episodes. We had the post-quantum cryptography lead from IBM. Apparently, we have till 2032, I think, until all shit hits the fan. So that's great. So great, great conversation, guys. What about a carryover question for our next guest? It could be on any topic, any subject. It doesn't have to be AI, it doesn't have to be quantum, it doesn't even have to be tech, philosophical, whatever you want.

Gaurav Vij One? I have a philosophical one. Yeah, like, maybe Zorub can add his ones. Like, straightforward question, are we in a simulation? And is there a way to figure that out in the next, like, 10 years, maybe, the next decade? Would we be able to figure that out? Like, if I would have been Mario in the Mario game, and there's no way I can come out of Mario game. Can I at least get the cheat codes? If you're eating the mushrooms, you're definitely in a simulation.

Saurabh Vij And if we can't hack out of the simulation, can we at least get the cheat codes? I love it, I love it. Gentlemen, thank you so, yeah, gentlemen, thank you so much for, oh, sorry, go ahead. I just have one question, you know, for the next audience, especially because they're from quantum computing world. Like, based on the, you know, announcements by Willow, the Google's quantum computer, one of the things that came out was, it was able to compute something that would take, that would have taken billions of years for a supercomputer.

It was able to do that, like, I think in five minutes or something. And one explanation, which is really weird and crazy right now is that it was able to go through multi-worlds. So it's a multi-world, you know, universe. And some of the people are saying that it's an evidence that we live in a multi-parallel universe kind of scenario. Now, the question, you know, for the next, you can say folks are, is there any other explanation that how was it able to, you know, really get the right answers within those five minutes?

Is this about, you can say, superposition of different scenarios? And collapsing onto a reality, or is it something really fundamentally different that is happening at the quantum level, which is actually able to deliver those results? Great question. Great question. I love it. Gentlemen, thanks so much for joining us. We'll be spreading the good word about Neo, keep up the great work and keep us posted on how things are developing over time.

Jeremy Gilbertson And maybe in, you know, a couple of quarters or whatever you guys pop back on and give us an update, but thanks so much for joining us. Awesome. Thank you so much. Have a great new year. Thank you. Take it easy, guys. Bye-bye. Thank you. Bye. All right. Disruptors and curious minds, you are now backstage with Mark and Jeremy as we slow our heads from melting about the idea of parsing off compute into multiple multiverses to have some compute happening over here, some happening over here, but it's all at the same time.

Mark Fielding Well, we spoke about that on Book Club when we were reading, which was quantum supremacy and maybe the order of time we spoke about that tunneling between worlds with information. And it's a great question that we can ask next week's guests. How are you feeling about Neo AI engineering and what you've just experienced and learned, Jeremy? It's really interesting. And it's a concept that's been throughout time.

Jeremy Gilbertson I think the idea of like automating and optimizing workflows that let us think about higher order things, I think is the important thing that landed with me. And Neo has apparently figured out a way to automate in a trustworthy way and an objectively provable way to engineers to offload certain tasks that they'd rather not do. And they can focus on solving bigger and more challenging problems.

I'm excited. I wanna learn more about, I wanna see that stuff in action. I wanna see someone use Neo to offload some stuff. And then as companies start having machine learning employees on staff to deal with these large data models and automating them and pointing agents to doing things. I don't know, what did you learn? What did you take away from this? And what did you not believe today?

What did you not believe today?

Mark Fielding Did you not believe anything today? Well, just on your point there, I saw a few videos which I'll put on the show when I write it up, thinking on paper.xyz, about people while using Monster API, which was their first company before Neo and they're using it to offload some of the work. And when I came into this doing the research, I was very surprised by how few engineers there are. I mean, they spoke about gaggle grandmasters.

I'm unsure what the difference between a gaggle grandmaster and one of the other 5,000 who are good. So I don't know what somebody who's number 5,000 could do as opposed to somebody who's a grandmaster. I mean, what the real difference is in their abilities. So when you're speaking about the greater creative endeavors, I think that just as in real normal life, away from AI, in any company, you get people who are tasked with creative thinking and people whose job is the mundane and the repetitive and the, what's the word that they said that you didn't like?

Oh, the grunt work. Grunt work. And I don't understand why there are so few engineers and it must be to do with that grunt work. And maybe it's just not much fun. And if you are a developer of any kind, there's more job satisfaction to be had elsewhere. Perhaps there's more money to be had in AI, but maybe there's more job satisfaction to be done elsewhere. And Neo taking away a lot of that, I see it as almost inevitable that it brings more people in and we get more of those developers that the world so desperately needs, apparently.

Like if there's gonna be a million AI companies, one thing about making predictions about the year 2030 that he says that there's gonna be 1 million AI companies, you could easily triple that, couldn't you? Quadruple it, I don't know, so. Yeah, well, how many startups have the AI nomenclature tied to them, right? And are they really AIs or is it just a little bolt-on that this but AI, right?

Jeremy Gilbertson So you have a lot of that. I think a call back to Julio Ottino, right? With Nexus and Nexus Thinking. And there are people that thrive in these divergent thinking spaces. And there are people that prefer more of the convergent mindset where they could be really 100% the best at one particular thing and do that one particular thing. And I think we're moving towards this polymathic tendency of society and of individuals trying to do interconnected things, I think.

And people are starting to be more open to that. The gig economy itself points into that, right? Yeah. Right, well, good stuff. This has been great. Hope you guys have enjoyed this new format. We're gonna continue experimenting with it. What didn't you believe? You asked me if I didn't believe anything. So that makes me assume that there was something that you didn't believe or you weren't sure of.

And as I just say that, there was maybe one thing that was interesting that I don't know if, it's not if I believe it or not, but he said the world wasn't ready for quantum computers. And I wonder if the world is ready for AI of this degree. I don't think the world is ready for AI. Like I think, but we can't, you know, we talked about it being a genie, right? And, you know, we can't stuff that thing back in the bottle, man.

And, you know, there's a lot of stuff that freaks me out a little bit about AI and, you know, offloading certain things. We talked about this a ton. Like if we push the easy button and we're susceptible as humans to pushing the easy button, we're optimization machines. We wanna try and find the shortcuts to everything in our thinking and all of that. And if we push this AI button more than pushing this brain button, like are we gonna lose our ability to creatively think and all that?

So that's always running in the back of my mind as we have these discussions. And I'm not saying AI, there won't be some benefit to it. I think there's tremendous benefit to it, but I think we gotta be careful. That's all I have on that.

Neo are not about that

Mark Fielding Just to be clear though, Neo aren't about making it easy, almost, I think about making it more difficult way.