Is AGI near? Let’s hope that profitability comes first
In September, OpenAI president Greg Brockman announced that they’d reached AGI, and it caused little stir because it’s no longer clear what exactly that means. And also because frontier model companies have spent so much money ‘flooding the zone’ with stories, including about how AI has a 10% chance of destroying humanity soon, people aren’t sure what to pay attention to.
What is artificial general intelligence? It’s probably best (now) described as human-equivalent intelligence, with reasoning and critical thinking enough to handle a wide range of tasks, personal and work, unobserved. By that level, we still aren’t there. Certainly not among the models now publicly available.
In the latest episode of 20VC, Scale Partner Rory O’Driscoll unpacks the AGI claim, what we can expect in terms of labor displacement, and why perhaps the harder question is, when will these models become profitable?
There will always be a role for humans
If you want a sense of what AI will do to skilled professions, look at what AI progenitor Geofrey Hinton said in 2016 of radiologists: “The profession is dead.” Fast forward to today and there have never been so many radiologists. Nor have they been paid so much. And this is despite the fact that much of the image processing and analysis can now be automated. What happened?
Partly, AI-affected professions will experience “induced demand,” whereby when a service becomes more accessible, or easier to use, it creates far more demand. This often overloads the system even more than before. For instance, adding lanes to a freeway often slows traffic further, because more people opt to drive.
According to Rory O’Driscoll, Scale Partner, despite the fact that 70% of radiology labs now use AI in some capacity:
“The remaining ‘5%’ of radiology work turned out to be more than enough to justify 100% of the radiologists.”
And partly, those professions are somewhat insulated by people’s desire to talk to other people—even if the artificial intelligence were to become general.
“You’re going to discover there are things that as humans, we prefer other humans do. At some point, you’re going to get a crappy diagnosis from a radiologist, as I’ve had, and you actually don’t want a machine telling you, ‘By the way you’re screwed, you’ve got cancer.’”
So too in coding—the same number of developers are still responsible for code even if they now rarely write it. And likewise in legal.
“You’re still going to have arguments with the opposing council and you're just not going to delegate it all to AI if it's significant.”
For startups pivoting and pricing in this space: Think about the portion of labor you can charge for augmenting that service professional.
“If you look at coding, there’s a credible argument that says every dollar you spend on labor, you spend fifty cents on coding. The annual subscription per lawyer is maybe $10k, plus or minus, and these lawyers are earning $200k, so that’s 5%. So it's a question of how much of that work can you eat up?”
AGI is fine, but does it generate economic value?
Ben Thompson of Stratechery said, ‘LLMs are the most scaled artifacts humanity has ever developed,” meaning, never before has so much knowledge been accessible in one service. It’s a profound collapsing of different repositories of information into one chat interface. The next step is of course figuring out how the companies who buy all that AI will rationalize that spend and measure its value.
“‘Most scaled artifacts’ is a great phrase, and it really stirs the imagination. I like that. We have this critical mass of companies using these things at scale. We’ll know what works because people will use it, because businesses are rational economic actors. And all these questions around AGI will be replaced by the question, ‘Is this model the one that generates the most economic value? In the most efficient fashion?’”
This is what some organizations are doing by trying to create indices to reveal token pricing and outputs. Investors should be talking to their portfolio companies about what they’re each using and how, for the truest sense of what’s economically useful. As a founder, it’s a great idea to keep in touch with peers and your team about the same: What are you finding useful enough to pay for?
What will AGI agents try that we would never consider?
Perhaps it’s trite to even cite it at this point, but back in 2016, the global champion of the 2,000-year-old game Go, Lee Sedol, lost to an artificial intelligence that made a move that no one had ever tried before. Which is remarkable, because there are 10¹⁸⁰ possible moves in that game, and in thousands of years, no human ever thought to attempt what the AI did.
This is predictive of what we’ve about to see as LLMs wander into that blurry realm of AGI:
“You really have two capabilities to consider here. They have this reasoning layer, but they also have a computational persistence to just keep trying things, exploring lots of alternatives. But because of the reasoning, it’s not just blindly trying a series of passwords, if it’s trying to get into something, it’s using deduction to try likely possibilities.”
This is why AGI, or anything close to it, poses such a grievous cybersecurity risk.
“If you say, ‘Hey I want to get tickets to the theater,’ maybe this agent that’s been trained on the entire corpus of the internet knows that the easiest path is to hack the reservation system and steal someone else’s seat. That's not a crazy option for it to try. So you end up writing rules and values to have it not do that, but you’ve never quite covered all the gaps. So it’s actually a very hard problem we’ll all be wrestling with. Especially with open source models from China running on servers in the Maldives doing whatever it takes to crack open your system. The threat level just goes exponential.”
Governmental regulations will do nothing to stop bad AGI actors
“We’re in the, ‘Lord help me before I sin again’ era with the frontier model companies. They’re saying they recognize their models are powerful, and can’t control them, ‘they now lie to us…’—and in the same statement, ‘We can’t stop because the other guys are going to have them anyway, so we need the government to step in and create some rules or code.’”
Rory isn’t sure government regulation will help.
“If we regulate OpenAI and Anthropic with all the noise that would have, I don’t know that helps with the problem, because if you’re worried about cyber, you’re worried about the North Koreans, the Iranians, the Russians, who don’t care anyway. So we’re going to need defenses that can deal with this, and maybe some kind of liability that attaches to running these models in ways that create dangers. I don’t know. But I don’t think it’s going to be a government agency that solves the problem.”
Physical AI takes longer
In September, Tesla’s autonomous Cybercabs launched in Austin with a modest 50 vehicles. Compared to Google Waymo’s LiDAR-enabled Jaguars, the camera-only Cybercabs are cheaper by far. But much less aware of their surroundings. That’s a bold bet with equal promise and risk.
“I think the Cybercab launch was a little underwhelming with 40-50 vehicles in Austin. The consensus was nice but slow wait times. Physical AI takes time, so the next step forward is a long journey, and you don’t often have the zero-to-one moment you sometimes get in the digital world.”
What Tesla does have going for it is they’re the only credible competitor to Google’s Waymo, after regulators shut down GM’s Cruise. But there will be more. Just this past month, Travis Kalanick of Uber fame just received funding from his old employer, Uber, for a driverless car startup. Uber was working on this for years while Kalanick was CEO there, but sold off the operations when they realized they just weren’t going to compete in such a capital-intensive operation against competitors like GM and Google.
“If you’re Travis, of course you’re going to want to scratch the itch of autonomy. And fine, you’ve got 100 million. But this hasn’t stopped being a very long-haul process. It’s probably smart to restart now that it may be near takeoff.”
Will the neolabs have viable businesses at scale?
No present LLM revenue stream seems like it will support neolab valuations.
“Neolabs are doing interesting stuff, but it’s not clear there’s a commercially viable standalone business here at scale. They still might be great investments, as the foundation model companies will go public and acquire for TAM expansion. But they won’t all be great investments.”
The fundamental problem is fundamentals. When the Adobe-Figma acquisition was scuttled by regulators, Figma could still fall back on a great revenue story.
“They could say, ‘Well, at least we’re doing a billion dollars in revenue. We’re growing at 40%. We’re worth something, goddamnit.’ Whereas many of the present AI businesses burning capital on compute, it can be tougher. When M&A deals fall through they have less to fall back on. If that were me and there was a backup bid, I’d hit that bid. Because otherwise it’s a long journey.”
Everyone is now in a risk-on race to AGI
Sometimes, the correct move is to exit quickly. That’s not always clear until in retrospect. But it’s worth remembering. Because as much as things feel like they’re changing now, they may only change more.
“I actually don’t think you should always stick it out. When I had my own business in the UK, I stuck it out for four years. Truly, I knew everything I needed in the first year. I shouldn’t have bothered for three more years, it was a waste of time. I look back and it’s so clear to me, right? But I stayed out of misguided loyalty.”
Right now, we’re seeing some companies that stuck it out and are finding the market has changed against them—Airtable being one. It recently sold equity at a valuation implying a worth of $2.25 billion, down from $11.7 billion some years prior. Or, InVision, which was one valued at $2 billion in its Series F round, which thereafter shut down in 2024. It didn’t sell off; it was so out-maneuvered by Figma that there wasn’t enough to sell.
“Airtable had run out of time and space, so I actually think the sale made sense. Especially when you are trying to be the platform. Just take Salesforce, the dominant SaaS company at $200 billion. Service cloud is worth 25%. The winner in the existing market is only worth $50 billion. So as you get these bigger market caps like Sierra, you have to go beyond the Service Cloud replacement to be a big company. You’re saying, ‘We want all your market cap, Mr. Salesforce, because the only way I can get to justify a $15-20 billion market cap is if I am the entire ecosystem and I go for your entire business.’ And then of course, everyone else has to follow."
