Keeping up with an industry as fast-moving as AI is a tall order. So until an AI can do it for you, here’s a handy roundup of recent stories in the world of machine learning, along with notable research and experiments we didn’t cover on their own.
This week, it was impossible to tune out — for this reporter included, much to my sleep-deprived brain’s dismay — the leadership controversy surrounding AI startup OpenAI. The board ousted Sam Altman, CEO and a co-founder, allegedly over what they saw as misplaced priorities on his part: commercializing AI at the expense of safety.
Altman was — in large part thanks to the efforts of Microsoft, a major OpenAI backer — reinstated as CEO and most of the original board replaced. But the saga illustrates the perils of AI companies, even those as large and influential as OpenAI, as the temptation to tap into… monetization-oriented sources of funding grows ever-stronger.
It’s not that AI labs necessarily want to become enmeshed with commercially-aligned, hungry-for-returns venture firms and tech giants. It’s that the sky-high costs of training and developing AI models makes it nigh impossible to avoid this fate.
According to CNBC, the process of training a large language model such as GPT-3, the predecessor to OpenAI’s flagship text-generating AI model, GPT-4, could cost over $4 million. That estimate doesn’t factor in the cost of hiring data scientists, AI experts and software engineers — all of whom command high salaries.
It’s no accident that many large AI labs have strategic agreements with public cloud providers; compute, especially at a time when the chips to train AI models are in short supply (benefiting vendors like Nvidia), has become more valuable than gold to these labs. Chief OpenAI rival Anthropic has taken on investments from both Google and Amazon. Cohere and Character.ai, meanwhile, have the backing of Google Cloud, which is also their exclusive compute infrastructure provider.
But, as this week showed, these investments come at a risk. Tech giants have their own agendas — and the weight to throw around to see their bidding done.
OpenAI attempted to maintain some independence with a unique, “capped-profit” structure that limits investors’ total returns. But Microsoft showed that compute can be just as valuable as capital in bringing a startup to heel; much of Microsoft’s investment in OpenAI is in the form of Azure cloud credits, and the threat of withholding these credits would be enough to get any board’s attention.
Barring massively increased investments in public supercomputing resources or AI grant programs, the status quo doesn’t look likely to change soon. AI startups of a certain size — like most startups — are forced to cede control over their destinies if they wish to grow. Hopefully, unlike OpenAI, they make a deal with the devil they know.
Here are some other AI stories of note from the past few days:
- OpenAI downfall exaggerated: Has OpenAI created AI tech with the potential to “threaten humanity”? From some recent headlines, it might seem so. But experts say there’s no cause for alarm here.
- California to regulate AI: California’s Privacy Protection Agency is preparing to impose regulations on AI usage. The state privacy regulator recently released draft regulations for how people’s data can be used for AI, taking inspiration from existing rules in the European Union.
- Bard’s new YouTube feature: Google has announced that its Bard AI chatbot can now answer questions about YouTube videos. Although Bard already had the ability to analyze YouTube videos with the launch of the YouTube Extension back in September, the chatbot can now give specific answers about queries related to the content of a video.
- Launch of X’s Grok: After screenshots emerged showing xAI’s chatbot Grok appearing on X’s web app, X owner Elon Musk confirmed that Grok would be available to all of the company’s Premium+ subscribers sometime this week. While Musk’s pronouncements about time frames for product deliveries haven’t always held up, code developments in X’s own app reveal that Grok integration is well underway.
- Stability AI’s video generator: AI startup Stability AI last week announced Stable Video Diffusion, an AI model that generates videos by animating existing images. Based on Stability’s existing Stable Diffusion text-to-image model, Stable Video Diffusion is one of the few video-generating models available in open source — or commercially, for that matter.
- Anthropic’s Claude 2.1 release: Anthropic recently released Claude 2.1, an improvement on its flagship large language model that keeps it competitive with OpenAI’s GPT series. Devin writes that the new update to Claude has three major improvements: context window, accuracy, and extensibility.
- OpenAI and the openness dilemma: Paul writes that the OpenAI debacle has highlighted the forces that control the burgeoning AI revolution, leading many to question what happens if you go all-in on a centralized proprietary player — and what happens if things then go belly-up.
- AI21 Labs secures funding: AI21 Labs, a company developing generative AI products along the lines of OpenAI’s GPT-4 and ChatGPT, last week raised $53 million — bringing its total raised to $336 million. A Tel Aviv-based startup creating a range of text-generating AI tools, AI21 Labs was founded in 2017 by Mobileye co-founder Amnon Shashua, Ori Goshen and Yoav Shoham, the startup’s other co-CEO.
More machine learnings
Making AI models more transparent about when they need more information to produce a confident answer is a difficult problem, since really, the model doesn’t know the difference between right and wrong. But by making the model expose its inner workings a bit, you can get a better sense of when it’s more likely to be uncertain.
This work by Purdue creates a human-readable “Reeb map” of how the neural network represents visual concepts in its vector space. Items it deems similar are grouped together, and overlaps with other areas could indicate either similarities between those groups or confusion on the model’s part. “What we’re doing is taking these complicated sets of information coming out of the network and giving people an ‘in’ into how the network sees the data at a macroscopic level,” said lead researcher David Gleich.