Last week, Facebook founder Mark Zuckerberg announced that he’s going to purchase hundreds of thousands of expensive AI processing chips — and experts are mighty worried about what he plans to use them for.

In the same Instagram post announcing his planned purchase of 350,000 Nvidia’s H100 graphics chips, which average about $30,000 apiece and are considered the gold standard for powering AI models, Zuckerberg said that he wants to build an open-source artificial AGI, the industry term for the point at which AI reaches or even surpasses human-level intelligence.

While there’s still an open debate about whether AGI is even possible, the prospect itself is enough to give some researchers pause.

“The thought of open source AGI being released before we have worked out how to regulate these very powerful AI systems is really very scary,” Dame Wendy Hall, a computer scientist at the University of Southampton in England and a member of the United Nations’ AI advisory board, told The Guardian. “In the wrong hands technology like this could do a great deal of harm. It is so irresponsible for a company to suggest it.”

AGI is likely still “many years away,” Hall added, which gives society time to get a regulatory framework in order. Nevertheless, “it is a matter of public safety that we progress this work with some urgency,” she said.

Open Scaries

Andrew Rogoyski, one of the directors of the Institute for People-Centred AI at the University of Surrey, echoed the Hall’s concerns about Meta being a potential player in AGI.

“There are deep and complex arguments about the merits of open-sourcing current AI models, pushing that into the realm of AGI could be world-saving or catastrophic,” Rogoyski told the newspaper. “These decisions need to be taken by international consensus, not in the boardroom of a tech giant.”

Meta’s open-source AI dreams have long drawn criticism. Last summer, Hall even told The Guardian that open-sourcing the company’s Llama 2 large language model (LLM) was a “bit like giving people a template to build a nuclear bomb” because of how accessible it is for anyone to use.

Meta, of course, is far from the only company trying to build AGI. But among its competitors like OpenAI and Google DeepMind, it seems to be the only one that wants to open source the potentially world-changing models — raising a host of concerns.