AI Weekly: China’s massive multimodal model highlights gaps in AI research


Elevate your technology and enterprise data strategy to Transform 2021.

This week, researchers at the Beijing Academy of Artificial Intelligence (BAAI) announced the release of Wu Dao 2.0, a multimodal AI model capable of generating indistinguishable text from man-made prose – and Moreover. Containing 1.75 trillion parameters, the parts of the machine learning model learned from historical training data, Wu Dao 2.0 is 10 times larger than OpenAI’s GPT-3 of 175 billion parameters.

Wu Dao 2.0 is the latest example of what OpenAI Policy Director Jack Clark calls model diffusion, or several state and private actors developing GPT-3 style AI models. For example, Russia and France are forming smaller-scale systems through LightOn’s Sberbank and PAGnol, while Naver Labs in Korea is investing in the newly created HyperCLOVA. Clark notes that because these models reflect and amplify the data they are trained on, different countries care about how their own cultures are represented in the models. The Wu Dao 2.0 announcement is therefore part of a general trend for nations to assert their own AI capabilities through frontier training models like GPT-3.

Wu Dao 2.0, which arrived three months after version 1.0 debuted in March, is built on an open source system similar to Google’s expert mix dubbed FastMoE. Mixture of Experts, a paradigm first proposed in the 1990s, maintains models specializing in different tasks within a larger model using a “control network”. BAAI states that Wu Dao 2.0 was formed with 4.9 terabytes of Chinese and English images and text on both supercomputer clusters and conventional GPUs, giving it more flexibility than Google’s system because FastMoE does not require proprietary hardware.

Wu Dao 2.0’s multimodal design gives it a range of skills, including the ability to perform natural language processing, text generation, image recognition, and image generation tasks. He can write essays, poems, and couplets in Traditional Chinese, as well as caption images and create almost photorealistic works of art, based on natural language descriptions. According to CommittedWu Dao 2.0 can also power “virtual idols” and predict 3D structures of proteins, like DeepMind’s AlphaFold.

“The path to general artificial intelligence is through large models and mainframe computers,” BAAI President Dr. Zhang Hongjiang said in a statement. “What we are building is a powerhouse for the future of AI. With big data, mega computing power and mega models, we can transform data to power the AI ​​applications of the future. “

AI nationalism

The release of Wu Dao 2.0 comes amid a rise in technological nationalism globally, particularly in China and parts of the eurozone. Last November, China imposed new rules on technology exports, with the country’s Commerce Ministry adding 23 items to its shortlist. Following Nvidia’s announcement of plans to acquire UK chipmaker Arm, the majority of UK region IT experts have said the government should step in to protect the country’s tech sector, according to to a survey by the industry’s professional body (The Chartered Institute for IT).

Former US chief technology officer Michael Kratsios, among others, has suggested that adversaries of the state pursue uses of AI technologies that “are not in line with American values.” In February, the White House mentionned that would bring non-defense AI investments to $ 2 billion per year by 2022, while US President Joe Biden has offers an increase in the amount of federal R&D spending to $ 300 billion over four years. And a U.S. Senate panel last month approved the Endless Frontier Act, a pending law that would authorize more than $ 110 billion for basic and advanced technological research over five years.

But America’s superiority in AI is an increasingly bleak prospect. France recently unveiled a $ 1.69 billion (€ 1.5 billion) initiative to transform the country into a “world leader” in AI research and training. In 2018, South Korea unveiled a multi-year, $ 1.95 billion (KRW 2.2 trillion) effort to boost its AI R&D, with the goal of establishing six AI-focused graduate schools by 2022 and train 5,000 AI specialists. And China, whose AI innovation action plan for colleges and universities called for the creation of 50 new AI institutions in 2020, is expected to overtake the European Union in the next few years if current trends change. continue.

BAAI is funded by the Beijing government, which invested 340 million yuan ($ 53.3 million) in the academy in 2018 and 2019 alone. A Beijing official has pledged to continue supporting in a 2019 speech.

In March, former Google CEO Eric Schmidt urged lawmakers to increase funding in the AI ​​space to prevent China from becoming the biggest player in the global AI market. Schmidt suggested doubling the national AI R&D budget each year until it reaches $ 32 billion in 2026. Citing the United States National Security Commission on Artificial Intelligence, Schmidt also said that lawmakers should encourage public-private partnerships to develop AI applications in government agencies.

“The government is not prepared today for this new technology,” Schmidt told CNN’s Fareed Zakaria, noting that the use of AI to produce and disseminate harmful information poses a “threat to democracy” and could ultimately be used as a weapon of war. “We believe this is a national emergency and a threat to our nation unless we take our steps to focus on AI in federal government and international security.”

So how can the United States make up for lost ground, despite the many challenges ahead? Last July, the Presidential Council of Science and Technology Advisors (PCAST) issued a report outlining what he says needs to happen for the United States to advance “industries of the future,” including AI. PCAST ​​recommended generating opportunities for AI education and training, in part by securing pledges to increase investment in the training and education of the US AI workforce; develop AI curricula and performance measures from Kindergarten to Grade 12 through postgraduate levels and for certificate and professional programs; create incentive, recruitment and retention programs for AI professors in universities; and increase the investments of the National Science Foundation and the Department of Education in AI educators, scientists and technologists at all levels.

Last August, in a step towards these goals, the White House established 12 new research institutes focused on AI and quantum information science. But the release of Wu Dao 2.0 highlights the work that needs to be done before the United States can close the AI ​​gap with other global superpowers.

For AI coverage, send topical advice to Kyle Wiggers – and be sure to subscribe to the AI ​​Weekly newsletter and bookmark our AI channel, The Machine.

Thanks for reading,

Kyle wiggers

IA personal writer


VentureBeat’s mission is to be a digital public place for technical decision-makers to learn about transformative technology and conduct transactions. Our site provides essential information on data technologies and strategies to guide you in managing your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the topics that interest you
  • our newsletters
  • Closed thought leader content and discounted access to our popular events, such as Transform 2021: Learn more
  • networking features, and more

Become a member

Source link


About Author

Rachel Amaral

Comments are closed.