A billion dollars worth of optimism.

  • 2 Replies
  • 209 Views
*

infurl

  • Trusty Member
  • *********
  • Terminator
  • *
  • 840
  • Humans will disappoint you.
    • Home Page
A billion dollars worth of optimism.
« on: May 20, 2020, 04:53:34 AM »
https://venturebeat.com/2020/05/19/openai-microsoft-azure-supercomputer-ai-model-training/

Quote
Roughly a year ago, Microsoft announced it would invest $1 billion in OpenAI to jointly develop new technologies for Microsoft’s Azure cloud platform and to “further extend” large-scale AI capabilities that “deliver on the promise” of artificial general intelligence (AGI). In exchange, OpenAI agreed to license some of its intellectual property to Microsoft, which the company would then commercialize and sell to partners, and to train and run AI models on Azure as OpenAI worked to develop next-generation computing hardware.

Quote
The new Azure-hosted, OpenAI-co-designed machine contains over 285,000 processor cores, 10,000 graphics cards, and 400 gigabits per second of connectivity for each graphics card server. It was designed to train single massive AI models, which are models that learn from ingesting billions of pages of text from self-published books, instruction manuals, history lessons, human resources guidelines, and other publicly available sources. Examples include a natural language processing (NLP) model from Nvidia that contains 8.3 billion parameters, or configurable variables internal to the model whose values are used in making predictions; Microsoft’s Turing NLG (17 billion parameters), which achieves state-of-the-art results on a number of language benchmarks; Facebook’s recently open-sourced Blender chatbot framework (9.4 billion parameters); and OpenAI’s own GPT-2 model (1.5 billion parameters), which generates impressively humanlike text given short prompts.

From the article, in recent years the amount of compute power needed to research AI at the cutting edge has been doubling approximately every four months. On the other hand, efficiency gains have seen the amount of compute power needed to obtain any given result halving every sixteen months. Therefore I would expect this kind of endeavour to continue to climb further and further out of reach of mere hobbyists like us for the foreseeable future, even taking into account Moore's law which observes that available compute power doubles every couple of years (or something like that).

However, there's no guarantee that this is the way to artificial general intelligence and certainly no guarantee that it's the only way. The best bets in my view are on some combination of symbolic and stochastic methods and the former are still well and truly within our reach.

*

LOCKSUIT

  • Emerged from nothing
  • Trusty Member
  • ******************
  • Hal 4000
  • *
  • 4113
  • First it wiggles, then it is rewarded.
    • Main Project Thread

*

Art

  • At the end of the game, the King and Pawn go into the same box.
  • Global Moderator
  • **********************
  • Colossus
  • *
  • 5807
Re: A billion dollars worth of optimism.
« Reply #2 on: May 25, 2020, 01:15:04 PM »
An interesting "recipe" in this article but read the exchange between the user and bot. Notice how the bot is able to make the inference or assumption near the end.

https://parl.ai/projects/recipes/
In the world of AI, it's the thought that counts!

 


Users Online

71 Guests, 1 User
Users active in past 15 minutes:
squarebear
[Trusty Member]

Most Online Today: 91. Most Online Ever: 340 (March 26, 2019, 09:47:57 PM)

Articles