Common Crawl

  • 1 Replies
  • 101 Views
*

infurl

  • Trusty Member
  • *********
  • Terminator
  • *
  • 764
  • Humans will disappoint you.
    • Home Page
Common Crawl
« on: March 02, 2020, 03:52:09 AM »
https://commoncrawl.org/

Quote
The Common Crawl corpus contains petabytes of data collected over the last 7 years. It contains raw web page data, extracted metadata and text extractions.

Need a lot of data off the internet to feed your artificial intelligence project? You can find it at Common Crawl, a completely free and open project to retrieve and archive vast numbers of web sites. Save yourself a huge amount of time and hassle by using their pre-packaged data. This is the closest thing to having the entire internet in a box that you're ever going to find.


*

Dat D

  • Bumblebee
  • **
  • 42
  • AI rocks!
Re: Common Crawl
« Reply #1 on: March 03, 2020, 02:22:54 AM »
https://commoncrawl.org/

Quote
The Common Crawl corpus contains petabytes of data collected over the last 7 years. It contains raw web page data, extracted metadata and text extractions.

Need a lot of data off the internet to feed your artificial intelligence project? You can find it at Common Crawl, a completely free and open project to retrieve and archive vast numbers of web sites. Save yourself a huge amount of time and hassle by using their pre-packaged data. This is the closest thing to having the entire internet in a box that you're ever going to find.
I sometimes crawl the net myself, Google Puppeteer is superb!

https://developers.google.com/web/tools/puppeteer

 


Terra Sentia - Robot
by krayvonk (Robotics News)
April 07, 2020, 03:47:09 PM
Understanding Mice.
by krayvonk (AI News )
April 07, 2020, 02:10:47 AM
Accelerating data-driven discoveries
by Tyler (Robotics News)
April 05, 2020, 12:01:32 PM
Q&A: Markus Buehler on setting coronavirus and AI-inspired proteins to music
by Tyler (Robotics News)
April 03, 2020, 12:01:02 PM
Walker
by krayvonk (Robotics News)
April 01, 2020, 03:23:26 AM
Cruzr Robot
by Art (Robotics News)
March 31, 2020, 08:41:34 PM

Users Online

11 Guests, 1 User
Users active in past 15 minutes:
infurl
[Trusty Member]

Most Online Today: 22. Most Online Ever: 340 (March 26, 2019, 09:47:57 PM)

Articles