[MikeB]

https://ai.meta.com/llama/
https://www.llama2.ai/

This release includes model weights and starting code for pretrained and fine-tuned Llama language models — ranging from 7B to 70B parameters.

Llama 2 outperforms other open source language models on many external benchmarks, including reasoning, coding, proficiency, and knowledge tests.

Scores 80.2 on the Winogrande benchmark. Benchmarks are listed in the first URL.

Very similar politician/spokes-person style to both ChatGPT and Google Bard.

[frankinstien]

Have you tried the local AI LLMs like:

https://github.com/qwopqwop200/GPTQ-for-LLaMa
or
FreedomGPT:


Trying to get a gpu from google colab is tricky at best. I did look into another cloud gpu service which was charging, for the new H100s, 1.89/hr. But, they wanted 3 years up front and it's a 24/7 charge, that's like $50,000!  You can get one on eBay for $30K But the A100s have come down in price to as low as $5000.

I recently purchased a RTX 3080 ti for $560 which replaced my M40. I like google colab and other cloud solutions but it becomes a real problem to get the high end gpus. If you're just needing inferencing a local gpu gets you in a better position. If you want to train those LLMs then the cloud solution is a good choice as long as you don't forget to signoff when your done, store your model on a cloud drive. But that's what I'm looking at now when it comes to customizing very Large LMs.

The LLMs can be converted to 4bit inferences which reduces the memory footprint quite a bit, so gpus like RTX 4090s look more interesting. Which now brings to mind some really interesting tools to literally build a virtual person:





Now integrate the new AR glasses like XReal and you're looking at a pretty cool humaniod robot that you can take with you anywhere! 

[Freddy]

I assume that the local version of FreedomGPT is limited in some way with just 5GB of data?

[frankinstien]

I assume that the local version of FreedomGPT is limited in some way with just 5GB of data?

Try it, just download the version that will work with your OS.

[Freddy]

I am reluctant to try it as I don't know what is in it.

[frankinstien]

Scan it with anti-virus. I tried and it worked fine.

[8pla.net]

I wonder if LLaMA2 works with llama-cpp-php by Bohuslav Šimek, and/or with llama.cpp by Georgi Gerganov ?

Reference: https://github.com/ggerganov/llama.cpp


Image source: Bohuslav Šimek <bohuslav@simek.si>  Accessed via https://packagist.org/packages/kambo/llama-cpp-php MIT License.  Permitted to use, copy, modify, merge publish, distribute, sublicense and sell copies of the software..

Image Modification:  Zoomed in for easier reading by older people for discussion purposes only: There is a scroll bar at the bottom of the image to read the entire image.

[8pla.net]

Yes!

https://github.com/ggerganov/llama.cpp

Supported platforms:
...
LLaMA 2