[frankinstien]

The fact that you can get a GPU for free is amazing but the horse power is pretty limited and the better GPUs are very expensive to rent let alone buy and hard to reserve. Particularly if you have to train a model for a month! I have an old server with dual Xeons and 128GB of ram and it's just sitting there doing nothing. I recently priced the Instinct MI25s that boast a 24 TFlops F16 with 16GB for $80! If I purchase 4 of those I'd have 96TFlops and 64GB of ram.  Of course the model's architectures would have to be MoE(Mixture of Experts) and the routing done by the CPU end. But in comparison to an A100 40GB on eBay at $7500 to $14,000 to a $320 expense I'm just about to pull the trigger.

The problem however is power, the 4 Instinct MI25 GPUs would draw 300 watts each. However...I do have some 2500 watts in solar panels hanging around.  Of course I could buy more GPUs, say double it which amounts to $640 that then leads to 192 TFlops F16 and 128GB of gpu ram, with power draw of 2400 watts! 

[WriterOfMinds]

The fact that you can get a GPU for free is amazing but the horse power is pretty limited

Where can one get a GPU for free, I'm intrigued. Are you just referring to the limited free trial/low-use plans of cloud services?

[frankinstien]

Google Collab...

[MagnusWootton]

my gpu on my comp is overheating bad,  cant do any largish workloads with it,  better not get knockoff video cards they have defects.