You are here: Home / Purchased 8x 32GB Nvidia GPUs Server Beast! 256GB Nvidia V100 AGX3 6u Server

Purchased 8x 32GB Nvidia GPUs Server Beast! 256GB Nvidia V100 AGX3 6u Server

Now own MONSTROUS 256GB VRAM 8x Nvidia V100 32GB VRAM GPUs each (AGX3), 6u just picked up to add to my personal at-home R&D Lab.
Purchased 8x 32GB Nvidia GPUs Server Beast! 256GB Nvidia V100 AGX3 6u Server

Wow!

Now own MONSTROUS 256GB VRAM 8x Nvidia V100 32GB VRAM GPUs each (AGX3), 6u just picked up to add to my personal at-home R&D Lab.

This beast is HEAVY, had to carry it myself downstairs.

Going to have to upgrade power in my basement considerably (6x 16 Amps 240v (not 110v) each per power supply! (96 amps total)) Electricians are coming out Monday for quotes to add some more power panels to my home.

Can't wait to get this up and running! Will take these following projects to a whole other level than my NVidia 5090, 4090, 4080, 4070, and 3090 DGPUNET (https://dgpunet.com) setup could do:

I have just enough (6u) slots left in the bottom of my rack for it (though it is going to stick a good 8-10 inches out the back it is so long)!

 

 

 

This reminds me when I had an e4000 Sun system on my desktop for months when I transitioned from ITI to Franklin Covey.

I have been so bottlenecked on my AI being limited to the Nvidia 5090 32GB limit. Even with my DGPUNET enhancements, which have helped a LOT to break through that limitation, I was still stuck below 100GB VRAM capacity.

Now with 256GB VRAM, I'll be able to effectively train 30B to even 70B models from scratch, and potentially larger models retraining (not from scratch).

And I should be able to potentially squeeze running up to 405B+ models too!

This should make it possible for me to run my full suite of:

  • AILCPH
  • SIIMPAF
  • RPEPTFS
  • and other tools

all simultaneously. Whereas before I could only run a piece here and there, and demo the components, now I can potentially run them all at once to show their full potential come to life!

This would cost me anywhere from $2k low-end to $30k+ PER MONTH at AWS or elsewhere "in the cloud"! I'm going to take a few hundred dollar a month hit on the power bill when it is running.

And it is going to cost me some thousands of dollars to upgrade my house to be able to even run this beast (need 96 Amps of 240volt!), but it is so much more affordable than the cloud, and opens up all kinds of possibilities with my development that I haven't been able to utilize previously!

Stay tuned for more to come soon!

 

Navigation