• 4 Posts
  • 79 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle

  • Reliability of connection to the drives, especially during unscheduled power cycles. USB is known for random drops, or not picking the drive up before all your other services have started, and can cause the need for extra troubleshooting. Can run fine… or it could not. This is in reference to storage drives, not OS drives.












  • Picked up an AMD instinct mi25 to try and do just that. Can get easy-diffusion working after some cussing and voodoo. Cannot get rocm to do ANY llm of any kind, feels like a waste of video ram

    Also have a tesla p4 that runs most text-to-image models rather well, but have been unsuccessful at any llm either, even oobabooga can’t seem to run on it.

    Have given up because the software stack keeps advancing and leaving my hardware behind. I don’t have $3000 for an a100 or $1300 for an mi100 sooo… until the models can run on older/less powerful hardware, I’m probably sitting out of this game. Even though I’d love to be elbow deep in this one.



  • Any old Dell desktop/workstation/server should reach those specs. Poweredge rX30 and up, precision XX20/30 and up or optiplex (don’t know or understand that product line). Most of them are being rescued from the landfill. Might have to spend a crap ton over your budget, like 5-10x over, but you will get those specs.

    Look at an r430 barebones, no cpu/ram and build from there using spec sheets from Dell on what it takes. I was able to get one for $400 3 years back, even came with 16gb of ram and a single 10 core Xeon e5-v3.

    Also, what are you doing that need these kinds of specs? Running more than 10 VMs at once? Cloud gaming? Form follows function.