Instructions here: https://github.com/ghobs91/Self-GPT

If you’ve ever wanted a ChatGPT-style assistant but fully self-hosted and open source, Self-GPT is a handy script that bundles Open WebUI (chat interface front end) with Ollama (LLM backend).

  • Privacy & Control: Unlike ChatGPT, everything runs locally, so your data stays with you—great for those concerned about data privacy.
  • Cost: Once set up, self-hosting avoids monthly subscription fees. You’ll need decent hardware (ideally a GPU), but there’s a range of model sizes to fit different setups.
  • Flexibility: Open WebUI and Ollama support multiple models and let you switch between them easily, so you’re not locked into one provider.
  • camilobotero@feddit.dkEnglish
    0·
    2 months ago

    I can confirm that it does not run (at least not smoothly) with an Nvidia 4080 12Gb. However, gemma2:27B runs pretty well. Do you think if we add another graphical card, a modest one, maybe the llama3.1:70B could run?

    • brucethemoose@lemmy.worldEnglish
      1·
      2 months ago

      No, but you can run Qwen 2.5 34B with 24GB total.

      Host it in TabbyAPI instead of ollama too. Use its native tensor parallelism and Q4 cache, it will fly.