Sunday, November 24, 2024

Local AI: Ollama, GPT4All + Open WebUI

 Exploring Local LLM Managers: LMStudio, Ollama, GPT4All, and AnythingLLM : r/LocalLLM

There are a few programs that let you run AI language models locally on your own computer. LM Studio, Ollama, GPT4All, and AnythingLLM are some options.

These programs make it easier for regular people to experiment with and use advanced AI language models on their home PCs.

Jan: Open source ChatGPT-alternative that runs 100% offline - Jan.ai
Jan is an open source ChatGPT-alternative that runs 100% offline.


host ALL your AI locally - YouTube




Ollama vs GPT4All on Ubuntu Linux: Discover The Truth - YouTube

Llama 3.2 Vision + Ollama: Chat with Images LOCALLY - YouTube


Download Ollama on Windows

ollama/ollama: Get up and running with Llama 3.2, Mistral, Gemma 2, and other large language models. @GitHub

Running Meta Llama on Windows | Llama Everywhere

Running Meta Llama on Mac | Llama Everywhere

Hardware costs to run 90B llama at home? : r/LocalLLaMA

ollama/ollama: Get up and running with Llama 3.2, Mistral, Gemma 2, and other large language models. @GitHub

Ollama @GitHub (go app, node.js, python)

ollama/docs/gpu.md at main · ollama/ollama 
"Ollama supports (some) Nvidia GPUs"
"Ollama supports the (some) AMD GPUs"


Ollama models search

llama3.1 is a state-of-the-art model from Meta available in 8B, 70B and 405B parameter sizes.
size: from 4.7GB

A working Ollama installation: For 7B models, at least 8GB RAM is recommended. For 13B models, at least 16GB RAM is recommended. For 70B models, at least 64GB RAM is recommended.

> ollama run llama3.1:8b  
# 4.7GB, parameters 8B params, quantization 4_0
>>> tell me all tom hanks movies
/bye
> ollama pull codegemma


docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway \ -v open-webui:/app/backend/data --name open-webui \ --restart always ghcr.io/open-webui/open-webui:main
open http://localhost:3000/


Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)