Hello, everyone.
Do you know that moment… you read a dry sheet… a single page and then your brain develops something like looking through a telescope at Andromeda and the vastness of the universe unfolds.
That’s how I felt after reading information about LLM Wiki according to Karpathy.
Who the hell is this Karpathy?
Oops, apparently a bright guy. He actually described in a rough sketch how we can split and divide information so that it can be preserved, evaluated, and used in the future:
Better known as LLM Wiki.
The basic principles, in my opinion:
-
The /raw data remains preserved, untouched, read-only, who knows what will come in 20 years.
-
Then, in a second layer, the current presentation, here pure text as an md file
-
In the third layer, the editing, processing, and evaluation by AI
Somehow, this ignited me. In my family, there is still a great-grandmother who is still somewhat fit to operate a dictation device.
But the great-grandchild at five years old is also already there, the old stories, experiences, and recipes and documents will disappear with almost certainty within 10 years.
Then we add a pinch of interest from me and it starts: A family wiki must be created, exactly according to these principles. The voice of the great-grandmother should be preserved, an AI should evaluate the data, create a wiki, and answer questions if necessary with the voice of the great-grandmother.
Exactly this is what I want to present to you here in a loose order:
A working model, simplest cost-effective hardware, it will take longer and sometimes the AI hallucinates or invents a lot, but let’s get started:
Condition:
Everything local, no internet, no registration, no API token, no one should care what one of the storytellers has to say.
Hardware: As a working model, existing hardware should be usable in exchange for time, so the mini computer may work through the night. I will go to sleep then.
In the end, not only a wiki should be created, but questions asked, for example: « Tell me a story about my grandmother with the voice of grandmother » should / must be possible.
Step 1:
Mini PC, Fedora system, 8 GB RAM.
No CPU, no GPU RAM… Snick snack for the pros 
Step 2:
We definitely need an Ollama, here as an example for Fedora, but at the same time solving the QNAP virtual switch difficulty (everything in the same network):
============================================================
2/3: Ollama (AI Server)
Runs the language models and the embedding model.
Started as the second.
============================================================
services:
test-ollama:
image: docker.io/ollama/ollama:latest
# This name is later used in AnythingLLM as an address:
# http://test-ollama:11434
container_name: test-ollama
restart: unless-stopped
user: "root"
ports:
# Accessible from the outside, e.g. for other devices in the home network.
# On Fedora, additionally release in the firewall:
# sudo firewall-cmd --permanent --add-port=11434/tcp
# sudo firewall-cmd --reload
- "11434:11434"
volumes:
# Downloaded models are stored here and remain after
# restarts. ":Z" because of SELinux.
- /srv/ki/ollama:/root/.ollama:Z
environment:
# Size of the text window per request. Ollama otherwise uses a
# small default value. 8192 is a compromise for 8 GB RAM,
# a larger value requires significantly more memory.
# Check after startup: podman exec -it test-ollama ollama ps
- OLLAMA_CONTEXT_LENGTH=8192
networks:
- my-ki-netz
networks:
my-ki-netz:
external: true
In Ollama, you pull in your model that fits your memory. With 8 GB RAM, you can expect: Long response time and many hallucinations
Step 3: Then you of course still need a Podman/Docker to display the wiki:
============================================================
Wiki-Viewer (SilverBullet)
Displays the wiki's Markdown pages in the browser so that
family members don't have to install anything. Read-only.
Call in the home network: http://:3000
Open firewall on Fedora:
firewall-cmd --permanent --add-port=3000/tcp
firewall-cmd --reload
Note: SilverBullet is designed for HTTPS. Display may only work
limitedly via plain HTTP. This only shows up in a browser test.
============================================================
services:
wiki-viewer-thomas:
# Full image name with registry, otherwise Podman will ask.
image:
container_name: wiki-viewer-thomas
restart: unless-stopped
ports:
# Left: Port on the server, right: Port in the container.
# For a second wiki (e.g., marlis) make a copy of this
# file with a different container_name and port, e.g. "3001:3000".
- "3000:3000"
volumes:
# Only the wiki/ folder is displayed, NOT raw/ (where the
# unchecked original texts are located).
# ":z" (small) instead of ":Z": The folder is also used by wiki.py and
# later possibly by a maintenance container. With ":Z"
# (capital) SELinux would give it exclusively to this one container.
- /srv/ki/llm-wiki/rueger/wiki:/space:z
environment:
# Access for the family as username:password.
# PLEASE CHANGE. Do not choose a password with a colon or
# space.
- SB_USER=familie:mannheim-2007
# Read-only: Pages can only be viewed in the browser,
# not changed. They are maintained exclusively by wiki.py.
- SB_READ_ONLY=true
- SB_INDEX_PAGE=web/index
… and what you get in return:
A refrigerator door, full of magnets and notes:
In the upper area, the round buttons are « magnets » with links to the people.
Below that, a series of « notes » with links to
Audio: List of all included audio recordings.
People: List of all AI-generated people
Places: The same, from the AI, all places
Events: The same, AI, all found events
Stories: All stories
Recipes: Grandma’s original recipes
Sources: Will be replaced with documents (marriage certificate, etc. etc.)
Below that:
Link to an input form, questions can be asked to the AI here, it answers depending on its model and RAM and time mostly correctly
In the lower area of our refrigerator door, a little help and support:
Coming Up: Still to be entered manually if a tag is discovered in the wiki, it will appear here in the month of the call, i.e.: If someone opens the wiki in November, they will get all the mentioned dates for the month of November, from the anniversary to the wedding day.
Below that, links for
Help
Data
Administration.
The path is basically always the same, the data comes to /raw. The AI gets the new files there, processes them and creates the wiki.
All of this with a single script… but that comes later.
As always: Suggestions, hints, additions welcome.
Greetings
Tom