starlette + aftercare frontend for simonw llm
Charlotte Som
77e7e0b0cf
simonw plugins (e.g. llm-anthropic, llm-gemini) don't send the system
prompt as a {role: system} message in build_messages, so the model will
'forget' its system intsructions if it's not retransmitted every time :/
|
||
|---|---|---|
| client | ||
| server | ||
| .gitignore | ||
| .prettierrc | ||
| _build_client.ts | ||
| deno.json | ||
| README.md | ||
| requirements.txt | ||
llm-py-web
starlette + aftercare ui for llm by simonw. requires uv, deno.
$ deno task setup
$ # any llm plugins: `uv pip install ...`
$ deno task run
i also use tsnsrv to expose the llm server internally in my tailnet:
$ # let's say $SOCK_PATH is /run/llm.sock
$ # in your llm-py-web systemd unit, have:
$ # Exec=/usr/bin/env deno task run --uds $SOCK_PATH
$ tsnsrv -name llm -stateDir /srv/llm/ts-state \
-ephemeral -suppressWhois \
-upstreamUnixAddr $SOCK_PATH http://llm-py-web