llm-py-web

starlette + aftercare ui for llm by simonw. requires uv, deno.

$ deno task setup
$ # any llm plugins: `uv pip install ...`
$ deno task run

i also use tsnsrv to expose the llm server internally in my tailnet:

$ # let's say $SOCK_PATH is /run/llm.sock
$ # in your llm-py-web systemd unit, have:
$ # Exec=/usr/bin/env deno task run --uds $SOCK_PATH
$ tsnsrv -name llm -stateDir /srv/llm/ts-state \
   -ephemeral -suppressWhois \
   -upstreamUnixAddr $SOCK_PATH http://llm-py-web