21 lines
540 B
Markdown
21 lines
540 B
Markdown
# llm-py-web
|
|
|
|
starlette + aftercare ui for llm by simonw.
|
|
requires uv, deno.
|
|
|
|
```shell
|
|
$ deno task setup
|
|
$ # any llm plugins: `uv pip install ...`
|
|
$ deno task run
|
|
```
|
|
|
|
i also use tsnsrv to expose the llm server internally in my tailnet:
|
|
|
|
```shell
|
|
$ # let's say $SOCK_PATH is /run/llm.sock
|
|
$ # in your llm-py-web systemd unit, have:
|
|
$ # Exec=/usr/bin/env deno task run --uds $SOCK_PATH
|
|
$ tsnsrv -name llm -stateDir /srv/llm/ts-state \
|
|
-ephemeral -suppressWhois \
|
|
-upstreamUnixAddr $SOCK_PATH http://llm-py-web
|
|
```
|