Genre mixes, discover UI, tracked playlist management and logging #1

Merged
jude merged 9 commits from discover into master 2026-11-21 21:56:31 +00:00

Adds genre mixes built from clustering recent listens, a web UI for listening to them alongside new songs that aren't in the library, a UI for managing tracked playlists, and proper logging.

Genre mixes

A new cluster_mix_rule clusters a Listenbrainz user's last few months of listens by MusicBrainz genre (pulling together tracks played in the same session) and keeps one Subsonic playlist per cluster. Each mix is a share of recently played tracks (familiar_ratio) plus unheard library songs from the best-matching library genres. Playlists are reused across rebuilds by genre overlap, so they keep their identity.

Discover UI (discover/)

A React app, served by playlistd when discover.ui is set.

  • Each rebuild also suggests songs that aren't in the library: Listenbrainz recommendations the user has never played, plus an LB Radio prompt per mix when listenbrainz.token is set. Each suggestion goes to the mix it fits best by genre.
  • Library songs stream from Subsonic; suggested songs are found on YouTube with yt-dlp. Both are proxied through playlistd with Range support, so seeking works.
  • "Play mix" plays the new songs with known songs mixed in at familiar_ratio.
  • Suggestions can be liked (kept in "Liked new songs") or hidden.

Tracked playlists UI

A "Tracked playlists" page, backed by new /api/tracked routes, lists, creates, edits, refreshes and removes tracked playlists and recurring playlist rules.

  • New playlists can be created in Subsonic or picked from existing ones.
  • Rule names preview live as you type.
  • Invalid name templates are rejected; they used to panic the create daemon.
  • Genre mix playlists can't be tracked, since two daemons would overwrite each other's songs.

The existing /api/playlist routes are unchanged.

Logging

  • env_logger is installed at startup. Set the level with log.level (or LOG_LEVEL); RUST_LOG overrides it.
  • Under systemd, lines carry their syslog priority, so journalctl -p warning works.
  • All println!s are now log macros.
  • Repeated failures are logged once, plus once when they recover.
  • Listenbrainz errors are readable, e.g. an unknown user. A period without stats yet (HTTP 204) leaves the playlist alone instead of failing to decode.
  • Fixes along the way:
    • daemons ran twice at startup
    • a failing playlist rule panicked its task

Docker Compose

  • compose.yaml runs playlistd with a Postgres database. Settings go in .env; see .env.example.
  • The Dockerfile builds the UI and server and runs them on Debian slim as an unprivileged user, with the latest yt-dlp and Deno (which yt-dlp needs for YouTube). Rebuild the image to update yt-dlp.
  • playlistd has no login, so its port is only published on 127.0.0.1 unless PLAYLISTD_BIND is set.
  • The image can't reach a database while building, so the sqlx query data is now committed in .sqlx/. After changing a query, run cargo sqlx prepare.

Upgrading

  • One new migration (20261113150000_cluster_mixes.sql) runs automatically.
  • New dependencies: rand, serde_json, futures-util, tokio-util, env_logger.
  • New optional config (see etc/playlistd/playlistd.toml):
    • listenbrainz.token
    • discover.ui
    • discover.ytdlp
    • discover.format
    • log.level
  • The .deb now includes the web UI, installed to /usr/share/playlistd/discover and served by the default config. The README's container build command builds the UI first; the Containerfile installs a pinned, checksum-verified Node.js for it.
  • The package recommends yt-dlp for playing suggested songs. Ubuntu 22.04's yt-dlp (2022.04) is too old for current YouTube, so install a recent one, e.g. from pip.

Testing

  • 11 unit tests: clustering, mix matching, genre fit, template and input validation, YouTube ID validation.
  • End to end against a mock Subsonic server whose library came from a real Listenbrainz user's history, with real Listenbrainz data. That covered:
    • mix builds and rebuilds
    • suggestions
    • streaming with seeking
    • like, hide and undo
    • the tracked playlist and rule flows
    • logging at each level and under journald
  • Browser checks with Playwright, in light, dark and phone layouts.
  • Built the Compose image and ran the stack with a test .env. Checked: Postgres healthy before playlistd starts, migrations run, UI served, Subsonic on the host reached through host.docker.internal, port not exposed beyond the host, data kept across down/up, yt-dlp detects Deno, and it runs as a non-root user.
  • Built the image from the Containerfile and the .deb with the README's command in it, installed the package on a clean Ubuntu 22.04 with apt, and checked the UI and API are served. cargo deb fails as intended when the UI hasn't been built.

Not tested here:

  • Navidrome: no real instance was reachable from the sandbox.
  • YouTube: no network access to it. A stand-in yt-dlp with the same arguments and output exercised that path, and the real yt-dlp accepted the arguments.
  • Listenbrainz with no stats yet (HTTP 204): not reproducible with the test account.

Generated with Claude Code

https://claude.ai/code/session_018opPJMJjLWr4RqdG6vuK6p

Adds genre mixes built from clustering recent listens, a web UI for listening to them alongside new songs that aren't in the library, a UI for managing tracked playlists, and proper logging. ## Genre mixes A new `cluster_mix_rule` clusters a Listenbrainz user's last few months of listens by MusicBrainz genre (pulling together tracks played in the same session) and keeps one Subsonic playlist per cluster. Each mix is a share of recently played tracks (`familiar_ratio`) plus unheard library songs from the best-matching library genres. Playlists are reused across rebuilds by genre overlap, so they keep their identity. ## Discover UI (`discover/`) A React app, served by playlistd when `discover.ui` is set. - Each rebuild also suggests songs that aren't in the library: Listenbrainz recommendations the user has never played, plus an LB Radio prompt per mix when `listenbrainz.token` is set. Each suggestion goes to the mix it fits best by genre. - Library songs stream from Subsonic; suggested songs are found on YouTube with yt-dlp. Both are proxied through playlistd with Range support, so seeking works. - "Play mix" plays the new songs with known songs mixed in at `familiar_ratio`. - Suggestions can be liked (kept in "Liked new songs") or hidden. ## Tracked playlists UI A "Tracked playlists" page, backed by new `/api/tracked` routes, lists, creates, edits, refreshes and removes tracked playlists and recurring playlist rules. - New playlists can be created in Subsonic or picked from existing ones. - Rule names preview live as you type. - Invalid name templates are rejected; they used to panic the create daemon. - Genre mix playlists can't be tracked, since two daemons would overwrite each other's songs. The existing `/api/playlist` routes are unchanged. ## Logging - env_logger is installed at startup. Set the level with `log.level` (or `LOG_LEVEL`); `RUST_LOG` overrides it. - Under systemd, lines carry their syslog priority, so `journalctl -p warning` works. - All `println!`s are now log macros. - Repeated failures are logged once, plus once when they recover. - Listenbrainz errors are readable, e.g. an unknown user. A period without stats yet (HTTP 204) leaves the playlist alone instead of failing to decode. - Fixes along the way: - daemons ran twice at startup - a failing playlist rule panicked its task ## Docker Compose - `compose.yaml` runs playlistd with a Postgres database. Settings go in `.env`; see `.env.example`. - The `Dockerfile` builds the UI and server and runs them on Debian slim as an unprivileged user, with the latest yt-dlp and Deno (which yt-dlp needs for YouTube). Rebuild the image to update yt-dlp. - playlistd has no login, so its port is only published on 127.0.0.1 unless `PLAYLISTD_BIND` is set. - The image can't reach a database while building, so the sqlx query data is now committed in `.sqlx/`. After changing a query, run `cargo sqlx prepare`. ## Upgrading - One new migration (`20261113150000_cluster_mixes.sql`) runs automatically. - New dependencies: `rand`, `serde_json`, `futures-util`, `tokio-util`, `env_logger`. - New optional config (see `etc/playlistd/playlistd.toml`): - `listenbrainz.token` - `discover.ui` - `discover.ytdlp` - `discover.format` - `log.level` - The `.deb` now includes the web UI, installed to `/usr/share/playlistd/discover` and served by the default config. The README's container build command builds the UI first; the Containerfile installs a pinned, checksum-verified Node.js for it. - The package recommends `yt-dlp` for playing suggested songs. Ubuntu 22.04's yt-dlp (2022.04) is too old for current YouTube, so install a recent one, e.g. from pip. ## Testing - 11 unit tests: clustering, mix matching, genre fit, template and input validation, YouTube ID validation. - End to end against a mock Subsonic server whose library came from a real Listenbrainz user's history, with real Listenbrainz data. That covered: - mix builds and rebuilds - suggestions - streaming with seeking - like, hide and undo - the tracked playlist and rule flows - logging at each level and under journald - Browser checks with Playwright, in light, dark and phone layouts. - Built the Compose image and ran the stack with a test `.env`. Checked: Postgres healthy before playlistd starts, migrations run, UI served, Subsonic on the host reached through `host.docker.internal`, port not exposed beyond the host, data kept across `down`/`up`, yt-dlp detects Deno, and it runs as a non-root user. - Built the image from the Containerfile and the `.deb` with the README's command in it, installed the package on a clean Ubuntu 22.04 with apt, and checked the UI and API are served. `cargo deb` fails as intended when the UI hasn't been built. Not tested here: - **Navidrome:** no real instance was reachable from the sandbox. - **YouTube:** no network access to it. A stand-in yt-dlp with the same arguments and output exercised that path, and the real yt-dlp accepted the arguments. - **Listenbrainz with no stats yet** (HTTP 204): not reproducible with the test account. Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018opPJMJjLWr4RqdG6vuK6p
claude Bot added 5 commits 2026-11-14 09:01:42 +00:00
A new cluster_mix_rule clusters a Listenbrainz user's listens from the
last few months by MusicBrainz genre (with tracks played in the same
session pulled together) and maintains one Subsonic playlist per cluster,
filled with the cluster's most played tracks plus unheard library songs
from the best matching library genres.

- listenbrainz: fetch raw listens since a timestamp and bulk recording
  genres, handling rate limits
- cluster: tag TF-IDF + co-listen smoothing, spherical k-means with k
  chosen by silhouette score
- mix: match cluster profiles to library genres and pick songs, with a
  per-artist cap; reuse playlists across rebuilds by genre overlap
- subsonic: getGenres, getSongsByGenre, rename/delete playlist, genre and
  MBID fields on tracks, API error detection
- daemon, migration, POST/GET /api/playlist/mix and README docs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018opPJMJjLWr4RqdG6vuK6p
Each mix rebuild now also finds songs that aren't in the library: the
user's Listenbrainz recommendations they've never played, plus an LB
Radio prompt per mix when a Listenbrainz token is configured. Each
candidate goes to the mix whose genre profile it fits best and is
skipped if it's already in the library or was played recently.

discover/ is a new React app to listen to the mixes: library songs
stream from Subsonic and suggested songs are found on YouTube with
yt-dlp, both proxied through playlistd with Range support for seeking.
Suggestions can be liked (kept in a "Liked new songs" list) or hidden.

- discover: candidate gathering, genre fit scoring, library filtering
- web: /api/discover routes, Subsonic/YouTube streaming proxy
- youtube: yt-dlp search (skipping hour-long results) and audio URLs
- mix tags now hold library genre names, for display in the UI
- config: listenbrainz.token, discover.ui, discover.ytdlp, discover.format

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018opPJMJjLWr4RqdG6vuK6p
"Play mix" in the discover UI used to spread new songs through the whole
library playlist, so the share of new songs depended on how many there
were. It now plays the new songs with songs the user has played mixed in
evenly at the rule's familiar_ratio (30% by default), the same ratio that
sets how much of the Subsonic playlist is familiar.

Which songs in each mix playlist were played recently is now recorded
when the mix is built (cluster_mix_playlist.known_song_ids) and returned
with the mix; the UI marks them as "Played". Mixes built before this fall
back to using the library songs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018opPJMJjLWr4RqdG6vuK6p
A "Tracked playlists" page in the web UI lists, creates, edits, refreshes
and removes tracked playlists, and manages the rules that create a new
tracked playlist each period.

- New /api/tracked routes. Input is validated with readable errors,
  including name templates, which made the create daemon panic when they
  had an invalid % code.
- New playlists can be created in Subsonic or picked from existing ones.
  Genre mix playlists can't be tracked, since two daemons would fight
  over their songs.
- "Refresh" updates a playlist straight away, and the page shows when
  some of the top songs couldn't be found in the library.
- Removing a playlist can also delete it from Subsonic.
- Rule names preview live as you type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018opPJMJjLWr4RqdG6vuK6p
Rocket's logger was the only one, so messages from the daemons depended on
it having launched, and most messages bypassed logging with println!.

- Install env_logger at startup: level from log.level (LOG_LEVEL), with
  RUST_LOG overriding it. Rocket's per-request logs and sqlx's Postgres
  notices are kept to warnings.
- Under systemd, prefix lines with their syslog priority so journald
  records levels (journalctl -p warning works).
- Replace every println! with a log macro at a fitting level.
- Recurring failures are a warning when they start or change, debug while
  they repeat, and an info message when they recover.
- Listenbrainz errors are readable: an unknown user is "Listenbrainz user
  "x" doesn't exist" rather than a JSON decode error, and a period without
  stats yet (204) leaves the playlist alone instead of failing to decode.
  The UI's Refresh shows the same messages.
- Warnings name the playlist or rule they're about.
- Songs missing from the library are debug output, not errors every tick.
- Daemons no longer run twice at startup (interval's first tick is
  immediate), and a failing playlist rule or daemon is logged instead of
  panicking the task.
- Subsonic errors display as messages rather than Debug output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018opPJMJjLWr4RqdG6vuK6p
claude Bot added 1 commit 2026-11-14 09:48:00 +00:00
The three migrations added on this branch (cluster mixes, mix suggestions,
known songs) haven't been released, so they're combined into
20261113150000_cluster_mixes.sql. The resulting schema is identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018opPJMJjLWr4RqdG6vuK6p
claude Bot added 1 commit 2026-11-14 10:02:43 +00:00
- Package discover/dist to /usr/share/playlistd/discover, and serve it
  from there in the default config. cargo deb refuses to package if the
  UI hasn't been built, rather than shipping without it.
- Install a pinned Node.js (22.23.4, checksum-verified) in the
  Containerfile; Ubuntu 22.04's is too old for Vite. The README's build
  command now builds the UI first.
- Recommend yt-dlp, for playing suggested songs from YouTube.
- If discover.ui points to a missing directory, warn instead of letting
  Rocket panic at startup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018opPJMJjLWr4RqdG6vuK6p
claude Bot added 1 commit 2026-11-14 10:22:26 +00:00
compose.yaml runs playlistd with a Postgres database. Settings come from
.env (see .env.example), and Postgres must be healthy before playlistd
starts.

The Dockerfile builds the web UI with Node and the server with Rust, then
copies both into a Debian slim image that runs as an unprivileged user. It
also installs the latest yt-dlp (with its YouTube scripts) and Deno, the
JavaScript runtime yt-dlp needs for YouTube. playlistd's port is only
published on 127.0.0.1 by default, since it has no login.

The image can't reach a database during the build, so the SQL query data
from `cargo sqlx prepare` is now committed in .sqlx/. Builds with
DATABASE_URL set still check queries against that database.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018opPJMJjLWr4RqdG6vuK6p
jude added 1 commit 2026-11-16 20:09:37 +00:00
The first visit to the UI creates the account from a username and password;
after that every API route needs a session cookie from logging in. Accounts
are stored in app_user (Argon2id password hashes) and sessions in
user_session (SHA-256 token hashes, 30 days).

The account page lets the user save a Listenbrainz token, checked with
Listenbrainz first. It takes priority over listenbrainz.token from the
config for LB Radio suggestions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jude merged commit f5cb5076ca into master 2026-11-21 21:56:31 +00:00
Sign in to join this conversation.
No Reviewers
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: jude/playlistd#1