Add support for S3-backed image storage

This commit is contained in:
jude committed 2026-10-11 17:16:37 +01:00
1 parent 7fcb2eb747
commit ea184c8056
24 files changed
+2443 -92

No files matched your search

+44
View File
@@ -76,6 +76,50 @@ __Other Variables__
* `THEME_COLOR` - default `8fb677`. Specifies the hex value of the color to use on info message
embeds
### Optional dashboard image uploads
Image URL inputs offer an **Upload image** button when all of these settings are supplied:
* `S3_BUCKET` — destination bucket name.
* `S3_REGION` — bucket region, for example `eu-west-2`.
* `S3_ACCESS_KEY_ID` and `S3_SECRET_ACCESS_KEY` — credentials with `s3:PutObject` and
`s3:DeleteObject` permissions for the bucket's `uploads/` prefix.
* `S3_PUBLIC_URL` — public base URL for the bucket or CDN, for example
`https://my-bucket.s3.eu-west-2.amazonaws.com`. Objects must be publicly readable through
this URL; the application does not change bucket policies or set object ACLs.
Optionally set `S3_ENDPOINT` for an S3-compatible provider; this uses path-style addressing.
The ONNX model in `assets/nsfw/model.onnx` is embedded in the binary at compile time using
`include_bytes!`; no model file, download or model-path setting is needed at runtime, including
in containers. The bundled model adds approximately 16.5 MiB to the binary and is initialized
once at startup only when S3 uploads are configured and NSFW checks are enabled.
See [model provenance](assets/nsfw/README.md).
Missing or invalid S3 configuration disables uploads while the dashboard continues to accept
image URLs. If the bundled model fails to initialize, uploads are disabled as well.
`NSFW_CHECK_ENABLED` defaults to `true`. Set it to `false`, `0` or `off` to skip NSFW
moderation and model initialization; `true`, `1` or `on` enables checks. Values are
case-insensitive. Empty or unrecognized values keep checks enabled. Restart the application
after changing this setting. Image format validation, size limits and uploader tracking still
apply when moderation is disabled.
When enabled, `POST /dashboard/api/images` accepts the image file as the raw request body with
`Content-Type: application/octet-stream`, using an authenticated dashboard session. It accepts
PNG, JPEG, GIF and WebP up to 8 MiB and 16 million pixels. Files are decoded, flattened onto a
white background and saved as static PNGs; animation and metadata are removed. By default, the `nsfw` crate
checks the exact pixels that will be published, rejecting images when the combined Porn, Hentai
and Sexy probability is at least 50%. Rejections return HTTP 451 and the dashboard shows an
information modal. Moderation failures reject the upload without publishing it.
Successful uploads return `{ "url": "https://.../uploads/....png" }` and record the uploader's
Discord ID, public URL and upload time in `image_uploads`. The database migration runs with the
other migrations. If recording fails, the application attempts to remove the S3 object and
returns an error. The endpoint is not mounted when uploads are disabled.
For local image-based moderation tests, place split image pairs in `tests/fixtures/nsfw/`
and `tests/fixtures/non_nsfw/`. The [fixture instructions](tests/fixtures/README.md) describe
the splitter script and the opt-in tests that reconstruct the images before classification.
## Running with Docker
A `compose.yml` file is provided to aid in running the bot agnostically using docker.