Add support for S3-backed image storage
This commit is contained in:
1 parent
7fcb2eb747
commit
8763a8e74a
55 files changed
+2611
-111
No files matched your search
@@ -50,7 +50,7 @@ or `PODMAN` to use another container command, for example `make PODMAN=docker`.
|
||||
5. Install `sqlx-cli`: `cargo install sqlx-cli`.
|
||||
6. Run migrations: `sqlx migrate run`.
|
||||
7. Set environment variables:
|
||||
* `DATABASE_URL` - the URL of your MySQL database (`mysql://user[:password]@domain/database`)
|
||||
- `DATABASE_URL` - the URL of your MySQL database (`mysql://user[:password]@domain/database`)
|
||||
8. Build: `cargo build --release`
|
||||
|
||||
### Configuring
|
||||
@@ -58,32 +58,76 @@ or `PODMAN` to use another container command, for example `make PODMAN=docker`.
|
||||
Reminder Bot reads a number of environment variables. Some are essential, and others have hardcoded
|
||||
fallbacks. Environment variables can be loaded from a .env file in the working directory.
|
||||
|
||||
__Required Variables__
|
||||
**Required Variables**
|
||||
|
||||
* `DATABASE_URL` - the URL of your MySQL database (`mysql://user[:password]@domain/database`)
|
||||
* `DISCORD_TOKEN` - your application's bot user's authorization token
|
||||
- `DATABASE_URL` - the URL of your MySQL database (`mysql://user[:password]@domain/database`)
|
||||
- `DISCORD_TOKEN` - your application's bot user's authorization token
|
||||
|
||||
__Other Variables__
|
||||
**Other Variables**
|
||||
|
||||
* `MIN_INTERVAL` - default `600`, defines the shortest interval the bot should accept
|
||||
* `LOCAL_TIMEZONE` - default `UTC`, necessary for calculations in the natural language processor
|
||||
* `SUBSCRIPTION_ROLES` - default `None`, accepts a list of Discord role IDs that are given to
|
||||
subscribed users
|
||||
* `CNC_GUILD` - default `None`, accepts a single Discord guild ID for the server that the
|
||||
subscription roles belong to
|
||||
* `PYTHON_LOCATION` - default `/usr/bin/python3`. Can be changed if your Python executable is
|
||||
located somewhere else
|
||||
* `THEME_COLOR` - default `8fb677`. Specifies the hex value of the color to use on info message
|
||||
embeds
|
||||
- `MIN_INTERVAL` - default `600`, defines the shortest interval the bot should accept
|
||||
- `LOCAL_TIMEZONE` - default `UTC`, necessary for calculations in the natural language processor
|
||||
- `SUBSCRIPTION_ROLES` - default `None`, accepts a list of Discord role IDs that are given to
|
||||
subscribed users
|
||||
- `CNC_GUILD` - default `None`, accepts a single Discord guild ID for the server that the
|
||||
subscription roles belong to
|
||||
- `PYTHON_LOCATION` - default `/usr/bin/python3`. Can be changed if your Python executable is
|
||||
located somewhere else
|
||||
- `THEME_COLOR` - default `8fb677`. Specifies the hex value of the color to use on info message
|
||||
embeds
|
||||
|
||||
### Dashboard image uploads
|
||||
|
||||
Image URL inputs offer an **Upload image** button when all of these settings are supplied:
|
||||
|
||||
- `S3_BUCKET` — destination bucket name.
|
||||
- `S3_REGION` — bucket region, for example `eu-west-2`.
|
||||
- `S3_ACCESS_KEY_ID` and `S3_SECRET_ACCESS_KEY` — credentials with `s3:PutObject` and
|
||||
`s3:DeleteObject` permissions for the bucket's `uploads/` prefix.
|
||||
- `S3_PUBLIC_URL` — public base URL for the bucket or CDN, for example
|
||||
`https://my-bucket.s3.eu-west-2.amazonaws.com`. Objects must be publicly readable through
|
||||
this URL; the application does not change bucket policies or set object ACLs.
|
||||
|
||||
`NSFW_CHECK_ENABLED` defaults to `true`, and enables the use of a small model to block uploading
|
||||
of NSFW images. Set it to `false` to skip NSFW moderation and model initialization. Empty or
|
||||
unrecognized values keep checks enabled.
|
||||
|
||||
## Running with Docker
|
||||
|
||||
### RustFS for local development
|
||||
|
||||
Start the local object store and initialize the image upload bucket:
|
||||
|
||||
```sh
|
||||
docker compose --env-file conf/rustfs.env -f compose.rustfs.yaml up -d
|
||||
```
|
||||
|
||||
Add the settings from `conf/rustfs.env` to your local `.env`, then restart the bot running on
|
||||
your host. RustFS serves the S3 API at `http://127.0.0.1:9000` and its console at
|
||||
`http://127.0.0.1:9001`. Log into the console with the access key and secret key from
|
||||
`conf/rustfs.env`. Both ports bind to localhost. The named data volume persists across restarts.
|
||||
|
||||
The initializer creates `reminder-images` if needed and permits anonymous reads of its
|
||||
`uploads/` objects. Uploads, deletes and bucket listing require credentials. These settings
|
||||
enable dashboard image uploads locally; `NSFW_CHECK_ENABLED` keeps its configured value.
|
||||
`S3_ENDPOINT` uses path-style addressing for RustFS, and `S3_PUBLIC_URL` includes the bucket
|
||||
name because the browser fetches objects through that public URL.
|
||||
|
||||
```sh
|
||||
docker compose --env-file conf/rustfs.env -f compose.rustfs.yaml logs rustfs-init
|
||||
docker compose --env-file conf/rustfs.env -f compose.rustfs.yaml down
|
||||
```
|
||||
|
||||
`down` keeps the data volume. This development stack is separate from the bot/database stack
|
||||
below. The [RustFS Docker documentation](https://docs.rustfs.com/en/installation/container/docker)
|
||||
describes the upstream container configuration.
|
||||
|
||||
A `compose.yml` file is provided to aid in running the bot agnostically using docker.
|
||||
|
||||
* Populate a `.env` file as in `conf/default.env`
|
||||
* Add the additional variable `ROCKET_SECRET_KEY` with a key generated from
|
||||
`head -c64 /dev/urandom | base64`
|
||||
* Run `docker compose up`
|
||||
- Populate a `.env` file as in `conf/default.env`
|
||||
- Add the additional variable `ROCKET_SECRET_KEY` with a key generated from
|
||||
`head -c64 /dev/urandom | base64`
|
||||
- Run `docker compose up`
|
||||
|
||||
Please note that this is _not_ production-ready when run via compose. We do not offer a way for
|
||||
backing up of your data, or a way to run the dashboard securely via HTTPS, which is required for
|
||||
|
||||
Reference in new issue
Block a user