68 lines
3.2 KiB
Markdown
68 lines
3.2 KiB
Markdown
The two tiny ONNX models are deterministic classifiers used by the image upload API tests.
|
|
They accept the same `[1, 224, 224, 3]` float input as the production nsfw model and return
|
|
`[Drawings, Hentai, Neutral, Porn, Sexy]` scores. One always returns Neutral = 1 and the
|
|
other always returns Porn = 1. They exercise the real nsfw inference path without requiring
|
|
image fixtures.
|
|
|
|
## Image fixtures for the bundled model
|
|
|
|
Add image pairs to these project directories:
|
|
|
|
* `tests/fixtures/nsfw/` — images expected to be rejected by moderation.
|
|
* `tests/fixtures/non_nsfw/` — images expected to be accepted by moderation.
|
|
|
|
Install the splitter's dependency with `python3 -m pip install Pillow`, then run from the
|
|
project root:
|
|
|
|
```sh
|
|
python3 scripts/split_nsfw_test_image.py /path/to/input.jpg --output-dir tests/fixtures/nsfw
|
|
python3 scripts/split_nsfw_test_image.py /path/to/safe.jpg --output-dir tests/fixtures/non_nsfw
|
|
```
|
|
|
|
Each command automatically selects the first unused positive index, starting with
|
|
`image_1a.png` and `image_1b.png`. Indices with either part already present are skipped.
|
|
You can override the selection with `--index 2`; existing parts are never overwritten.
|
|
Each part contains random-looking
|
|
RGBA pixels. The original decoded pixels, including alpha, are recovered by XORing the two
|
|
parts. Only the first frame of animated input is used. Keep both files unchanged: resizing,
|
|
re-encoding with loss or flattening their alpha channels prevents correct reconstruction.
|
|
Both parts together reconstruct the original image, so splitting is not access control.
|
|
|
|
Run the splitter's round-trip and overwrite checks with
|
|
`python3 -m unittest discover -s scripts -p 'test_*.py'`.
|
|
|
|
The directories initially contain no images. The two dataset tests are explicitly ignored
|
|
until you provide fixtures. To run them after adding pairs to both directories:
|
|
|
|
```sh
|
|
SQLX_OFFLINE=true cargo test --locked directory_images -- --ignored
|
|
```
|
|
|
|
The tests load all numbered pairs, recombine them in memory and pass the reconstructed PNG
|
|
through the production upload moderation function using the bundled model and its current
|
|
threshold. They also check that disabling moderation accepts the reconstructed images.
|
|
Empty directories, missing partners, mismatched dimensions, unexpected names and unexpected
|
|
moderation results fail the tests. Recombined images are never written to disk.
|
|
|
|
## Deterministic model fixtures
|
|
|
|
To regenerate them with Python's `onnx` package:
|
|
|
|
```python
|
|
from pathlib import Path
|
|
from onnx import TensorProto, checker, helper
|
|
|
|
for name, scores in [("safe", [0., 0., 1., 0., 0.]), ("unsafe", [0., 0., 0., 1., 0.])]:
|
|
tensor = helper.make_tensor("probabilities", TensorProto.FLOAT, [1, 5], scores)
|
|
graph = helper.make_graph(
|
|
[helper.make_node("Identity", ["probabilities"], ["output"])],
|
|
"constant classifier for upload endpoint tests",
|
|
[helper.make_tensor_value_info("input", TensorProto.FLOAT, [1, 224, 224, 3])],
|
|
[helper.make_tensor_value_info("output", TensorProto.FLOAT, [1, 5])],
|
|
[tensor],
|
|
)
|
|
model = helper.make_model(graph, opset_imports=[helper.make_opsetid("", 13)], ir_version=8)
|
|
checker.check_model(model)
|
|
Path(f"tests/fixtures/image-upload-{name}.onnx").write_bytes(model.SerializeToString())
|
|
```
|