Compare commits

..

3 Commits

Author SHA1 Message Date
shamoon
198ee6ea09 Just some stuff 2024-12-20 22:13:53 -08:00
shamoon
112b39c4de Lol this works 2024-12-20 20:09:30 -08:00
shamoon
01706dd391 Just messing around 2024-12-20 17:08:59 -08:00
626 changed files with 83985 additions and 122227 deletions

117
.devcontainer/README.md Normal file
View File

@@ -0,0 +1,117 @@
# Paperless NGX Development Environment
## Overview
Welcome to the Paperless NGX development environment! This setup uses VSCode DevContainers to provide a consistent and seamless development experience.
### What are DevContainers?
DevContainers are a feature in VSCode that allows you to develop within a Docker container. This ensures that your development environment is consistent across different machines and setups. By defining a containerized environment, you can eliminate the "works on my machine" problem.
### Advantages of DevContainers
- **Consistency**: Same environment for all developers.
- **Isolation**: Separate development environment from your local machine.
- **Reproducibility**: Easily recreate the environment on any machine.
- **Pre-configured Tools**: Include all necessary tools and dependencies in the container.
## DevContainer Setup
The DevContainer configuration provides up all the necessary services for Paperless NGX, including:
- Redis
- Gotenberg
- Tika
Data is stored using Docker volumes to ensure persistence across container restarts.
## Configuration Files
The setup includes debugging configurations (`launch.json`) and tasks (`tasks.json`) to help you manage and debug various parts of the project:
- **Backend Debugging:**
- `manage.py runserver`
- `manage.py document-consumer`
- `celery`
- **Maintenance Tasks:**
- Create superuser
- Run migrations
- Recreate virtual environment (`.venv` with pipenv)
- Compile frontend assets
## Getting Started
### Step 1: Running the DevContainer
To start the DevContainer:
1. Open VSCode.
2. Open the project folder.
3. Open the command palette:
- **Windows/Linux**: `Ctrl+Shift+P`
- **Mac**: `Cmd+Shift+P`
4. Type and select `Dev Containers: Rebuild and Reopen in Container`.
VSCode will build and start the DevContainer environment.
### Step 2: Initial Setup
Once the DevContainer is up and running, perform the following steps:
1. **Compile Frontend Assets**:
- Open the command palette:
- **Windows/Linux**: `Ctrl+Shift+P`
- **Mac**: `Cmd+Shift+P`
- Select `Tasks: Run Task`.
- Choose `Frontend Compile`.
2. **Run Database Migrations**:
- Open the command palette:
- **Windows/Linux**: `Ctrl+Shift+P`
- **Mac**: `Cmd+Shift+P`
- Select `Tasks: Run Task`.
- Choose `Migrate Database`.
3. **Create Superuser**:
- Open the command palette:
- **Windows/Linux**: `Ctrl+Shift+P`
- **Mac**: `Cmd+Shift+P`
- Select `Tasks: Run Task`.
- Choose `Create Superuser`.
### Debugging and Running Services
You can start and debug backend services either as debugging sessions via `launch.json` or as tasks.
#### Using `launch.json`:
1. Press `F5` or go to the **Run and Debug** view in VSCode.
2. Select the desired configuration:
- `Runserver`
- `Document Consumer`
- `Celery`
#### Using Tasks:
1. Open the command palette:
- **Windows/Linux**: `Ctrl+Shift+P`
- **Mac**: `Cmd+Shift+P`
2. Select `Tasks: Run Task`.
3. Choose the desired task:
- `Runserver`
- `Document Consumer`
- `Celery`
### Additional Maintenance Tasks
Additional tasks are available for common maintenance operations:
- **Recreate .venv**: For setting up the virtual environment using pipenv.
- **Migrate Database**: To apply database migrations.
- **Create Superuser**: To create an admin user for the application.
## Let's Get Started!
Follow the steps above to get your development environment up and running. Happy coding!

View File

@@ -3,26 +3,14 @@
"dockerComposeFile": "docker-compose.devcontainer.sqlite-tika.yml",
"service": "paperless-development",
"workspaceFolder": "/usr/src/paperless/paperless-ngx",
"postCreateCommand": "pipenv install --dev && pipenv run pre-commit install",
"postCreateCommand": "/bin/bash -c pre-commit install && pipenv install --dev",
"customizations": {
"vscode": {
"extensions": [
"mhutchie.git-graph",
"ms-python.python",
"ms-vscode.js-debug-nightly",
"eamodio.gitlens",
"yzhang.markdown-all-in-one"
],
"settings": {
"python.defaultInterpreterPath": "/usr/src/paperless/paperless-ngx/.venv/bin/python",
"python.pythonPath": "/usr/src/paperless/paperless-ngx/.venv/bin/python",
"python.terminal.activateEnvInCurrentTerminal": true,
"editor.formatOnPaste": false,
"editor.formatOnSave": true,
"editor.formatOnType": true,
"files.trimTrailingWhitespace": true
}
"extensions": [
"mhutchie.git-graph",
"ms-python.python"
]
}
},
"remoteUser": "paperless"
}
},
"remoteUser": "paperless"
}

View File

@@ -27,7 +27,7 @@ services:
image: docker.io/library/redis:7
restart: unless-stopped
volumes:
- ./redisdata:/data
- redisdata:/data
# No ports need to be exposed; the VSCode DevContainer plugin manages them.
paperless-development:
@@ -43,16 +43,14 @@ services:
volumes:
- ..:/usr/src/paperless/paperless-ngx:delegated
- ../.devcontainer/vscode:/usr/src/paperless/paperless-ngx/.vscode:delegated # VSCode config files
- pipenv:/usr/src/paperless/paperless-ngx/.venv
- pipenv:/usr/src/paperless/paperless-ngx/.venv # Pipenv environment persisted in volume
- /usr/src/paperless/paperless-ngx/src/documents/static/frontend # Static frontend files exist only in container
- /usr/src/paperless/paperless-ngx/src/.pytest_cache
- /usr/src/paperless/paperless-ngx/.ruff_cache
- /usr/src/paperless/paperless-ngx/htmlcov
- /usr/src/paperless/paperless-ngx/.coverage
- ./data:/usr/src/paperless/paperless-ngx/data
- ./media:/usr/src/paperless/paperless-ngx/media
- ./consume:/usr/src/paperless/paperless-ngx/consume
- ~/.gitconfig:/usr/src/paperless/.gitconfig:ro
- data:/usr/src/paperless/paperless-ngx/data
- media:/usr/src/paperless/paperless-ngx/media
environment:
PAPERLESS_REDIS: redis://broker:6379
PAPERLESS_TIKA_ENABLED: 1
@@ -80,4 +78,7 @@ services:
restart: unless-stopped
volumes:
data:
media:
redisdata:
pipenv:

View File

@@ -2,57 +2,42 @@
"version": "0.2.0",
"configurations": [
{
"name": "Chrome: Debug Angular Frontend",
"description": "Debug the Angular Dev Frontend in Chrome",
"type": "chrome",
"request": "launch",
"url": "http://localhost:4200",
"webRoot": "${workspaceFolder}/src-ui",
"preLaunchTask": "Start: Frontend Angular"
},
{
"name": "Debug: Backend Server (manage.py runserver)",
"description": "Debug the Django Backend Server",
"name": "manage.py runserver",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/src/manage.py",
"args": [
"runserver"
],
"django": true,
"console": "integratedTerminal",
"env": {
"PYTHONPATH": "${workspaceFolder}/src"
},
"python": "${workspaceFolder}/.venv/bin/python"
"justMyCode": true,
"args": ["runserver"],
"django": true
},
{
"name": "Debug: Consumer Service (manage.py document_consumer)",
"description": "Debug the Consumer Service which processes files from a directory",
"name": "manage.py document_consumer",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/src/manage.py",
"args": [
"document_consumer"
],
"django": true,
"console": "integratedTerminal",
"justMyCode": true,
"args": ["document_consumer"],
"django": true
},
{
"name": "celery",
"type": "python",
"cwd": "${workspaceFolder}/src",
"request": "launch",
"module": "celery",
"console": "integratedTerminal",
"env": {
"PYTHONPATH": "${workspaceFolder}/src"
},
"python": "${workspaceFolder}/.venv/bin/python"
}
],
"compounds": [
{
"name": "Debug: FullStack",
"description": "Debug run the Angular dev frontend, Django backend, and consumer service",
"configurations": [
"Chrome: Debug Angular Frontend",
"Debug: Backend Server (manage.py runserver)",
"Debug: Consumer Service (manage.py document_consumer)"
],
"preLaunchTask": "Start: Celery Worker"
},
"args": [
"-A",
"paperless",
"worker",
"-l",
"DEBUG"
]
}
]
}

View File

@@ -1,84 +1,27 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "Start: Celery Worker",
"description": "Start the Celery Worker which processes background and consume tasks",
"type": "shell",
"command": "pipenv run celery --app paperless worker -l DEBUG",
"isBackground": true,
"options": {
"cwd": "${workspaceFolder}/src"
},
"problemMatcher": [
{
"owner": "custom",
"pattern": [
{
"regexp": ".",
"file": 1,
"location": 2,
"message": 3
}
],
"background": {
"activeOnStart": true,
"beginsPattern": "celery.*",
"endsPattern": "ready"
}
}
]
{
"label": "manage.py document_consumer",
"type": "shell",
"command": "pipenv run python manage.py document_consumer",
"group": "build",
"presentation": {
"echo": true,
"reveal": "always",
"focus": false,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}/src"
}
},
{
"label": "Start: Frontend Angular",
"description": "Start the Frontend Angular Dev Server",
"type": "shell",
"command": "npm start",
"isBackground": true,
"options": {
"cwd": "${workspaceFolder}/src-ui"
},
"problemMatcher": [
{
"owner": "custom",
"pattern": [
{
"regexp": ".",
"file": 1,
"location": 2,
"message": 3
}
],
"background": {
"activeOnStart": true,
"beginsPattern": ".*",
"endsPattern": "Compiled successfully"
}
}
]
},
{
"label": "Start: Consumer Service (manage.py document_consumer)",
"description": "Start the Consumer Service which processes files from a directory",
"type": "shell",
"command": "pipenv run python manage.py document_consumer",
"group": "build",
"presentation": {
"echo": true,
"reveal": "always",
"focus": false,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}/src"
}
},
{
"label": "Start: Backend Server (manage.py runserver)",
"description": "Start the Backend Server which serves the Django API and the compiled Angular frontend",
"label": "manage.py runserver",
"type": "shell",
"command": "pipenv run python manage.py runserver",
"group": "build",
@@ -94,130 +37,100 @@
"options": {
"cwd": "${workspaceFolder}/src"
}
},
{
"label": "Maintenance: manage.py migrate",
"description": "Apply database migrations",
"type": "shell",
"command": "pipenv run python manage.py migrate",
"group": "none",
"presentation": {
"echo": true,
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}/src"
}
{
"label": "Maintenance: manage.py migrate",
"type": "shell",
"command": "pipenv run python manage.py migrate",
"group": "none",
"presentation": {
"echo": true,
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
{
"label": "Maintenance: Build Documentation",
"description": "Build the documentation with MkDocs",
"type": "shell",
"command": "pipenv run mkdocs build --config-file mkdocs.yml && pipenv run mkdocs serve",
"group": "none",
"presentation": {
"echo": true,
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}"
}
},
{
"label": "Maintenance: manage.py createsuperuser",
"description": "Create a superuser",
"type": "shell",
"command": "pipenv run python manage.py createsuperuser",
"group": "none",
"presentation": {
"echo": true,
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}/src"
}
},
{
"label": "Maintenance: recreate .venv",
"description": "Recreate the python virtual environment and install python dependencies",
"type": "shell",
"command": "rm -R -v .venv/* || pipenv install --dev",
"group": "none",
"presentation": {
"echo": true,
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}"
}
},
{
"label": "Maintenance: Install Frontend Dependencies",
"description": "Install frontend (npm) dependencies",
"type": "npm",
"script": "install",
"path": "src-ui",
"group": "clean",
"problemMatcher": [],
"detail": "install dependencies from package"
},
{
"description": "Clean install frontend dependencies and build the frontend for production",
"label": "Maintenance: Compile frontend for production",
"type": "shell",
"command": "npm ci && ./node_modules/.bin/ng build --configuration production",
"group": "none",
"presentation": {
"echo": true,
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}/src-ui"
}
},
{
"label": "Project Setup: Run all Init Tasks",
"description": "Runs all init tasks to setup the project including migrate the database, create a superuser and compile the frontend for production",
"dependsOrder": "sequence",
"dependsOn": [
"Maintenance: manage.py migrate",
"Maintenance: manage.py createsuperuser",
"Maintenance: Compile frontend for production"
]
},
{
"label": "Project Start: Run all Services",
"description": "Runs all services required to start the project including the Celery Worker, the Consumer Service and the Backend Server",
"dependsOn": [
"Start: Celery Worker",
"Start: Consumer Service (manage.py document_consumer)",
"Start: Backend Server (manage.py runserver)"
]
"options": {
"cwd": "${workspaceFolder}/src"
}
},
{
"label": "Maintenance: manage.py createsuperuser",
"type": "shell",
"command": "pipenv run python manage.py createsuperuser",
"group": "none",
"presentation": {
"echo": true,
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}/src"
}
},
{
"label": "compile frontend",
"type": "shell",
"command": "npm ci && ./node_modules/.bin/ng build --configuration production",
"group": "none",
"presentation": {
"echo": true,
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}/src-ui"
}
},
{
"label": "Maintenance: recreate .venv",
"type": "shell",
"command": "rm -R -v .venv/* || pipenv install --dev",
"group": "none",
"presentation": {
"echo": true,
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}"
}
},
{
"label": "Celery Worker",
"type": "shell",
"command": "pipenv run celery --app paperless worker -l DEBUG",
"group": {
"kind": "build",
"isDefault": true
},
"presentation": {
"echo": true,
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"clear": true,
"revealProblems": "onProblem"
},
"options": {
"cwd": "${workspaceFolder}/src"
}
}
]
}
}

View File

@@ -26,5 +26,3 @@
./dist
./scripts
./resources
# Other stuff
**/*.drawio.png

View File

@@ -2,10 +2,10 @@ blank_issues_enabled: false
contact_links:
- name: 🤔 Questions and Help
url: https://github.com/paperless-ngx/paperless-ngx/discussions
about: General questions or support for using Paperless-ngx.
about: This issue tracker is not for support questions. Please refer to our Discussions.
- name: 💬 Chat
url: https://matrix.to/#/#paperlessngx:matrix.org
about: Want to discuss Paperless-ngx with others? Check out our chat.
- name: 🚀 Feature Request
url: https://github.com/paperless-ngx/paperless-ngx/discussions/new?category=feature-requests
about: Remember to search for existing feature requests and "up-vote" those that you like.
about: Remember to search for existing feature requests and "up-vote" any you like

View File

@@ -16,7 +16,7 @@ on:
env:
# This is the version of pipenv all the steps will use
# If changing this, change Dockerfile
DEFAULT_PIP_ENV_VERSION: "2024.4.1"
DEFAULT_PIP_ENV_VERSION: "2024.4.0"
# This is the default version of Python to use in most steps which aren't specific
DEFAULT_PYTHON_VERSION: "3.11"
@@ -131,7 +131,7 @@ jobs:
-
name: Configure ImageMagick
run: |
sudo cp docker/rootfs/etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-6/policy.xml
sudo cp docker/imagemagick-policy.xml /etc/ImageMagick-6/policy.xml
-
name: Install Python dependencies
run: |

6
.gitignore vendored
View File

@@ -100,9 +100,3 @@ scripts/nuke
# celery schedule file
celerybeat-schedule*
# ignore .devcontainer sub folders
/.devcontainer/consume/
/.devcontainer/data/
/.devcontainer/media/
/.devcontainer/redisdata/

View File

@@ -29,10 +29,10 @@ repos:
- id: check-case-conflict
- id: detect-private-key
- repo: https://github.com/codespell-project/codespell
rev: v2.4.0
rev: v2.3.0
hooks:
- id: codespell
exclude: "(^src-ui/src/locale/)|(^src-ui/e2e/)|(^src/paperless_mail/tests/samples/)"
exclude: "(^src-ui/src/locale/)|(^src-ui/e2e/)|(^src/paperless_mail/tests/samples/)|(^src/paperless_einvoice/tests/samples/)|(^src/paperless_einvoice/templates/)"
exclude_types:
- pofile
- json
@@ -51,7 +51,7 @@ repos:
- 'prettier-plugin-organize-imports@4.1.0'
# Python hooks
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.9.6
rev: v0.8.3
hooks:
- id: ruff
- id: ruff-format

View File

@@ -32,25 +32,33 @@ extend-select = [
"RUF", # https://docs.astral.sh/ruff/rules/#ruff-specific-rules-ruf
"FLY", # https://docs.astral.sh/ruff/rules/#flynt-fly
"PTH", # https://docs.astral.sh/ruff/rules/#flake8-use-pathlib-pth
"FBT", # https://docs.astral.sh/ruff/rules/#flake8-boolean-trap-fbt
]
ignore = ["DJ001", "SIM105", "RUF012"]
[lint.per-file-ignores]
".github/scripts/*.py" = ["E501", "INP001", "SIM117"]
"docker/wait-for-redis.py" = ["INP001", "T201"]
"src/documents/barcodes.py" = ["PTH"] # TODO Enable & remove
"src/documents/classifier.py" = ["PTH"] # TODO Enable & remove
"src/documents/consumer.py" = ["PTH"] # TODO Enable & remove
"src/documents/file_handling.py" = ["PTH"] # TODO Enable & remove
"src/documents/index.py" = ["PTH"] # TODO Enable & remove
"src/documents/management/commands/decrypt_documents.py" = ["PTH"] # TODO Enable & remove
"src/documents/management/commands/document_consumer.py" = ["PTH"] # TODO Enable & remove
"src/documents/management/commands/document_exporter.py" = ["PTH"] # TODO Enable & remove
"src/documents/management/commands/document_importer.py" = ["PTH"] # TODO Enable & remove
"src/documents/migrations/0012_auto_20160305_0040.py" = ["PTH"] # TODO Enable & remove
"src/documents/migrations/0014_document_checksum.py" = ["PTH"] # TODO Enable & remove
"src/documents/migrations/1003_mime_types.py" = ["PTH"] # TODO Enable & remove
"src/documents/migrations/1012_fix_archive_files.py" = ["PTH"] # TODO Enable & remove
"src/documents/migrations/1037_webp_encrypted_thumbnail_conversion.py" = ["PTH"] # TODO Enable & remove
"src/documents/models.py" = ["SIM115", "PTH"] # TODO PTH Enable & remove
"src/documents/parsers.py" = ["PTH"] # TODO Enable & remove
"src/documents/signals/handlers.py" = ["PTH"] # TODO Enable & remove
"src/documents/tasks.py" = ["PTH"] # TODO Enable & remove
"src/documents/tests/test_api_app_config.py" = ["PTH"] # TODO Enable & remove
"src/documents/tests/test_api_bulk_download.py" = ["PTH"] # TODO Enable & remove
"src/documents/tests/test_api_documents.py" = ["PTH"] # TODO Enable & remove
"src/documents/tests/test_classifier.py" = ["PTH"] # TODO Enable & remove
"src/documents/tests/test_consumer.py" = ["PTH"] # TODO Enable & remove
"src/documents/tests/test_file_handling.py" = ["PTH"] # TODO Enable & remove
@@ -76,12 +84,8 @@ ignore = ["DJ001", "SIM105", "RUF012"]
"src/paperless_tesseract/tests/test_parser.py" = ["RUF001", "PTH"] # TODO PTH Enable & remove
"src/paperless_tika/tests/test_live_tika.py" = ["PTH"] # TODO Enable & remove
"src/paperless_tika/tests/test_tika_parser.py" = ["PTH"] # TODO Enable & remove
# Testing
"*/tests/*.py" = ["E501", "SIM117"]
# Migrations
"*/migrations/*.py" = ["E501", "SIM", "T201"]
# Docker specific
"docker/rootfs/usr/local/bin/wait-for-redis.py" = ["INP001", "T201"]
[lint.isort]
force-single-line = true

View File

@@ -39,70 +39,15 @@ COPY Pipfile* ./
RUN set -eux \
&& echo "Installing pipenv" \
&& python3 -m pip install --no-cache-dir --upgrade pipenv==2024.4.1 \
&& python3 -m pip install --no-cache-dir --upgrade pipenv==2024.4.0 \
&& echo "Generating requirement.txt" \
&& pipenv requirements > requirements.txt
# Stage: s6-overlay-base
# Purpose: Installs s6-overlay and rootfs
# Comments:
# - Don't leave anything extra in here either
FROM docker.io/python:3.12-slim-bookworm AS s6-overlay-base
WORKDIR /usr/src/s6
# https://github.com/just-containers/s6-overlay#customizing-s6-overlay-behaviour
ENV \
S6_BEHAVIOUR_IF_STAGE2_FAILS=2 \
S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0 \
S6_VERBOSITY=1 \
PATH=/command:$PATH
# Buildx provided, must be defined to use though
ARG TARGETARCH
ARG TARGETVARIANT
# Lock this version
ARG S6_OVERLAY_VERSION=3.2.0.2
ARG S6_BUILD_TIME_PKGS="curl \
xz-utils"
RUN set -eux \
&& echo "Installing build time packages" \
&& apt-get update \
&& apt-get install --yes --quiet --no-install-recommends ${S6_BUILD_TIME_PKGS} \
&& echo "Determining arch" \
&& S6_ARCH="" \
&& if [ "${TARGETARCH}${TARGETVARIANT}" = "amd64" ]; then S6_ARCH="x86_64"; \
elif [ "${TARGETARCH}${TARGETVARIANT}" = "arm64" ]; then S6_ARCH="aarch64"; fi\
&& if [ -z "${S6_ARCH}" ]; then { echo "Error: Not able to determine arch"; exit 1; }; fi \
&& echo "Installing s6-overlay for ${S6_ARCH}" \
&& curl --fail --silent --no-progress-meter --show-error --location --remote-name-all --parallel --parallel-max 4 \
"https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz" \
"https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz.sha256" \
"https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-${S6_ARCH}.tar.xz" \
"https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-${S6_ARCH}.tar.xz.sha256" \
&& echo "Validating s6-archive checksums" \
&& sha256sum --check ./*.sha256 \
&& echo "Unpacking archives" \
&& tar --directory / -Jxpf s6-overlay-noarch.tar.xz \
&& tar --directory / -Jxpf s6-overlay-${S6_ARCH}.tar.xz \
&& echo "Removing downloaded archives" \
&& rm ./*.tar.xz \
&& rm ./*.sha256 \
&& echo "Cleaning up image" \
&& apt-get --yes purge ${S6_BUILD_TIME_PKGS} \
&& apt-get --yes autoremove --purge \
&& rm -rf /var/lib/apt/lists/*
# Copy our service defs and filesystem
COPY ./docker/rootfs /
# Stage: main-app
# Purpose: The final image
# Comments:
# - Don't leave anything extra in here
FROM s6-overlay-base AS main-app
FROM docker.io/python:3.12-slim-bookworm AS main-app
LABEL org.opencontainers.image.authors="paperless-ngx team <hello@paperless-ngx.com>"
LABEL org.opencontainers.image.documentation="https://docs.paperless-ngx.com/"
@@ -116,7 +61,7 @@ ARG DEBIAN_FRONTEND=noninteractive
ARG TARGETARCH
# Can be workflow provided, defaults set for manual building
ARG JBIG2ENC_VERSION=0.30
ARG JBIG2ENC_VERSION=0.29
ARG QPDF_VERSION=11.9.0
ARG GS_VERSION=10.03.1
@@ -182,39 +127,91 @@ RUN set -eux \
&& apt-get update \
&& apt-get install --yes --quiet --no-install-recommends ${RUNTIME_PACKAGES} \
&& echo "Installing pre-built updates" \
&& curl --fail --silent --no-progress-meter --show-error --location --remote-name-all --parallel --parallel-max 4 \
https://github.com/paperless-ngx/builder/releases/download/qpdf-${QPDF_VERSION}/libqpdf29_${QPDF_VERSION}-1_${TARGETARCH}.deb \
https://github.com/paperless-ngx/builder/releases/download/qpdf-${QPDF_VERSION}/qpdf_${QPDF_VERSION}-1_${TARGETARCH}.deb \
https://github.com/paperless-ngx/builder/releases/download/ghostscript-${GS_VERSION}/libgs10_${GS_VERSION}.dfsg-1_${TARGETARCH}.deb \
https://github.com/paperless-ngx/builder/releases/download/ghostscript-${GS_VERSION}/ghostscript_${GS_VERSION}.dfsg-1_${TARGETARCH}.deb \
https://github.com/paperless-ngx/builder/releases/download/ghostscript-${GS_VERSION}/libgs10-common_${GS_VERSION}.dfsg-1_all.deb \
https://github.com/paperless-ngx/builder/releases/download/jbig2enc-${JBIG2ENC_VERSION}/jbig2enc_${JBIG2ENC_VERSION}-1_${TARGETARCH}.deb \
&& echo "Installing qpdf ${QPDF_VERSION}" \
&& curl --fail --silent --show-error --location \
--output libqpdf29_${QPDF_VERSION}-1_${TARGETARCH}.deb \
https://github.com/paperless-ngx/builder/releases/download/qpdf-${QPDF_VERSION}/libqpdf29_${QPDF_VERSION}-1_${TARGETARCH}.deb \
&& curl --fail --silent --show-error --location \
--output qpdf_${QPDF_VERSION}-1_${TARGETARCH}.deb \
https://github.com/paperless-ngx/builder/releases/download/qpdf-${QPDF_VERSION}/qpdf_${QPDF_VERSION}-1_${TARGETARCH}.deb \
&& dpkg --install ./libqpdf29_${QPDF_VERSION}-1_${TARGETARCH}.deb \
&& dpkg --install ./qpdf_${QPDF_VERSION}-1_${TARGETARCH}.deb \
&& echo "Installing Ghostscript ${GS_VERSION}" \
&& curl --fail --silent --show-error --location \
--output libgs10_${GS_VERSION}.dfsg-1_${TARGETARCH}.deb \
https://github.com/paperless-ngx/builder/releases/download/ghostscript-${GS_VERSION}/libgs10_${GS_VERSION}.dfsg-1_${TARGETARCH}.deb \
&& curl --fail --silent --show-error --location \
--output ghostscript_${GS_VERSION}.dfsg-1_${TARGETARCH}.deb \
https://github.com/paperless-ngx/builder/releases/download/ghostscript-${GS_VERSION}/ghostscript_${GS_VERSION}.dfsg-1_${TARGETARCH}.deb \
&& curl --fail --silent --show-error --location \
--output libgs10-common_${GS_VERSION}.dfsg-1_all.deb \
https://github.com/paperless-ngx/builder/releases/download/ghostscript-${GS_VERSION}/libgs10-common_${GS_VERSION}.dfsg-1_all.deb \
&& dpkg --install ./libgs10-common_${GS_VERSION}.dfsg-1_all.deb \
&& dpkg --install ./libgs10_${GS_VERSION}.dfsg-1_${TARGETARCH}.deb \
&& dpkg --install ./ghostscript_${GS_VERSION}.dfsg-1_${TARGETARCH}.deb \
&& echo "Installing jbig2enc" \
&& curl --fail --silent --show-error --location \
--output jbig2enc_${JBIG2ENC_VERSION}-1_${TARGETARCH}.deb \
https://github.com/paperless-ngx/builder/releases/download/jbig2enc-${JBIG2ENC_VERSION}/jbig2enc_${JBIG2ENC_VERSION}-1_${TARGETARCH}.deb \
&& dpkg --install ./jbig2enc_${JBIG2ENC_VERSION}-1_${TARGETARCH}.deb \
&& echo "Configuring imagemagick" \
&& cp /etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-6/policy.xml \
&& echo "Cleaning up image layer" \
&& rm --force --verbose *.deb \
&& rm --recursive --force --verbose /var/lib/apt/lists/*
&& rm --recursive --force --verbose /var/lib/apt/lists/* \
&& echo "Installing supervisor" \
&& python3 -m pip install --default-timeout=1000 --upgrade --no-cache-dir supervisor==4.2.5
# Copy gunicorn config
# Changes very infrequently
WORKDIR /usr/src/paperless/
COPY --chown=1000:1000 gunicorn.conf.py /usr/src/paperless/gunicorn.conf.py
COPY gunicorn.conf.py .
# setup docker-specific things
# These change sometimes, but rarely
WORKDIR /usr/src/paperless/src/docker/
COPY [ \
"docker/imagemagick-policy.xml", \
"docker/supervisord.conf", \
"docker/docker-entrypoint.sh", \
"docker/docker-prepare.sh", \
"docker/paperless_cmd.sh", \
"docker/wait-for-redis.py", \
"docker/env-from-file.sh", \
"docker/management_script.sh", \
"docker/flower-conditional.sh", \
"docker/install_management_commands.sh", \
"/usr/src/paperless/src/docker/" \
]
RUN set -eux \
&& echo "Configuring ImageMagick" \
&& mv imagemagick-policy.xml /etc/ImageMagick-6/policy.xml \
&& echo "Configuring supervisord" \
&& mkdir /var/log/supervisord /var/run/supervisord \
&& mv supervisord.conf /etc/supervisord.conf \
&& echo "Setting up Docker scripts" \
&& mv docker-entrypoint.sh /sbin/docker-entrypoint.sh \
&& chmod 755 /sbin/docker-entrypoint.sh \
&& mv docker-prepare.sh /sbin/docker-prepare.sh \
&& chmod 755 /sbin/docker-prepare.sh \
&& mv wait-for-redis.py /sbin/wait-for-redis.py \
&& chmod 755 /sbin/wait-for-redis.py \
&& mv env-from-file.sh /sbin/env-from-file.sh \
&& chmod 755 /sbin/env-from-file.sh \
&& mv paperless_cmd.sh /usr/local/bin/paperless_cmd.sh \
&& chmod 755 /usr/local/bin/paperless_cmd.sh \
&& mv flower-conditional.sh /usr/local/bin/flower-conditional.sh \
&& chmod 755 /usr/local/bin/flower-conditional.sh \
&& echo "Installing management commands" \
&& chmod +x install_management_commands.sh \
&& ./install_management_commands.sh
WORKDIR /usr/src/paperless/src/
# Python dependencies
# Change pretty frequently
COPY --chown=1000:1000 --from=pipenv-base /usr/src/pipenv/requirements.txt ./
COPY --from=pipenv-base /usr/src/pipenv/requirements.txt ./
# Packages needed only for building a few quick Python
# dependencies
@@ -227,22 +224,20 @@ ARG BUILD_PACKAGES="\
default-libmysqlclient-dev \
pkg-config"
ARG ZXING_VERSION=2.3.0
ARG PSYCOPG_VERSION=3.2.4
# hadolint ignore=DL3042
RUN --mount=type=cache,target=/root/.cache/pip/,id=pip-cache \
set -eux \
&& echo "Installing build system packages" \
&& apt-get update \
&& apt-get install --yes --quiet --no-install-recommends ${BUILD_PACKAGES} \
&& python3 -m pip install --upgrade wheel \
&& python3 -m pip install --no-cache-dir --upgrade wheel \
&& echo "Installing Python requirements" \
&& curl --fail --silent --no-progress-meter --show-error --location --remote-name-all --parallel --parallel-max 4 \
https://github.com/paperless-ngx/builder/releases/download/psycopg-${PSYCOPG_VERSION}/psycopg_c-${PSYCOPG_VERSION}-cp312-cp312-linux_x86_64.whl \
https://github.com/paperless-ngx/builder/releases/download/psycopg-${PSYCOPG_VERSION}/psycopg_c-${PSYCOPG_VERSION}-cp312-cp312-linux_aarch64.whl \
https://github.com/paperless-ngx/builder/releases/download/zxing-${ZXING_VERSION}/zxing_cpp-${ZXING_VERSION}-cp312-cp312-linux_aarch64.whl \
https://github.com/paperless-ngx/builder/releases/download/zxing-${ZXING_VERSION}/zxing_cpp-${ZXING_VERSION}-cp312-cp312-linux_x86_64.whl \
&& curl --fail --silent --show-error --location \
--output psycopg_c-3.2.3-cp312-cp312-linux_x86_64.whl \
https://github.com/paperless-ngx/builder/releases/download/psycopg-3.2.3/psycopg_c-3.2.3-cp312-cp312-linux_x86_64.whl \
&& curl --fail --silent --show-error --location \
--output psycopg_c-3.2.3-cp312-cp312-linux_aarch64.whl \
https://github.com/paperless-ngx/builder/releases/download/psycopg-3.2.3/psycopg_c-3.2.3-cp312-cp312-linux_aarch64.whl \
&& python3 -m pip install --default-timeout=1000 --find-links . --requirement requirements.txt \
&& echo "Installing NLTK data" \
&& python3 -W ignore::RuntimeWarning -m nltk.downloader -d "/usr/share/nltk_data" snowball_data \
@@ -281,16 +276,18 @@ RUN set -eux \
&& echo "Adjusting all permissions" \
&& chown --from root:root --changes --recursive paperless:paperless /usr/src/paperless \
&& echo "Collecting static files" \
&& s6-setuidgid paperless python3 manage.py collectstatic --clear --no-input --link \
&& s6-setuidgid paperless python3 manage.py compilemessages
&& gosu paperless python3 manage.py collectstatic --clear --no-input --link \
&& gosu paperless python3 manage.py compilemessages
VOLUME ["/usr/src/paperless/data", \
"/usr/src/paperless/media", \
"/usr/src/paperless/consume", \
"/usr/src/paperless/export"]
ENTRYPOINT ["/init"]
ENTRYPOINT ["/sbin/docker-entrypoint.sh"]
EXPOSE 8000
CMD ["/usr/local/bin/paperless_cmd.sh"]
HEALTHCHECK --interval=30s --timeout=10s --retries=5 CMD [ "curl", "-fs", "-S", "--max-time", "2", "http://localhost:8000" ]

13
Pipfile
View File

@@ -7,21 +7,20 @@ name = "pypi"
dateparser = "~=1.2"
# WARNING: django does not use semver.
# Only patch versions are guaranteed to not introduce breaking changes.
django = "~=5.1.5"
django = "~=5.1.4"
django-allauth = {extras = ["mfa", "socialaccount"], version = "*"}
django-auditlog = "*"
django-celery-results = "*"
django-compression-middleware = "*"
django-cors-headers = "*"
django-extensions = "*"
django-filter = "~=25.1"
django-filter = "~=24.3"
django-guardian = "*"
django-multiselectfield = "*"
django-soft-delete = "*"
djangorestframework = "~=3.15.2"
djangorestframework-guardian = "*"
drf-spectacular = "*"
drf-spectacular-sidecar = "*"
drafthorse = "*"
drf-writable-nested = "*"
bleach = "*"
celery = {extras = ["redis"], version = "*"}
@@ -39,7 +38,7 @@ jinja2 = "~=3.1"
langdetect = "*"
mysqlclient = "*"
nltk = "*"
ocrmypdf = "~=16.9"
ocrmypdf = "~=16.7"
pathvalidate = "*"
pdf2image = "*"
psycopg = {version = "*", extras = ["c"]}
@@ -58,9 +57,9 @@ tqdm = "*"
# See https://github.com/paperless-ngx/paperless-ngx/issues/5494
uvicorn = {extras = ["standard"], version = "==0.25.0"}
watchdog = "~=6.0"
whitenoise = "~=6.9"
whitenoise = "~=6.8"
whoosh = "~=2.7"
zxing-cpp = "*"
zxing-cpp = {version = "*", platform_machine = "== 'x86_64'"}
[dev-packages]

3957
Pipfile.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -55,7 +55,7 @@ A full list of [features](https://docs.paperless-ngx.com/#features) and [screens
# Getting started
The easiest way to deploy paperless is `docker compose`. The files in the [`/docker/compose` directory](https://github.com/paperless-ngx/paperless-ngx/tree/main/docker/compose) are configured to pull the image from the GitHub container registry.
The easiest way to deploy paperless is `docker compose`. The files in the [`/docker/compose` directory](https://github.com/paperless-ngx/paperless-ngx/tree/main/docker/compose) are configured to pull the image from GitHub Packages.
If you'd like to jump right in, you can configure a `docker compose` environment with our install script:

View File

@@ -1,17 +1,26 @@
###############################################################################
# Paperless-ngx settings #
###############################################################################
# See http://docs.paperless-ngx.com/configuration/ for all available options.
# The UID and GID of the user used to run paperless in the container. Set this
# to your UID and GID on the host so that you have write access to the
# consumption directory.
#USERMAP_UID=1000
#USERMAP_GID=1000
# See the documentation linked above for all options. A few commonly adjusted settings
# are provided below.
# Additional languages to install for text recognition, separated by a
# whitespace. Note that this is
# different from PAPERLESS_OCR_LANGUAGE (default=eng), which defines the
# language used for OCR.
# The container installs English, German, Italian, Spanish and French by
# default.
# See https://packages.debian.org/search?keywords=tesseract-ocr-&searchon=names&suite=buster
# for available languages.
#PAPERLESS_OCR_LANGUAGES=tur ces
###############################################################################
# Paperless-specific settings #
###############################################################################
# All settings defined in the paperless.conf.example can be used here. The
# Docker setup does not use the configuration file.
# A few commonly adjusted settings are provided below.
# This is required if you will be exposing Paperless-ngx on a public domain
# (if doing so please consider security measures such as reverse proxy)
@@ -21,17 +30,13 @@
# be a very long sequence of random characters. You don't need to remember it.
#PAPERLESS_SECRET_KEY=change-me
# Use this variable to set a timezone for the Paperless Docker containers. Defaults to UTC.
# Use this variable to set a timezone for the Paperless Docker containers. If not specified, defaults to UTC.
#PAPERLESS_TIME_ZONE=America/Los_Angeles
# The default language to use for OCR. Set this to the language most of your
# documents are written in.
#PAPERLESS_OCR_LANGUAGE=eng
# Additional languages to install for text recognition, separated by a whitespace.
# Note that this is different from PAPERLESS_OCR_LANGUAGE (default=eng), which defines
# the language used for OCR.
# The container installs English, German, Italian, Spanish and French by default.
# See https://packages.debian.org/search?keywords=tesseract-ocr-&searchon=names&suite=buster
# for available languages.
#PAPERLESS_OCR_LANGUAGES=tur ces
# Set if accessing paperless via a domain subpath e.g. https://domain.com/PATHPREFIX and using a reverse-proxy like traefik or nginx
#PAPERLESS_FORCE_SCRIPT_NAME=/PATHPREFIX
#PAPERLESS_STATIC_URL=/PATHPREFIX/static/ # trailing slash required

120
docker/docker-prepare.sh Executable file
View File

@@ -0,0 +1,120 @@
#!/usr/bin/env bash
set -e
wait_for_postgres() {
local attempt_num=1
local -r max_attempts=5
echo "Waiting for PostgreSQL to start..."
local -r host="${PAPERLESS_DBHOST:-localhost}"
local -r port="${PAPERLESS_DBPORT:-5432}"
# Disable warning, host and port can't have spaces
# shellcheck disable=SC2086
while [ ! "$(pg_isready --host ${host} --port ${port})" ]; do
if [ $attempt_num -eq $max_attempts ]; then
echo "Unable to connect to database."
exit 1
else
echo "Attempt $attempt_num failed! Trying again in 5 seconds..."
fi
attempt_num=$(("$attempt_num" + 1))
sleep 5
done
echo "Connected to PostgreSQL"
}
wait_for_mariadb() {
echo "Waiting for MariaDB to start..."
local -r host="${PAPERLESS_DBHOST:=localhost}"
local -r port="${PAPERLESS_DBPORT:=3306}"
local attempt_num=1
local -r max_attempts=5
# Disable warning, host and port can't have spaces
# shellcheck disable=SC2086
while ! true > /dev/tcp/$host/$port; do
if [ $attempt_num -eq $max_attempts ]; then
echo "Unable to connect to database."
exit 1
else
echo "Attempt $attempt_num failed! Trying again in 5 seconds..."
fi
attempt_num=$(("$attempt_num" + 1))
sleep 5
done
echo "Connected to MariaDB"
}
wait_for_redis() {
# We use a Python script to send the Redis ping
# instead of installing redis-tools just for 1 thing
if ! python3 /sbin/wait-for-redis.py; then
exit 1
fi
}
migrations() {
(
# flock is in place to prevent multiple containers from doing migrations
# simultaneously. This also ensures that the db is ready when the command
# of the current container starts.
flock 200
echo "Apply database migrations..."
python3 manage.py migrate --skip-checks --no-input
) 200>"${DATA_DIR}/migration_lock"
}
django_checks() {
# Explicitly run the Django system checks
echo "Running Django checks"
python3 manage.py check
}
search_index() {
local -r index_version=9
local -r index_version_file=${DATA_DIR}/.index_version
if [[ (! -f "${index_version_file}") || $(<"${index_version_file}") != "$index_version" ]]; then
echo "Search index out of date. Updating..."
python3 manage.py document_index reindex --no-progress-bar
echo ${index_version} | tee "${index_version_file}" >/dev/null
fi
}
superuser() {
if [[ -n "${PAPERLESS_ADMIN_USER}" ]]; then
python3 manage.py manage_superuser
fi
}
do_work() {
if [[ "${PAPERLESS_DBENGINE}" == "mariadb" ]]; then
wait_for_mariadb
elif [[ -n "${PAPERLESS_DBHOST}" ]]; then
wait_for_postgres
fi
wait_for_redis
migrations
django_checks
search_index
superuser
}
do_work

42
docker/env-from-file.sh Normal file
View File

@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# Scans the environment variables for those with the suffix _FILE
# When located, checks the file exists, and exports the contents
# of the file as the same name, minus the suffix
# This allows the use of Docker secrets or mounted files
# to fill in any of the settings configurable via environment
# variables
set -eu
for line in $(printenv)
do
# Extract the name of the environment variable
env_name=${line%%=*}
# Check if it starts with "PAPERLESS_" and ends in "_FILE"
if [[ ${env_name} == PAPERLESS_*_FILE ]]; then
# This should have been named different..
if [[ ${env_name} == "PAPERLESS_OCR_SKIP_ARCHIVE_FILE" || ${env_name} == "PAPERLESS_MODEL_FILE" ]]; then
continue
fi
# Extract the value of the environment
env_value=${line#*=}
# Check the file exists
if [[ -f ${env_value} ]]; then
# Trim off the _FILE suffix
non_file_env_name=${env_name%"_FILE"}
echo "Setting ${non_file_env_name} from file"
# Reads the value from th file
val="$(< "${!env_name}")"
# Sets the normal name to the read file contents
export "${non_file_env_name}"="${val}"
else
echo "File ${env_value} referenced by ${env_name} doesn't exist"
fi
fi
done

View File

@@ -0,0 +1,12 @@
#!/usr/bin/env bash
echo "Checking if we should start flower..."
if [[ -n "${PAPERLESS_ENABLE_FLOWER}" ]]; then
# Small delay to allow celery to be up first
echo "Starting flower in 5s"
sleep 5
celery --app paperless flower --conf=/usr/src/paperless/src/paperless/flowerconfig.py
else
echo "Not starting flower"
fi

Binary file not shown.

Before

Width:  |  Height:  |  Size: 30 KiB

View File

@@ -1,7 +1,5 @@
#!/usr/bin/env bash
# Run this script to generate the management commands again (for example if a new command is create or the template is updated)
set -eu
for command in decrypt_documents \
@@ -21,6 +19,6 @@ for command in decrypt_documents \
prune_audit_logs;
do
echo "installing $command..."
sed "s/management_command/$command/g" management_script.sh >"$PWD/rootfs/usr/local/bin/$command"
chmod +x "$PWD/rootfs/usr/local/bin/$command"
sed "s/management_command/$command/g" management_script.sh > /usr/local/bin/$command
chmod +x /usr/local/bin/$command
done

View File

@@ -1,13 +1,17 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
#!/usr/bin/env bash
set -e
cd "${PAPERLESS_SRC_DIR}"
cd /usr/src/paperless/src/
# This ensures environment is setup
# shellcheck disable=SC1091
source /sbin/env-from-file.sh
if [[ $(id -u) == 0 ]]; then
s6-setuidgid paperless python3 manage.py management_command "$@"
elif [[ $(id -un) == "paperless" ]]; then
if [[ $(id -u) == 0 ]] ;
then
gosu paperless python3 manage.py management_command "$@"
elif [[ $(id -un) == "paperless" ]] ;
then
python3 manage.py management_command "$@"
else
echo "Unknown user."

16
docker/paperless_cmd.sh Executable file
View File

@@ -0,0 +1,16 @@
#!/usr/bin/env bash
SUPERVISORD_WORKING_DIR="${PAPERLESS_SUPERVISORD_WORKING_DIR:-$PWD}"
rootless_args=()
if [ "$(id -u)" == "$(id -u paperless)" ]; then
rootless_args=(
--user
paperless
--logfile
"${SUPERVISORD_WORKING_DIR}/supervisord.log"
--pidfile
"${SUPERVISORD_WORKING_DIR}/supervisord.pid"
)
fi
exec /usr/local/bin/supervisord -c /etc/supervisord.conf "${rootless_args[@]}"

View File

@@ -1,8 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-complete]"
declare -r end_time=$(date +%s)
declare -r start_time=${PAPERLESS_START_TIME_S}
echo "${log_prefix} paperless-ngx docker container init completed in $(($end_time-$start_time)) seconds"
echo "${log_prefix} Starting services"

View File

@@ -1 +0,0 @@
oneshot

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-complete/run

View File

@@ -1,44 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[custom-init]"
# Mostly borrowed from the LinuxServer.io base image
# https://github.com/linuxserver/docker-baseimage-ubuntu/tree/bionic/root/etc/cont-init.d
declare -r custom_script_dir="/custom-cont-init.d"
# Tamper checking.
# Don't run files which are owned by anyone except root
# Don't run files which are writeable by others
if [ -d "${custom_script_dir}" ]; then
if [ -n "$(/usr/bin/find "${custom_script_dir}" -maxdepth 1 ! -user root)" ]; then
echo "${log_prefix} **** Potential tampering with custom scripts detected ****"
echo "${log_prefix} **** The folder '${custom_script_dir}' must be owned by root ****"
exit 0
fi
if [ -n "$(/usr/bin/find "${custom_script_dir}" -maxdepth 1 -perm -o+w)" ]; then
echo "${log_prefix} **** The folder '${custom_script_dir}' or some of contents have write permissions for others, which is a security risk. ****"
echo "${log_prefix} **** Please review the permissions and their contents to make sure they are owned by root, and can only be modified by root. ****"
exit 0
fi
# Make sure custom init directory has files in it
if [ -n "$(/bin/ls --almost-all "${custom_script_dir}" 2>/dev/null)" ]; then
echo "${log_prefix} files found in ${custom_script_dir} executing"
# Loop over files in the directory
for SCRIPT in "${custom_script_dir}"/*; do
NAME="$(basename "${SCRIPT}")"
if [ -f "${SCRIPT}" ]; then
echo "${log_prefix} ${NAME}: executing..."
/command/with-contenv /bin/bash "${SCRIPT}"
echo "${log_prefix} ${NAME}: exited $?"
elif [ ! -f "${SCRIPT}" ]; then
echo "${log_prefix} ${NAME}: is not a file"
fi
done
else
echo "${log_prefix} no custom files found exiting..."
fi
else
echo "${log_prefix} ${custom_script_dir} doesn't exist, nothing to do"
fi

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-custom-init/run

View File

@@ -1,30 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[env-init]"
echo "${log_prefix} Checking for environment from files"
if find /run/s6/container_environment/*"_FILE" -maxdepth 1 > /dev/null 2>&1; then
for FILENAME in /run/s6/container_environment/*; do
if [[ "${FILENAME##*/}" == PAPERLESS_*_FILE ]]; then
# This should have been named different..
if [[ ${FILENAME} == "PAPERLESS_OCR_SKIP_ARCHIVE_FILE" || ${FILENAME} == "PAPERLESS_MODEL_FILE" ]]; then
continue
fi
SECRETFILE=$(cat "${FILENAME}")
# Check the file exists
if [[ -f ${SECRETFILE} ]]; then
# Trim off trailing _FILE
FILESTRIP=${FILENAME//_FILE/}
# Set environment variable
cat "${SECRETFILE}" > "${FILESTRIP}"
echo "${log_prefix} ${FILESTRIP##*/} set from ${FILENAME##*/}"
else
echo "${log_prefix} cannot find secret in ${FILENAME##*/}"
fi
fi
done
else
echo "${log_prefix} No *_FILE environment found"
fi

View File

@@ -1 +0,0 @@
oneshot

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-env-file/run

View File

@@ -1,33 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-folders]"
declare -r export_dir="/usr/src/paperless/export"
declare -r data_dir="${PAPERLESS_DATA_DIR:-/usr/src/paperless/data}"
declare -r media_root_dir="${PAPERLESS_MEDIA_ROOT:-/usr/src/paperless/media}"
declare -r consume_dir="${PAPERLESS_CONSUMPTION_DIR:-/usr/src/paperless/consume}"
declare -r tmp_dir="${PAPERLESS_SCRATCH_DIR:=/tmp/paperless}"
echo "${log_prefix} Checking for folder existence"
for dir in \
"${export_dir}" \
"${data_dir}" "${data_dir}/index" \
"${media_root_dir}" "${media_root_dir}/documents" "${media_root_dir}/documents/originals" "${media_root_dir}/documents/thumbnails" \
"${consume_dir}" \
"${tmp_dir}"; do
if [[ ! -d "${dir}" ]]; then
mkdir --parents --verbose "${dir}"
fi
done
echo "${log_prefix} Adjusting file and folder permissions"
for dir in \
"${export_dir}" \
"${data_dir}" \
"${media_root_dir}" \
"${consume_dir}" \
"${tmp_dir}"; do
find "${dir}" -not \( -user paperless -and -group paperless \) -exec chown --changes paperless:paperless {} +
done

View File

@@ -1 +0,0 @@
oneshot

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-folders/run

View File

@@ -1,20 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-migrations]"
declare -r data_dir="${PAPERLESS_DATA_DIR:-/usr/src/paperless/data}"
(
# flock is in place to prevent multiple containers from doing migrations
# simultaneously. This also ensures that the db is ready when the command
# of the current container starts.
flock 200
echo "${log_prefix} Apply database migrations..."
cd "${PAPERLESS_SRC_DIR}"
if [[ -n "${USER_IS_NON_ROOT}" ]]; then
exec python3 manage.py migrate --skip-checks --no-input
else
exec s6-setuidgid paperless python3 manage.py migrate --skip-checks --no-input
fi
) 200>"${data_dir}/migration_lock"

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-migrations/run

View File

@@ -1,22 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-user]"
declare -r usermap_original_uid=$(id -u paperless)
declare -r usermap_original_gid=$(id -g paperless)
declare -r usermap_new_uid=${USERMAP_UID:-$usermap_original_uid}
declare -r usermap_new_gid=${USERMAP_GID:-${usermap_original_gid:-$usermap_new_uid}}
if [[ ${usermap_new_uid} != "${usermap_original_uid}" ]]; then
echo "${log_prefix} Mapping UID for paperless to $usermap_new_uid"
usermod --non-unique --uid "${usermap_new_uid}" paperless
else
echo "${log_prefix} No UID changes for paperless"
fi
if [[ ${usermap_new_gid} != "${usermap_original_gid}" ]]; then
echo "${log_prefix} Mapping GID for paperless to $usermap_new_gid"
groupmod --non-unique --gid "${usermap_new_gid}" paperless
else
echo "${log_prefix} No GID changes for paperless"
fi

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-modify-user/run

View File

@@ -1,28 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-index]"
declare -r index_version=9
declare -r data_dir="${PAPERLESS_DATA_DIR:-/usr/src/paperless/data}"
declare -r index_version_file="${data_dir}/.index_version"
update_index () {
echo "${log_prefix} Search index out of date. Updating..."
cd "${PAPERLESS_SRC_DIR}"
if [[ -n "${USER_IS_NON_ROOT}" ]]; then
python3 manage.py document_index reindex --no-progress-bar
echo ${index_version} | tee "${index_version_file}" > /dev/null
else
s6-setuidgid paperless python3 manage.py document_index reindex --no-progress-bar
echo ${index_version} | s6-setuidgid paperless tee "${index_version_file}" > /dev/null
fi
}
if [[ (! -f "${index_version_file}") ]]; then
echo "${log_prefix} No index version file found"
update_index
elif [[ $(<"${index_version_file}") != "$index_version" ]]; then
echo "${log_prefix} index version updated"
update_index
fi

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-search-index/run

View File

@@ -1,19 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-start]"
echo "${log_prefix} paperless-ngx docker container starting..."
# Set some directories into environment for other steps to access via environment
# Sort of like variables for later
printf "/usr/src/paperless/src" > /var/run/s6/container_environment/PAPERLESS_SRC_DIR
echo $(date +%s) > /var/run/s6/container_environment/PAPERLESS_START_TIME_S
# Check if we're starting as a non-root user
if [ $(id -u) == $(id -u paperless) ]; then
printf "true" > /var/run/s6/container_environment/USER_IS_NON_ROOT
echo "${log_prefix} paperless-ngx docker container running under a user"
else
echo "${log_prefix} paperless-ngx docker container starting init as root"
fi

View File

@@ -1 +0,0 @@
oneshot

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-start/run

View File

@@ -1,20 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-superuser]"
if [[ -n "${PAPERLESS_ADMIN_USER}" ]]; then
echo "${log_prefix} Creating superuser..."
cd "${PAPERLESS_SRC_DIR}"
if [[ -n "${USER_IS_NON_ROOT}" ]]; then
python3 manage.py manage_superuser
else
s6-setuidgid paperless python3 manage.py manage_superuser
fi
echo "${log_prefix} Superuser creation done"
else
echo "${log_prefix} Not creating superuser"
fi

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-superuser/run

View File

@@ -1,15 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-checks]"
# Explicitly run the Django system checks
echo "${log_prefix} Running Django checks"
cd "${PAPERLESS_SRC_DIR}"
if [[ -n "${USER_IS_NON_ROOT}" ]]; then
python3 manage.py check
else
s6-setuidgid paperless python3 manage.py check
fi

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-system-checks/run

View File

@@ -1,65 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-tesseract-langs]"
install_languages() {
echo "Installing languages..."
read -ra langs <<<"$1"
# Check that it is not empty
if [ ${#langs[@]} -eq 0 ]; then
return
fi
# Build list of packages to install
to_install=()
for lang in "${langs[@]}"; do
pkg="tesseract-ocr-$lang"
if dpkg --status "$pkg" &>/dev/null; then
echo "${log_prefix} Package $pkg already installed!"
continue
else
to_install+=("$pkg")
fi
done
# Use apt only when we install packages
if [ ${#to_install[@]} -gt 0 ]; then
# Warn the user if they're not root, but try anyway
if [[ -n "${USER_IS_NON_ROOT}" ]]; then
echo "${log_prefix} ERROR: Unable to install language ${pkg} as non-root, startup may fail"
fi
apt-get --quiet update &>/dev/null
for pkg in "${to_install[@]}"; do
if ! apt-cache --quiet show "$pkg" &>/dev/null; then
echo "${log_prefix} Skipped $pkg: Package not found! :("
continue
fi
echo "${log_prefix} Installing package $pkg..."
if ! apt-get --quiet --assume-yes install "$pkg" &>/dev/null; then
echo "${log_prefix} Could not install $pkg"
exit 1
else
echo "${log_prefix} Installed $pkg"
fi
done
fi
}
echo "${log_prefix} Checking if additional teseract languages needed"
# Install additional languages if specified
if [[ -n "$PAPERLESS_OCR_LANGUAGES" ]]; then
install_languages "$PAPERLESS_OCR_LANGUAGES"
echo "${log_prefix} Additional packages installed"
else
echo "${log_prefix} No additional installs requested"
fi

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-tesseract-langs/run

View File

@@ -1,70 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-db-wait]"
wait_for_postgres() {
local attempt_num=1
local -r max_attempts=5
echo "${log_prefix} Waiting for PostgreSQL to start..."
local -r host="${PAPERLESS_DBHOST:-localhost}"
local -r port="${PAPERLESS_DBPORT:-5432}"
local -r user="${PAPERLESS_DBUSER:-paperless}"
# Disable warning, host and port can't have spaces
# shellcheck disable=SC2086
while [ ! "$(pg_isready -h ${host} -p ${port} --username ${user})" ]; do
if [ $attempt_num -eq $max_attempts ]; then
echo "${log_prefix} Unable to connect to database."
exit 1
else
echo "${log_prefix} Attempt $attempt_num failed! Trying again in 5 seconds..."
fi
attempt_num=$(("$attempt_num" + 1))
sleep 5
done
# Extra in case this is a first start
sleep 5
echo "Connected to PostgreSQL"
}
wait_for_mariadb() {
echo "${log_prefix} Waiting for MariaDB to start..."
local -r host="${PAPERLESS_DBHOST:=localhost}"
local -r port="${PAPERLESS_DBPORT:=3306}"
local attempt_num=1
local -r max_attempts=5
# Disable warning, host and port can't have spaces
# shellcheck disable=SC2086
while ! true > /dev/tcp/$host/$port; do
if [ $attempt_num -eq $max_attempts ]; then
echo "${log_prefix} Unable to connect to database."
exit 1
else
echo "${log_prefix} Attempt $attempt_num failed! Trying again in 5 seconds..."
fi
attempt_num=$(("$attempt_num" + 1))
sleep 5
done
echo "Connected to MariaDB"
}
if [[ "${PAPERLESS_DBENGINE}" == "mariadb" ]]; then
echo "${log_prefix} Waiting for MariaDB to report ready"
wait_for_mariadb
elif [[ -n "${PAPERLESS_DBHOST}" ]]; then
echo "${log_prefix} Waiting for postgresql to report ready"
wait_for_postgres
fi
echo "${log_prefix} Database is ready"

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-wait-for-db/run

View File

@@ -1,14 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[init-redis-wait]"
echo "${log_prefix} Waiting for Redis to report ready"
# We use a Python script to send the Redis ping
# instead of installing redis-tools just for 1 thing
if ! python3 /usr/local/bin/wait-for-redis.py; then
exit 1
else
echo "${log_prefix} Redis ready"
fi

View File

@@ -1 +0,0 @@
/etc/s6-overlay/s6-rc.d/init-wait-for-redis/run

View File

@@ -1,10 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
cd ${PAPERLESS_SRC_DIR}
if [[ -n "${USER_IS_NON_ROOT}" ]]; then
exec python3 manage.py document_consumer
else
exec s6-setuidgid paperless python3 manage.py document_consumer
fi

View File

@@ -1 +0,0 @@
longrun

View File

@@ -1,24 +0,0 @@
#!/command/with-contenv /usr/bin/bash
# shellcheck shell=bash
declare -r log_prefix="[svc-flower]"
echo "${log_prefix} Checking if we should start flower..."
if [[ -n "${PAPERLESS_ENABLE_FLOWER}" ]]; then
# Small delay to allow celery to be up first
echo "${log_prefix} Starting flower in 5s"
sleep 5
cd ${PAPERLESS_SRC_DIR}
if [[ -n "${USER_IS_NON_ROOT}" ]]; then
exec /usr/local/bin/celery --app paperless flower --conf=${PAPERLESS_SRC_DIR}/paperless/flowerconfig.py
else
exec s6-setuidgid paperless /usr/local/bin/celery --app paperless flower --conf=${PAPERLESS_SRC_DIR}/paperless/flowerconfig.py
fi
else
echo "${log_prefix} Not starting flower"
# https://skarnet.org/software/s6/s6-svc.html
s6-svc -Od .
fi

View File

@@ -1 +0,0 @@
longrun

Some files were not shown because too many files have changed in this diff Show More