Commit Graph
27 Commits
Author SHA1 Message Date
Erik Arvstedt be2cbebaf7 Stop tests from writing to the source tree 2018-07-19 23:48:23 +02:00
Erik ArvstedtandDaniel Quinn 742b01d1f5 Update Consumer class documentation 2018-06-17 20:17:40 +01:00
Erik ArvstedtandDaniel Quinn b3624f6375 Improve password check
1. Fail when the db contains encrypted docs and no password is set.
   Previously, this case wasn't detected.

2. Exit with an error instead of showing warnings.
   This ensures that we never store docs with different encryption passwords.
2018-06-17 20:07:32 +01:00
Erik Arvstedt fc560b8c04 Fix unclickable checkbox in documents view
1. Clicks to the document selection checkbox were captured by the onclick
   handler of the document item header. This is now fixed.

2. Reexpose the doc title link to mouse events by putting it on top of
   the header link layer.
2018-06-01 14:07:34 +02:00
Erik Arvstedt 48738dab9f Fix incompatibility with Python versions < 3.6
Direct index access to a match was only added in 3.6.

Fixes #359
2018-06-01 00:45:59 +02:00
Erik Arvstedt 11db87fa11 Add field 'added' to documents
This field indicates when the document was added to the database
2018-05-31 10:17:03 +02:00
Erik Arvstedt f96e7f7895 fixup: mention inotify in 'utilities.rst' 2018-05-22 01:22:41 +02:00
Erik Arvstedt 8218b1aa51 Documentation: Replace 'PDF' with 'document'
There are more supported file formats than just PDF.
2018-05-22 01:22:38 +02:00
Erik Arvstedt 0559204be4 fixup: require usage of PAPERLESS_EMAIL_SECRET 2018-05-21 12:11:56 +02:00
Erik Arvstedt bccac5017c fixup: remove helper fn 'make_dirs' 2018-05-21 00:45:00 +02:00
Erik Arvstedt 3e8038577d fixup: break up complex if condition 2018-05-21 00:44:58 +02:00
Erik Arvstedt 7e1d59377a Add inotify support 2018-05-11 14:14:50 +02:00
Erik Arvstedt 7357471b9e Consumer loop: make sleep duration dynamic
Make the sleep duration dynamic to account for the time spent in
loop_step.
This improves responsiveness when repeatedly consuming newly
arriving docs.

Use float epoch seconds (time.time()) as the time type for
MailFetcher.last_checked to allow for natural time arithmetic.
2018-05-11 14:14:50 +02:00
Erik Arvstedt bd75a65866 Refactor: renamings, extract fn 'loop'
Renamings:
loop -> loop_step
delta -> next_mail_time (this variable names a point in time, not a duration)

Extracting the 'loop' fn is a preparation for later commits where a
second type of loop is added.
2018-05-11 14:14:25 +02:00
Erik Arvstedt e65e27d11f Consider mtime of ignored files, garbage-collect ignore list
1. Store the mtime of ignored files so that we can reconsider them if
they have changed.

2. Regularly reset the ignore list to files that still exist in the
consumption dir. Previously, the list could grow indefinitely.
2018-05-11 14:05:30 +02:00
Erik Arvstedt 12488c9634 Simplify ignoring docs 2018-05-11 14:05:29 +02:00
Erik Arvstedt 61cd050e24 Ensure docs have been unmodified for some time before consuming
Previously, the second mtime check for new files usually happened right
after the first one, which could have caused consumption of docs that
were still being modified.

We're now waiting for at least FILES_MIN_UNMODIFIED_DURATION (0.5s).

This also cleans up the logic by eliminating the consumer.stats attribute
and the weird double call to consumer.run().

Additionally, this a fixes memory leak in consumer.stats where paths could be
added but never removed if the corresponding files disappeared from
the consumer dir before being considered ready.
2018-05-11 14:05:29 +02:00
Erik Arvstedt f018e8e54f Refactor: extract fn try_consume_file
The main purpose of this change is to make the following commits more
readable.
2018-05-11 14:05:28 +02:00
Erik Arvstedt a56a3eb86d Use os.scandir instead of os.listdir
It's simpler and better suited for use cases introduced in later commits.
2018-05-11 14:05:25 +02:00
Erik Arvstedt 2fe7df8ca0 Consume documents in order of increasing mtime
This increases overall usability, especially for multi-page scans.
Previously, the consumption order was undefined (see os.listdir())
2018-05-11 14:04:37 +02:00
Erik Arvstedt 873c98dddb Refactor: extract fn 'make_dirs' 2018-05-11 14:04:36 +02:00
Erik Arvstedt ea287e0db2 Fix list out of bounds error in mail message parsing
Check list length before accessing the first two elements of
'dispositions'.
The list may have only a single element ('inline') or may be empty in
mailformed emails.
2018-05-11 14:04:36 +02:00
Erik Arvstedt 4babfa1a5b Set default empty PAPERLESS_EMAIL_SECRET
Previously, if the user didn't set PAPERLESS_EMAIL_SECRET, Paperless
failed with an error in check_body() because self.SECRET was None.
2018-05-11 14:04:31 +02:00
Erik Arvstedt aa2fc84d7f Mail fetching: Only catch internal errors
Previously, all errors raised during mail fetching were silently caught
and printed without backtrace.

To increase robustness and ease debugging, we now fail with a backtrace
on unexpected errors.
2018-05-11 14:01:09 +02:00
Erik Arvstedt 8d5ae64aff Increase link area in documents listing
Increase the link area to include the whole visual header.

Fixes #335
2018-05-11 13:50:09 +02:00
Erik Arvstedt fcdcf62c2c Fix runtime warning when adding log entries
LogEntry.action_time expects a Django timezone object instead of a builtin datetime.

This fixes a runtime warning of the following kind:
RuntimeWarning: DateTimeField LogEntry.action_time received a naive datetime (2018-03-28 20:53:01.714173) while time zone support is active.
2018-03-30 00:15:52 +02:00
Erik Arvstedt 1f2079f65a Use --noreload for permanent server start commands
Without it, the server is highly resource-intensive even when
running idle
2018-03-23 11:13:20 +01:00