reworking the documentation.

docs/usage_overview.rst

@@ -0,0 +1,216 @@
+**************
+Usage Overview
+**************
+
+Paperless is an application that manages your personal documents. With
+the help of a document scanner (see :ref:`scanners`), paperless transforms
+your wieldy physical document binders into a searchable archive and
+provices many utilities for finding and managing your documents.
+
+
+Terms and definitions
+#####################
+
+Paperless esentially consists of two different parts for managing your
+documents:
+
+* The *consumer* watches a specified folder and adds all documents in that
+  folder to paperless.
+* The *web server* provides a UI that you use to manage and search for your
+  scanned documents.
+
+Each document has a couple of fields that you can assign to them:
+
+* A *Document* is a piece of paper that sometimes contains valuable
+  information.
+* The *correspondent* of a document is the person, institution or company that
+  a document either originates form, or is sent to.
+* A *tag* is a label that you can assign to documents. Think of labels as more
+  powerful folders: Multiple documents can be grouped together with a single
+  tag, however, a single document can also have multiple tags. This is not 
+  possible with folders. The reason folders are not implemented in paperless
+  is simply that tags are much more versatile than folders.
+* A *document type* is used to demarkate the type of a document such as letter,
+  bank statement, invoice, contract, etc. It is used to identify what a document
+  is about.
+* The *date added* of a document is the date the document was scanned into
+  paperless. You cannot and should not change this date.
+* The *date created* of a document is the date the document was intially issued.
+  This can be the date you bought a product, the date you signed a contract, or
+  the date a letter was sent to you.
+* The *archive serial number* (short: ASN) of a document is the identifier of
+  the document in your physical document binders. See
+  :ref:`usage-recommended_workflow` below.
+* The *content* of a document is the text that was OCR'ed from the document.
+  This text is fed into the search engine and is used for matching tags,
+  correspondents and document types.
+
+.. TODO: hyperref
+
+Frontend overview
+#################
+
+.. warning::
+
+    TBD. Add some fancy screenshots!
+
+Adding documents to paperless
+#############################
+
+Once you've got Paperless setup, you need to start feeding documents into it.
+Currently, there are three options: the consumption directory, IMAP (email), and
+HTTP POST.
+
+
+The consumption directory
+=========================
+
+The primary method of getting documents into your database is by putting them in
+the consumption directory.  The consumer runs in an infinite
+loop looking for new additions to this directory and when it finds them, it goes
+about the process of parsing them with the OCR, indexing what it finds, and storing
+it in the media directory.
+
+Getting stuff into this directory is up to you.  If you're running Paperless
+on your local computer, you might just want to drag and drop files there, but if
+you're running this on a server and want your scanner to automatically push
+files to this directory, you'll need to setup some sort of service to accept the
+files from the scanner.  Typically, you're looking at an FTP server like
+`Proftpd`_ or a Windows folder share with `Samba`_.
+
+.. _Proftpd: http://www.proftpd.org/
+.. _Samba: http://www.samba.org/
+
+.. TODO: hyperref to configuration of the location of this magic folder.
+
+
+IMAP (Email)
+============
+
+Another handy way to get documents into your database is to email them to
+yourself.  The typical use-case would be to be out for lunch and want to send a
+copy of the receipt back to your system at home.  Paperless can be taught to
+pull emails down from an arbitrary account and dump them into the consumption
+directory where the consumer will follow the
+usual pattern on consuming the document.
+
+Some things you need to know about this feature:
+
+* It's disabled by default. By setting the values below it will be enabled.
+* It's been tested in a limited environment, so it may not work for you (please
+  submit a pull request if you can!)
+* It's designed to **delete mail from the server once consumed**.  So don't go
+  pointing this to your personal email account and wonder where all your stuff
+  went.
+* Currently, only one photo (attachment) per email will work.
+
+So, with all that in mind, here's what you do to get it running:
+
+1. Setup a new email account somewhere, or if you're feeling daring, create a
+   folder in an existing email box and note the path to that folder.
+2. In ``/etc/paperless.conf`` set all of the appropriate values in
+   ``PATHS AND FOLDERS`` and ``SECURITY``.
+   If you decided to use a subfolder of an existing account, then make sure you
+   set ``PAPERLESS_CONSUME_MAIL_INBOX`` accordingly here.  You also have to set
+   the ``PAPERLESS_EMAIL_SECRET`` to something you can remember 'cause you'll
+   have to include that in every email you send.
+3. Restart paperless.  Paperless will check
+   the configured email account at startup and from then on every 10 minutes
+   for something new and pulls down whatever it finds.
+4. Send yourself an email!  Note that the subject is treated as the file name,
+   so if you set the subject to ``Correspondent - Title - tag,tag,tag``, you'll
+   get what you expect.  Also, you must include the aforementioned secret
+   string in every email so the fetcher knows that it's safe to import.
+   Note that Paperless only allows the email title to consist of safe characters
+   to be imported. These consist of alpha-numeric characters and ``-_ ,.'``.
+
+
+REST API
+========
+
+You can also submit a document using the REST API, see the API section for details.
+
+
+.. _usage-recommended_workflow:
+
+The recommended workflow
+########################
+
+Once you have familiarized yourself with paperless and are ready to use it
+for all your documents, the recommended workflow for managing your documents
+is as follows. This workflow also takes into account that some documents
+have to be kept in physical form, but still ensures that you get all the
+advantages for these documents as well.
+
+Preparations in paperless
+=========================
+
+* Create an inbox tag that gets assigned to all new documents.
+* Create a TODO tag.
+
+Processing of the physical documents
+====================================
+
+Keep a physical inbox. Whenever you receive a document that you need to 
+archive, put it into your inbox. Regulary, do the following for all documents
+in your inbox:
+
+1.  For each document, decide if you need to keep the document in physical
+    form. This applies to certain important documents, such as contracts and
+    certificates.
+2.  If you need to keep the document, write a running number on the document
+    before scanning, starting at one and counting upwards. This is the archive
+    serial number, or ASN in short.
+3.  Scan the document.
+4.  If the document has an ASN assigned, store it in a *single* binder, sorted
+    by ASN. Don't order this binder in any other way.
+5.  If the document has no ASN, throw it away. Yay!
+
+Over time, you will notice that your physical binder will fill up. If it is
+full, label the binder with the range of ASNs in this binder (i.e., "Documents
+1 to 343"), store the binder in your cellar or elsewhere, and start a new
+binder.
+
+The idea behind this process is that you will never have to use the physical
+binders to find a document. If you need a specific physical document, you
+may find this document by:
+
+1.  Searching in paperless for the document.
+2.  Identify the ASN of the document, since it appears on the scan.
+3.  Grab the relevant document binder and get the document. This is easy since
+    they are sorted by ASN.
+
+Processing of documents in paperless
+====================================
+
+Once you have scanned in a document, proceed in paperless as follows.
+
+1.  If the document has an ASN, assign the ASN to the document.
+2.  Assign a correspondent to the document (i.e., your employer, bank, etc)
+    This isnt strictly necessary but helps in finding a document when you need
+    it.
+3.  Assign a document type (i.e., invoice, bank statement, etc) to the document
+    This isnt strictly necessary but helps in finding a document when you need
+    it.
+4.  Assign a proper title to the document (the name of an item you bought, the
+    subject of the letter, etc)
+5.  Check that the date of the document is corrent. Paperless tries to read
+    the date from the content of the document, but this fails sometimes if the
+    OCR is bad or multiple dates appear on the document.
+6.  Remove inbox tags from the documents.
+
+
+Task management
+===============
+
+Some documents require attention and require you to act on the document. You
+may take two different approaches to handle these documents based on how
+regularly you intent to use paperless and scan documents.
+
+* If you scan and process your documents in paperless regularly, assign a
+  TODO tag to all scanned documents that you need to process. Create a saved
+  view on the dashboard that shows all documents with this tag.
+* If you do not scan documents regularly and use paperless solely for archiving,
+  create a physical todo box next to your physical inbox and put documents you
+  need to process in the TODO box. When you performed the task associated with
+  the document, move it to the inbox.