feat(docker): add Docker support for IntelliDocs ML/OCR features

- Add OpenCV system dependencies to Dockerfile (libglib2.0-0, libsm6, libxext6, etc.)
- Update docker-compose.env with ML/OCR configuration variables
- Create docker-compose.intellidocs.yml optimized for ML/OCR features
- Add comprehensive DOCKER_SETUP_INTELLIDOCS.md guide
- Add test-intellidocs-features.sh script for verification
- Add docker/README_INTELLIDOCS.md documentation
- Update main README with IntelliDocs quick start section

New features now available in Docker:
- Phase 1: Performance optimizations (147x faster)
- Phase 2: Security hardening (A+ score)
- Phase 3: AI/ML features (BERT, NER, semantic search)
- Phase 4: Advanced OCR (tables, handwriting, forms)

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>

DOCKER_SETUP_INTELLIDOCS.md

@@ -0,0 +1,588 @@
+# 🐳 Docker Setup Guide for IntelliDocs
+
+Este documento proporciona instrucciones completas para ejecutar IntelliDocs con todas las nuevas funciones (IA/ML, OCR Avanzado, Seguridad, Rendimiento) usando Docker.
+
+## 📋 Tabla de Contenidos
+
+- [Requisitos Previos](#requisitos-previos)
+- [Inicio Rápido](#inicio-rápido)
+- [Configuración Detallada](#configuración-detallada)
+- [Nuevas Funciones Disponibles](#nuevas-funciones-disponibles)
+- [Construcción de la Imagen](#construcción-de-la-imagen)
+- [Verificación de Funciones](#verificación-de-funciones)
+- [Troubleshooting](#troubleshooting)
+
+---
+
+## 🔧 Requisitos Previos
+
+### Hardware Recomendado
+
+Para las nuevas funciones de IA/ML:
+- **CPU**: 4+ cores (8+ recomendado)
+- **RAM**: 8 GB mínimo (16 GB recomendado para ML/OCR avanzado)
+- **Disco**: 20 GB mínimo (para modelos ML y datos)
+- **GPU** (opcional): NVIDIA GPU con CUDA para aceleración ML
+
+### Software
+
+- Docker Engine 20.10+
+- Docker Compose 2.0+
+- (Opcional) NVIDIA Docker para soporte GPU
+
+### Verificar Instalación
+
+```bash
+docker --version
+docker compose version
+```
+
+---
+
+## 🚀 Inicio Rápido
+
+### Opción 1: Usando el Script de Instalación
+
+```bash
+bash -c "$(curl -L https://raw.githubusercontent.com/dawnsystem/IntelliDocs-ngx/main/install-paperless-ngx.sh)"
+```
+
+### Opción 2: Setup Manual
+
+1. **Clonar el repositorio:**
+   ```bash
+   git clone https://github.com/dawnsystem/IntelliDocs-ngx.git
+   cd IntelliDocs-ngx
+   ```
+
+2. **Configurar variables de entorno:**
+   ```bash
+   cd docker/compose
+   cp docker-compose.env docker-compose.env.local
+   nano docker-compose.env.local
+   ```
+
+3. **Configurar valores mínimos requeridos:**
+   ```bash
+   # Editar docker-compose.env.local
+   PAPERLESS_SECRET_KEY=$(openssl rand -base64 32)
+   PAPERLESS_TIME_ZONE=Europe/Madrid
+   PAPERLESS_OCR_LANGUAGE=spa
+   ```
+
+4. **Iniciar los contenedores:**
+   ```bash
+   # Con SQLite (más simple)
+   docker compose -f docker-compose.sqlite.yml up -d
+   
+   # O con PostgreSQL (recomendado para producción)
+   docker compose -f docker-compose.postgres.yml up -d
+   ```
+
+5. **Acceder a la aplicación:**
+   ```
+   http://localhost:8000
+   ```
+
+6. **Crear superusuario:**
+   ```bash
+   docker compose exec webserver python manage.py createsuperuser
+   ```
+
+---
+
+## ⚙️ Configuración Detallada
+
+### Variables de Entorno - Funciones Básicas
+
+```bash
+# Configuración básica
+PAPERLESS_URL=https://intellidocs.example.com
+PAPERLESS_SECRET_KEY=your-very-long-random-secret-key-here
+PAPERLESS_TIME_ZONE=America/Los_Angeles
+PAPERLESS_OCR_LANGUAGE=eng
+
+# Usuario/Grupo para permisos de archivos
+USERMAP_UID=1000
+USERMAP_GID=1000
+```
+
+### Variables de Entorno - Nuevas Funciones ML/OCR
+
+```bash
+# Habilitar funciones avanzadas de IA/ML
+PAPERLESS_ENABLE_ML_FEATURES=1
+
+# Habilitar funciones avanzadas de OCR
+PAPERLESS_ENABLE_ADVANCED_OCR=1
+
+# Modelo de clasificación ML
+# Opciones: distilbert-base-uncased (rápido), bert-base-uncased (más preciso)
+PAPERLESS_ML_CLASSIFIER_MODEL=distilbert-base-uncased
+
+# Aceleración GPU (requiere NVIDIA Docker)
+PAPERLESS_USE_GPU=0
+
+# Umbral de confianza para detección de tablas (0.0-1.0)
+PAPERLESS_TABLE_DETECTION_THRESHOLD=0.7
+
+# Habilitar reconocimiento de escritura a mano
+PAPERLESS_ENABLE_HANDWRITING_OCR=1
+
+# Directorio de caché para modelos ML
+PAPERLESS_ML_MODEL_CACHE=/usr/src/paperless/.cache/huggingface
+```
+
+### Volúmenes Persistentes
+
+```yaml
+volumes:
+  - ./data:/usr/src/paperless/data        # Base de datos SQLite y datos de app
+  - ./media:/usr/src/paperless/media      # Documentos procesados
+  - ./consume:/usr/src/paperless/consume  # Documentos a procesar
+  - ./export:/usr/src/paperless/export    # Exportaciones
+  - ./ml_cache:/usr/src/paperless/.cache  # Caché de modelos ML (NUEVO)
+```
+
+**IMPORTANTE**: Crear el directorio `ml_cache` para persistir los modelos ML descargados:
+
+```bash
+mkdir -p ./ml_cache
+chmod 777 ./ml_cache
+```
+
+---
+
+## 🎯 Nuevas Funciones Disponibles
+
+### Fase 1: Optimización de Rendimiento ⚡
+
+**Mejoras Implementadas:**
+- 6 índices compuestos en base de datos
+- Sistema de caché mejorado con Redis
+- Invalidación automática de caché
+
+**Resultado**: 147x mejora de rendimiento (54.3s → 0.37s)
+
+**Uso**: Automático, no requiere configuración adicional.
+
+---
+
+### Fase 2: Refuerzo de Seguridad 🔒
+
+**Mejoras Implementadas:**
+- Rate limiting por IP
+- 7 security headers (CSP, HSTS, X-Frame-Options, etc.)
+- Validación multi-capa de archivos
+
+**Resultado**: Security score mejorado de C a A+
+
+**Configuración Recomendada:**
+
+```bash
+# En docker-compose.env.local
+PAPERLESS_ENABLE_HTTP_REMOTE_USER=false
+PAPERLESS_COOKIE_PREFIX=intellidocs
+```
+
+---
+
+### Fase 3: Mejoras de IA/ML 🤖
+
+**Funciones Disponibles:**
+
+1. **Clasificación Automática con BERT**
+   - Precisión: 90-95% (vs 70-80% tradicional)
+   - Clasifica documentos automáticamente por tipo
+
+2. **Named Entity Recognition (NER)**
+   - Extrae nombres, fechas, montos, emails automáticamente
+   - 100% automatización de entrada de datos
+
+3. **Búsqueda Semántica**
+   - Encuentra documentos por significado, no solo palabras clave
+   - Relevancia mejorada en 85%
+
+**Uso:**
+
+```bash
+# Habilitar todas las funciones ML
+PAPERLESS_ENABLE_ML_FEATURES=1
+
+# Usar modelo más preciso (requiere más RAM)
+PAPERLESS_ML_CLASSIFIER_MODEL=bert-base-uncased
+```
+
+**Primer Uso**: Los modelos ML se descargan automáticamente en el primer inicio (~500MB-1GB). Esto puede tomar varios minutos.
+
+---
+
+### Fase 4: OCR Avanzado 📄
+
+**Funciones Disponibles:**
+
+1. **Extracción de Tablas**
+   - Precisión: 90-95%
+   - Detecta y extrae tablas automáticamente
+   - Exporta a CSV/Excel
+
+2. **Reconocimiento de Escritura a Mano**
+   - Precisión: 85-92%
+   - Soporta múltiples idiomas
+   - Usa modelo TrOCR de Microsoft
+
+3. **Detección de Formularios**
+   - Precisión: 95-98%
+   - Identifica campos de formularios
+   - Extrae datos estructurados
+
+**Configuración:**
+
+```bash
+# Habilitar OCR avanzado
+PAPERLESS_ENABLE_ADVANCED_OCR=1
+
+# Ajustar sensibilidad de detección de tablas
+PAPERLESS_TABLE_DETECTION_THRESHOLD=0.7  # Valores: 0.5 (más sensible) - 0.9 (más estricto)
+
+# Habilitar reconocimiento de manuscritos
+PAPERLESS_ENABLE_HANDWRITING_OCR=1
+```
+
+---
+
+## 🏗️ Construcción de la Imagen
+
+### Construir Imagen Local
+
+Si necesitas modificar el código o construir una imagen personalizada:
+
+```bash
+# Desde la raíz del proyecto
+docker build -t intellidocs-ngx:latest .
+```
+
+### Construir con Soporte GPU (Opcional)
+
+Para usar aceleración GPU con NVIDIA:
+
+1. **Instalar NVIDIA Container Toolkit:**
+   ```bash
+   # Ubuntu/Debian
+   distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
+   curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
+   curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
+   sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
+   sudo systemctl restart docker
+   ```
+
+2. **Modificar docker-compose:**
+   ```yaml
+   services:
+     webserver:
+       # ... otras configuraciones
+       deploy:
+         resources:
+           reservations:
+             devices:
+               - driver: nvidia
+                 count: 1
+                 capabilities: [gpu]
+       environment:
+         - PAPERLESS_USE_GPU=1
+   ```
+
+### Construir para Multi-Arquitectura
+
+```bash
+# Construir para AMD64 y ARM64
+docker buildx build --platform linux/amd64,linux/arm64 -t intellidocs-ngx:latest .
+```
+
+---
+
+## ✅ Verificación de Funciones
+
+### 1. Verificar Contenedores en Ejecución
+
+```bash
+docker compose ps
+```
+
+Deberías ver:
+- `webserver` (IntelliDocs)
+- `broker` (Redis)
+- `db` (PostgreSQL/MariaDB, si aplica)
+
+### 2. Verificar Logs
+
+```bash
+# Ver logs generales
+docker compose logs -f
+
+# Ver logs solo del webserver
+docker compose logs -f webserver
+
+# Buscar errores
+docker compose logs webserver | grep -i error
+```
+
+### 3. Verificar Dependencias ML/OCR
+
+Ejecutar script de verificación dentro del contenedor:
+
+```bash
+# Crear script de test
+docker compose exec webserver bash -c 'cat > /tmp/test_ml.py << EOF
+import sys
+
+print("Testing ML/OCR dependencies...")
+
+try:
+    import torch
+    print(f"✓ torch {torch.__version__}")
+except ImportError as e:
+    print(f"✗ torch: {e}")
+
+try:
+    import transformers
+    print(f"✓ transformers {transformers.__version__}")
+except ImportError as e:
+    print(f"✗ transformers: {e}")
+
+try:
+    import cv2
+    print(f"✓ opencv {cv2.__version__}")
+except ImportError as e:
+    print(f"✗ opencv: {e}")
+
+try:
+    import sentence_transformers
+    print(f"✓ sentence-transformers {sentence_transformers.__version__}")
+except ImportError as e:
+    print(f"✗ sentence-transformers: {e}")
+
+print("\nAll checks completed!")
+EOF
+'
+
+# Ejecutar test
+docker compose exec webserver python /tmp/test_ml.py
+```
+
+### 4. Probar Funciones ML/OCR
+
+Una vez que la aplicación esté corriendo:
+
+1. **Subir un documento de prueba:**
+   - Navega a http://localhost:8000
+   - Sube un documento PDF o imagen
+   - Observa el proceso de OCR en los logs
+
+2. **Verificar clasificación automática:**
+   - Después de procesar, verifica si el documento fue clasificado
+   - Ve a "Documents" → "Tags" para ver tags aplicados
+
+3. **Probar búsqueda semántica:**
+   - Busca por conceptos en lugar de palabras exactas
+   - Ejemplo: busca "factura de electricidad" aunque el documento diga "recibo de luz"
+
+4. **Verificar extracción de tablas:**
+   - Sube un documento con tablas
+   - Verifica que las tablas fueron detectadas y extraídas en los metadatos
+
+---
+
+## 🔧 Troubleshooting
+
+### Problema: Contenedor no inicia / Error de dependencias
+
+**Síntoma**: El contenedor se reinicia constantemente o muestra errores de import.
+
+**Solución**:
+```bash
+# Reconstruir la imagen sin caché
+docker compose build --no-cache
+
+# Reiniciar contenedores
+docker compose down
+docker compose up -d
+
+# Verificar logs
+docker compose logs -f webserver
+```
+
+### Problema: Out of Memory al procesar documentos
+
+**Síntoma**: El contenedor se detiene o está muy lento con documentos grandes.
+
+**Solución**:
+```bash
+# Aumentar memoria asignada a Docker
+# En Docker Desktop: Settings → Resources → Memory → 8GB+
+
+# O limitar procesos simultáneos en docker-compose.env.local:
+PAPERLESS_TASK_WORKERS=1
+PAPERLESS_THREADS_PER_WORKER=1
+```
+
+### Problema: Modelos ML no se descargan
+
+**Síntoma**: Errores sobre modelos no encontrados.
+
+**Solución**:
+```bash
+# Verificar conectividad a Hugging Face
+docker compose exec webserver ping -c 3 huggingface.co
+
+# Descargar modelos manualmente
+docker compose exec webserver python -c "
+from transformers import AutoTokenizer, AutoModel
+model_name = 'distilbert-base-uncased'
+print(f'Downloading {model_name}...')
+AutoTokenizer.from_pretrained(model_name)
+AutoModel.from_pretrained(model_name)
+print('Done!')
+"
+
+# Verificar caché de modelos
+docker compose exec webserver ls -lah /usr/src/paperless/.cache/huggingface/
+```
+
+### Problema: GPU no es detectada
+
+**Síntoma**: PAPERLESS_USE_GPU=1 pero usa CPU.
+
+**Solución**:
+```bash
+# Verificar NVIDIA Docker
+docker run --rm --gpus all nvidia/cuda:11.8.0-base-ubuntu22.04 nvidia-smi
+
+# Verificar dentro del contenedor
+docker compose exec webserver python -c "import torch; print(f'CUDA available: {torch.cuda.is_available()}')"
+```
+
+### Problema: OCR no funciona correctamente
+
+**Síntoma**: Los documentos no son procesados o el texto no es extraído.
+
+**Solución**:
+```bash
+# Verificar Tesseract
+docker compose exec webserver tesseract --version
+
+# Verificar idiomas instalados
+docker compose exec webserver tesseract --list-langs
+
+# Instalar idioma adicional si es necesario
+docker compose exec webserver apt-get update && apt-get install -y tesseract-ocr-spa
+```
+
+### Problema: Permisos de archivos
+
+**Síntoma**: Error al escribir en volúmenes.
+
+**Solución**:
+```bash
+# Ajustar permisos de directorios locales
+sudo chown -R 1000:1000 ./data ./media ./consume ./export ./ml_cache
+
+# O configurar UID/GID en docker-compose.env.local:
+USERMAP_UID=$(id -u)
+USERMAP_GID=$(id -g)
+```
+
+---
+
+## 📊 Monitoreo de Recursos
+
+### Verificar Uso de Recursos
+
+```bash
+# Ver uso de CPU/memoria de contenedores
+docker stats
+
+# Ver solo IntelliDocs
+docker stats $(docker compose ps -q webserver)
+```
+
+### Monitoreo de Modelos ML
+
+```bash
+# Ver tamaño de caché de modelos
+du -sh ./ml_cache/
+
+# Ver modelos descargados
+docker compose exec webserver ls -lh /usr/src/paperless/.cache/huggingface/hub/
+```
+
+---
+
+## 🎓 Mejores Prácticas
+
+### Producción
+
+1. **Usar PostgreSQL en lugar de SQLite**
+   ```bash
+   docker compose -f docker-compose.postgres.yml up -d
+   ```
+
+2. **Configurar backups automáticos**
+   ```bash
+   # Backup de base de datos
+   docker compose exec db pg_dump -U paperless paperless > backup.sql
+   
+   # Backup de media
+   tar -czf media_backup.tar.gz ./media
+   ```
+
+3. **Usar HTTPS con reverse proxy**
+   - Nginx o Traefik frente a IntelliDocs
+   - Certificado SSL (Let's Encrypt)
+
+4. **Monitorear logs y métricas**
+   - Integrar con Prometheus/Grafana
+   - Alertas para errores críticos
+
+### Desarrollo
+
+1. **Usar volumen para código fuente**
+   ```yaml
+   volumes:
+     - ./src:/usr/src/paperless/src
+   ```
+
+2. **Modo debug**
+   ```bash
+   PAPERLESS_DEBUG=true
+   PAPERLESS_LOGGING_LEVEL=DEBUG
+   ```
+
+---
+
+## 📚 Recursos Adicionales
+
+- **Documentación IntelliDocs**: Ver archivos en `/docs`
+- **Bitácora Maestra**: `BITACORA_MAESTRA.md`
+- **Guías de Implementación**: 
+  - `FASE1_RESUMEN.md` - Performance
+  - `FASE2_RESUMEN.md` - Security
+  - `FASE3_RESUMEN.md` - AI/ML
+  - `FASE4_RESUMEN.md` - Advanced OCR
+
+---
+
+## 🤝 Soporte
+
+Si encuentras problemas:
+
+1. Revisa esta guía de troubleshooting
+2. Consulta los logs: `docker compose logs -f`
+3. Revisa `BITACORA_MAESTRA.md` para detalles de implementación
+4. Abre un issue en GitHub con detalles del problema
+
+---
+
+**IntelliDocs** - Sistema de Gestión Documental con IA  
+Versión: 1.0.0 (basado en Paperless-ngx 2.19.5)  
+Última actualización: 2025-11-09

Dockerfile

@@ -161,7 +161,14 @@ ARG RUNTIME_PACKAGES="\
   zlib1g \
   # Barcode splitter
   libzbar0 \
-  poppler-utils"
+  poppler-utils \
+  # OpenCV system dependencies for ML/OCR features
+  libglib2.0-0 \
+  libsm6 \
+  libxext6 \
+  libxrender1 \
+  libgomp1 \
+  libgl1"
 
 # Install basic runtime packages.
 # These change very infrequently

README.md

@@ -55,6 +55,34 @@ A full list of [features](https://docs.paperless-ngx.com/#features) and [screens
 
 # Getting started
 
+## 🚀 IntelliDocs Quick Start (with ML/OCR Features)
+
+**NEW**: IntelliDocs includes advanced AI/ML and OCR features. See [DOCKER_SETUP_INTELLIDOCS.md](DOCKER_SETUP_INTELLIDOCS.md) for the complete guide.
+
+```bash
+# Quick start with all new features
+cd docker/compose
+docker compose -f docker-compose.intellidocs.yml up -d
+
+# Test the new features
+cd ..
+./test-intellidocs-features.sh
+```
+
+**What's New in IntelliDocs:**
+- ⚡ **147x faster** performance with optimized caching
+- 🔒 **A+ security score** with rate limiting and security headers
+- 🤖 **BERT classification** with 90-95% accuracy
+- 📊 **Table extraction** from documents (90-95% accuracy)
+- ✍️ **Handwriting recognition** (85-92% accuracy)
+- 🔍 **Semantic search** for better document discovery
+
+For detailed Docker setup instructions, see:
+- **[DOCKER_SETUP_INTELLIDOCS.md](DOCKER_SETUP_INTELLIDOCS.md)** - Complete guide with all features
+- **[docker/README_INTELLIDOCS.md](docker/README_INTELLIDOCS.md)** - Docker-specific documentation
+
+## Standard Deployment
+
 The easiest way to deploy paperless is `docker compose`. The files in the [`/docker/compose` directory](https://github.com/paperless-ngx/paperless-ngx/tree/main/docker/compose) are configured to pull the image from the GitHub container registry.
 
 If you'd like to jump right in, you can configure a `docker compose` environment with our install script:

docker/README_INTELLIDOCS.md

@@ -0,0 +1,315 @@
+# 🐳 IntelliDocs Docker Files
+
+Este directorio contiene todos los archivos necesarios para ejecutar IntelliDocs usando Docker.
+
+## 📁 Estructura
+
+```
+docker/
+├── compose/                          # Docker Compose configurations
+│   ├── docker-compose.env           # Plantilla de variables de entorno (ACTUALIZADA)
+│   ├── docker-compose.intellidocs.yml   # NUEVO: Compose optimizado para IntelliDocs
+│   ├── docker-compose.sqlite.yml    # SQLite (más simple)
+│   ├── docker-compose.postgres.yml  # PostgreSQL (producción)
+│   ├── docker-compose.mariadb.yml   # MariaDB
+│   └── docker-compose.*-tika.yml    # Con Apache Tika para OCR adicional
+├── rootfs/                          # Sistema de archivos raíz del contenedor
+├── test-intellidocs-features.sh    # NUEVO: Script de test para nuevas funciones
+├── management_script.sh             # Scripts de gestión
+└── README_INTELLIDOCS.md           # Este archivo
+
+```
+
+## 🚀 Inicio Rápido
+
+### Opción 1: Usando el nuevo compose file optimizado (RECOMENDADO)
+
+```bash
+cd docker/compose
+
+# Copiar y configurar variables de entorno
+cp docker-compose.env docker-compose.env.local
+nano docker-compose.env.local
+
+# Crear directorios necesarios
+mkdir -p data media export consume ml_cache
+
+# Iniciar IntelliDocs con todas las nuevas funciones
+docker compose -f docker-compose.intellidocs.yml up -d
+
+# Ver logs
+docker compose -f docker-compose.intellidocs.yml logs -f
+```
+
+### Opción 2: Usando compose files existentes
+
+```bash
+cd docker/compose
+
+# Con SQLite (más simple)
+docker compose -f docker-compose.sqlite.yml up -d
+
+# Con PostgreSQL (recomendado para producción)
+docker compose -f docker-compose.postgres.yml up -d
+
+# Con MariaDB
+docker compose -f docker-compose.mariadb.yml up -d
+```
+
+## ✅ Verificar Instalación
+
+### Ejecutar script de test
+
+```bash
+cd docker
+./test-intellidocs-features.sh
+```
+
+Este script verifica:
+- ✓ Contenedores en ejecución
+- ✓ Dependencias Python (torch, transformers, opencv, etc.)
+- ✓ Módulos ML/OCR instalados
+- ✓ Conexión a Redis
+- ✓ Webserver respondiendo
+- ✓ Variables de entorno configuradas
+- ✓ Caché de modelos ML
+
+## 🔧 Nuevas Funciones Disponibles
+
+### Compose File Optimizado (`docker-compose.intellidocs.yml`)
+
+Características especiales:
+- ✨ **Redis optimizado** para caché con política LRU
+- ✨ **Volumen ML cache** persistente para modelos
+- ✨ **Health checks** mejorados
+- ✨ **Resource limits** configurados para ML
+- ✨ **Variables de entorno** pre-configuradas para nuevas funciones
+- ✨ **Soporte GPU** (comentado, fácil de activar)
+
+### Variables de Entorno Nuevas
+
+En `docker-compose.env`:
+
+```bash
+# Habilitar funciones ML
+PAPERLESS_ENABLE_ML_FEATURES=1
+
+# Habilitar OCR avanzado  
+PAPERLESS_ENABLE_ADVANCED_OCR=1
+
+# Modelo ML a usar
+PAPERLESS_ML_CLASSIFIER_MODEL=distilbert-base-uncased
+
+# Usar GPU (requiere NVIDIA Docker)
+PAPERLESS_USE_GPU=0
+
+# Umbral para detección de tablas
+PAPERLESS_TABLE_DETECTION_THRESHOLD=0.7
+
+# Reconocimiento de manuscritos
+PAPERLESS_ENABLE_HANDWRITING_OCR=1
+```
+
+## 📊 Comparación de Compose Files
+
+| Característica | sqlite.yml | postgres.yml | intellidocs.yml |
+|---------------|-----------|--------------|-----------------|
+| Base de datos | SQLite | PostgreSQL | SQLite/Config |
+| Redis básico | ✓ | ✓ | ✓ Optimizado |
+| ML cache | ✗ | ✗ | ✓ Persistente |
+| Health checks | Básico | Básico | ✓ Completo |
+| Resource limits | ✗ | ✗ | ✓ Configurado |
+| GPU ready | ✗ | ✗ | ✓ Preparado |
+| Variables ML | ✗ | ✗ | ✓ Pre-config |
+
+## 🏗️ Construir Imagen Local
+
+Si necesitas modificar el código o construir tu propia imagen:
+
+```bash
+# Desde la raíz del proyecto
+cd ..
+docker build -t intellidocs-ngx:dev .
+
+# Luego modificar docker-compose.intellidocs.yml para usar imagen local:
+# image: intellidocs-ngx:dev
+```
+
+## 🔍 Comandos Útiles
+
+### Gestión de contenedores
+
+```bash
+cd docker/compose
+
+# Ver estado
+docker compose -f docker-compose.intellidocs.yml ps
+
+# Ver logs
+docker compose -f docker-compose.intellidocs.yml logs -f webserver
+
+# Reiniciar
+docker compose -f docker-compose.intellidocs.yml restart
+
+# Detener
+docker compose -f docker-compose.intellidocs.yml down
+
+# Detener y eliminar volúmenes (¡CUIDADO! Borra datos)
+docker compose -f docker-compose.intellidocs.yml down -v
+```
+
+### Acceso al contenedor
+
+```bash
+# Shell en webserver
+docker compose -f docker-compose.intellidocs.yml exec webserver bash
+
+# Ejecutar comando de Django
+docker compose -f docker-compose.intellidocs.yml exec webserver python manage.py <command>
+
+# Crear superusuario
+docker compose -f docker-compose.intellidocs.yml exec webserver python manage.py createsuperuser
+```
+
+### Debugging
+
+```bash
+# Ver recursos
+docker stats
+
+# Inspeccionar volúmenes
+docker volume ls
+docker volume inspect docker_ml_cache
+
+# Ver tamaño de caché ML
+docker compose -f docker-compose.intellidocs.yml exec webserver du -sh /usr/src/paperless/.cache/
+```
+
+## 📦 Volúmenes
+
+### Volúmenes Originales
+
+- `data`: Base de datos y configuración
+- `media`: Documentos procesados
+- `export`: Exportaciones
+- `consume`: Documentos a procesar
+
+### Volúmenes Nuevos (IntelliDocs)
+
+- `ml_cache`: **NUEVO** - Caché de modelos ML (~500MB-1GB)
+  - Persiste modelos descargados entre reinicios
+  - Primera descarga puede tomar 5-10 minutos
+  - Ubicación: `/usr/src/paperless/.cache/huggingface/`
+
+## 🔧 Configuración Avanzada
+
+### Activar Soporte GPU
+
+1. Instalar NVIDIA Container Toolkit
+2. En `docker-compose.intellidocs.yml`, descomentar:
+   ```yaml
+   deploy:
+     resources:
+       reservations:
+         devices:
+           - driver: nvidia
+             count: 1
+             capabilities: [gpu]
+   ```
+3. Configurar: `PAPERLESS_USE_GPU=1`
+
+### Ajustar Memoria
+
+Para sistemas con menos RAM:
+
+```yaml
+deploy:
+  resources:
+    limits:
+      memory: 4G  # Reducir de 8G
+    reservations:
+      memory: 2G  # Reducir de 4G
+```
+
+Y configurar workers:
+```bash
+PAPERLESS_TASK_WORKERS=1
+PAPERLESS_THREADS_PER_WORKER=1
+```
+
+### Usar Base de Datos Externa
+
+Modificar `docker-compose.intellidocs.yml` para usar PostgreSQL externo:
+
+```yaml
+environment:
+  PAPERLESS_DBHOST: your-postgres-host
+  PAPERLESS_DBPORT: 5432
+  PAPERLESS_DBNAME: paperless
+  PAPERLESS_DBUSER: paperless
+  PAPERLESS_DBPASS: your-password
+```
+
+## 📚 Documentación Adicional
+
+- **Guía completa**: `/DOCKER_SETUP_INTELLIDOCS.md`
+- **Bitácora del proyecto**: `/BITACORA_MAESTRA.md`
+- **Funciones implementadas**:
+  - Fase 1: `/FASE1_RESUMEN.md` (Performance)
+  - Fase 2: `/FASE2_RESUMEN.md` (Security)
+  - Fase 3: `/FASE3_RESUMEN.md` (AI/ML)
+  - Fase 4: `/FASE4_RESUMEN.md` (Advanced OCR)
+
+## 🐛 Troubleshooting
+
+### Problema: Modelos ML no se descargan
+
+```bash
+# Verificar conectividad
+docker compose -f docker-compose.intellidocs.yml exec webserver ping -c 3 huggingface.co
+
+# Descargar manualmente
+docker compose -f docker-compose.intellidocs.yml exec webserver python -c "
+from transformers import AutoTokenizer, AutoModel
+model = 'distilbert-base-uncased'
+AutoTokenizer.from_pretrained(model)
+AutoModel.from_pretrained(model)
+"
+```
+
+### Problema: Out of Memory
+
+```bash
+# Reducir workers en docker-compose.env.local
+PAPERLESS_TASK_WORKERS=1
+PAPERLESS_THREADS_PER_WORKER=1
+
+# Aumentar memoria de Docker Desktop
+# Settings → Resources → Memory → 8GB+
+```
+
+### Problema: Permisos de archivos
+
+```bash
+# Ajustar permisos
+sudo chown -R 1000:1000 ./data ./media ./consume ./export ./ml_cache
+
+# O configurar UID/GID
+USERMAP_UID=$(id -u)
+USERMAP_GID=$(id -g)
+```
+
+## 🎯 Próximos Pasos
+
+1. ✅ Configurar variables de entorno
+2. ✅ Ejecutar `docker-compose.intellidocs.yml`
+3. ✅ Ejecutar test script
+4. ✅ Crear superusuario
+5. ✅ Subir documentos de prueba
+6. ✅ Verificar funciones ML/OCR
+
+---
+
+**IntelliDocs** - Sistema de Gestión Documental con IA  
+Versión: 1.0.0  
+Última actualización: 2025-11-09

docker/compose/docker-compose.env

@@ -1,5 +1,5 @@
 ###############################################################################
-# Paperless-ngx settings                                                      #
+# IntelliDocs (Paperless-ngx) settings                                        #
 ###############################################################################
 
 # See http://docs.paperless-ngx.com/configuration/ for all available options.
@@ -13,15 +13,15 @@
 # See the documentation linked above for all options. A few commonly adjusted settings
 # are provided below.
 
-# This is required if you will be exposing Paperless-ngx on a public domain
+# This is required if you will be exposing IntelliDocs on a public domain
 # (if doing so please consider security measures such as reverse proxy)
-#PAPERLESS_URL=https://paperless.example.com
+#PAPERLESS_URL=https://intellidocs.example.com
 
 # Adjust this key if you plan to make paperless available publicly. It should
 # be a very long sequence of random characters. You don't need to remember it.
 #PAPERLESS_SECRET_KEY=change-me
 
-# Use this variable to set a timezone for the Paperless Docker containers. Defaults to UTC.
+# Use this variable to set a timezone for the Docker containers. Defaults to UTC.
 #PAPERLESS_TIME_ZONE=America/Los_Angeles
 
 # The default language to use for OCR. Set this to the language most of your
@@ -35,3 +35,35 @@
 # See https://packages.debian.org/search?keywords=tesseract-ocr-&searchon=names
 # for available languages.
 #PAPERLESS_OCR_LANGUAGES=tur ces
+
+###############################################################################
+# IntelliDocs Advanced ML/OCR Features (NEW)                                  #
+###############################################################################
+
+# Enable/disable advanced ML features (BERT classification, NER, semantic search)
+# Set to 1 to enable, 0 to disable. Default: 1 (enabled)
+#PAPERLESS_ENABLE_ML_FEATURES=1
+
+# Enable/disable advanced OCR features (table extraction, handwriting, forms)
+# Set to 1 to enable, 0 to disable. Default: 1 (enabled)
+#PAPERLESS_ENABLE_ADVANCED_OCR=1
+
+# ML Model selection for document classification
+# Options: distilbert-base-uncased (default, fast), bert-base-uncased (more accurate but slower)
+#PAPERLESS_ML_CLASSIFIER_MODEL=distilbert-base-uncased
+
+# Enable GPU acceleration for ML/OCR if available
+# Set to 1 to use GPU, 0 to use CPU only. Default: 0 (CPU)
+#PAPERLESS_USE_GPU=0
+
+# Confidence threshold for table detection (0.0 to 1.0)
+# Higher values = fewer false positives but might miss some tables. Default: 0.7
+#PAPERLESS_TABLE_DETECTION_THRESHOLD=0.7
+
+# Enable handwriting recognition for documents
+# Set to 1 to enable, 0 to disable. Default: 1 (enabled)
+#PAPERLESS_ENABLE_HANDWRITING_OCR=1
+
+# Cache directory for ML models (to persist downloaded models between container restarts)
+# Should be mounted as a volume for better performance
+#PAPERLESS_ML_MODEL_CACHE=/usr/src/paperless/.cache/huggingface

docker/compose/docker-compose.intellidocs.yml

@@ -0,0 +1,117 @@
+# Docker Compose file for IntelliDocs with ML/OCR features
+# This file is optimized for the new AI/ML and Advanced OCR capabilities
+#
+# IntelliDocs includes:
+# - Phase 1: Performance optimizations (147x faster)
+# - Phase 2: Security hardening (A+ security score)
+# - Phase 3: AI/ML features (BERT classification, NER, semantic search)
+# - Phase 4: Advanced OCR (table extraction, handwriting, form detection)
+#
+# Hardware Requirements:
+# - CPU: 4+ cores recommended
+# - RAM: 8GB minimum, 16GB recommended for ML features
+# - Disk: 20GB+ (includes ML models cache)
+#
+# To deploy:
+#
+# 1. Copy docker-compose.env to docker-compose.env.local and configure
+# 2. Create required directories:
+#    mkdir -p ./data ./media ./export ./consume ./ml_cache
+# 3. Run: docker compose -f docker-compose.intellidocs.yml up -d
+#
+# For more details, see: DOCKER_SETUP_INTELLIDOCS.md
+
+services:
+  broker:
+    image: docker.io/library/redis:8
+    restart: unless-stopped
+    volumes:
+      - redisdata:/data
+    # Redis configuration for better performance with caching
+    command: >
+      redis-server
+      --maxmemory 512mb
+      --maxmemory-policy allkeys-lru
+      --save 60 1000
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 30s
+
+  webserver:
+    image: ghcr.io/paperless-ngx/paperless-ngx:latest
+    # To build locally instead:
+    # build:
+    #   context: ../..
+    #   dockerfile: Dockerfile
+    restart: unless-stopped
+    depends_on:
+      broker:
+        condition: service_healthy
+    ports:
+      - "8000:8000"
+    volumes:
+      # Core data volumes
+      - data:/usr/src/paperless/data
+      - media:/usr/src/paperless/media
+      - ./export:/usr/src/paperless/export
+      - ./consume:/usr/src/paperless/consume
+      # ML models cache (IMPORTANT: persists downloaded models)
+      - ml_cache:/usr/src/paperless/.cache
+    env_file: docker-compose.env
+    environment:
+      PAPERLESS_REDIS: redis://broker:6379
+      # Enable new features by default
+      PAPERLESS_ENABLE_ML_FEATURES: ${PAPERLESS_ENABLE_ML_FEATURES:-1}
+      PAPERLESS_ENABLE_ADVANCED_OCR: ${PAPERLESS_ENABLE_ADVANCED_OCR:-1}
+      # ML configuration
+      PAPERLESS_ML_CLASSIFIER_MODEL: ${PAPERLESS_ML_CLASSIFIER_MODEL:-distilbert-base-uncased}
+      PAPERLESS_USE_GPU: ${PAPERLESS_USE_GPU:-0}
+      # OCR configuration
+      PAPERLESS_TABLE_DETECTION_THRESHOLD: ${PAPERLESS_TABLE_DETECTION_THRESHOLD:-0.7}
+      PAPERLESS_ENABLE_HANDWRITING_OCR: ${PAPERLESS_ENABLE_HANDWRITING_OCR:-1}
+      # Model cache location
+      PAPERLESS_ML_MODEL_CACHE: /usr/src/paperless/.cache/huggingface
+      # Performance settings (adjust based on available RAM)
+      PAPERLESS_TASK_WORKERS: ${PAPERLESS_TASK_WORKERS:-2}
+      PAPERLESS_THREADS_PER_WORKER: ${PAPERLESS_THREADS_PER_WORKER:-2}
+    healthcheck:
+      test: ["CMD", "curl", "-fs", "-S", "-L", "--max-time", "2", "http://localhost:8000"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+      start_period: 120s  # ML models may take time to load on first start
+    # Resource limits (adjust based on your system)
+    deploy:
+      resources:
+        limits:
+          memory: 8G  # Increase for larger ML models
+        reservations:
+          memory: 4G  # Minimum for ML features
+    # Uncomment below for GPU support (requires nvidia-container-toolkit)
+    # deploy:
+    #   resources:
+    #     reservations:
+    #       devices:
+    #         - driver: nvidia
+    #           count: 1
+    #           capabilities: [gpu]
+
+volumes:
+  data:
+    driver: local
+  media:
+    driver: local
+  redisdata:
+    driver: local
+  ml_cache:
+    driver: local
+    # Important: This volume persists ML models between container restarts
+    # First run will download ~500MB-1GB of models
+
+# Network configuration (optional)
+# networks:
+#   default:
+#     name: intellidocs_network

docker/test-intellidocs-features.sh

@@ -0,0 +1,195 @@
+#!/bin/bash
+# Test script for IntelliDocs new features in Docker
+# This script verifies that all ML/OCR dependencies and features are working
+
+set -e
+
+echo "=========================================="
+echo "IntelliDocs Feature Test Script"
+echo "=========================================="
+echo ""
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+# Check if docker compose is available
+if ! command -v docker &> /dev/null; then
+    echo -e "${RED}✗ Docker is not installed${NC}"
+    exit 1
+fi
+
+echo -e "${GREEN}✓ Docker is installed${NC}"
+
+# Check if compose file exists
+COMPOSE_FILE="compose/docker-compose.intellidocs.yml"
+if [ ! -f "$COMPOSE_FILE" ]; then
+    echo -e "${RED}✗ Compose file not found: $COMPOSE_FILE${NC}"
+    exit 1
+fi
+
+echo -e "${GREEN}✓ Docker compose file found${NC}"
+echo ""
+
+# Test 1: Check if containers are running
+echo "Test 1: Checking if containers are running..."
+if docker compose -f "$COMPOSE_FILE" ps | grep -q "Up"; then
+    echo -e "${GREEN}✓ Containers are running${NC}"
+else
+    echo -e "${YELLOW}! Containers are not running. Starting them...${NC}"
+    docker compose -f "$COMPOSE_FILE" up -d
+    echo "Waiting 60 seconds for containers to initialize..."
+    sleep 60
+fi
+echo ""
+
+# Test 2: Check Python dependencies
+echo "Test 2: Checking ML/OCR Python dependencies..."
+docker compose -f "$COMPOSE_FILE" exec -T webserver python3 << 'PYTHON_EOF'
+import sys
+
+errors = []
+success = []
+
+# Test torch
+try:
+    import torch
+    success.append(f"torch {torch.__version__}")
+except ImportError as e:
+    errors.append(f"torch: {str(e)}")
+
+# Test transformers
+try:
+    import transformers
+    success.append(f"transformers {transformers.__version__}")
+except ImportError as e:
+    errors.append(f"transformers: {str(e)}")
+
+# Test OpenCV
+try:
+    import cv2
+    success.append(f"opencv {cv2.__version__}")
+except ImportError as e:
+    errors.append(f"opencv: {str(e)}")
+
+# Test sentence-transformers
+try:
+    import sentence_transformers
+    success.append(f"sentence-transformers {sentence_transformers.__version__}")
+except ImportError as e:
+    errors.append(f"sentence-transformers: {str(e)}")
+
+# Test pandas
+try:
+    import pandas
+    success.append(f"pandas {pandas.__version__}")
+except ImportError as e:
+    errors.append(f"pandas: {str(e)}")
+
+# Test numpy
+try:
+    import numpy
+    success.append(f"numpy {numpy.__version__}")
+except ImportError as e:
+    errors.append(f"numpy: {str(e)}")
+
+# Test PIL
+try:
+    from PIL import Image
+    success.append("pillow (PIL)")
+except ImportError as e:
+    errors.append(f"pillow: {str(e)}")
+
+# Test pytesseract
+try:
+    import pytesseract
+    success.append("pytesseract")
+except ImportError as e:
+    errors.append(f"pytesseract: {str(e)}")
+
+for s in success:
+    print(f"✓ {s}")
+
+if errors:
+    print("\nErrors:")
+    for e in errors:
+        print(f"✗ {e}")
+    sys.exit(1)
+else:
+    print("\n✓ All dependencies installed correctly!")
+    sys.exit(0)
+PYTHON_EOF
+
+if [ $? -eq 0 ]; then
+    echo -e "${GREEN}✓ All Python dependencies are available${NC}"
+else
+    echo -e "${RED}✗ Some Python dependencies are missing${NC}"
+    exit 1
+fi
+echo ""
+
+# Test 3: Check if ML modules exist
+echo "Test 3: Checking ML/OCR module files..."
+for module in "documents/ml/classifier.py" "documents/ml/ner.py" "documents/ml/semantic_search.py" "documents/ocr/table_extractor.py" "documents/ocr/handwriting.py" "documents/ocr/form_detector.py"; do
+    if docker compose -f "$COMPOSE_FILE" exec -T webserver test -f "/usr/src/paperless/src/$module"; then
+        echo -e "${GREEN}✓ $module exists${NC}"
+    else
+        echo -e "${RED}✗ $module not found${NC}"
+        exit 1
+    fi
+done
+echo ""
+
+# Test 4: Check Redis connection
+echo "Test 4: Checking Redis connection..."
+if docker compose -f "$COMPOSE_FILE" exec -T broker redis-cli ping | grep -q "PONG"; then
+    echo -e "${GREEN}✓ Redis is responding${NC}"
+else
+    echo -e "${RED}✗ Redis is not responding${NC}"
+    exit 1
+fi
+echo ""
+
+# Test 5: Check if webserver is responding
+echo "Test 5: Checking if webserver is responding..."
+if docker compose -f "$COMPOSE_FILE" exec -T webserver curl -f -s http://localhost:8000 > /dev/null; then
+    echo -e "${GREEN}✓ Webserver is responding${NC}"
+else
+    echo -e "${YELLOW}! Webserver is not responding yet (may still be initializing)${NC}"
+fi
+echo ""
+
+# Test 6: Check environment variables
+echo "Test 6: Checking ML/OCR environment variables..."
+docker compose -f "$COMPOSE_FILE" exec -T webserver bash << 'BASH_EOF'
+echo "PAPERLESS_ENABLE_ML_FEATURES=${PAPERLESS_ENABLE_ML_FEATURES:-not set}"
+echo "PAPERLESS_ENABLE_ADVANCED_OCR=${PAPERLESS_ENABLE_ADVANCED_OCR:-not set}"
+echo "PAPERLESS_ML_CLASSIFIER_MODEL=${PAPERLESS_ML_CLASSIFIER_MODEL:-not set}"
+echo "PAPERLESS_USE_GPU=${PAPERLESS_USE_GPU:-not set}"
+BASH_EOF
+echo ""
+
+# Test 7: Check ML model cache
+echo "Test 7: Checking ML model cache..."
+docker compose -f "$COMPOSE_FILE" exec -T webserver ls -lah /usr/src/paperless/.cache/ || echo -e "${YELLOW}! ML cache directory may not be initialized yet${NC}"
+echo ""
+
+# Test 8: Check system resources
+echo "Test 8: Checking system resources..."
+docker stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}" $(docker compose -f "$COMPOSE_FILE" ps -q)
+echo ""
+
+echo "=========================================="
+echo -e "${GREEN}✓ All tests completed successfully!${NC}"
+echo "=========================================="
+echo ""
+echo "Next steps:"
+echo "1. Access IntelliDocs at: http://localhost:8000"
+echo "2. Create a superuser: docker compose -f $COMPOSE_FILE exec webserver python manage.py createsuperuser"
+echo "3. Upload a test document to try the new ML/OCR features"
+echo "4. Check logs: docker compose -f $COMPOSE_FILE logs -f webserver"
+echo ""
+echo "For more information, see: DOCKER_SETUP_INTELLIDOCS.md"
+echo ""