Commit graph

10738 commits

Author SHA1 Message Date
dawnsystem
04ced421b8
Revert "Implement webhook system for AI events notifications" 2025-11-14 16:40:31 +01:00
dawnsystem
f00424f7c4
Merge pull request #54 from dawnsystem/copilot/add-webhook-system-ai-events
Implement webhook system for AI events notifications
2025-11-14 16:37:04 +01:00
dawnsystem
21db1cc6d6
Merge branch 'dev' into copilot/add-webhook-system-ai-events 2025-11-14 16:30:15 +01:00
copilot-swe-agent[bot]
5ae18e03b5 Merge branch 'dev' into copilot/add-webhook-system-ai-events
Resolved merge conflicts in:
- src/documents/ai_deletion_manager.py: Kept webhook integration alongside dev changes
- src/documents/ai_scanner.py: Kept webhook integration and applied_fields tracking
- src/documents/models.py: Integrated AISuggestionFeedback model with webhook imports

All conflicts resolved maintaining both webhook functionality and new AI suggestions features from dev branch.

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-14 15:26:54 +00:00
copilot-swe-agent[bot]
ebc906b713 chore(webhooks): Remove unused imports (time, timedelta)
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-14 15:17:05 +00:00
copilot-swe-agent[bot]
7b8c45c44f feat(webhooks): Complete webhook system implementation with admin and docs
- Add AIWebhookEvent and AIWebhookConfig models to Django admin
- Import webhook models in documents/models.py for Django recognition
- Create comprehensive AI_WEBHOOKS_DOCUMENTATION.md with:
  - Setup and configuration guide
  - Event payload examples
  - Security and HMAC signature verification
  - Integration examples (Slack, Discord, custom endpoints)
  - Troubleshooting guide
  - API reference
- Admin interface for webhook management and monitoring
- All webhook events are tracked and logged

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-14 15:15:17 +00:00
dawnsystem
f76a546bea
Merge pull request #43 from dawnsystem/copilot/add-ai-suggestions-endpoints
Add API endpoints for AI suggestions with tracking and statistics
2025-11-13 19:08:36 +01:00
copilot-swe-agent[bot]
e9d64e6aac refactor: Apply code review improvements to AI suggestions API
- Remove unused imports (Document from serializers, override_settings from tests)
- Add explanatory comments to all empty except clauses
- Create SUGGESTION_TYPE_CHOICES constant and SuggestionSerializerMixin for DRY
- Implement type-specific validation (value_id for ID types, value_text for text types)
- Remove redundant Document.DoesNotExist handler (already handled by DRF)
- Optimize ai_suggestion_stats query (21 queries → 3 queries using aggregation)
- Update documentation to clarify unsupported custom_field/workflow in apply endpoint

Addresses all 14 code review comments from Copilot PR reviewer.

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-13 08:22:57 +00:00
copilot-swe-agent[bot]
075f58734a security: Fix stack trace exposure vulnerabilities in AI suggestions API
- Remove internal error details from API responses
- Log full error details server-side only
- Provide user-friendly error messages
- Fixed 5 CodeQL security alerts (py/stack-trace-exposure)

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-13 06:12:47 +00:00
copilot-swe-agent[bot]
c7a129cbea docs: Add comprehensive API documentation for AI Suggestions endpoints
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-13 06:09:30 +00:00
copilot-swe-agent[bot]
9542c32fb8 feat(ai): Add AI Suggestions API endpoints with tracking and statistics
- Created serializers for AI suggestions (AISuggestionsSerializer, ApplySuggestionSerializer, etc.)
- Added AISuggestionFeedback model for tracking applied/rejected suggestions
- Implemented 4 new API endpoints:
  - GET /api/documents/{id}/ai-suggestions/ - Get AI suggestions for a document
  - POST /api/documents/{id}/apply-suggestion/ - Apply a suggestion and record feedback
  - POST /api/documents/{id}/reject-suggestion/ - Reject a suggestion and record feedback
  - GET /api/documents/ai-suggestion-stats/ - Get accuracy statistics
- Created database migration for AISuggestionFeedback model
- Added comprehensive test suite (28 test cases) covering all endpoints
- Supports tracking for tags, correspondents, document types, storage paths, titles
- Calculates accuracy rates and confidence scores for AI improvement

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-13 06:07:11 +00:00
dawnsystem
9cb2064460
Merge pull request #52 from dawnsystem/copilot/add-deletion-request-endpoints
Add API endpoints for deletion request approval workflow
2025-11-13 07:01:00 +01:00
copilot-swe-agent[bot]
b8d554631c fix: Remove duplicate deletion approval code and fix formatting
- Removed duplicate DeletionApprovalView from views.py
- Removed duplicate DeletionApprovalSerializer from serialisers.py
- Removed duplicate URL route /api/ai/deletions/approve/
- Fixed missing blank line in serialisers.py before AISuggestionsRequestSerializer
- Kept DeletionRequestViewSet as the correct implementation

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-13 05:59:16 +00:00
dawnsystem
d9a4edf9c5
Merge branch 'dev' into copilot/add-deletion-request-endpoints 2025-11-13 06:51:58 +01:00
copilot-swe-agent[bot]
dd3536d3ff docs: Update BITACORA_MAESTRA with API endpoints implementation details
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-13 05:46:32 +00:00
dawnsystem
86060730e5
Merge pull request #69 from dawnsystem/copilot/optimize-database-indexes
feat(db): Add performance indexes for DeletionRequest queries
2025-11-13 06:40:12 +01:00
dawnsystem
00f8f2c5d8
Merge pull request #66 from dawnsystem/copilot/create-django-migration-deletion-request
Create Django migration for DeletionRequest model
2025-11-13 02:04:07 +01:00
copilot-swe-agent[bot]
3ae23cddfa docs: add migration documentation for DeletionRequest
Created comprehensive documentation for migration 1076 including:
- Overview and migration details
- Complete field descriptions
- How to apply and rollback
- Testing instructions
- Performance and security considerations

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-13 00:52:30 +00:00
copilot-swe-agent[bot]
ad2df8c7ff feat(migrations): add Django migration for DeletionRequest model
Created migration 1076_add_deletion_request.py to add the DeletionRequest model
to the database schema. This model tracks AI-initiated deletion requests that
require explicit user approval before any documents are deleted.

Migration includes:
- All model fields (created_at, updated_at, requested_by_ai, ai_reason, status, etc.)
- ManyToMany relationship with Document model
- Foreign keys to User model (user and reviewed_by)
- Custom indexes for common query patterns (status+user, created_at)
- Proper ordering and metadata

Fixes #10

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-13 00:51:41 +00:00
dawnsystem
487f6ff75d
Merge pull request #41 from dawnsystem/copilot/add-integration-tests-ai-scanner
Add integration tests for AI Scanner consumer pipeline
2025-11-13 01:40:56 +01:00
dawnsystem
9c1cd2638d
Merge pull request #42 from dawnsystem/copilot/add-granular-ai-permissions
Add granular permissions for AI features
2025-11-13 01:38:53 +01:00
copilot-swe-agent[bot]
7658a5571b Address code review feedback: fix imports, docstrings, and add comments
- Move all imports to module level (removed from inside methods)
- Add missing `status` import from rest_framework
- Fix docstring formatting to comply with PEP 257
- Add explanatory comments to empty except clauses
- Improve error message for document not found
- Add warning comment about thread-safety in config update
- Add TODO comments for storing approval/rejection reasons
- Remove unused StoragePath import from tests
- Remove duplicate permission imports at end of file

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 17:24:43 +00:00
copilot-swe-agent[bot]
7bb516a6a5 refactor: Address code review feedback for AI Scanner integration tests
- Add PAPERLESS_ENABLE_AI_SCANNER check in consumer._run_ai_scanner() method
- Move AIScanResult import to top of test file to follow DRY principle
- Create helper methods: create_empty_scan_result_mock() and get_test_file_with_name()
- Reduce code duplication by using helper methods across all tests
- Improve performance test threshold from 10s to 1s for mocked operations
- Add TODO comment for proper performance testing with real ML models
- Remove duplicate 'from documents.ai_scanner import AIScanResult' imports from 7 test methods

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 16:23:11 +00:00
copilot-swe-agent[bot]
cd5c7afdcd Changes before error encountered
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 15:42:23 +00:00
copilot-swe-agent[bot]
b16394fdbb feat(db): Add performance indexes for DeletionRequest model
- Add composite index (user, status, created_at) for listing queries
- Add index for reviewed_at field for filtering reviewed requests
- Add index for completed_at field for filtering completed requests
- Create migration 1076_add_deletionrequest_performance_indexes
- Expected improvement: list queries <100ms, filter queries <50ms

Addresses: [AI Scanner] Índices de Performance para DeletionRequest
Epic: Migraciones de Base de Datos

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 15:42:00 +00:00
copilot-swe-agent[bot]
d159277707 Changes before error encountered
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 15:38:37 +00:00
copilot-swe-agent[bot]
f67dd152e6 Changes before error encountered
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 15:36:48 +00:00
dawnsystem
79d9338ab6
Merge branch 'dev' into copilot/add-integration-tests-ai-scanner 2025-11-12 16:19:57 +01:00
dawnsystem
114c4c89ed
Merge pull request #40 from dawnsystem/copilot/fix-linting-and-formatting
fix(ai-scanner): Correct linting violations and standardize code quality
2025-11-12 16:16:24 +01:00
copilot-swe-agent[bot]
fe679ed6d4 Initial plan 2025-11-12 15:14:43 +00:00
copilot-swe-agent[bot]
f8f2a72c6d Initial plan 2025-11-12 15:14:41 +00:00
copilot-swe-agent[bot]
780decf543 Initial plan 2025-11-12 15:13:08 +00:00
copilot-swe-agent[bot]
275ff4d1d4 Initial plan 2025-11-12 15:13:05 +00:00
dawnsystem
c7a338ef0c
Merge pull request #39 from dawnsystem/copilot/add-tests-for-ai-deletion-manager
Add unit tests for AI Deletion Manager and DeletionRequest model
2025-11-12 16:06:10 +01:00
copilot-swe-agent[bot]
b86f541a0d Remove unused mock import from test_ai_deletion_manager.py
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 14:03:08 +00:00
copilot-swe-agent[bot]
426e7b8e16 Add comprehensive integration tests for AI API endpoints
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 14:01:55 +00:00
copilot-swe-agent[bot]
0eb883287c Add AI API endpoints with permission-protected views
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 14:00:25 +00:00
copilot-swe-agent[bot]
476b08a23b Add AI permission classes and comprehensive tests
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 13:56:47 +00:00
copilot-swe-agent[bot]
dcd9d6cff3 Initial plan 2025-11-12 13:49:28 +00:00
copilot-swe-agent[bot]
b397ebfb21 docs: Actualizar BITACORA_MAESTRA.md con sesión de linting
- Añadir TSK-AI-SCANNER-LINTING al historial completado
- Documentar sesión detallada con timestamps y acciones
- Registrar decisiones de diseño sobre correcciones aplicadas
- Actualizar WIP a estado de espera

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 13:19:18 +00:00
copilot-swe-agent[bot]
8a5ece9e2b fix: Corregir todos los warnings de linting (ruff, black)
- Corregir import ordering (TC002) moviendo User a TYPE_CHECKING
- Corregir type hints implícitos (RUF013) usando | None
- Corregir boolean traps (FBT001/FBT002) usando keyword-only args
- Corregir logging warnings (G201) usando logger.exception()
- Formatear código con ruff format y black

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 13:17:08 +00:00
copilot-swe-agent[bot]
c691de5e87 feat(tests): Add comprehensive AI Scanner integration tests for consumer pipeline
- Add TestConsumerAIScannerIntegration test class with 10 integration tests
- Test 1: End-to-end integration (upload → consumption → AI scan → metadata)
- Test 2: ML components disabled (graceful degradation)
- Test 3: AI scanner failures (error handling)
- Test 4a-4c: Different document types (PDF, image, text)
- Test 5: Performance verification (<2s additional time)
- Test 6: Transactions and rollbacks
- Test 7: Multiple concurrent documents
- Test 8: AI scanner disabled via settings
- All tests use mocks to verify integration without requiring actual ML models
- Tests follow existing patterns in test_consumer.py
- Update BITACORA_MAESTRA.md with session details per agents.md requirements

Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 13:14:28 +00:00
copilot-swe-agent[bot]
d31bdaaab8 Add comprehensive unit tests for AI deletion manager and DeletionRequest model
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 13:10:14 +00:00
copilot-swe-agent[bot]
2d7345f0bc Initial exploration: AI Scanner linting and pre-commit hooks
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 13:09:03 +00:00
copilot-swe-agent[bot]
2933cb27dc Initial plan 2025-11-12 13:05:47 +00:00
copilot-swe-agent[bot]
496a9e7b7b Initial plan 2025-11-12 13:05:45 +00:00
copilot-swe-agent[bot]
a31305b330 Initial plan 2025-11-12 13:01:31 +00:00
dawnsystem
911f16fa41
Merge pull request #38 from dawnsystem/copilot/create-unit-tests-ai-scanner
Tests Unitarios para AI Scanner - 91 comprehensive tests with >90% coverage
2025-11-12 12:41:43 +01:00
copilot-swe-agent[bot]
2f36ac7a7b Fix code review issues: remove unused imports and variables
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 11:23:54 +00:00
copilot-swe-agent[bot]
f63c7faa92 Add comprehensive unit and integration tests for AI Scanner
Co-authored-by: dawnsystem <42047891+dawnsystem@users.noreply.github.com>
2025-11-12 10:05:58 +00:00