88 Commits

Author SHA1 Message Date
thR CIrcU5
c746afcf76 Bulk create HTML snapshots (#1132)
* Add option to create HTML snapshot for bulk edit

* Add the prerequisite for displaying the "Create HTML Snapshot" bulk action option

* Add test case

This test case covers the scenario where the bulk actions panel displays the corresponding options when the HTML snapshot feature is enabled.

* Use the existing `tasks.create_html_snapshots()` instead of the for loop

* Fix the exposure of `settings.LD_ENABLE_SNAPSHOTS` within `BookmarkListContext`

* add service tests

* cleanup context

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2025-08-12 23:06:23 +02:00
Sascha Ißbrücker
1e56b0e6f3 Ignore tags that exceed length limit during import (#1153) 2025-08-10 15:05:10 +02:00
Sascha Ißbrücker
a8623d11ef Update order when deleting bundle (#1114) 2025-07-01 07:09:02 +02:00
hkclark
7a4236d179 Automatically compress uploads with gzip (#1087)
* Gzip .html upload, tests for .html & .gz uploads

* Gzip all file types that aren't already gzips

* Show filename of what user uploaded before compression

* Remove line I thought we need but we don't

* cleanup and fix tests

---------

Co-authored-by: kclark <kclark@autoverify.net>
Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2025-06-20 06:15:25 +02:00
Sascha Ißbrücker
bb796c9bdb Add date filters for REST API (#1080)
* Add modified_since query parameter

* Add added_since parameter

* update date_modified when assets change
2025-05-30 10:24:19 +02:00
Sascha Ißbrücker
1b0684bd6c Allow auto tagging rules to match URL fragments (#1045) 2025-04-13 09:43:11 +02:00
Sascha Ißbrücker
6bba4f35c8 Prefer local snapshot over web archive link in bookmark list links (#1021)
* Prefer local snapshot over web archive link

* Update latest snapshot when it is deleted

* fix filter in migration

* improve migration performance
2025-03-22 19:07:05 +01:00
Josh S
a23c357f2f Add bulk and single bookmark metadata refresh (#999)
* Add url create/edit query paramter to clear cache

* Add refresh bookmark metadata button in create/edit bookmark page

* Fix refresh bookmark metadata when editing existing bookmark

* Add bulk refresh metadata functionality

* Fix test cases for bulk view dropdown selection list

* Allow bulk metadata refresh when background tasks are disabled

* Move load preview image call on refresh metadata

* Update bookmark modified time on metadata refresh

* Rename function to align with convention

* Add tests for refresh task

* Add tests for bookmarks service refresh metadata

* Add tests for bookmarks api disable cache on check

* Remove bulk refresh metadata when background tasks disabled

* Refactor refresh metadata task

* Remove unnecessary call

* Fix testing mock name

* Abstract clearing metadata cache

* Add test to check if load page is called twice when cache disabled

* Remove refresh button for new bookmarks

* Remove strict disable cache is true check

* Refactor refresh metadata form logic into its own function

* move button and highlight changes

* polish and update tests

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2025-03-22 11:34:10 +01:00
Sascha Ißbrücker
1a1092d03a Fix some type hints 2025-03-09 11:30:13 +01:00
Sascha Ißbrücker
2d3bd13a12 Merge siteroot application 2025-03-09 05:50:05 +01:00
Sascha Ißbrücker
2d81ea6f6e Add REST endpoint for uploading snapshots from the Singlefile extension (#996)
* Extract asset logic

* Allow disabling HTML snapshot when creating bookmark

* Add endpoint for uploading singlefile snapshots

* Add URL parameter to disable HTML snapshots

* Allow using asset list in base Docker image

* Expose app version through profile
2025-02-23 22:58:14 +01:00
ixzhao
1f2cf21585 Add LAST_MODIFIED attribute when exporting (#860)
* add LAST_MODIFIED attribute when exporting

* complement test_exporter for LAST_MODIFIED attribute

* parse LAST_MODIFIED attribute when importing

* use bookmark date_added when no modified date is parsed, otherwise use parsed datetime.

* complement test_parser and test_importer for LAST_MODIFIED attribute

* cleanup tests a bit

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-10-02 22:21:08 +02:00
Sascha Ißbrücker
52400feacf Improve error handling for auto tagging (#855) 2024-09-24 17:26:01 +02:00
Sascha Ißbrücker
7b405c054d Do not clear fields in POST requests (API behavior change) (#852) 2024-09-24 11:37:50 +02:00
Sascha Ißbrücker
fe7ddbe645 Allow bookmarks to have empty title and description (#843)
* add migration for merging fields

* remove usage of website title and description

* keep empty website title and description in API for compatibility

* restore scraping in API and add option for disabling it

* document API scraping behavior

* remove deprecated fields from API docs

* improve form layout

* cleanup migration

* cleanup website loader

* update tests
2024-09-22 07:52:00 +02:00
Leonhard Markert
76c65566cf Rename "SingeFileError" to "SingleFileError" (#823) 2024-09-14 11:37:03 +02:00
Sascha Ißbrücker
7572aa5bc9 Fix auto-tagging when URL includes port (#820) 2024-09-10 21:19:20 +02:00
Sascha Ißbrücker
749bc1ef63 Generate fallback URLs for web archive links (#804)
* generate fallback web archive URL if none exists

* remove fallback web archive snapshot creation

* fix test
2024-08-29 22:45:10 +02:00
Viacheslav Slinko
fa5f78cf71 Automatically add tags to bookmarks based on URL pattern (#736)
* [WIP] DSL

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* dsl2

* full feature

* upd

* upd

* upd

* upd

* rename to auto_tagging_rules

* update migration after rebase

* add REST API tests

* improve settings view

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-05-17 09:39:46 +02:00
Sascha Ißbrücker
0f9ba57fef Load missing thumbnails after enabling the feature (#725) 2024-05-10 09:50:19 +02:00
Viacheslav Slinko
b4376a9ff1 Load bookmark thumbnails after import (#724)
* Update thumbnails after import

* Safer way to download thumbnails

* small test improvements

* add missing tests

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-05-10 09:19:00 +02:00
Viacheslav Slinko
87cd4061cb Add support for bookmark thumbnails (#721)
* Preview Image

* fix tests

* add test

* download preview image

* relative path

* gst

* details view

* fix tests

* Improve preview image styles

* Remove preview image URL from model

* Revert form changes

* update tests

* make it work in uwsgi

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-05-07 18:58:52 +02:00
Sascha Ißbrücker
5d2acca122 Allow uploading custom files for bookmarks (#713) 2024-04-20 12:14:11 +02:00
Sascha Ißbrücker
df9f0095cc Add button for creating missing HTML snapshots (#696)
* add button for creating missing HTML snapshots

* refactor messages in settings view

* show alternative text when there are no missing snapshots
2024-04-14 13:21:15 +02:00
Sascha Ißbrücker
25470edb2c Remove ads and cookie banners from HTML snapshots (#695)
* integrate ublock with single-file

* reuse chromium profile
2024-04-14 13:09:46 +02:00
pettijohn
2b342c0d56 Add option for passing arguments to single-file command (#691)
* Promoting singlefile timeout to env variable

* Promoting singlefile timeout to env variable

* add tests

* Add LD_SINGLEFILE_OPTIONS support

* add tests

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-04-09 20:22:14 +02:00
pettijohn
2d22d6871e Add option for customizing single-file timeout (#688)
* Promoting singlefile timeout to env variable

* Promoting singlefile timeout to env variable

* add tests

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-04-07 20:21:59 +02:00
Sascha Ißbrücker
5e8f5b2c58 Truncate snapshot filename for long URLs (#687) 2024-04-07 18:13:28 +02:00
Sascha Ißbrücker
a6f35119cd Replace django-background-tasks with huey (#657)
* Replace django-background-tasks with huey

* Add command for migrating tasks

* Add custom admin view

* fix dockerfile

* fix tests

* fix tests in CI

* fix task tests

* implement retries

* improve config

* workaround to avoid running singlefile tasks in parallel

* properly kill single-file sub-processes

* use period task for HTML snapshots

* clear task lock when starting task consumer

* remove obsolete cleanup task command
2024-04-07 11:11:14 +02:00
Sascha Ißbrücker
4280ab40c6 Archive snapshots of websites locally (#672)
* Add basic HTML snapshots

* Implement asset list

* Add snapshot creation tests

* Add deletion tests

* Show file size

* Remove snapshots

* Create new snapshots

* Switch to single-file

* CSS tweak

* Remove auto refresh

* Show delete link when there is no file yet

* Add current date to display name

* Add flag for snapshot support

* Add option for disabling automatic snapshots

* Make snapshots sharable

* Document image variants

* Update README.md

* Add migrations

* Fix tests
2024-04-01 15:19:38 +02:00
Sascha Ißbrücker
98b9a9c1a0 Add black code formatter 2024-01-27 11:29:16 +01:00
Jonathan Sundqvist
150dfecc6f Support Open Graph description (#602)
* Support pytest for running tests

* Support extracting description from meta og:description property

* Revert changes to TOC

* Add test

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-01-27 10:28:46 +01:00
Sascha Ißbrücker
935189ecc2 Improve bulk tag performance (#612) 2024-01-27 09:13:21 +01:00
Sascha Ißbrücker
a9512b2333 Include archived bookmarks in export (#579) 2023-11-24 09:21:23 +01:00
Sascha Ißbrücker
28acf3299c Add support for exporting/importing bookmark notes (#532) 2023-09-10 23:37:37 +02:00
Sascha Ißbrücker
e2e5930985 Allow bulk editing unread and shared state of bookmarks (#517)
* Move bulk actions into select

* Update tests

* Implement bulk read / unread actions

* Implement bulk share/unshare actions

* Show correct archiving actions

* Allow selecting bookmarks across pages

* Dynamically update select across checkbox

* Filter available bulk actions

* Refactor tag autocomplete toggling
2023-08-25 13:54:23 +02:00
Sascha Ißbrücker
8206705876 Add support for PRIVATE flag in import and export (#505)
* Add support for PRIVATE attribute in import

* Add support for PRIVATE attribute in export

* Update import sync tests
2023-08-20 11:44:53 +02:00
Sascha Ißbrücker
5d9e487ec1 Various improvements to favicons (#504)
* Update default favicon provider

* Add domain placeholder for favicon providers

* Fix favicon loader to handle streaming response

* Handle different mime types for favicons

* Use 32px size by default

* Update documentation

* Skip mime-type test for now

* Manually configure image/x-icon mime type
2023-08-15 16:49:58 +02:00
Sascha Ißbrücker
4220ea0b4c Fix website loader content encoding detection (#482) 2023-05-30 22:04:54 +02:00
Sascha Ißbrücker
43115fd8f2 Add notes to bookmarks (#472)
* Add basic bookmark notes

* Add bookmark list JS to shared bookmarks page

* Allow testing through ngrok

* Improve CSS

* Set notes through API

* Improve notes editing

* Improve notes icon

* Remove transitions for now

* Update keyboard shortcut

* Add bookmark list tests

* Add setting for showing notes permanently

* Add test for toggling notes

* Update API docs

* Allow searching for notes content

* Skip test
2023-05-20 11:54:26 +02:00
Sascha Ißbrücker
74134d3896 Escape texts in exported HTML (#429) 2023-02-18 18:25:54 +01:00
Sascha Ißbrücker
6b4664117b Fix favicon being cleared by web archive snapshot task (#405) 2023-01-22 14:07:06 +01:00
Sascha Ißbrücker
814401be2e Add option for showing bookmark favicons (#390)
* Implement favicon loader

* Implement load favicon task

* Show favicons in bookmark list

* Add missing migration

* Load missing favicons on import

* Automatically refresh favicons

* Add enable favicon setting

* Update uwsgi config to host favicons

* Improve settings wording

* Fix favicon loader test setup

* Document LD_FAVICON_PROVIDER setting

* Add refresh favicons button
2023-01-21 16:36:10 +01:00
Sascha Ißbrücker
30da1880a5 Cache website metadata to avoid duplicate scraping (#401)
* Cache website metadata to avoid duplicate scraping

* fix test setup
2023-01-20 22:28:44 +01:00
Sascha Ißbrücker
021d1cd673 Fix bookmark website metadata not being updated when URL changes (#400) 2023-01-20 20:59:09 +01:00
Sascha Ißbrücker
43d52642a6 Fix website loader test 2023-01-14 12:26:04 +01:00
Sascha Ißbrücker
4f9170c48d Improve website loader logging 2023-01-14 11:24:09 +01:00
Luca
c2d8cde86b Trim website metadata title and description (#383)
* feat: trim fetched metadata placeholders

* feat: implement trimming serverside

* Add website loader tests

* Address review comments

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2023-01-12 21:06:36 +01:00
Sascha Ißbrücker
2fd7704816 Limit document size for website scraper (#354)
Limits the size of scraped HTML documents to prevent out of memory errors. The scraper will stop reading from the response when it encounters the closing head tag, or if the read content's size exceeds a max limit.

Fixes #345
2022-10-07 21:18:18 +02:00
Sascha Ißbrücker
b94eaee833 Setup logging for background tasks 2022-09-11 07:50:08 +02:00