Compare commits

..

114 Commits

Author SHA1 Message Date
Sascha Ißbrücker
695b0dc300 Bump version 2024-06-16 11:45:06 +02:00
Sascha Ißbrücker
fe40139838 Make backup include preview images 2024-06-16 10:37:02 +02:00
Sascha Ißbrücker
44b49a4cfe Preview auto tags in bookmark form (#737) 2024-06-16 10:04:38 +02:00
dependabot[bot]
469883a674 --- (#740)
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-16 10:01:36 +02:00
Viacheslav Slinko
fa5f78cf71 Automatically add tags to bookmarks based on URL pattern (#736)
* [WIP] DSL

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* upd

* dsl2

* full feature

* upd

* upd

* upd

* upd

* rename to auto_tagging_rules

* update migration after rebase

* add REST API tests

* improve settings view

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-05-17 09:39:46 +02:00
Viacheslav Slinko
e03f536925 Add option for disabling tag grouping (#735)
* Configurable tag grouping

* update tag group name

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-05-17 08:38:08 +02:00
Viacheslav Slinko
a92a35cfb8 Thumbnails lazy loading (#734) 2024-05-16 09:44:38 +02:00
Viacheslav Slinko
ff334e0888 Hide tooltip on mobile (#733) 2024-05-15 09:06:30 +02:00
Sascha Ißbrücker
0f9ba57fef Load missing thumbnails after enabling the feature (#725) 2024-05-10 09:50:19 +02:00
Viacheslav Slinko
b4376a9ff1 Load bookmark thumbnails after import (#724)
* Update thumbnails after import

* Safer way to download thumbnails

* small test improvements

* add missing tests

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-05-10 09:19:00 +02:00
Viacheslav Slinko
87cd4061cb Add support for bookmark thumbnails (#721)
* Preview Image

* fix tests

* add test

* download preview image

* relative path

* gst

* details view

* fix tests

* Improve preview image styles

* Remove preview image URL from model

* Revert form changes

* update tests

* make it work in uwsgi

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-05-07 18:58:52 +02:00
Sascha Ißbrücker
e2415f652b Remove leading/trailing whitespace in description 2024-04-21 18:56:01 +02:00
Sascha Ißbrücker
9cf5eb5ec0 Use temp dir for favicon loader tests 2024-04-20 19:45:57 +02:00
Sascha Ißbrücker
023a213ba6 Update CHANGELOG.md 2024-04-20 19:23:21 +02:00
Sascha Ißbrücker
23d97db016 Bump version 2024-04-20 14:11:14 +02:00
dependabot[bot]
0fb1bbd0e2 Bump sqlparse from 0.4.4 to 0.5.0 (#704)
Bumps [sqlparse](https://github.com/andialbrecht/sqlparse) from 0.4.4 to 0.5.0.
- [Changelog](https://github.com/andialbrecht/sqlparse/blob/master/CHANGELOG)
- [Commits](https://github.com/andialbrecht/sqlparse/compare/0.4.4...0.5.0)

---
updated-dependencies:
- dependency-name: sqlparse
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-20 12:17:01 +02:00
Sascha Ißbrücker
5d2acca122 Allow uploading custom files for bookmarks (#713) 2024-04-20 12:14:11 +02:00
Sascha Ißbrücker
0cbaf927e4 Add reader mode (#703)
* Add reader mode view

* Show link for latest snapshot instead
2024-04-20 09:18:57 +02:00
ab623
0586983602 Show proper name for bookmark assets in admin (#708) 2024-04-17 23:18:23 +02:00
ab623
9dc3521d5e Add option for marking bookmarks as unread by default (#706)
* Added new option to set Mark as unread with a default

* Added additional test

* tweak test a bit

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-04-17 23:08:18 +02:00
Sascha Ißbrücker
a1822e2091 Close bookmark details with escape (#702) 2024-04-15 19:41:18 +02:00
Sascha Ißbrücker
22ffecbb9d Make blocking cookie banners more reliable (#699) 2024-04-15 19:33:25 +02:00
Sascha Ißbrücker
d9096eacd6 Update CHANGELOG.md 2024-04-14 21:10:27 +02:00
Sascha Ißbrücker
e50912df12 Bump version 2024-04-14 20:48:30 +02:00
Sascha Ißbrücker
393d688247 Fix directory name 2024-04-14 20:31:53 +02:00
Sascha Ißbrücker
6e38587174 Fix missing home directory in background tasks 2024-04-14 20:28:39 +02:00
dependabot[bot]
123c6fe02a Bump idna from 3.6 to 3.7 (#694)
Bumps [idna](https://github.com/kjd/idna) from 3.6 to 3.7.
- [Release notes](https://github.com/kjd/idna/releases)
- [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst)
- [Commits](https://github.com/kjd/idna/compare/v3.6...v3.7)

---
updated-dependencies:
- dependency-name: idna
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-14 14:41:53 +02:00
Sascha Ißbrücker
1b7731e506 Refresh file list when there are queued snapshots (#697)
* add destroy hook

* refresh details modal in interval

* refactor to refresh assets list

* disable create snapshot button when there is a pending snapshot
2024-04-14 14:41:22 +02:00
Sascha Ißbrücker
df9f0095cc Add button for creating missing HTML snapshots (#696)
* add button for creating missing HTML snapshots

* refactor messages in settings view

* show alternative text when there are no missing snapshots
2024-04-14 13:21:15 +02:00
Sascha Ißbrücker
25470edb2c Remove ads and cookie banners from HTML snapshots (#695)
* integrate ublock with single-file

* reuse chromium profile
2024-04-14 13:09:46 +02:00
Sascha Ißbrücker
22a1fc80ad Update README.md 2024-04-14 06:44:08 +02:00
Sascha Ißbrücker
65f0eb2a04 Refactor client-side fetch logic (#693)
* extract generic behaviors

* preserve query string when refreshing content

* refactor details modal refresh

* refactor bulk edit

* update tests

* restore tag modal

* Make IntelliJ aware of custom attributes

* improve e2e test coverage
2024-04-11 19:07:20 +02:00
Sascha Ißbrücker
82f86bf537 Update CHANGELOG.md 2024-04-09 20:46:59 +02:00
Sascha Ißbrücker
639629ddfe Bump version 2024-04-09 20:28:35 +02:00
pettijohn
2b342c0d56 Add option for passing arguments to single-file command (#691)
* Promoting singlefile timeout to env variable

* Promoting singlefile timeout to env variable

* add tests

* Add LD_SINGLEFILE_OPTIONS support

* add tests

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-04-09 20:22:14 +02:00
Sascha Ißbrücker
3ffec72d3e Fix jumping tag auto complete 2024-04-09 19:41:14 +02:00
tianheg
edd958fff6 Update backup.md (#689) 2024-04-08 08:11:48 +02:00
pettijohn
2d22d6871e Add option for customizing single-file timeout (#688)
* Promoting singlefile timeout to env variable

* Promoting singlefile timeout to env variable

* add tests

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-04-07 20:21:59 +02:00
Sascha Ißbrücker
5e8f5b2c58 Truncate snapshot filename for long URLs (#687) 2024-04-07 18:13:28 +02:00
Sascha Ißbrücker
d5a83722de Add full backup method (#686) 2024-04-07 17:49:30 +02:00
Jan Hendrik Lübke
5d8fdebb7c Add option to disable SSL verification for OIDC (#684)
* Add setting OIDC_VERIFY_SSL

Passtrough the setting OIDC_VERIFY_SSL in order to allow self-signed certificates/custom certificate authority for the OIDC provider

* Update Options.md to include the new setting OIDC_VERIFY_SSL

* add default setting test

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-04-07 16:33:29 +02:00
Sascha Ißbrücker
f7bd6ccb31 Update CHANGELOG.md 2024-04-07 13:09:37 +02:00
Sascha Ißbrücker
e4ee0171be Bump version 2024-04-07 13:06:51 +02:00
Hugo van Rijswijk
53d1f0c91b Add Authelia OIDC example to docs (#675) 2024-04-07 11:12:12 +02:00
Sascha Ißbrücker
a6f35119cd Replace django-background-tasks with huey (#657)
* Replace django-background-tasks with huey

* Add command for migrating tasks

* Add custom admin view

* fix dockerfile

* fix tests

* fix tests in CI

* fix task tests

* implement retries

* improve config

* workaround to avoid running singlefile tasks in parallel

* properly kill single-file sub-processes

* use period task for HTML snapshots

* clear task lock when starting task consumer

* remove obsolete cleanup task command
2024-04-07 11:11:14 +02:00
Sascha Ißbrücker
68c163d943 Fix HTML snapshot errors related to single-file-cli (#683)
* Install node 20 on debian image

* use singlefile fork
2024-04-07 11:05:48 +02:00
Sascha Ißbrücker
bb6c5ca29e Update CHANGELOG.md 2024-04-01 16:04:40 +02:00
Sascha Ißbrücker
c919e79759 Bump version 2024-04-01 15:47:03 +02:00
Sascha Ißbrücker
8ff9b42a79 Update README.md 2024-04-01 15:27:58 +02:00
Sascha Ißbrücker
4280ab40c6 Archive snapshots of websites locally (#672)
* Add basic HTML snapshots

* Implement asset list

* Add snapshot creation tests

* Add deletion tests

* Show file size

* Remove snapshots

* Create new snapshots

* Switch to single-file

* CSS tweak

* Remove auto refresh

* Show delete link when there is no file yet

* Add current date to display name

* Add flag for snapshot support

* Add option for disabling automatic snapshots

* Make snapshots sharable

* Document image variants

* Update README.md

* Add migrations

* Fix tests
2024-04-01 15:19:38 +02:00
tianheg
db1906942a Update Railway hosting option (#670) 2024-03-31 16:25:24 +02:00
Sascha Ißbrücker
69877a32e5 Add how to for increasing the font size (#667) 2024-03-30 11:43:15 +01:00
Sascha Ißbrücker
e5a9a772f0 Update CHANGELOG.md 2024-03-30 11:07:40 +01:00
tianheg
2f56d418cf Add new hosting option (#661)
* Add new hosting option

* Update Railway template url
2024-03-30 10:37:17 +01:00
Sascha Ißbrücker
a4df586a8a Bump version 2024-03-30 10:27:28 +01:00
Sascha Ißbrücker
d9b7996e06 Make bookmark list actions configurable (#666)
* Make bookmark list actions configurable

* Add upgrade notice
2024-03-29 23:07:11 +01:00
Sascha Ißbrücker
92f62d3ded Fix CSS sub-pixel issues 2024-03-29 20:49:07 +01:00
Sascha Ißbrücker
9c48085829 Add bookmark details view (#665)
* Experiment with bookmark details

* Add basic tests

* Refactor details into modal

* Implement edit and delete button

* Remove slide down animation

* Add fallback details view

* Add status actions

* Improve dark theme

* Improve return URLs

* Make bookmark details sharable

* Fix E2E tests
2024-03-29 12:37:20 +01:00
Sascha Ißbrücker
77e1525402 Fix flaky E2E tests 2024-03-24 22:16:09 +01:00
Sascha Ißbrücker
9df80e01de Add option for showing bookmark description as separate block (#663)
* Add option for showing bookmark description as separate block

* Use context
2024-03-24 21:31:15 +01:00
Sascha Ißbrücker
ec34cc523f Run formatter 2024-03-24 11:50:02 +01:00
Sascha Ißbrücker
eb0b092d17 Disable pointer-events on bookmark tooltip 2024-03-22 23:55:46 +01:00
dependabot[bot]
39e8f03345 Bump black from 24.1.1 to 24.3.0 (#662)
Bumps [black](https://github.com/psf/black) from 24.1.1 to 24.3.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/24.1.1...24.3.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-21 23:12:58 +01:00
Sascha Ißbrücker
d43b97e0c0 Update CHANGELOG.md 2024-03-19 10:02:44 +01:00
Sascha Ißbrücker
d6484ba8e9 Add release script 2024-03-18 22:54:22 +01:00
dependabot[bot]
4c26d66177 Bump django from 5.0.2 to 5.0.3 (#658)
Bumps [django](https://github.com/django/django) from 5.0.2 to 5.0.3.
- [Commits](https://github.com/django/django/compare/5.0.2...5.0.3)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-18 22:51:29 +01:00
Sascha Ißbrücker
c51dcafa40 Fix docker build 2024-03-18 22:50:07 +01:00
Sascha Ißbrücker
262dd2b28f Update OIDC configuration defaults 2024-03-18 22:41:25 +01:00
Sascha Ißbrücker
01ad7f4d9e Bump version 2024-03-17 12:00:30 +01:00
Sascha Ißbrücker
d0d5c15345 Add RSS feeds for shared bookmarks (#656)
* Add shared bookmarks feed

* Add public shared bookmarks feed
2024-03-17 11:55:34 +01:00
Sascha Ißbrücker
afb752765d Include web archive link in /api/bookmarks/ (#655) 2024-03-17 10:04:05 +01:00
Sascha Ißbrücker
ce213775b6 Try fix workflow config 2024-03-17 09:59:04 +01:00
Bruno Henriques
fd1bbadcf3 Update backup location to safe directory (#653)
The previous directory may not share the same directory as the user that runs the container. `/etc/linkding/data` is a safe directory.

Fixes #626
2024-03-17 09:06:07 +01:00
Sascha Ißbrücker
83c2530df4 Add option for custom CSS (#652)
* Add option for adding custom CSS

* add missing migration
2024-03-17 01:11:59 +01:00
ηg
39782e75e7 Add support for OIDC (#389)
* added support for oidc auth

* fixed oidc usernames

* hiding password for users that aren't logged in via local auth

* add dependency, update settings

* keep change password link

* add tests

* add docs

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-03-16 23:42:46 +01:00
Hugo van Rijswijk
4bee104b62 Build improvements (#649)
* Improve PWA capabilities

* Invert background_color theme logic

* Revert build changes

* Revert "Revert build changes"

This reverts commit 1ab640fda1.

* update

* revert svelte component changes
2024-03-16 15:57:23 +01:00
Sascha Ißbrücker
f4ecffbb7f Fix flaky bulk edit E2E test (#650) 2024-03-16 15:35:22 +01:00
Hugo van Rijswijk
6f52bafda8 Improve PWA capabilities (#630)
* Improve PWA capabilities

* Invert background_color theme logic

* Revert build changes
2024-03-16 15:20:22 +01:00
Sascha Ißbrücker
2deecc5c91 Bump version 2024-03-16 11:27:17 +01:00
Sascha Ißbrücker
54cfa13861 Fix logout button (#648) 2024-03-16 11:24:17 +01:00
Sascha Ißbrücker
ee4f99261f Update CHANGELOG.md 2024-03-16 10:48:25 +01:00
Sascha Ißbrücker
d2fa0a8f5a Bump version 2024-03-16 09:28:57 +01:00
Juan Gilsanz Polo
02a15c9460 Added a new Linkding client to community section (#638)
* Added linkdy entry to readme

* Fix alphabetic order

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@googlemail.com>
2024-03-16 08:26:21 +01:00
Jack Halford
7a6428c037 Add k8s setup to community section (#633)
* Update README.md

* Move to community section

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@googlemail.com>
2024-03-16 08:24:12 +01:00
dependabot[bot]
c6001aa7b8 Bump django from 5.0.1 to 5.0.2 (#625)
Bumps [django](https://github.com/django/django) from 5.0.1 to 5.0.2.
- [Commits](https://github.com/django/django/compare/5.0.1...5.0.2)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-16 07:21:58 +01:00
Sascha Ißbrücker
eefbefd714 Enable workflow runs from pull requests 2024-03-16 07:19:40 +01:00
Jonathan Sundqvist
683cf529d7 Group ideographic characters in tag cloud (#613)
* Fix #588, Ideographic characters should be grouped together.
Following the suggestion of using regex to find the ideographic
range in this SO answer https://stackoverflow.com/a/2718203/554903

We group the ideographic characters together, while keeping other
chinese, japanese and korean characters apart.

* cleanup

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-03-16 07:09:37 +01:00
Sascha Ißbrücker
38204c87cf Persist secret key in data folder (#620)
* Persist secret key in data folder

* use random secret key by default in prod

* fix e2e test
2024-01-28 23:58:03 +01:00
Sascha Ißbrücker
96ee4746ad Fix JS bundle caching 2024-01-28 23:07:38 +01:00
Sascha Ißbrücker
d7c1afa2a5 Bump dependencies (#618)
* Bump dependencies

* Make it work with Python 3.10

* replace psycopg2-binary with psycopg2 in Docker build
2024-01-28 22:50:51 +01:00
Sascha Ißbrücker
16ed6ef200 Update CHANGELOG.md 2024-01-27 20:01:48 +01:00
Sascha Ißbrücker
98b9a9c1a0 Add black code formatter 2024-01-27 11:29:16 +01:00
Sascha Ißbrücker
6775633be5 Bump version 2024-01-27 10:58:21 +01:00
Jonathan Sundqvist
150dfecc6f Support Open Graph description (#602)
* Support pytest for running tests

* Support extracting description from meta og:description property

* Revert changes to TOC

* Add test

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-01-27 10:28:46 +01:00
Jonathan Sundqvist
81ae55bc1c Add tooltip to truncated bookmark titles (#607)
* Add title to link so you can see the entire title when hover

* Tweak JS, styles

* Fix snapshot tests

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2024-01-27 10:16:44 +01:00
Sascha Ißbrücker
935189ecc2 Improve bulk tag performance (#612) 2024-01-27 09:13:21 +01:00
JnsDornbusch
7997f20d89 Adjust archive.org donation link in general.html (#603)
Adjust archive.org donation link due to broken link.
2024-01-23 22:57:50 +01:00
Jonathan Sundqvist
ae27500cde Bump playwright dependencies (#601) 2024-01-23 22:45:25 +01:00
Adam Shand
71d853999e Add CapRover as managed hosting option (#585)
Add a note that I've created a one-click app for Linkding in CapRover (Scalable PaaS, automated Docker+nginx).
2024-01-23 22:37:51 +01:00
Sebastian Ruml
70288d6865 Increase tag limit in tag autocomplete (#581)
- increas tag limit to 5000

Co-authored-by: Sebastian Ruml <sebastian@sebastianruml.name>
2024-01-23 22:32:16 +01:00
Sascha Ißbrücker
e83d519cab Bump version 2023-12-08 22:01:24 +01:00
Sascha Ißbrücker
6355d8dff1 Properly encode search query param (#587) 2023-12-08 21:53:54 +01:00
Sascha Ißbrücker
227cfdb063 Update CHANGELOG.md 2023-11-24 10:23:44 +01:00
Sascha Ißbrücker
2d4da099c7 Bump version 2023-11-24 09:33:36 +01:00
Sascha Ißbrücker
a9512b2333 Include archived bookmarks in export (#579) 2023-11-24 09:21:23 +01:00
Oleksandr Perepadia
47e944e6c5 Update README.md (#574)
* Update README.md

Correct Firefox addon links to direct to the English language page

* Update firefox addon links to not presume any language
2023-11-14 17:08:17 +01:00
Sascha Ißbrücker
6c7ce91d53 Add backup CLI command (#571) 2023-11-05 19:27:48 +01:00
Sascha Ißbrücker
87020de917 Add Alpine based Docker image (experimental) (#570)
* use alpine as base image

* try fix missing mime types

* Extract separate Dockerfile

* Restore playwright dev dependency

* Add info to README.md
2023-11-05 14:18:26 +01:00
Sascha Ißbrücker
a130daa0f0 Update backup docs 2023-11-05 13:53:08 +01:00
Sascha Ißbrücker
d7c68c2818 Update CHANGELOG.md 2023-11-04 12:26:30 +01:00
Sascha Ißbrücker
1daad2c86c Bump version 2023-11-04 10:17:01 +01:00
dependabot[bot]
251def2583 Bump django from 4.1.10 to 4.1.13 (#567)
Bumps [django](https://github.com/django/django) from 4.1.10 to 4.1.13.
- [Commits](https://github.com/django/django/compare/4.1.10...4.1.13)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-04 09:57:08 +01:00
Vitor Marçal
560769f068 Fix RSS feed not handling None values (#569)
Previously, the 'sanitize' function would throw an error when 'text' was None. This commit fixes the issue by adding a check to handle the case where 'text' is None, returning an empty string instead.

Closes #568
2023-11-04 09:56:06 +01:00
Sascha Ißbrücker
dc9799cc53 Update CHANGELOG.md 2023-10-27 21:18:08 +02:00
259 changed files with 18843 additions and 6254 deletions

View File

@@ -14,7 +14,7 @@
"forwardPorts": [8000],
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "pip3 install --user -r requirements.txt && npm install && mkdir -p data && python3 manage.py migrate",
"postCreateCommand": "pip3 install --user -r requirements.txt -r requirements.dev.txt && npm install && mkdir -p data && python3 manage.py migrate",
// Configure tool-specific properties.
"customizations": {

View File

@@ -1,34 +1,21 @@
# Remove project files, data, tmp files, build files
/.env
/.idea
/data
/node_modules
/tmp
/docs
/static
/scripts
/build
/out
/.git
/.devcontainer
# Ignore everything
*
/.dockerignore
/.gitignore
/.gitattributes
/Dockerfile
/docker-compose.yml
/*.sh
/*.iml
/*.patch
/*.md
/*.js
/*.log
/*.pid
# Include files required for build or at runtime
!/bookmarks
!/siteroot
# Whitelist files needed in build or prod image
!/rollup.config.js
!/bootstrap.sh
!/background-tasks-wrapper.sh
!/LICENSE.txt
!/manage.py
!/package.json
!/package-lock.json
!/requirements.dev.txt
!/requirements.txt
!/rollup.config.mjs
!/supervisord.conf
!/uwsgi.ini
!/version.txt
# Remove development settings
# Remove dev settings
/siteroot/settings/dev.py

View File

@@ -1,46 +1,55 @@
name: linkding CI
on: [push]
on:
pull_request:
push:
branches:
- master
jobs:
unit_tests:
name: Unit Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Set up Node
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: 18
node-version: 20
cache: 'npm'
- name: Install Node dependencies
run: npm install
run: npm ci
- name: Setup Python environment
run: pip install -r requirements.txt
run: |
pip install -r requirements.txt -r requirements.dev.txt
mkdir data
- name: Run tests
run: python manage.py test bookmarks.tests
e2e_tests:
name: E2E Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Set up Node
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: 18
node-version: 20
cache: 'npm'
- name: Install Node dependencies
run: npm install
run: npm ci
- name: Setup Python environment
run: |
pip install -r requirements.txt
pip install -r requirements.txt -r requirements.dev.txt
playwright install chromium
mkdir data
- name: Run build
run: |
npm run build

5
.gitignore vendored
View File

@@ -191,3 +191,8 @@ typings/
/tmp
# Database file
/data
# ublock + chromium
/uBlock0.chromium
/chromium-profile
# direnv
/.direnv

View File

@@ -1,5 +1,232 @@
# Changelog
## v1.30.0 (20/04/2024)
### What's Changed
* Add reader mode by @sissbruecker in https://github.com/sissbruecker/linkding/pull/703
* Allow uploading custom files for bookmarks by @sissbruecker in https://github.com/sissbruecker/linkding/pull/713
* Add option for marking bookmarks as unread by default by @ab623 in https://github.com/sissbruecker/linkding/pull/706
* Make blocking cookie banners more reliable by @sissbruecker in https://github.com/sissbruecker/linkding/pull/699
* Close bookmark details with escape by @sissbruecker in https://github.com/sissbruecker/linkding/pull/702
* Show proper name for bookmark assets in admin by @ab623 in https://github.com/sissbruecker/linkding/pull/708
* Bump sqlparse from 0.4.4 to 0.5.0 by @dependabot in https://github.com/sissbruecker/linkding/pull/704
### New Contributors
* @ab623 made their first contribution in https://github.com/sissbruecker/linkding/pull/706
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.29.0...v1.30.0
---
## v1.29.0 (14/04/2024)
### What's Changed
* Remove ads and cookie banners from HTML snapshots by @sissbruecker in https://github.com/sissbruecker/linkding/pull/695
* Add button for creating missing HTML snapshots by @sissbruecker in https://github.com/sissbruecker/linkding/pull/696
* Refresh file list when there are queued snapshots by @sissbruecker in https://github.com/sissbruecker/linkding/pull/697
* Bump idna from 3.6 to 3.7 by @dependabot in https://github.com/sissbruecker/linkding/pull/694
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.28.0...v1.29.0
---
## v1.28.0 (09/04/2024)
### What's Changed
* Add option to disable SSL verification for OIDC by @akaSyntaax in https://github.com/sissbruecker/linkding/pull/684
* Add full backup method by @sissbruecker in https://github.com/sissbruecker/linkding/pull/686
* Truncate snapshot filename for long URLs by @sissbruecker in https://github.com/sissbruecker/linkding/pull/687
* Add option for customizing single-file timeout by @pettijohn in https://github.com/sissbruecker/linkding/pull/688
* Add option for passing arguments to single-file command by @pettijohn in https://github.com/sissbruecker/linkding/pull/691
* Fix typo by @tianheg in https://github.com/sissbruecker/linkding/pull/689
### New Contributors
* @akaSyntaax made their first contribution in https://github.com/sissbruecker/linkding/pull/684
* @pettijohn made their first contribution in https://github.com/sissbruecker/linkding/pull/688
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.27.1...v1.28.0
---
## v1.27.1 (07/04/2024)
### What's Changed
* Fix HTML snapshot errors related to single-file-cli by @sissbruecker in https://github.com/sissbruecker/linkding/pull/683
* Replace django-background-tasks with huey by @sissbruecker in https://github.com/sissbruecker/linkding/pull/657
* Add Authelia OIDC example to docs by @hugo-vrijswijk in https://github.com/sissbruecker/linkding/pull/675
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.27.0...v1.27.1
---
## v1.27.0 (01/04/2024)
### What's Changed
* Archive snapshots of websites locally by @sissbruecker in https://github.com/sissbruecker/linkding/pull/672
* Add Railway hosting option by @tianheg in https://github.com/sissbruecker/linkding/pull/661
* Add how to for increasing the font size by @sissbruecker in https://github.com/sissbruecker/linkding/pull/667
### New Contributors
* @tianheg made their first contribution in https://github.com/sissbruecker/linkding/pull/661
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.26.0...v1.27.0
---
## v1.26.0 (30/03/2024)
### What's Changed
* Add option for showing bookmark description as separate block by @sissbruecker in https://github.com/sissbruecker/linkding/pull/663
* Add bookmark details view by @sissbruecker in https://github.com/sissbruecker/linkding/pull/665
* Make bookmark list actions configurable by @sissbruecker in https://github.com/sissbruecker/linkding/pull/666
* Bump black from 24.1.1 to 24.3.0 by @dependabot in https://github.com/sissbruecker/linkding/pull/662
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.25.0...v1.26.0
---
## v1.25.0 (18/03/2024)
### What's Changed
* Improve PWA capabilities by @hugo-vrijswijk in https://github.com/sissbruecker/linkding/pull/630
* build improvements by @hugo-vrijswijk in https://github.com/sissbruecker/linkding/pull/649
* Add support for oidc by @Nighmared in https://github.com/sissbruecker/linkding/pull/389
* Add option for custom CSS by @sissbruecker in https://github.com/sissbruecker/linkding/pull/652
* Update backup location to safe directory by @bphenriques in https://github.com/sissbruecker/linkding/pull/653
* Include web archive link in /api/bookmarks/ by @sissbruecker in https://github.com/sissbruecker/linkding/pull/655
* Add RSS feeds for shared bookmarks by @sissbruecker in https://github.com/sissbruecker/linkding/pull/656
* Bump django from 5.0.2 to 5.0.3 by @dependabot in https://github.com/sissbruecker/linkding/pull/658
### New Contributors
* @hugo-vrijswijk made their first contribution in https://github.com/sissbruecker/linkding/pull/630
* @Nighmared made their first contribution in https://github.com/sissbruecker/linkding/pull/389
* @bphenriques made their first contribution in https://github.com/sissbruecker/linkding/pull/653
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.24.2...v1.25.0
---
## v1.24.2 (16/03/2024)
### What's Changed
* Fix logout button by @sissbruecker in https://github.com/sissbruecker/linkding/pull/648
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.24.1...v1.24.2
---
## v1.24.1 (16/03/2024)
### What's Changed
* Bump dependencies by @sissbruecker in https://github.com/sissbruecker/linkding/pull/618
* Persist secret key in data folder by @sissbruecker in https://github.com/sissbruecker/linkding/pull/620
* Group ideographic characters in tag cloud by @jonathan-s in https://github.com/sissbruecker/linkding/pull/613
* Bump django from 5.0.1 to 5.0.2 by @dependabot in https://github.com/sissbruecker/linkding/pull/625
* Add k8s setup to community section by @jzck in https://github.com/sissbruecker/linkding/pull/633
* Added a new Linkding client to community section by @JGeek00 in https://github.com/sissbruecker/linkding/pull/638
### New Contributors
* @jzck made their first contribution in https://github.com/sissbruecker/linkding/pull/633
* @JGeek00 made their first contribution in https://github.com/sissbruecker/linkding/pull/638
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.24.0...v1.24.1
---
## v1.24.0 (27/01/2024)
### What's Changed
* Support Open Graph description by @jonathan-s in https://github.com/sissbruecker/linkding/pull/602
* Add tooltip to truncated bookmark titles by @jonathan-s in https://github.com/sissbruecker/linkding/pull/607
* Improve bulk tag performance by @sissbruecker in https://github.com/sissbruecker/linkding/pull/612
* Increase tag limit in tag autocomplete by @hypebeast in https://github.com/sissbruecker/linkding/pull/581
* Add CapRover as managed hosting option by @adamshand in https://github.com/sissbruecker/linkding/pull/585
* Bump playwright dependencies by @jonathan-s in https://github.com/sissbruecker/linkding/pull/601
* Adjust archive.org donation link in general.html by @JnsDornbusch in https://github.com/sissbruecker/linkding/pull/603
### New Contributors
* @hypebeast made their first contribution in https://github.com/sissbruecker/linkding/pull/581
* @adamshand made their first contribution in https://github.com/sissbruecker/linkding/pull/585
* @jonathan-s made their first contribution in https://github.com/sissbruecker/linkding/pull/601
* @JnsDornbusch made their first contribution in https://github.com/sissbruecker/linkding/pull/603
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.23.1...v1.24.0
---
## v1.23.1 (08/12/2023)
### What's Changed
* Properly encode search query param by @sissbruecker in https://github.com/sissbruecker/linkding/pull/587
> [!WARNING]
> *This resolves a security vulnerability in linkding. Everyone is encouraged to upgrade to the latest version as soon as possible.*
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.23.0...v1.23.1
---
## v1.23.0 (24/11/2023)
### What's Changed
* Add Alpine based Docker image (experimental) by @sissbruecker in https://github.com/sissbruecker/linkding/pull/570
* Add backup CLI command by @sissbruecker in https://github.com/sissbruecker/linkding/pull/571
* Update browser extension links by @OPerepadia in https://github.com/sissbruecker/linkding/pull/574
* Include archived bookmarks in export by @sissbruecker in https://github.com/sissbruecker/linkding/pull/579
### New Contributors
* @OPerepadia made their first contribution in https://github.com/sissbruecker/linkding/pull/574
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.22.3...v1.23.0
---
## v1.22.3 (04/11/2023)
### What's Changed
* Fix RSS feed not handling None values by @vitormarcal in https://github.com/sissbruecker/linkding/pull/569
* Bump django from 4.1.10 to 4.1.13 by @dependabot in https://github.com/sissbruecker/linkding/pull/567
### New Contributors
* @vitormarcal made their first contribution in https://github.com/sissbruecker/linkding/pull/569
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.22.2...v1.22.3
---
## v1.22.2 (27/10/2023)
### What's Changed
* Fix search options not opening on iOS by @sissbruecker in https://github.com/sissbruecker/linkding/pull/549
* Bump urllib3 from 1.26.11 to 1.26.17 by @dependabot in https://github.com/sissbruecker/linkding/pull/542
* Add iOS shortcut to community section by @andrewdolphin in https://github.com/sissbruecker/linkding/pull/550
* Disable editing of search preferences in user admin by @sissbruecker in https://github.com/sissbruecker/linkding/pull/555
* Add feed2linkding to community section by @Strubbl in https://github.com/sissbruecker/linkding/pull/544
* Sanitize RSS feed to remove control characters by @sissbruecker in https://github.com/sissbruecker/linkding/pull/565
* Bump urllib3 from 1.26.17 to 1.26.18 by @dependabot in https://github.com/sissbruecker/linkding/pull/560
### New Contributors
* @andrewdolphin made their first contribution in https://github.com/sissbruecker/linkding/pull/550
* @Strubbl made their first contribution in https://github.com/sissbruecker/linkding/pull/544
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.22.1...v1.22.2
---
## v1.22.1 (06/10/2023)
### What's Changed
* Fix memory leak with SQLite by @sissbruecker in https://github.com/sissbruecker/linkding/pull/548
**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.22.0...v1.22.1
---
## v1.22.0 (01/10/2023)
### What's Changed

View File

@@ -1,84 +0,0 @@
FROM node:18.18.0-alpine AS node-build
WORKDIR /etc/linkding
# install build dependencies
COPY package.json package-lock.json ./
RUN npm install -g npm && \
npm install
# compile JS components
COPY . .
RUN npm run build
FROM python:3.10.6-slim-buster AS python-base
RUN apt-get update && apt-get -y install build-essential libpq-dev
WORKDIR /etc/linkding
FROM python-base AS python-build
# install build dependencies
COPY requirements.txt requirements.txt
RUN pip install -U pip && pip install -Ur requirements.txt
# run Django part of the build
COPY --from=node-build /etc/linkding .
RUN python manage.py compilescss && \
python manage.py collectstatic --ignore=*.scss && \
python manage.py compilescss --delete-files
FROM python-base AS prod-deps
COPY requirements.prod.txt ./requirements.txt
RUN mkdir /opt/venv && \
python -m venv --upgrade-deps --copies /opt/venv && \
/opt/venv/bin/pip install --upgrade pip wheel && \
/opt/venv/bin/pip install -Ur requirements.txt
FROM python-base AS compile-icu
RUN apt-get update && apt-get -y install libicu-dev libsqlite3-dev wget unzip
WORKDIR /etc/linkding
# Defines SQLite version
# Since this is only needed for downloading the header files this probably
# doesn't need to be up-to-date, assuming the SQLite APIs used by the ICU
# extension do not change
ARG SQLITE_RELEASE_YEAR=2023
ARG SQLITE_RELEASE=3430000
# Compile the ICU extension needed for case-insensitive search and ordering
# with SQLite. This does:
# - Download SQLite amalgamation for header files
# - Download ICU extension source file
# - Compile ICU extension
RUN wget https://www.sqlite.org/${SQLITE_RELEASE_YEAR}/sqlite-amalgamation-${SQLITE_RELEASE}.zip && \
unzip sqlite-amalgamation-${SQLITE_RELEASE}.zip && \
cp sqlite-amalgamation-${SQLITE_RELEASE}/sqlite3.h ./sqlite3.h && \
cp sqlite-amalgamation-${SQLITE_RELEASE}/sqlite3ext.h ./sqlite3ext.h && \
wget https://www.sqlite.org/src/raw/ext/icu/icu.c?name=91c021c7e3e8bbba286960810fa303295c622e323567b2e6def4ce58e4466e60 -O icu.c && \
gcc -fPIC -shared icu.c `pkg-config --libs --cflags icu-uc icu-io` -o libicu.so
FROM python:3.10.6-slim-buster as final
RUN apt-get update && apt-get -y install mime-support libpq-dev libicu-dev curl
WORKDIR /etc/linkding
# copy prod dependencies
COPY --from=prod-deps /opt/venv /opt/venv
# copy output from build stage
COPY --from=python-build /etc/linkding/static static/
# copy compiled icu extension
COPY --from=compile-icu /etc/linkding/libicu.so libicu.so
# copy application code
COPY . .
# Expose uwsgi server at port 9090
EXPOSE 9090
# Activate virtual env
ENV VIRTUAL_ENV /opt/venv
ENV PATH /opt/venv/bin:$PATH
# Allow running containers as an an arbitrary user in the root group, to support deployment scenarios like OpenShift, Podman
RUN ["chmod", "g+w", "."]
# Run bootstrap logic
RUN ["chmod", "+x", "./bootstrap.sh"]
HEALTHCHECK --interval=30s --retries=3 --timeout=1s \
CMD curl -f http://localhost:${LD_SERVER_PORT:-9090}/${LD_CONTEXT_PATH}health || exit 1
CMD ["./bootstrap.sh"]

15
Makefile Normal file
View File

@@ -0,0 +1,15 @@
.PHONY: serve
serve:
python manage.py runserver
tasks:
python manage.py process_tasks
test:
pytest
format:
black bookmarks
black siteroot
npx prettier bookmarks/frontend --write

View File

@@ -9,11 +9,11 @@
## Overview
- [Introduction](#introduction)
- [Installation](#installation)
- [Using Docker](#using-docker)
- [Using Docker Compose](#using-docker-compose)
- [User Setup](#user-setup)
- [Reverse Proxy Setup](#reverse-proxy-setup)
- [Managed Hosting Options](#managed-hosting-options)
- [Using Docker](#using-docker)
- [Using Docker Compose](#using-docker-compose)
- [User Setup](#user-setup)
- [Reverse Proxy Setup](#reverse-proxy-setup)
- [Managed Hosting Options](#managed-hosting-options)
- [Documentation](#documentation)
- [Browser Extension](#browser-extension)
- [Community](#community)
@@ -33,21 +33,19 @@ The name comes from:
**Feature Overview:**
- Clean UI optimized for readability
- Organize bookmarks with tags
- Add notes using Markdown
- Read it later functionality
- Share bookmarks with other users
- Bulk editing
- Bulk editing, Markdown notes, read it later functionality
- Share bookmarks with other users or guests
- Automatically provides titles, descriptions and icons of bookmarked websites
- Automatically creates snapshots of bookmarked websites on [the Internet Archive Wayback Machine](https://archive.org/web/)
- Automatically archive websites, either as local HTML file or on Internet Archive
- Import and export bookmarks in Netscape HTML format
- Extensions for [Firefox](https://addons.mozilla.org/de/firefox/addon/linkding-extension/) and [Chrome](https://chrome.google.com/webstore/detail/linkding-extension/beakmhbijpdhipnjhnclmhgjlddhidpe), as well as a bookmarklet
- Light and dark themes
- Installable as a Progressive Web App (PWA)
- Extensions for [Firefox](https://addons.mozilla.org/firefox/addon/linkding-extension/) and [Chrome](https://chrome.google.com/webstore/detail/linkding-extension/beakmhbijpdhipnjhnclmhgjlddhidpe), as well as a bookmarklet
- SSO support via OIDC or authentication proxies
- REST API for developing 3rd party apps
- Admin panel for user self-service and raw data access
- Easy setup using Docker and a SQLite database, with PostgreSQL as an option
**Demo:** https://demo.linkding.link/ ([see here](https://github.com/sissbruecker/linkding/issues/408) if you have trouble accessing it)
**Demo:** https://demo.linkding.link/
**Screenshot:**
@@ -58,12 +56,48 @@ The name comes from:
linkding is designed to be run with container solutions like [Docker](https://docs.docker.com/get-started/).
The Docker image is compatible with ARM platforms, so it can be run on a Raspberry Pi.
By default, linkding uses SQLite as a database.
linkding uses an SQLite database by default.
Alternatively linkding supports PostgreSQL, see the [database options](docs/Options.md#LD_DB_ENGINE) for more information.
### Using Docker
To install linkding using Docker you can just run the [latest image](https://hub.docker.com/repository/docker/sissbruecker/linkding) from Docker Hub:
The Docker image comes in several variants. To use a different image than the default, replace `latest` with the desired tag in the commands below, or in the docker-compose file.
<table>
<thead>
<tr>
<th>Tag</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>latest</code></td>
<td>Provides the basic functionality of linkding</td>
</tr>
<tr>
<td><code>latest-plus</code></td>
<td>
Includes feature for archiving websites as HTML snapshots
<ul>
<li>Significantly larger image size as it includes a Chromium installation</li>
<li>Requires more runtime memory to run Chromium</li>
<li>Requires more disk space for storing HTML snapshots</li>
</ul>
</td>
</tr>
<tr>
<td><code>latest-alpine</code></td>
<td><code>latest</code>, but based on Alpine Linux. 🧪 Experimental</td>
</tr>
<tr>
<td><code>latest-plus-alpine</code></td>
<td><code>latest-plus</code>, but based on Alpine Linux. 🧪 Experimental</td>
</tr>
</tbody>
</table>
To install linkding using Docker you can just run the image from [Docker Hub](https://hub.docker.com/repository/docker/sissbruecker/linkding):
```shell
docker run --name linkding -p 9090:9090 -v {host-data-folder}:/etc/linkding/data -d sissbruecker/linkding:latest
```
@@ -85,7 +119,7 @@ docker-compose up -d
To complete the setup, you still have to [create an initial user](#user-setup), so that you can access your installation.
### User setup
### User Setup
For security reasons, the linkding Docker image does not provide an initial user, so you have to create one after setting up an installation. To do so, replace the credentials in the following command and run it:
@@ -101,7 +135,7 @@ docker-compose exec linkding python manage.py createsuperuser --username=joe --e
The command will prompt you for a secure password. After the command has completed you can start using the application by logging into the UI with your credentials.
Alternatively you can automatically create an initial superuser on startup using the [`LD_SUPERUSER_NAME` option](docs/Options.md#LD_SUPERUSER_NAME).
Alternatively you can automatically create an initial superuser on startup using the [`LD_SUPERUSER_NAME` option](docs/Options.md#LD_SUPERUSER_NAME).
### Reverse Proxy Setup
@@ -164,6 +198,8 @@ Self-hosting web applications still requires a lot of technical know-how and com
- [linkding on fly.io](https://github.com/fspoettel/linkding-on-fly) - Guide for hosting a linkding installation on [fly.io](https://fly.io). By [fspoettel](https://github.com/fspoettel)
- [PikaPods.com](https://www.pikapods.com/) - Managed hosting for linkding, EU and US regions available. [1-click setup link](https://www.pikapods.com/pods?run=linkding) ([Disclosure](#pikapods))
- [CapRover](https://caprover.com/) - Linkding is included as a default one-click app
- [linkding on railway.app](https://github.com/tianheg/linkding-on-railway) - Guide for hosting a linkding installation on [railway.app](https://railway.app/). By [tianheg](https://github.com/tianheg)
## Documentation
@@ -180,7 +216,7 @@ Self-hosting web applications still requires a lot of technical know-how and com
## Browser Extension
linkding comes with an official browser extension that allows to quickly add bookmarks, and search bookmarks through the browser's address bar. You can get the extension here:
- [Mozilla Addon Store](https://addons.mozilla.org/de/firefox/addon/linkding-extension/)
- [Mozilla Addon Store](https://addons.mozilla.org/firefox/addon/linkding-extension/)
- [Chrome Web Store](https://chrome.google.com/webstore/detail/linkding-extension/beakmhbijpdhipnjhnclmhgjlddhidpe)
The extension is open-source as well, and can be found [here](https://github.com/sissbruecker/linkding-extension).
@@ -192,11 +228,13 @@ This section lists community projects around using linkding, in alphabetical ord
- [aiolinkding](https://github.com/bachya/aiolinkding) A Python3, async library to interact with the linkding REST API. By [bachya](https://github.com/bachya)
- [feed2linkding](https://codeberg.org/strubbl/feed2linkding) A commandline utility to add all web feed item links to linkding via API call. By [Strubbl](https://github.com/Strubbl)
- [Helm Chart](https://charts.pascaliske.dev/charts/linkding/) Helm Chart for deploying linkding inside a Kubernetes cluster. By [pascaliske](https://github.com/pascaliske)
- [iOS Shortcut using API and Tagging](https://gist.github.com/andrewdolphin/a7dff49505e588d940bec55132fab8ad) An iOS shortcut using the Linkding API (no extra logins required) that pulls previously used tags and allows tagging at the time of link creation.
- [iOS Shortcut using API and Tagging](https://gist.github.com/andrewdolphin/a7dff49505e588d940bec55132fab8ad) An iOS shortcut using the Linkding API (no extra logins required) that pulls previously used tags and allows tagging at the time of link creation.
- [k8s + s3](https://github.com/jzck/linkding-k8s-s3) - Setup for hosting stateless linkding on k8s with sqlite replicated to s3. By [jzck](https://github.com/jzck)
- [Linka!](https://github.com/cmsax/linka) Web app (also a PWA) for quickly searching & opening bookmarks in linkding, support multi keywords, exclude mode and other advance options. By [cmsax](https://github.com/cmsax)
- [linkding-cli](https://github.com/bachya/linkding-cli) A command-line interface (CLI) to interact with the linkding REST API. Powered by [aiolinkding](https://github.com/bachya/aiolinkding). By [bachya](https://github.com/bachya)
- [linkding-extension](https://github.com/jeroenpardon/linkding-extension) Chromium compatible extension that wraps the linkding bookmarklet. Tested with Chrome, Edge, Brave. By [jeroenpardon](https://github.com/jeroenpardon)
- [linkding-injector](https://github.com/Fivefold/linkding-injector) Injects search results from linkding into the sidebar of search pages like google and duckduckgo. Tested with Firefox and Chrome. By [Fivefold](https://github.com/Fivefold)
- [Linkdy](https://github.com/JGeek00/linkdy): An open source mobile and desktop (not yet) client created with Flutter. Available at the [Google Play Store](https://play.google.com/store/apps/details?id=com.jgeek00.linkdy). By [JGeek00](https://github.com/JGeek00).
- [LinkThing](https://apps.apple.com/us/app/linkthing/id1666031776) An iOS client for linkding. By [amoscardino](https://github.com/amoscardino)
- [Open all links bookmarklet](https://gist.github.com/ukcuddlyguy/336dd7339e6d35fc64a75ccfc9323c66) A browser bookmarklet to open all links on the current Linkding page in new tabs. By [ukcuddlyguy](https://github.com/ukcuddlyguy)
- [Postman collection](https://gist.github.com/gingerbeardman/f0b42502f3bc9344e92ce63afd4360d3) a group of saved request templates for API testing. By [gingerbeardman](https://github.com/gingerbeardman)
@@ -205,7 +243,7 @@ This section lists community projects around using linkding, in alphabetical ord
### PikaPods
[PikaPods](https://www.pikapods.com/) has a revenue sharing agreement with this project, sharing some of their revenue from hosting linkding instances. I do not intend to profit from this project financially, so I am in turn donating that revenue. Big thanks to PikaPods for making this possible.
[PikaPods](https://www.pikapods.com/) has a revenue sharing agreement with this project, sharing some of their revenue from hosting linkding instances. I do not intend to profit from this project financially, so I am in turn donating that revenue. Big thanks to PikaPods for making this possible.
See the table below for a list of donations.
@@ -237,7 +275,7 @@ source ~/environments/linkding/bin/activate[.csh|.fish]
```
Within the active environment install the application dependencies from the application folder:
```
pip3 install -Ur requirements.txt
pip3 install -r requirements.txt -r requirements.dev.txt
```
Install frontend dependencies:
```
@@ -262,6 +300,20 @@ python3 manage.py runserver
```
The frontend is now available under http://localhost:8000
### Tests
Run all tests with pytest:
```
pytest
```
### Formatting
Format Python code with black, and JavaScript code with prettier:
```
make format
```
### DevContainers
This repository also supports DevContainers: [![Open in Remote - Containers](https://img.shields.io/static/v1?label=Remote%20-%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=git@github.com:sissbruecker/linkding.git)

View File

@@ -1,5 +0,0 @@
#!/usr/bin/env bash
# Wrapper script used by supervisord to first clear task locks before starting the background task processor
python manage.py clean_tasks
exec python manage.py process_tasks

View File

@@ -1,93 +1,223 @@
from background_task.admin import TaskAdmin, CompletedTaskAdmin
from background_task.models import Task, CompletedTask
from django.contrib import admin, messages
from django.contrib.admin import AdminSite
from django.contrib.auth.admin import UserAdmin
from django.contrib.auth.models import User
from django.core.paginator import Paginator
from django.db.models import Count, QuerySet
from django.shortcuts import render
from django.urls import path
from django.utils.translation import ngettext, gettext
from huey.contrib.djhuey import HUEY as huey
from rest_framework.authtoken.admin import TokenAdmin
from rest_framework.authtoken.models import TokenProxy
from bookmarks.models import Bookmark, Tag, UserProfile, Toast, FeedToken
from bookmarks.models import Bookmark, BookmarkAsset, Tag, UserProfile, Toast, FeedToken
from bookmarks.services.bookmarks import archive_bookmark, unarchive_bookmark
# Custom paginator to paginate through Huey tasks
class TaskPaginator(Paginator):
def __init__(self):
super().__init__(self, 100)
self.task_count = huey.storage.queue_size()
@property
def count(self):
return self.task_count
def page(self, number):
limit = self.per_page
offset = (number - 1) * self.per_page
return self._get_page(
self.enqueued_items(limit, offset),
number,
self,
)
# Copied from Huey's SqliteStorage with some modifications to allow pagination
def enqueued_items(self, limit, offset):
to_bytes = lambda b: bytes(b) if not isinstance(b, bytes) else b
sql = "select data from task where queue=? order by priority desc, id limit ? offset ?"
params = (huey.storage.name, limit, offset)
serialized_tasks = [
to_bytes(i) for i, in huey.storage.sql(sql, params, results=True)
]
return [huey.deserialize_task(task) for task in serialized_tasks]
# Custom view to display Huey tasks in the admin
def background_task_view(request):
page_number = int(request.GET.get("p", 1))
paginator = TaskPaginator()
page = paginator.get_page(page_number)
page_range = paginator.get_elided_page_range(page_number, on_each_side=2, on_ends=2)
context = {
**linkding_admin_site.each_context(request),
"title": "Background tasks",
"page": page,
"page_range": page_range,
"tasks": page.object_list,
}
return render(request, "admin/background_tasks.html", context)
class LinkdingAdminSite(AdminSite):
site_header = 'linkding administration'
site_title = 'linkding Admin'
site_header = "linkding administration"
site_title = "linkding Admin"
def get_urls(self):
urls = super().get_urls()
custom_urls = [
path("tasks/", background_task_view, name="background_tasks"),
]
return custom_urls + urls
def get_app_list(self, request, app_label=None):
app_list = super().get_app_list(request, app_label)
app_list += [
{
"name": "Huey",
"app_label": "huey_app",
"models": [
{
"name": "Queued tasks",
"object_name": "background_tasks",
"admin_url": "/admin/tasks/",
"view_only": True,
}
],
}
]
return app_list
class AdminBookmark(admin.ModelAdmin):
list_display = ('resolved_title', 'url', 'is_archived', 'owner', 'date_added')
search_fields = ('title', 'description', 'website_title', 'website_description', 'url', 'tags__name')
list_filter = ('owner__username', 'is_archived', 'unread', 'tags',)
ordering = ('-date_added',)
actions = ['delete_selected_bookmarks', 'archive_selected_bookmarks', 'unarchive_selected_bookmarks', 'mark_as_read', 'mark_as_unread']
list_display = ("resolved_title", "url", "is_archived", "owner", "date_added")
search_fields = (
"title",
"description",
"website_title",
"website_description",
"url",
"tags__name",
)
list_filter = (
"owner__username",
"is_archived",
"unread",
"tags",
)
ordering = ("-date_added",)
actions = [
"delete_selected_bookmarks",
"archive_selected_bookmarks",
"unarchive_selected_bookmarks",
"mark_as_read",
"mark_as_unread",
]
def get_actions(self, request):
actions = super().get_actions(request)
# Remove default delete action, which gets replaced by delete_selected_bookmarks below
# The default action shows a confirmation page which can fail in production when selecting all bookmarks and the
# number of objects to delete exceeds the value in DATA_UPLOAD_MAX_NUMBER_FIELDS (1000 by default)
del actions['delete_selected']
del actions["delete_selected"]
return actions
def delete_selected_bookmarks(self, request, queryset: QuerySet):
bookmarks_count = queryset.count()
for bookmark in queryset:
bookmark.delete()
self.message_user(request, ngettext(
'%d bookmark was successfully deleted.',
'%d bookmarks were successfully deleted.',
bookmarks_count,
) % bookmarks_count, messages.SUCCESS)
self.message_user(
request,
ngettext(
"%d bookmark was successfully deleted.",
"%d bookmarks were successfully deleted.",
bookmarks_count,
)
% bookmarks_count,
messages.SUCCESS,
)
def archive_selected_bookmarks(self, request, queryset: QuerySet):
for bookmark in queryset:
archive_bookmark(bookmark)
bookmarks_count = queryset.count()
self.message_user(request, ngettext(
'%d bookmark was successfully archived.',
'%d bookmarks were successfully archived.',
bookmarks_count,
) % bookmarks_count, messages.SUCCESS)
self.message_user(
request,
ngettext(
"%d bookmark was successfully archived.",
"%d bookmarks were successfully archived.",
bookmarks_count,
)
% bookmarks_count,
messages.SUCCESS,
)
def unarchive_selected_bookmarks(self, request, queryset: QuerySet):
for bookmark in queryset:
unarchive_bookmark(bookmark)
bookmarks_count = queryset.count()
self.message_user(request, ngettext(
'%d bookmark was successfully unarchived.',
'%d bookmarks were successfully unarchived.',
bookmarks_count,
) % bookmarks_count, messages.SUCCESS)
self.message_user(
request,
ngettext(
"%d bookmark was successfully unarchived.",
"%d bookmarks were successfully unarchived.",
bookmarks_count,
)
% bookmarks_count,
messages.SUCCESS,
)
def mark_as_read(self, request, queryset: QuerySet):
bookmarks_count = queryset.count()
queryset.update(unread=False)
self.message_user(request, ngettext(
'%d bookmark marked as read.',
'%d bookmarks marked as read.',
bookmarks_count,
) % bookmarks_count, messages.SUCCESS)
self.message_user(
request,
ngettext(
"%d bookmark marked as read.",
"%d bookmarks marked as read.",
bookmarks_count,
)
% bookmarks_count,
messages.SUCCESS,
)
def mark_as_unread(self, request, queryset: QuerySet):
bookmarks_count = queryset.count()
queryset.update(unread=True)
self.message_user(request, ngettext(
'%d bookmark marked as unread.',
'%d bookmarks marked as unread.',
bookmarks_count,
) % bookmarks_count, messages.SUCCESS)
self.message_user(
request,
ngettext(
"%d bookmark marked as unread.",
"%d bookmarks marked as unread.",
bookmarks_count,
)
% bookmarks_count,
messages.SUCCESS,
)
class AdminBookmarkAsset(admin.ModelAdmin):
@admin.display(description="Display Name")
def custom_display_name(self, obj):
return str(obj)
list_display = ("custom_display_name", "date_created", "status")
search_fields = (
"custom_display_name",
"file",
)
list_filter = ("status",)
class AdminTag(admin.ModelAdmin):
list_display = ('name', 'bookmarks_count', 'owner', 'date_added')
search_fields = ('name', 'owner__username')
list_filter = ('owner__username',)
ordering = ('-date_added',)
actions = ['delete_unused_tags']
list_display = ("name", "bookmarks_count", "owner", "date_added")
search_fields = ("name", "owner__username")
list_filter = ("owner__username",)
ordering = ("-date_added",)
actions = ["delete_unused_tags"]
def get_queryset(self, request):
queryset = super().get_queryset(request)
@@ -97,7 +227,7 @@ class AdminTag(admin.ModelAdmin):
def bookmarks_count(self, obj):
return obj.bookmarks_count
bookmarks_count.admin_order_field = 'bookmarks_count'
bookmarks_count.admin_order_field = "bookmarks_count"
def delete_unused_tags(self, request, queryset: QuerySet):
unused_tags = queryset.filter(bookmark__isnull=True)
@@ -106,23 +236,33 @@ class AdminTag(admin.ModelAdmin):
tag.delete()
if unused_tags_count > 0:
self.message_user(request, ngettext(
'%d unused tag was successfully deleted.',
'%d unused tags were successfully deleted.',
unused_tags_count,
) % unused_tags_count, messages.SUCCESS)
self.message_user(
request,
ngettext(
"%d unused tag was successfully deleted.",
"%d unused tags were successfully deleted.",
unused_tags_count,
)
% unused_tags_count,
messages.SUCCESS,
)
else:
self.message_user(request, gettext(
'There were no unused tags in the selection',
), messages.SUCCESS)
self.message_user(
request,
gettext(
"There were no unused tags in the selection",
),
messages.SUCCESS,
)
class AdminUserProfileInline(admin.StackedInline):
model = UserProfile
can_delete = False
verbose_name_plural = 'Profile'
fk_name = 'user'
readonly_fields = ('search_preferences', )
verbose_name_plural = "Profile"
fk_name = "user"
readonly_fields = ("search_preferences",)
class AdminCustomUser(UserAdmin):
inlines = (AdminUserProfileInline,)
@@ -134,23 +274,22 @@ class AdminCustomUser(UserAdmin):
class AdminToast(admin.ModelAdmin):
list_display = ('key', 'message', 'owner', 'acknowledged')
search_fields = ('key', 'message')
list_filter = ('owner__username',)
list_display = ("key", "message", "owner", "acknowledged")
search_fields = ("key", "message")
list_filter = ("owner__username",)
class AdminFeedToken(admin.ModelAdmin):
list_display = ('key', 'user')
search_fields = ['key']
list_filter = ('user__username',)
list_display = ("key", "user")
search_fields = ["key"]
list_filter = ("user__username",)
linkding_admin_site = LinkdingAdminSite()
linkding_admin_site.register(Bookmark, AdminBookmark)
linkding_admin_site.register(BookmarkAsset, AdminBookmarkAsset)
linkding_admin_site.register(Tag, AdminTag)
linkding_admin_site.register(User, AdminCustomUser)
linkding_admin_site.register(TokenProxy, TokenAdmin)
linkding_admin_site.register(Toast, AdminToast)
linkding_admin_site.register(FeedToken, AdminFeedToken)
linkding_admin_site.register(Task, TaskAdmin)
linkding_admin_site.register(CompletedTask, CompletedTaskAdmin)

View File

@@ -5,18 +5,29 @@ from rest_framework.response import Response
from rest_framework.routers import DefaultRouter
from bookmarks import queries
from bookmarks.api.serializers import BookmarkSerializer, TagSerializer, UserProfileSerializer
from bookmarks.api.serializers import (
BookmarkSerializer,
TagSerializer,
UserProfileSerializer,
)
from bookmarks.models import Bookmark, BookmarkSearch, Tag, User
from bookmarks.services.bookmarks import archive_bookmark, unarchive_bookmark, website_loader
from bookmarks.services import auto_tagging
from bookmarks.services.bookmarks import (
archive_bookmark,
unarchive_bookmark,
website_loader,
)
from bookmarks.services.website_loader import WebsiteMetadata
class BookmarkViewSet(viewsets.GenericViewSet,
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
mixins.CreateModelMixin,
mixins.UpdateModelMixin,
mixins.DestroyModelMixin):
class BookmarkViewSet(
viewsets.GenericViewSet,
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
mixins.CreateModelMixin,
mixins.UpdateModelMixin,
mixins.DestroyModelMixin,
):
serializer_class = BookmarkSerializer
def get_permissions(self):
@@ -24,7 +35,7 @@ class BookmarkViewSet(viewsets.GenericViewSet,
# The shared action should still filter bookmarks so that
# unauthenticated users only see bookmarks from users that have public
# sharing explicitly enabled
if self.action == 'shared':
if self.action == "shared":
return [AllowAny()]
# Otherwise use default permissions which should require authentication
@@ -33,7 +44,7 @@ class BookmarkViewSet(viewsets.GenericViewSet,
def get_queryset(self):
user = self.request.user
# For list action, use query set that applies search and tag projections
if self.action == 'list':
if self.action == "list":
search = BookmarkSearch.from_request(self.request.GET)
return queries.query_bookmarks(user, user.profile, search)
@@ -41,9 +52,9 @@ class BookmarkViewSet(viewsets.GenericViewSet,
return Bookmark.objects.all().filter(owner=user)
def get_serializer_context(self):
return {'user': self.request.user}
return {"user": self.request.user}
@action(methods=['get'], detail=False)
@action(methods=["get"], detail=False)
def archived(self, request):
user = request.user
search = BookmarkSearch.from_request(request.GET)
@@ -53,51 +64,72 @@ class BookmarkViewSet(viewsets.GenericViewSet,
data = serializer(page, many=True).data
return self.get_paginated_response(data)
@action(methods=['get'], detail=False)
@action(methods=["get"], detail=False)
def shared(self, request):
search = BookmarkSearch.from_request(request.GET)
user = User.objects.filter(username=search.user).first()
public_only = not request.user.is_authenticated
query_set = queries.query_shared_bookmarks(user, request.user_profile, search, public_only)
query_set = queries.query_shared_bookmarks(
user, request.user_profile, search, public_only
)
page = self.paginate_queryset(query_set)
serializer = self.get_serializer_class()
data = serializer(page, many=True).data
return self.get_paginated_response(data)
@action(methods=['post'], detail=True)
@action(methods=["post"], detail=True)
def archive(self, request, pk):
bookmark = self.get_object()
archive_bookmark(bookmark)
return Response(status=status.HTTP_204_NO_CONTENT)
@action(methods=['post'], detail=True)
@action(methods=["post"], detail=True)
def unarchive(self, request, pk):
bookmark = self.get_object()
unarchive_bookmark(bookmark)
return Response(status=status.HTTP_204_NO_CONTENT)
@action(methods=['get'], detail=False)
@action(methods=["get"], detail=False)
def check(self, request):
url = request.GET.get('url')
url = request.GET.get("url")
bookmark = Bookmark.objects.filter(owner=request.user, url=url).first()
existing_bookmark_data = self.get_serializer(bookmark).data if bookmark else None
existing_bookmark_data = (
self.get_serializer(bookmark).data if bookmark else None
)
# Either return metadata from existing bookmark, or scrape from URL
if bookmark:
metadata = WebsiteMetadata(url, bookmark.website_title, bookmark.website_description)
metadata = WebsiteMetadata(
url,
bookmark.website_title,
bookmark.website_description,
None,
)
else:
metadata = website_loader.load_website_metadata(url)
return Response({
'bookmark': existing_bookmark_data,
'metadata': metadata.to_dict()
}, status=status.HTTP_200_OK)
# Return tags that would be automatically applied to the bookmark
profile = request.user.profile
auto_tags = []
if profile.auto_tagging_rules:
auto_tags = auto_tagging.get_tags(profile.auto_tagging_rules, url)
return Response(
{
"bookmark": existing_bookmark_data,
"metadata": metadata.to_dict(),
"auto_tags": auto_tags,
},
status=status.HTTP_200_OK,
)
class TagViewSet(viewsets.GenericViewSet,
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
mixins.CreateModelMixin):
class TagViewSet(
viewsets.GenericViewSet,
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
mixins.CreateModelMixin,
):
serializer_class = TagSerializer
def get_queryset(self):
@@ -105,16 +137,16 @@ class TagViewSet(viewsets.GenericViewSet,
return Tag.objects.all().filter(owner=user)
def get_serializer_context(self):
return {'user': self.request.user}
return {"user": self.request.user}
class UserViewSet(viewsets.GenericViewSet):
@action(methods=['get'], detail=False)
@action(methods=["get"], detail=False)
def profile(self, request):
return Response(UserProfileSerializer(request.user.profile).data)
router = DefaultRouter()
router.register(r'bookmarks', BookmarkViewSet, basename='bookmark')
router.register(r'tags', TagViewSet, basename='tag')
router.register(r'user', UserViewSet, basename='user')
router.register(r"bookmarks", BookmarkViewSet, basename="bookmark")
router.register(r"tags", TagViewSet, basename="tag")
router.register(r"user", UserViewSet, basename="user")

View File

@@ -14,7 +14,7 @@ class TagListField(serializers.ListField):
class BookmarkListSerializer(ListSerializer):
def to_representation(self, data):
# Prefetch nested relations to avoid n+1 queries
prefetch_related_objects(data, 'tags')
prefetch_related_objects(data, "tags")
return super().to_representation(data)
@@ -23,32 +23,34 @@ class BookmarkSerializer(serializers.ModelSerializer):
class Meta:
model = Bookmark
fields = [
'id',
'url',
'title',
'description',
'notes',
'website_title',
'website_description',
'is_archived',
'unread',
'shared',
'tag_names',
'date_added',
'date_modified'
"id",
"url",
"title",
"description",
"notes",
"website_title",
"website_description",
"web_archive_snapshot_url",
"is_archived",
"unread",
"shared",
"tag_names",
"date_added",
"date_modified",
]
read_only_fields = [
'website_title',
'website_description',
'date_added',
'date_modified'
"website_title",
"website_description",
"web_archive_snapshot_url",
"date_added",
"date_modified",
]
list_serializer_class = BookmarkListSerializer
# Override optional char fields to provide default value
title = serializers.CharField(required=False, allow_blank=True, default='')
description = serializers.CharField(required=False, allow_blank=True, default='')
notes = serializers.CharField(required=False, allow_blank=True, default='')
title = serializers.CharField(required=False, allow_blank=True, default="")
description = serializers.CharField(required=False, allow_blank=True, default="")
notes = serializers.CharField(required=False, allow_blank=True, default="")
is_archived = serializers.BooleanField(required=False, default=False)
unread = serializers.BooleanField(required=False, default=False)
shared = serializers.BooleanField(required=False, default=False)
@@ -57,38 +59,38 @@ class BookmarkSerializer(serializers.ModelSerializer):
def create(self, validated_data):
bookmark = Bookmark()
bookmark.url = validated_data['url']
bookmark.title = validated_data['title']
bookmark.description = validated_data['description']
bookmark.notes = validated_data['notes']
bookmark.is_archived = validated_data['is_archived']
bookmark.unread = validated_data['unread']
bookmark.shared = validated_data['shared']
tag_string = build_tag_string(validated_data['tag_names'])
return create_bookmark(bookmark, tag_string, self.context['user'])
bookmark.url = validated_data["url"]
bookmark.title = validated_data["title"]
bookmark.description = validated_data["description"]
bookmark.notes = validated_data["notes"]
bookmark.is_archived = validated_data["is_archived"]
bookmark.unread = validated_data["unread"]
bookmark.shared = validated_data["shared"]
tag_string = build_tag_string(validated_data["tag_names"])
return create_bookmark(bookmark, tag_string, self.context["user"])
def update(self, instance: Bookmark, validated_data):
# Update fields if they were provided in the payload
for key in ['url', 'title', 'description', 'notes', 'unread', 'shared']:
for key in ["url", "title", "description", "notes", "unread", "shared"]:
if key in validated_data:
setattr(instance, key, validated_data[key])
# Use tag string from payload, or use bookmark's current tags as fallback
tag_string = build_tag_string(instance.tag_names)
if 'tag_names' in validated_data:
tag_string = build_tag_string(validated_data['tag_names'])
if "tag_names" in validated_data:
tag_string = build_tag_string(validated_data["tag_names"])
return update_bookmark(instance, tag_string, self.context['user'])
return update_bookmark(instance, tag_string, self.context["user"])
class TagSerializer(serializers.ModelSerializer):
class Meta:
model = Tag
fields = ['id', 'name', 'date_added']
read_only_fields = ['date_added']
fields = ["id", "name", "date_added"]
read_only_fields = ["date_added"]
def create(self, validated_data):
return get_or_create_tag(validated_data['name'], self.context['user'])
return get_or_create_tag(validated_data["name"], self.context["user"])
class UserProfileSerializer(serializers.ModelSerializer):

View File

@@ -2,7 +2,7 @@ from django.apps import AppConfig
class BookmarksConfig(AppConfig):
name = 'bookmarks'
name = "bookmarks"
def ready(self):
# Register signal handlers

View File

@@ -5,28 +5,32 @@ from bookmarks import utils
def toasts(request):
user = request.user
toast_messages = Toast.objects.filter(owner=user, acknowledged=False) if user.is_authenticated else []
toast_messages = (
Toast.objects.filter(owner=user, acknowledged=False)
if user.is_authenticated
else []
)
has_toasts = len(toast_messages) > 0
return {
'has_toasts': has_toasts,
'toast_messages': toast_messages,
"has_toasts": has_toasts,
"toast_messages": toast_messages,
}
def public_shares(request):
# Only check for public shares for anonymous users
if not request.user.is_authenticated:
query_set = queries.query_shared_bookmarks(None, request.user_profile, BookmarkSearch(), True)
query_set = queries.query_shared_bookmarks(
None, request.user_profile, BookmarkSearch(), True
)
has_public_shares = query_set.count() > 0
return {
'has_public_shares': has_public_shares,
"has_public_shares": has_public_shares,
}
return {}
def app_version(request):
return {
'app_version': utils.app_version
}
return {"app_version": utils.app_version}

View File

@@ -0,0 +1,176 @@
from django.test import override_settings
from django.urls import reverse
from playwright.sync_api import sync_playwright, expect
from bookmarks.e2e.helpers import LinkdingE2ETestCase
from bookmarks.models import Bookmark
class BookmarkDetailsModalE2ETestCase(LinkdingE2ETestCase):
def test_show_details(self):
bookmark = self.setup_bookmark()
with sync_playwright() as p:
self.open(reverse("bookmarks:index"), p)
details_modal = self.open_details_modal(bookmark)
title = details_modal.locator("h2")
expect(title).to_have_text(bookmark.title)
def test_close_details(self):
bookmark = self.setup_bookmark()
with sync_playwright() as p:
self.open(reverse("bookmarks:index"), p)
# close with close button
details_modal = self.open_details_modal(bookmark)
details_modal.locator("button.close").click()
expect(details_modal).to_be_hidden()
# close with backdrop
details_modal = self.open_details_modal(bookmark)
overlay = details_modal.locator(".modal-overlay")
overlay.click(position={"x": 0, "y": 0})
expect(details_modal).to_be_hidden()
# close with escape
details_modal = self.open_details_modal(bookmark)
self.page.keyboard.press("Escape")
expect(details_modal).to_be_hidden()
def test_toggle_archived(self):
bookmark = self.setup_bookmark()
with sync_playwright() as p:
# archive
url = reverse("bookmarks:index")
self.open(url, p)
details_modal = self.open_details_modal(bookmark)
details_modal.get_by_text("Archived", exact=False).click()
expect(self.locate_bookmark(bookmark.title)).not_to_be_visible()
self.assertReloads(0)
# unarchive
url = reverse("bookmarks:archived")
self.page.goto(self.live_server_url + url)
self.resetReloads()
details_modal = self.open_details_modal(bookmark)
details_modal.get_by_text("Archived", exact=False).click()
expect(self.locate_bookmark(bookmark.title)).not_to_be_visible()
self.assertReloads(0)
def test_toggle_unread(self):
bookmark = self.setup_bookmark()
with sync_playwright() as p:
# mark as unread
url = reverse("bookmarks:index")
self.open(url, p)
details_modal = self.open_details_modal(bookmark)
details_modal.get_by_text("Unread").click()
bookmark_item = self.locate_bookmark(bookmark.title)
expect(bookmark_item.get_by_text("Unread")).to_be_visible()
self.assertReloads(0)
# mark as read
details_modal.get_by_text("Unread").click()
bookmark_item = self.locate_bookmark(bookmark.title)
expect(bookmark_item.get_by_text("Unread")).not_to_be_visible()
self.assertReloads(0)
def test_toggle_shared(self):
profile = self.get_or_create_test_user().profile
profile.enable_sharing = True
profile.save()
bookmark = self.setup_bookmark()
with sync_playwright() as p:
# share bookmark
url = reverse("bookmarks:index")
self.open(url, p)
details_modal = self.open_details_modal(bookmark)
details_modal.get_by_text("Shared").click()
bookmark_item = self.locate_bookmark(bookmark.title)
expect(bookmark_item.get_by_text("Shared")).to_be_visible()
self.assertReloads(0)
# unshare bookmark
details_modal.get_by_text("Shared").click()
bookmark_item = self.locate_bookmark(bookmark.title)
expect(bookmark_item.get_by_text("Shared")).not_to_be_visible()
self.assertReloads(0)
def test_edit_return_url(self):
bookmark = self.setup_bookmark()
with sync_playwright() as p:
url = reverse("bookmarks:index") + f"?q={bookmark.title}"
self.open(url, p)
details_modal = self.open_details_modal(bookmark)
# Navigate to edit page
with self.page.expect_navigation():
details_modal.get_by_text("Edit").click()
# Cancel edit, verify return url
with self.page.expect_navigation(url=self.live_server_url + url):
self.page.get_by_text("Nevermind").click()
def test_delete(self):
bookmark = self.setup_bookmark()
with sync_playwright() as p:
url = reverse("bookmarks:index") + f"?q={bookmark.title}"
self.open(url, p)
details_modal = self.open_details_modal(bookmark)
# Delete bookmark, verify return url
with self.page.expect_navigation(url=self.live_server_url + url):
details_modal.get_by_text("Delete...").click()
details_modal.get_by_text("Confirm").click()
# verify bookmark is deleted
self.locate_bookmark(bookmark.title)
expect(self.locate_bookmark(bookmark.title)).not_to_be_visible()
self.assertEqual(Bookmark.objects.count(), 0)
@override_settings(LD_ENABLE_SNAPSHOTS=True)
def test_create_snapshot_remove_snapshot(self):
bookmark = self.setup_bookmark()
with sync_playwright() as p:
url = reverse("bookmarks:index") + f"?q={bookmark.title}"
self.open(url, p)
details_modal = self.open_details_modal(bookmark)
asset_list = details_modal.locator(".assets")
# No snapshots initially
snapshot = asset_list.get_by_text("HTML snapshot from", exact=False)
expect(snapshot).not_to_be_visible()
# Create snapshot
details_modal.get_by_text("Create HTML snapshot", exact=False).click()
self.assertReloads(0)
# Has new snapshots
expect(snapshot).to_be_visible()
# Create snapshot
asset_list.get_by_text("Remove", exact=False).click()
asset_list.get_by_text("Confirm", exact=False).click()
# Snapshot is removed
expect(snapshot).not_to_be_visible()
self.assertReloads(0)

View File

@@ -0,0 +1,37 @@
from django.urls import reverse
from playwright.sync_api import sync_playwright
from bookmarks.e2e.helpers import LinkdingE2ETestCase
class BookmarkDetailsViewE2ETestCase(LinkdingE2ETestCase):
def test_edit_return_url(self):
bookmark = self.setup_bookmark()
with sync_playwright() as p:
self.open(reverse("bookmarks:details", args=[bookmark.id]), p)
# Navigate to edit page
with self.page.expect_navigation():
self.page.get_by_text("Edit").click()
# Cancel edit, verify return url
with self.page.expect_navigation(
url=self.live_server_url
+ reverse("bookmarks:details", args=[bookmark.id])
):
self.page.get_by_text("Nevermind").click()
def test_delete_return_url(self):
bookmark = self.setup_bookmark()
with sync_playwright() as p:
self.open(reverse("bookmarks:details", args=[bookmark.id]), p)
# Trigger delete, verify return url
# Should probably return to last bookmark list page, but for now just returns to index
with self.page.expect_navigation(
url=self.live_server_url + reverse("bookmarks:index")
):
self.page.get_by_text("Delete...").click()
self.page.get_by_text("Confirm").click()

View File

@@ -6,38 +6,54 @@ from bookmarks.e2e.helpers import LinkdingE2ETestCase
class BookmarkFormE2ETestCase(LinkdingE2ETestCase):
def test_create_should_check_for_existing_bookmark(self):
existing_bookmark = self.setup_bookmark(title='Existing title',
description='Existing description',
notes='Existing notes',
tags=[self.setup_tag(name='tag1'), self.setup_tag(name='tag2')],
website_title='Existing website title',
website_description='Existing website description',
unread=True)
tag_names = ' '.join(existing_bookmark.tag_names)
existing_bookmark = self.setup_bookmark(
title="Existing title",
description="Existing description",
notes="Existing notes",
tags=[self.setup_tag(name="tag1"), self.setup_tag(name="tag2")],
website_title="Existing website title",
website_description="Existing website description",
unread=True,
)
tag_names = " ".join(existing_bookmark.tag_names)
with sync_playwright() as p:
browser = self.setup_browser(p)
page = browser.new_page()
page.goto(self.live_server_url + reverse('bookmarks:new'))
page.goto(self.live_server_url + reverse("bookmarks:new"))
# Enter bookmarked URL
page.get_by_label('URL').fill(existing_bookmark.url)
page.get_by_label("URL").fill(existing_bookmark.url)
# Already bookmarked hint should be visible
page.get_by_text('This URL is already bookmarked.').wait_for(timeout=2000)
page.get_by_text("This URL is already bookmarked.").wait_for(timeout=2000)
# Form should be pre-filled with data from existing bookmark
self.assertEqual(existing_bookmark.title, page.get_by_label('Title').input_value())
self.assertEqual(existing_bookmark.description, page.get_by_label('Description').input_value())
self.assertEqual(existing_bookmark.notes, page.get_by_label('Notes').input_value())
self.assertEqual(existing_bookmark.website_title, page.get_by_label('Title').get_attribute('placeholder'))
self.assertEqual(existing_bookmark.website_description,
page.get_by_label('Description').get_attribute('placeholder'))
self.assertEqual(tag_names, page.get_by_label('Tags').input_value())
self.assertTrue(tag_names, page.get_by_label('Mark as unread').is_checked())
self.assertEqual(
existing_bookmark.title, page.get_by_label("Title").input_value()
)
self.assertEqual(
existing_bookmark.description,
page.get_by_label("Description").input_value(),
)
self.assertEqual(
existing_bookmark.notes, page.get_by_label("Notes").input_value()
)
self.assertEqual(
existing_bookmark.website_title,
page.get_by_label("Title").get_attribute("placeholder"),
)
self.assertEqual(
existing_bookmark.website_description,
page.get_by_label("Description").get_attribute("placeholder"),
)
self.assertEqual(tag_names, page.get_by_label("Tags").input_value())
self.assertTrue(tag_names, page.get_by_label("Mark as unread").is_checked())
# Enter non-bookmarked URL
page.get_by_label('URL').fill('https://example.com/unknown')
page.get_by_label("URL").fill("https://example.com/unknown")
# Already bookmarked hint should be hidden
page.get_by_text('This URL is already bookmarked.').wait_for(state='hidden', timeout=2000)
page.get_by_text("This URL is already bookmarked.").wait_for(
state="hidden", timeout=2000
)
browser.close()
@@ -47,21 +63,47 @@ class BookmarkFormE2ETestCase(LinkdingE2ETestCase):
with sync_playwright() as p:
browser = self.setup_browser(p)
page = browser.new_page()
page.goto(self.live_server_url + reverse('bookmarks:edit', args=[bookmark.id]))
page.goto(
self.live_server_url + reverse("bookmarks:edit", args=[bookmark.id])
)
page.wait_for_timeout(timeout=1000)
page.get_by_text('This URL is already bookmarked.').wait_for(state='hidden')
page.get_by_text("This URL is already bookmarked.").wait_for(state="hidden")
def test_enter_url_of_existing_bookmark_should_show_notes(self):
bookmark = self.setup_bookmark(notes='Existing notes', description='Existing description')
bookmark = self.setup_bookmark(
notes="Existing notes", description="Existing description"
)
with sync_playwright() as p:
browser = self.setup_browser(p)
page = browser.new_page()
page.goto(self.live_server_url + reverse('bookmarks:new'))
page.goto(self.live_server_url + reverse("bookmarks:new"))
details = page.locator('details.notes')
expect(details).not_to_have_attribute('open', value='')
details = page.locator("details.notes")
expect(details).not_to_have_attribute("open", value="")
page.get_by_label('URL').fill(bookmark.url)
expect(details).to_have_attribute('open', value='')
page.get_by_label("URL").fill(bookmark.url)
expect(details).to_have_attribute("open", value="")
def test_create_should_preview_auto_tags(self):
profile = self.get_or_create_test_user().profile
profile.auto_tagging_rules = "github.com dev github"
profile.save()
with sync_playwright() as p:
# Open page with URL that should have auto tags
browser = self.setup_browser(p)
page = browser.new_page()
url = self.live_server_url + reverse("bookmarks:new")
url += f"?url=https%3A%2F%2Fgithub.com%2Fsissbruecker%2Flinkding"
page.goto(url)
auto_tags_hint = page.locator(".form-input-hint.auto-tags")
expect(auto_tags_hint).to_be_visible()
expect(auto_tags_hint).to_have_text("Auto tags: dev github")
# Change to URL without auto tags
page.get_by_label("URL").fill("https://example.com")
expect(auto_tags_hint).to_be_hidden()

View File

@@ -9,15 +9,15 @@ from bookmarks.e2e.helpers import LinkdingE2ETestCase
class BookmarkItemE2ETestCase(LinkdingE2ETestCase):
@skip("Fails in CI, needs investigation")
def test_toggle_notes_should_show_hide_notes(self):
bookmark = self.setup_bookmark(notes='Test notes')
bookmark = self.setup_bookmark(notes="Test notes")
with sync_playwright() as p:
page = self.open(reverse('bookmarks:index'), p)
page = self.open(reverse("bookmarks:index"), p)
notes = self.locate_bookmark(bookmark.title).locator('.notes')
notes = self.locate_bookmark(bookmark.title).locator(".notes")
expect(notes).to_be_hidden()
toggle_notes = page.locator('li button.toggle-notes')
toggle_notes = page.locator("li button.toggle-notes")
toggle_notes.click()
expect(notes).to_be_visible()

View File

@@ -9,100 +9,192 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
def setup_test_data(self):
self.setup_numbered_bookmarks(50)
self.setup_numbered_bookmarks(50, archived=True)
self.setup_numbered_bookmarks(50, prefix='foo')
self.setup_numbered_bookmarks(50, archived=True, prefix='foo')
self.setup_numbered_bookmarks(50, prefix="foo")
self.setup_numbered_bookmarks(50, archived=True, prefix="foo")
self.assertEqual(50, Bookmark.objects.filter(is_archived=False, title__startswith='Bookmark').count())
self.assertEqual(50, Bookmark.objects.filter(is_archived=True, title__startswith='Archived Bookmark').count())
self.assertEqual(50, Bookmark.objects.filter(is_archived=False, title__startswith='foo').count())
self.assertEqual(50, Bookmark.objects.filter(is_archived=True, title__startswith='foo').count())
self.assertEqual(
50,
Bookmark.objects.filter(
is_archived=False, title__startswith="Bookmark"
).count(),
)
self.assertEqual(
50,
Bookmark.objects.filter(
is_archived=True, title__startswith="Archived Bookmark"
).count(),
)
self.assertEqual(
50,
Bookmark.objects.filter(is_archived=False, title__startswith="foo").count(),
)
self.assertEqual(
50,
Bookmark.objects.filter(is_archived=True, title__startswith="foo").count(),
)
def test_active_bookmarks_bulk_select_across(self):
self.setup_test_data()
with sync_playwright() as p:
self.open(reverse('bookmarks:index'), p)
self.open(reverse("bookmarks:index"), p)
bookmark_list = self.locate_bookmark_list()
self.locate_bulk_edit_toggle().click()
self.locate_bulk_edit_select_all().click()
self.locate_bulk_edit_select_across().click()
self.select_bulk_action('Delete')
self.locate_bulk_edit_bar().get_by_text('Execute').click()
self.locate_bulk_edit_bar().get_by_text('Confirm').click()
self.select_bulk_action("Delete")
self.locate_bulk_edit_bar().get_by_text("Execute").click()
self.locate_bulk_edit_bar().get_by_text("Confirm").click()
# Wait until bookmark list is updated (old reference becomes invisible)
expect(bookmark_list).not_to_be_visible()
self.assertEqual(0, Bookmark.objects.filter(is_archived=False, title__startswith='Bookmark').count())
self.assertEqual(50, Bookmark.objects.filter(is_archived=True, title__startswith='Archived Bookmark').count())
self.assertEqual(0, Bookmark.objects.filter(is_archived=False, title__startswith='foo').count())
self.assertEqual(50, Bookmark.objects.filter(is_archived=True, title__startswith='foo').count())
self.assertEqual(
0,
Bookmark.objects.filter(
is_archived=False, title__startswith="Bookmark"
).count(),
)
self.assertEqual(
50,
Bookmark.objects.filter(
is_archived=True, title__startswith="Archived Bookmark"
).count(),
)
self.assertEqual(
0,
Bookmark.objects.filter(is_archived=False, title__startswith="foo").count(),
)
self.assertEqual(
50,
Bookmark.objects.filter(is_archived=True, title__startswith="foo").count(),
)
def test_archived_bookmarks_bulk_select_across(self):
self.setup_test_data()
with sync_playwright() as p:
self.open(reverse('bookmarks:archived'), p)
self.open(reverse("bookmarks:archived"), p)
bookmark_list = self.locate_bookmark_list()
self.locate_bulk_edit_toggle().click()
self.locate_bulk_edit_select_all().click()
self.locate_bulk_edit_select_across().click()
self.select_bulk_action('Delete')
self.locate_bulk_edit_bar().get_by_text('Execute').click()
self.locate_bulk_edit_bar().get_by_text('Confirm').click()
self.select_bulk_action("Delete")
self.locate_bulk_edit_bar().get_by_text("Execute").click()
self.locate_bulk_edit_bar().get_by_text("Confirm").click()
# Wait until bookmark list is updated (old reference becomes invisible)
expect(bookmark_list).not_to_be_visible()
self.assertEqual(50, Bookmark.objects.filter(is_archived=False, title__startswith='Bookmark').count())
self.assertEqual(0, Bookmark.objects.filter(is_archived=True, title__startswith='Archived Bookmark').count())
self.assertEqual(50, Bookmark.objects.filter(is_archived=False, title__startswith='foo').count())
self.assertEqual(0, Bookmark.objects.filter(is_archived=True, title__startswith='foo').count())
self.assertEqual(
50,
Bookmark.objects.filter(
is_archived=False, title__startswith="Bookmark"
).count(),
)
self.assertEqual(
0,
Bookmark.objects.filter(
is_archived=True, title__startswith="Archived Bookmark"
).count(),
)
self.assertEqual(
50,
Bookmark.objects.filter(is_archived=False, title__startswith="foo").count(),
)
self.assertEqual(
0,
Bookmark.objects.filter(is_archived=True, title__startswith="foo").count(),
)
def test_active_bookmarks_bulk_select_across_respects_query(self):
self.setup_test_data()
with sync_playwright() as p:
self.open(reverse('bookmarks:index') + '?q=foo', p)
self.open(reverse("bookmarks:index") + "?q=foo", p)
bookmark_list = self.locate_bookmark_list()
self.locate_bulk_edit_toggle().click()
self.locate_bulk_edit_select_all().click()
self.locate_bulk_edit_select_across().click()
self.select_bulk_action('Delete')
self.locate_bulk_edit_bar().get_by_text('Execute').click()
self.locate_bulk_edit_bar().get_by_text('Confirm').click()
self.select_bulk_action("Delete")
self.locate_bulk_edit_bar().get_by_text("Execute").click()
self.locate_bulk_edit_bar().get_by_text("Confirm").click()
# Wait until bookmark list is updated (old reference becomes invisible)
expect(bookmark_list).not_to_be_visible()
self.assertEqual(50, Bookmark.objects.filter(is_archived=False, title__startswith='Bookmark').count())
self.assertEqual(50, Bookmark.objects.filter(is_archived=True, title__startswith='Archived Bookmark').count())
self.assertEqual(0, Bookmark.objects.filter(is_archived=False, title__startswith='foo').count())
self.assertEqual(50, Bookmark.objects.filter(is_archived=True, title__startswith='foo').count())
self.assertEqual(
50,
Bookmark.objects.filter(
is_archived=False, title__startswith="Bookmark"
).count(),
)
self.assertEqual(
50,
Bookmark.objects.filter(
is_archived=True, title__startswith="Archived Bookmark"
).count(),
)
self.assertEqual(
0,
Bookmark.objects.filter(is_archived=False, title__startswith="foo").count(),
)
self.assertEqual(
50,
Bookmark.objects.filter(is_archived=True, title__startswith="foo").count(),
)
def test_archived_bookmarks_bulk_select_across_respects_query(self):
self.setup_test_data()
with sync_playwright() as p:
self.open(reverse('bookmarks:archived') + '?q=foo', p)
self.open(reverse("bookmarks:archived") + "?q=foo", p)
bookmark_list = self.locate_bookmark_list()
self.locate_bulk_edit_toggle().click()
self.locate_bulk_edit_select_all().click()
self.locate_bulk_edit_select_across().click()
self.select_bulk_action('Delete')
self.locate_bulk_edit_bar().get_by_text('Execute').click()
self.locate_bulk_edit_bar().get_by_text('Confirm').click()
self.select_bulk_action("Delete")
self.locate_bulk_edit_bar().get_by_text("Execute").click()
self.locate_bulk_edit_bar().get_by_text("Confirm").click()
# Wait until bookmark list is updated (old reference becomes invisible)
expect(bookmark_list).not_to_be_visible()
self.assertEqual(50, Bookmark.objects.filter(is_archived=False, title__startswith='Bookmark').count())
self.assertEqual(50, Bookmark.objects.filter(is_archived=True, title__startswith='Archived Bookmark').count())
self.assertEqual(50, Bookmark.objects.filter(is_archived=False, title__startswith='foo').count())
self.assertEqual(0, Bookmark.objects.filter(is_archived=True, title__startswith='foo').count())
self.assertEqual(
50,
Bookmark.objects.filter(
is_archived=False, title__startswith="Bookmark"
).count(),
)
self.assertEqual(
50,
Bookmark.objects.filter(
is_archived=True, title__startswith="Archived Bookmark"
).count(),
)
self.assertEqual(
50,
Bookmark.objects.filter(is_archived=False, title__startswith="foo").count(),
)
self.assertEqual(
0,
Bookmark.objects.filter(is_archived=True, title__startswith="foo").count(),
)
def test_select_all_toggles_all_checkboxes(self):
self.setup_numbered_bookmarks(5)
with sync_playwright() as p:
url = reverse('bookmarks:index')
url = reverse("bookmarks:index")
page = self.open(url, p)
self.locate_bulk_edit_toggle().click()
checkboxes = page.locator('label[ld-bulk-edit-checkbox] input')
checkboxes = page.locator("label.bulk-edit-checkbox input")
self.assertEqual(6, checkboxes.count())
for i in range(checkboxes.count()):
expect(checkboxes.nth(i)).not_to_be_checked()
@@ -121,7 +213,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
self.setup_numbered_bookmarks(5)
with sync_playwright() as p:
url = reverse('bookmarks:index')
url = reverse("bookmarks:index")
self.open(url, p)
self.locate_bulk_edit_toggle().click()
@@ -138,7 +230,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
self.setup_numbered_bookmarks(5)
with sync_playwright() as p:
url = reverse('bookmarks:index')
url = reverse("bookmarks:index")
self.open(url, p)
self.locate_bulk_edit_toggle().click()
@@ -160,7 +252,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
self.setup_numbered_bookmarks(5)
with sync_playwright() as p:
url = reverse('bookmarks:index')
url = reverse("bookmarks:index")
self.open(url, p)
self.locate_bulk_edit_toggle().click()
@@ -171,38 +263,41 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
expect(self.locate_bulk_edit_select_across()).to_be_checked()
# Hide select across by toggling a single bookmark
self.locate_bookmark('Bookmark 1').locator('label[ld-bulk-edit-checkbox]').click()
self.locate_bookmark("Bookmark 1").locator(
"label.bulk-edit-checkbox"
).click()
expect(self.locate_bulk_edit_select_across()).not_to_be_visible()
# Show select across again, verify it is unchecked
self.locate_bookmark('Bookmark 1').locator('label[ld-bulk-edit-checkbox]').click()
self.locate_bookmark("Bookmark 1").locator(
"label.bulk-edit-checkbox"
).click()
expect(self.locate_bulk_edit_select_across()).not_to_be_checked()
def test_execute_resets_all_checkboxes(self):
self.setup_numbered_bookmarks(100)
with sync_playwright() as p:
url = reverse('bookmarks:index')
url = reverse("bookmarks:index")
page = self.open(url, p)
bookmark_list = self.locate_bookmark_list()
# Select all bookmarks, enable select across
self.locate_bulk_edit_toggle().click()
self.locate_bulk_edit_select_all().click()
self.locate_bulk_edit_select_across().click()
# Get reference for bookmark list
bookmark_list = page.locator('ul[ld-bookmark-list]')
# Execute bulk action
self.select_bulk_action('Mark as unread')
self.locate_bulk_edit_bar().get_by_text('Execute').click()
self.locate_bulk_edit_bar().get_by_text('Confirm').click()
self.select_bulk_action("Mark as unread")
self.locate_bulk_edit_bar().get_by_text("Execute").click()
self.locate_bulk_edit_bar().get_by_text("Confirm").click()
# Wait until bookmark list is updated (old reference becomes invisible)
expect(bookmark_list).not_to_be_visible()
# Verify bulk edit checkboxes are reset
checkboxes = page.locator('label[ld-bulk-edit-checkbox] input')
checkboxes = page.locator("label.bulk-edit-checkbox input")
self.assertEqual(31, checkboxes.count())
for i in range(checkboxes.count()):
expect(checkboxes.nth(i)).not_to_be_checked()
@@ -215,18 +310,26 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
self.setup_numbered_bookmarks(100)
with sync_playwright() as p:
url = reverse('bookmarks:index')
url = reverse("bookmarks:index")
self.open(url, p)
bookmark_list = self.locate_bookmark_list()
self.locate_bulk_edit_toggle().click()
self.locate_bulk_edit_select_all().click()
expect(self.locate_bulk_edit_bar().get_by_text('All pages (100 bookmarks)')).to_be_visible()
expect(
self.locate_bulk_edit_bar().get_by_text("All pages (100 bookmarks)")
).to_be_visible()
self.select_bulk_action('Delete')
self.locate_bulk_edit_bar().get_by_text('Execute').click()
self.locate_bulk_edit_bar().get_by_text('Confirm').click()
self.select_bulk_action("Delete")
self.locate_bulk_edit_bar().get_by_text("Execute").click()
self.locate_bulk_edit_bar().get_by_text("Confirm").click()
# Wait until bookmark list is updated (old reference becomes invisible)
expect(bookmark_list).not_to_be_visible()
expect(self.locate_bulk_edit_select_all()).not_to_be_checked()
self.locate_bulk_edit_select_all().click()
expect(self.locate_bulk_edit_bar().get_by_text('All pages (70 bookmarks)')).to_be_visible()
expect(
self.locate_bulk_edit_bar().get_by_text("All pages (70 bookmarks)")
).to_be_visible()

View File

@@ -16,13 +16,15 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
# verify correct data is loaded on update
self.setup_numbered_bookmarks(3, with_tags=True)
self.setup_numbered_bookmarks(3, with_tags=True, archived=True)
self.setup_numbered_bookmarks(3,
shared=True,
prefix="Joe's Bookmark",
user=self.setup_user(enable_sharing=True))
self.setup_numbered_bookmarks(
3,
shared=True,
prefix="Joe's Bookmark",
user=self.setup_user(enable_sharing=True),
)
def assertVisibleBookmarks(self, titles: List[str]):
bookmark_tags = self.page.locator('li[ld-bookmark-item]')
bookmark_tags = self.page.locator("li[ld-bookmark-item]")
expect(bookmark_tags).to_have_count(len(titles))
for title in titles:
@@ -30,7 +32,7 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
expect(matching_tag).to_be_visible()
def assertVisibleTags(self, titles: List[str]):
tag_tags = self.page.locator('.tag-cloud .unselected-tags a')
tag_tags = self.page.locator(".tag-cloud .unselected-tags a")
expect(tag_tags).to_have_count(len(titles))
for title in titles:
@@ -38,65 +40,67 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
expect(matching_tag).to_be_visible()
def test_partial_update_respects_query(self):
self.setup_numbered_bookmarks(5, prefix='foo')
self.setup_numbered_bookmarks(5, prefix='bar')
self.setup_numbered_bookmarks(5, prefix="foo")
self.setup_numbered_bookmarks(5, prefix="bar")
with sync_playwright() as p:
url = reverse('bookmarks:index') + '?q=foo'
url = reverse("bookmarks:index") + "?q=foo"
self.open(url, p)
self.assertVisibleBookmarks(['foo 1', 'foo 2', 'foo 3', 'foo 4', 'foo 5'])
self.assertVisibleBookmarks(["foo 1", "foo 2", "foo 3", "foo 4", "foo 5"])
self.locate_bookmark('foo 2').get_by_text('Archive').click()
self.assertVisibleBookmarks(['foo 1', 'foo 3', 'foo 4', 'foo 5'])
self.locate_bookmark("foo 2").get_by_text("Archive").click()
self.assertVisibleBookmarks(["foo 1", "foo 3", "foo 4", "foo 5"])
def test_partial_update_respects_sort(self):
self.setup_numbered_bookmarks(5, prefix='foo')
self.setup_numbered_bookmarks(5, prefix="foo")
with sync_playwright() as p:
url = reverse('bookmarks:index') + '?sort=title_asc'
url = reverse("bookmarks:index") + "?sort=title_asc"
page = self.open(url, p)
first_item = page.locator('li[ld-bookmark-item]').first
expect(first_item).to_contain_text('foo 1')
first_item = page.locator("li[ld-bookmark-item]").first
expect(first_item).to_contain_text("foo 1")
first_item.get_by_text('Archive').click()
first_item.get_by_text("Archive").click()
first_item = page.locator('li[ld-bookmark-item]').first
expect(first_item).to_contain_text('foo 2')
first_item = page.locator("li[ld-bookmark-item]").first
expect(first_item).to_contain_text("foo 2")
def test_partial_update_respects_page(self):
# add a suffix, otherwise 'foo 1' also matches 'foo 10'
self.setup_numbered_bookmarks(50, prefix='foo', suffix='-')
self.setup_numbered_bookmarks(50, prefix="foo", suffix="-")
with sync_playwright() as p:
url = reverse('bookmarks:index') + '?q=foo&page=2'
url = reverse("bookmarks:index") + "?q=foo&page=2"
self.open(url, p)
# with descending sort, page two has 'foo 1' to 'foo 20'
expected_titles = [f'foo {i}-' for i in range(1, 21)]
expected_titles = [f"foo {i}-" for i in range(1, 21)]
self.assertVisibleBookmarks(expected_titles)
self.locate_bookmark('foo 20-').get_by_text('Archive').click()
self.locate_bookmark("foo 20-").get_by_text("Archive").click()
expected_titles = [f'foo {i}-' for i in range(1, 20)]
expected_titles = [f"foo {i}-" for i in range(1, 20)]
self.assertVisibleBookmarks(expected_titles)
def test_multiple_partial_updates(self):
self.setup_numbered_bookmarks(5)
with sync_playwright() as p:
url = reverse('bookmarks:index')
url = reverse("bookmarks:index")
self.open(url, p)
self.locate_bookmark('Bookmark 1').get_by_text('Archive').click()
self.assertVisibleBookmarks(['Bookmark 2', 'Bookmark 3', 'Bookmark 4', 'Bookmark 5'])
self.locate_bookmark("Bookmark 1").get_by_text("Archive").click()
self.assertVisibleBookmarks(
["Bookmark 2", "Bookmark 3", "Bookmark 4", "Bookmark 5"]
)
self.locate_bookmark('Bookmark 2').get_by_text('Archive').click()
self.assertVisibleBookmarks(['Bookmark 3', 'Bookmark 4', 'Bookmark 5'])
self.locate_bookmark("Bookmark 2").get_by_text("Archive").click()
self.assertVisibleBookmarks(["Bookmark 3", "Bookmark 4", "Bookmark 5"])
self.locate_bookmark('Bookmark 3').get_by_text('Archive').click()
self.assertVisibleBookmarks(['Bookmark 4', 'Bookmark 5'])
self.locate_bookmark("Bookmark 3").get_by_text("Archive").click()
self.assertVisibleBookmarks(["Bookmark 4", "Bookmark 5"])
self.assertReloads(0)
@@ -104,185 +108,201 @@ class BookmarkPagePartialUpdatesE2ETestCase(LinkdingE2ETestCase):
self.setup_fixture()
with sync_playwright() as p:
self.open(reverse('bookmarks:index'), p)
self.open(reverse("bookmarks:index"), p)
self.locate_bookmark('Bookmark 2').get_by_text('Archive').click()
self.locate_bookmark("Bookmark 2").get_by_text("Archive").click()
self.assertVisibleBookmarks(['Bookmark 1', 'Bookmark 3'])
self.assertVisibleTags(['Tag 1', 'Tag 3'])
self.assertVisibleBookmarks(["Bookmark 1", "Bookmark 3"])
self.assertVisibleTags(["Tag 1", "Tag 3"])
self.assertReloads(0)
def test_active_bookmarks_partial_update_on_delete(self):
self.setup_fixture()
with sync_playwright() as p:
self.open(reverse('bookmarks:index'), p)
self.open(reverse("bookmarks:index"), p)
self.locate_bookmark('Bookmark 2').get_by_text('Remove').click()
self.locate_bookmark('Bookmark 2').get_by_text('Confirm').click()
self.locate_bookmark("Bookmark 2").get_by_text("Remove").click()
self.locate_bookmark("Bookmark 2").get_by_text("Confirm").click()
self.assertVisibleBookmarks(['Bookmark 1', 'Bookmark 3'])
self.assertVisibleTags(['Tag 1', 'Tag 3'])
self.assertVisibleBookmarks(["Bookmark 1", "Bookmark 3"])
self.assertVisibleTags(["Tag 1", "Tag 3"])
self.assertReloads(0)
def test_active_bookmarks_partial_update_on_mark_as_read(self):
self.setup_fixture()
bookmark2 = self.get_numbered_bookmark('Bookmark 2')
bookmark2 = self.get_numbered_bookmark("Bookmark 2")
bookmark2.unread = True
bookmark2.save()
with sync_playwright() as p:
self.open(reverse('bookmarks:index'), p)
self.open(reverse("bookmarks:index"), p)
expect(self.locate_bookmark('Bookmark 2')).to_have_class('unread')
self.locate_bookmark('Bookmark 2').get_by_text('Unread').click()
self.locate_bookmark('Bookmark 2').get_by_text('Yes').click()
expect(self.locate_bookmark("Bookmark 2")).to_have_class("unread")
self.locate_bookmark("Bookmark 2").get_by_text("Unread").click()
self.locate_bookmark("Bookmark 2").get_by_text("Yes").click()
expect(self.locate_bookmark('Bookmark 2')).not_to_have_class('unread')
expect(self.locate_bookmark("Bookmark 2")).not_to_have_class("unread")
self.assertReloads(0)
def test_active_bookmarks_partial_update_on_unshare(self):
self.setup_fixture()
bookmark2 = self.get_numbered_bookmark('Bookmark 2')
bookmark2 = self.get_numbered_bookmark("Bookmark 2")
bookmark2.shared = True
bookmark2.save()
with sync_playwright() as p:
self.open(reverse('bookmarks:index'), p)
self.open(reverse("bookmarks:index"), p)
expect(self.locate_bookmark('Bookmark 2')).to_have_class('shared')
self.locate_bookmark('Bookmark 2').get_by_text('Shared').click()
self.locate_bookmark('Bookmark 2').get_by_text('Yes').click()
expect(self.locate_bookmark("Bookmark 2")).to_have_class("shared")
self.locate_bookmark("Bookmark 2").get_by_text("Shared").click()
self.locate_bookmark("Bookmark 2").get_by_text("Yes").click()
expect(self.locate_bookmark('Bookmark 2')).not_to_have_class('shared')
expect(self.locate_bookmark("Bookmark 2")).not_to_have_class("shared")
self.assertReloads(0)
def test_active_bookmarks_partial_update_on_bulk_archive(self):
self.setup_fixture()
with sync_playwright() as p:
self.open(reverse('bookmarks:index'), p)
self.open(reverse("bookmarks:index"), p)
self.locate_bulk_edit_toggle().click()
self.locate_bookmark('Bookmark 2').locator('label[ld-bulk-edit-checkbox]').click()
self.select_bulk_action('Archive')
self.locate_bulk_edit_bar().get_by_text('Execute').click()
self.locate_bulk_edit_bar().get_by_text('Confirm').click()
self.locate_bookmark("Bookmark 2").locator(
"label.bulk-edit-checkbox"
).click()
self.select_bulk_action("Archive")
self.locate_bulk_edit_bar().get_by_text("Execute").click()
self.locate_bulk_edit_bar().get_by_text("Confirm").click()
self.assertVisibleBookmarks(['Bookmark 1', 'Bookmark 3'])
self.assertVisibleTags(['Tag 1', 'Tag 3'])
self.assertVisibleBookmarks(["Bookmark 1", "Bookmark 3"])
self.assertVisibleTags(["Tag 1", "Tag 3"])
self.assertReloads(0)
def test_active_bookmarks_partial_update_on_bulk_delete(self):
self.setup_fixture()
with sync_playwright() as p:
self.open(reverse('bookmarks:index'), p)
self.open(reverse("bookmarks:index"), p)
self.locate_bulk_edit_toggle().click()
self.locate_bookmark('Bookmark 2').locator('label[ld-bulk-edit-checkbox]').click()
self.select_bulk_action('Delete')
self.locate_bulk_edit_bar().get_by_text('Execute').click()
self.locate_bulk_edit_bar().get_by_text('Confirm').click()
self.locate_bookmark("Bookmark 2").locator(
"label.bulk-edit-checkbox"
).click()
self.select_bulk_action("Delete")
self.locate_bulk_edit_bar().get_by_text("Execute").click()
self.locate_bulk_edit_bar().get_by_text("Confirm").click()
self.assertVisibleBookmarks(['Bookmark 1', 'Bookmark 3'])
self.assertVisibleTags(['Tag 1', 'Tag 3'])
self.assertVisibleBookmarks(["Bookmark 1", "Bookmark 3"])
self.assertVisibleTags(["Tag 1", "Tag 3"])
self.assertReloads(0)
def test_archived_bookmarks_partial_update_on_unarchive(self):
self.setup_fixture()
with sync_playwright() as p:
self.open(reverse('bookmarks:archived'), p)
self.open(reverse("bookmarks:archived"), p)
self.locate_bookmark('Archived Bookmark 2').get_by_text('Unarchive').click()
self.locate_bookmark("Archived Bookmark 2").get_by_text("Unarchive").click()
self.assertVisibleBookmarks(['Archived Bookmark 1', 'Archived Bookmark 3'])
self.assertVisibleTags(['Archived Tag 1', 'Archived Tag 3'])
self.assertVisibleBookmarks(["Archived Bookmark 1", "Archived Bookmark 3"])
self.assertVisibleTags(["Archived Tag 1", "Archived Tag 3"])
self.assertReloads(0)
def test_archived_bookmarks_partial_update_on_delete(self):
self.setup_fixture()
with sync_playwright() as p:
self.open(reverse('bookmarks:archived'), p)
self.open(reverse("bookmarks:archived"), p)
self.locate_bookmark('Archived Bookmark 2').get_by_text('Remove').click()
self.locate_bookmark('Archived Bookmark 2').get_by_text('Confirm').click()
self.locate_bookmark("Archived Bookmark 2").get_by_text("Remove").click()
self.locate_bookmark("Archived Bookmark 2").get_by_text("Confirm").click()
self.assertVisibleBookmarks(['Archived Bookmark 1', 'Archived Bookmark 3'])
self.assertVisibleTags(['Archived Tag 1', 'Archived Tag 3'])
self.assertVisibleBookmarks(["Archived Bookmark 1", "Archived Bookmark 3"])
self.assertVisibleTags(["Archived Tag 1", "Archived Tag 3"])
self.assertReloads(0)
def test_archived_bookmarks_partial_update_on_bulk_unarchive(self):
self.setup_fixture()
with sync_playwright() as p:
self.open(reverse('bookmarks:archived'), p)
self.open(reverse("bookmarks:archived"), p)
self.locate_bulk_edit_toggle().click()
self.locate_bookmark('Archived Bookmark 2').locator('label[ld-bulk-edit-checkbox]').click()
self.select_bulk_action('Unarchive')
self.locate_bulk_edit_bar().get_by_text('Execute').click()
self.locate_bulk_edit_bar().get_by_text('Confirm').click()
self.locate_bookmark("Archived Bookmark 2").locator(
"label.bulk-edit-checkbox"
).click()
self.select_bulk_action("Unarchive")
self.locate_bulk_edit_bar().get_by_text("Execute").click()
self.locate_bulk_edit_bar().get_by_text("Confirm").click()
self.assertVisibleBookmarks(['Archived Bookmark 1', 'Archived Bookmark 3'])
self.assertVisibleTags(['Archived Tag 1', 'Archived Tag 3'])
self.assertVisibleBookmarks(["Archived Bookmark 1", "Archived Bookmark 3"])
self.assertVisibleTags(["Archived Tag 1", "Archived Tag 3"])
self.assertReloads(0)
def test_archived_bookmarks_partial_update_on_bulk_delete(self):
self.setup_fixture()
with sync_playwright() as p:
self.open(reverse('bookmarks:archived'), p)
self.open(reverse("bookmarks:archived"), p)
self.locate_bulk_edit_toggle().click()
self.locate_bookmark('Archived Bookmark 2').locator('label[ld-bulk-edit-checkbox]').click()
self.select_bulk_action('Delete')
self.locate_bulk_edit_bar().get_by_text('Execute').click()
self.locate_bulk_edit_bar().get_by_text('Confirm').click()
self.locate_bookmark("Archived Bookmark 2").locator(
"label.bulk-edit-checkbox"
).click()
self.select_bulk_action("Delete")
self.locate_bulk_edit_bar().get_by_text("Execute").click()
self.locate_bulk_edit_bar().get_by_text("Confirm").click()
self.assertVisibleBookmarks(['Archived Bookmark 1', 'Archived Bookmark 3'])
self.assertVisibleTags(['Archived Tag 1', 'Archived Tag 3'])
self.assertVisibleBookmarks(["Archived Bookmark 1", "Archived Bookmark 3"])
self.assertVisibleTags(["Archived Tag 1", "Archived Tag 3"])
self.assertReloads(0)
def test_shared_bookmarks_partial_update_on_unarchive(self):
self.setup_fixture()
self.setup_numbered_bookmarks(3, shared=True, prefix="My Bookmark", with_tags=True)
self.setup_numbered_bookmarks(
3, shared=True, prefix="My Bookmark", with_tags=True
)
with sync_playwright() as p:
self.open(reverse('bookmarks:shared'), p)
self.open(reverse("bookmarks:shared"), p)
self.locate_bookmark('My Bookmark 2').get_by_text('Archive').click()
self.locate_bookmark("My Bookmark 2").get_by_text("Archive").click()
# Shared bookmarks page also shows archived bookmarks, though it probably shouldn't
self.assertVisibleBookmarks([
'My Bookmark 1',
'My Bookmark 2',
'My Bookmark 3',
"Joe's Bookmark 1",
"Joe's Bookmark 2",
"Joe's Bookmark 3",
])
self.assertVisibleTags(['Shared Tag 1', 'Shared Tag 2', 'Shared Tag 3'])
self.assertVisibleBookmarks(
[
"My Bookmark 1",
"My Bookmark 2",
"My Bookmark 3",
"Joe's Bookmark 1",
"Joe's Bookmark 2",
"Joe's Bookmark 3",
]
)
self.assertVisibleTags(["Shared Tag 1", "Shared Tag 2", "Shared Tag 3"])
self.assertReloads(0)
def test_shared_bookmarks_partial_update_on_delete(self):
self.setup_fixture()
self.setup_numbered_bookmarks(3, shared=True, prefix="My Bookmark", with_tags=True)
self.setup_numbered_bookmarks(
3, shared=True, prefix="My Bookmark", with_tags=True
)
with sync_playwright() as p:
self.open(reverse('bookmarks:shared'), p)
self.open(reverse("bookmarks:shared"), p)
self.locate_bookmark('My Bookmark 2').get_by_text('Remove').click()
self.locate_bookmark('My Bookmark 2').get_by_text('Confirm').click()
self.locate_bookmark("My Bookmark 2").get_by_text("Remove").click()
self.locate_bookmark("My Bookmark 2").get_by_text("Confirm").click()
self.assertVisibleBookmarks([
'My Bookmark 1',
'My Bookmark 3',
"Joe's Bookmark 1",
"Joe's Bookmark 2",
"Joe's Bookmark 3",
])
self.assertVisibleTags(['Shared Tag 1', 'Shared Tag 3'])
self.assertVisibleBookmarks(
[
"My Bookmark 1",
"My Bookmark 3",
"Joe's Bookmark 1",
"Joe's Bookmark 2",
"Joe's Bookmark 3",
]
)
self.assertVisibleTags(["Shared Tag 1", "Shared Tag 3"])
self.assertReloads(0)

View File

@@ -9,11 +9,11 @@ class GlobalShortcutsE2ETestCase(LinkdingE2ETestCase):
with sync_playwright() as p:
browser = self.setup_browser(p)
page = browser.new_page()
page.goto(self.live_server_url + reverse('bookmarks:index'))
page.goto(self.live_server_url + reverse("bookmarks:index"))
page.press('body', 's')
page.press("body", "s")
expect(page.get_by_placeholder('Search for words or #tags')).to_be_focused()
expect(page.get_by_placeholder("Search for words or #tags")).to_be_focused()
browser.close()
@@ -21,10 +21,10 @@ class GlobalShortcutsE2ETestCase(LinkdingE2ETestCase):
with sync_playwright() as p:
browser = self.setup_browser(p)
page = browser.new_page()
page.goto(self.live_server_url + reverse('bookmarks:index'))
page.goto(self.live_server_url + reverse("bookmarks:index"))
page.press('body', 'n')
page.press("body", "n")
expect(page).to_have_url(self.live_server_url + reverse('bookmarks:new'))
expect(page).to_have_url(self.live_server_url + reverse("bookmarks:new"))
browser.close()

View File

@@ -2,6 +2,7 @@ from django.urls import reverse
from playwright.sync_api import sync_playwright, expect
from bookmarks.e2e.helpers import LinkdingE2ETestCase
from bookmarks.models import UserProfile
class SettingsGeneralE2ETestCase(LinkdingE2ETestCase):
@@ -9,12 +10,14 @@ class SettingsGeneralE2ETestCase(LinkdingE2ETestCase):
with sync_playwright() as p:
browser = self.setup_browser(p)
page = browser.new_page()
page.goto(self.live_server_url + reverse('bookmarks:settings.general'))
page.goto(self.live_server_url + reverse("bookmarks:settings.general"))
enable_sharing = page.get_by_label('Enable bookmark sharing')
enable_sharing_label = page.get_by_text('Enable bookmark sharing')
enable_public_sharing = page.get_by_label('Enable public bookmark sharing')
enable_public_sharing_label = page.get_by_text('Enable public bookmark sharing')
enable_sharing = page.get_by_label("Enable bookmark sharing")
enable_sharing_label = page.get_by_text("Enable bookmark sharing")
enable_public_sharing = page.get_by_label("Enable public bookmark sharing")
enable_public_sharing_label = page.get_by_text(
"Enable public bookmark sharing"
)
# Public sharing is disabled by default
expect(enable_sharing).not_to_be_checked()
@@ -37,3 +40,49 @@ class SettingsGeneralE2ETestCase(LinkdingE2ETestCase):
expect(enable_sharing).not_to_be_checked()
expect(enable_public_sharing).not_to_be_checked()
expect(enable_public_sharing).to_be_disabled()
def test_should_not_show_bookmark_description_max_lines_when_display_inline(self):
profile = self.get_or_create_test_user().profile
profile.bookmark_description_display = (
UserProfile.BOOKMARK_DESCRIPTION_DISPLAY_INLINE
)
profile.save()
with sync_playwright() as p:
browser = self.setup_browser(p)
page = browser.new_page()
page.goto(self.live_server_url + reverse("bookmarks:settings.general"))
max_lines = page.get_by_label("Bookmark description max lines")
expect(max_lines).to_be_hidden()
def test_should_show_bookmark_description_max_lines_when_display_separate(self):
profile = self.get_or_create_test_user().profile
profile.bookmark_description_display = (
UserProfile.BOOKMARK_DESCRIPTION_DISPLAY_SEPARATE
)
profile.save()
with sync_playwright() as p:
browser = self.setup_browser(p)
page = browser.new_page()
page.goto(self.live_server_url + reverse("bookmarks:settings.general"))
max_lines = page.get_by_label("Bookmark description max lines")
expect(max_lines).to_be_visible()
def test_should_update_bookmark_description_max_lines_when_changing_display(self):
with sync_playwright() as p:
browser = self.setup_browser(p)
page = browser.new_page()
page.goto(self.live_server_url + reverse("bookmarks:settings.general"))
max_lines = page.get_by_label("Bookmark description max lines")
expect(max_lines).to_be_hidden()
display = page.get_by_label("Bookmark description", exact=True)
display.select_option("separate")
expect(max_lines).to_be_visible()
display.select_option("inline")
expect(max_lines).to_be_hidden()

View File

@@ -0,0 +1,76 @@
from django.test import override_settings
from django.urls import reverse
from playwright.sync_api import sync_playwright, expect, Locator
from bookmarks.e2e.helpers import LinkdingE2ETestCase
from bookmarks.models import Bookmark
class TagCloudModalE2ETestCase(LinkdingE2ETestCase):
def test_show_modal_close_modal(self):
self.setup_bookmark(tags=[self.setup_tag(name="cooking")])
self.setup_bookmark(tags=[self.setup_tag(name="hiking")])
with sync_playwright() as p:
page = self.open(reverse("bookmarks:index"), p)
# use smaller viewport to make tags button visible
page.set_viewport_size({"width": 375, "height": 812})
# open tag cloud modal
modal_trigger = page.locator(".content-area-header").get_by_role(
"button", name="Tags"
)
modal_trigger.click()
# verify modal is visible
modal = page.locator(".modal")
expect(modal).to_be_visible()
expect(modal.locator(".modal-title")).to_have_text("Tags")
# close with close button
modal.locator("button.close").click()
expect(modal).to_be_hidden()
# open modal again
modal_trigger.click()
# close with backdrop
backdrop = modal.locator(".modal-overlay")
backdrop.click(position={"x": 0, "y": 0})
expect(modal).to_be_hidden()
def test_select_tag(self):
self.setup_bookmark(tags=[self.setup_tag(name="cooking")])
self.setup_bookmark(tags=[self.setup_tag(name="hiking")])
with sync_playwright() as p:
page = self.open(reverse("bookmarks:index"), p)
# use smaller viewport to make tags button visible
page.set_viewport_size({"width": 375, "height": 812})
# open tag cloud modal
modal_trigger = page.locator(".content-area-header").get_by_role(
"button", name="Tags"
)
modal_trigger.click()
# verify tags are displayed
modal = page.locator(".modal")
unselected_tags = modal.locator(".unselected-tags")
expect(unselected_tags.get_by_text("cooking")).to_be_visible()
expect(unselected_tags.get_by_text("hiking")).to_be_visible()
# select tag
unselected_tags.get_by_text("cooking").click()
# open modal again
modal_trigger.click()
# verify tag is selected, other tag is not visible anymore
selected_tags = modal.locator(".selected-tags")
expect(selected_tags.get_by_text("cooking")).to_be_visible()
expect(unselected_tags.get_by_text("cooking")).not_to_be_visible()
expect(unselected_tags.get_by_text("hiking")).not_to_be_visible()

View File

@@ -1,5 +1,6 @@
from django.contrib.staticfiles.testing import LiveServerTestCase
from playwright.sync_api import BrowserContext, Playwright, Page
from playwright.sync_api import expect
from bookmarks.tests.helpers import BookmarkFactoryMixin
@@ -7,24 +8,28 @@ from bookmarks.tests.helpers import BookmarkFactoryMixin
class LinkdingE2ETestCase(LiveServerTestCase, BookmarkFactoryMixin):
def setUp(self) -> None:
self.client.force_login(self.get_or_create_test_user())
self.cookie = self.client.cookies['sessionid']
self.cookie = self.client.cookies["sessionid"]
def setup_browser(self, playwright) -> BrowserContext:
browser = playwright.chromium.launch(headless=True)
context = browser.new_context()
context.add_cookies([{
'name': 'sessionid',
'value': self.cookie.value,
'domain': self.live_server_url.replace('http:', ''),
'path': '/'
}])
context.add_cookies(
[
{
"name": "sessionid",
"value": self.cookie.value,
"domain": self.live_server_url.replace("http:", ""),
"path": "/",
}
]
)
return context
def open(self, url: str, playwright: Playwright) -> Page:
browser = self.setup_browser(playwright)
self.page = browser.new_page()
self.page.goto(self.live_server_url + url)
self.page.on('load', self.on_load)
self.page.on("load", self.on_load)
self.num_loads = 0
return self.page
@@ -34,21 +39,43 @@ class LinkdingE2ETestCase(LiveServerTestCase, BookmarkFactoryMixin):
def assertReloads(self, count: int):
self.assertEqual(self.num_loads, count)
def resetReloads(self):
self.num_loads = 0
def locate_bookmark_list(self):
return self.page.locator("ul[ld-bookmark-list]")
def locate_bookmark(self, title: str):
bookmark_tags = self.page.locator('li[ld-bookmark-item]')
bookmark_tags = self.page.locator("li[ld-bookmark-item]")
return bookmark_tags.filter(has_text=title)
def locate_details_modal(self):
return self.page.locator(".modal.bookmark-details")
def open_details_modal(self, bookmark):
details_button = self.locate_bookmark(bookmark.title).get_by_text("View")
details_button.click()
details_modal = self.locate_details_modal()
expect(details_modal).to_be_visible()
return details_modal
def locate_bulk_edit_bar(self):
return self.page.locator('.bulk-edit-bar')
return self.page.locator(".bulk-edit-bar")
def locate_bulk_edit_select_all(self):
return self.locate_bulk_edit_bar().locator('label[ld-bulk-edit-checkbox][all]')
return self.locate_bulk_edit_bar().locator("label.bulk-edit-checkbox.all")
def locate_bulk_edit_select_across(self):
return self.locate_bulk_edit_bar().locator('label.select-across')
return self.locate_bulk_edit_bar().locator("label.select-across")
def locate_bulk_edit_toggle(self):
return self.page.get_by_title('Bulk edit')
return self.page.get_by_title("Bulk edit")
def select_bulk_action(self, value: str):
return self.locate_bulk_edit_bar().locator('select[name="bulk_action"]').select_option(value)
return (
self.locate_bulk_edit_bar()
.locator('select[name="bulk_action"]')
.select_option(value)
)

View File

@@ -6,26 +6,32 @@ from django.db.models import QuerySet
from django.urls import reverse
from bookmarks import queries
from bookmarks.models import Bookmark, BookmarkSearch, FeedToken
from bookmarks.models import Bookmark, BookmarkSearch, FeedToken, UserProfile
@dataclass
class FeedContext:
feed_token: FeedToken
feed_token: FeedToken | None
query_set: QuerySet[Bookmark]
def sanitize(text: str):
if not text:
return ""
# remove control characters
valid_chars = ['\n', '\r', '\t']
return ''.join(ch for ch in text if ch in valid_chars or unicodedata.category(ch)[0] != 'C')
valid_chars = ["\n", "\r", "\t"]
return "".join(
ch for ch in text if ch in valid_chars or unicodedata.category(ch)[0] != "C"
)
class BaseBookmarksFeed(Feed):
def get_object(self, request, feed_key: str):
feed_token = FeedToken.objects.get(key__exact=feed_key)
search = BookmarkSearch(q=request.GET.get('q', ''))
query_set = queries.query_bookmarks(feed_token.user, feed_token.user.profile, search)
search = BookmarkSearch(q=request.GET.get("q", ""))
query_set = queries.query_bookmarks(
feed_token.user, feed_token.user.profile, search
)
return FeedContext(feed_token, query_set)
def item_title(self, item: Bookmark):
@@ -42,22 +48,58 @@ class BaseBookmarksFeed(Feed):
class AllBookmarksFeed(BaseBookmarksFeed):
title = 'All bookmarks'
description = 'All bookmarks'
title = "All bookmarks"
description = "All bookmarks"
def link(self, context: FeedContext):
return reverse('bookmarks:feeds.all', args=[context.feed_token.key])
return reverse("bookmarks:feeds.all", args=[context.feed_token.key])
def items(self, context: FeedContext):
return context.query_set
class UnreadBookmarksFeed(BaseBookmarksFeed):
title = 'Unread bookmarks'
description = 'All unread bookmarks'
title = "Unread bookmarks"
description = "All unread bookmarks"
def link(self, context: FeedContext):
return reverse('bookmarks:feeds.unread', args=[context.feed_token.key])
return reverse("bookmarks:feeds.unread", args=[context.feed_token.key])
def items(self, context: FeedContext):
return context.query_set.filter(unread=True)
class SharedBookmarksFeed(BaseBookmarksFeed):
title = "Shared bookmarks"
description = "All shared bookmarks"
def get_object(self, request, feed_key: str):
feed_token = FeedToken.objects.get(key__exact=feed_key)
search = BookmarkSearch(q=request.GET.get("q", ""))
query_set = queries.query_shared_bookmarks(
None, feed_token.user.profile, search, False
)
return FeedContext(feed_token, query_set)
def link(self, context: FeedContext):
return reverse("bookmarks:feeds.shared", args=[context.feed_token.key])
def items(self, context: FeedContext):
return context.query_set
class PublicSharedBookmarksFeed(BaseBookmarksFeed):
title = "Public shared bookmarks"
description = "All public shared bookmarks"
def get_object(self, request):
search = BookmarkSearch(q=request.GET.get("q", ""))
default_profile = UserProfile()
query_set = queries.query_shared_bookmarks(None, default_profile, search, True)
return FeedContext(None, query_set)
def link(self, context: FeedContext):
return reverse("bookmarks:feeds.public_shared")
def items(self, context: FeedContext):
return context.query_set

View File

@@ -1,68 +1,23 @@
import { registerBehavior, swap } from "./index";
import { Behavior, registerBehavior } from "./index";
class BookmarkPage {
class BookmarkItem extends Behavior {
constructor(element) {
this.element = element;
this.form = element.querySelector("form.bookmark-actions");
this.form.addEventListener("submit", this.onFormSubmit.bind(this));
this.bookmarkList = element.querySelector(".bookmark-list-container");
this.tagCloud = element.querySelector(".tag-cloud-container");
}
async onFormSubmit(event) {
event.preventDefault();
const url = this.form.action;
const formData = new FormData(this.form);
formData.append(event.submitter.name, event.submitter.value);
await fetch(url, {
method: "POST",
body: formData,
redirect: "manual", // ignore redirect
});
await this.refresh();
}
async refresh() {
const query = window.location.search;
const bookmarksUrl = this.element.getAttribute("bookmarks-url");
const tagsUrl = this.element.getAttribute("tags-url");
Promise.all([
fetch(`${bookmarksUrl}${query}`).then((response) => response.text()),
fetch(`${tagsUrl}${query}`).then((response) => response.text()),
]).then(([bookmarkListHtml, tagCloudHtml]) => {
swap(this.bookmarkList, bookmarkListHtml);
swap(this.tagCloud, tagCloudHtml);
// Dispatch list updated event
const listElement = this.bookmarkList.querySelector(
"ul[data-bookmarks-total]",
);
const bookmarksTotal =
(listElement && listElement.dataset.bookmarksTotal) || 0;
this.bookmarkList.dispatchEvent(
new CustomEvent("bookmark-list-updated", {
bubbles: true,
detail: { bookmarksTotal },
}),
);
});
}
}
registerBehavior("ld-bookmark-page", BookmarkPage);
class BookmarkItem {
constructor(element) {
this.element = element;
super(element);
// Toggle notes
const notesToggle = element.querySelector(".toggle-notes");
if (notesToggle) {
notesToggle.addEventListener("click", this.onToggleNotes.bind(this));
}
// Add tooltip to title if it is truncated
const titleAnchor = element.querySelector(".title > a");
const titleSpan = titleAnchor.querySelector("span");
requestAnimationFrame(() => {
if (titleSpan.offsetWidth > titleAnchor.offsetWidth) {
titleAnchor.dataset.tooltip = titleSpan.textContent;
}
});
}
onToggleNotes(event) {

View File

@@ -1,46 +1,60 @@
import { registerBehavior } from "./index";
import { Behavior, registerBehavior } from "./index";
class BulkEdit {
class BulkEdit extends Behavior {
constructor(element) {
this.element = element;
super(element);
this.active = false;
this.actionSelect = element.querySelector("select[name='bulk_action']");
this.tagAutoComplete = element.querySelector(".tag-autocomplete");
this.selectAcross = element.querySelector("label.select-across");
element.addEventListener(
"bulk-edit-toggle-active",
this.onToggleActive.bind(this),
);
element.addEventListener(
"bulk-edit-toggle-all",
this.onToggleAll.bind(this),
);
element.addEventListener(
"bulk-edit-toggle-bookmark",
this.onToggleBookmark.bind(this),
);
element.addEventListener(
"bookmark-list-updated",
this.onListUpdated.bind(this),
);
this.onToggleActive = this.onToggleActive.bind(this);
this.onToggleAll = this.onToggleAll.bind(this);
this.onToggleBookmark = this.onToggleBookmark.bind(this);
this.onActionSelected = this.onActionSelected.bind(this);
this.actionSelect.addEventListener(
"change",
this.onActionSelected.bind(this),
);
this.init();
// Reset when bookmarks are refreshed
document.addEventListener("refresh-bookmark-list-done", () => this.init());
}
get allCheckbox() {
return this.element.querySelector("[ld-bulk-edit-checkbox][all] input");
}
init() {
// Update elements
this.activeToggle = this.element.querySelector(".bulk-edit-active-toggle");
this.actionSelect = this.element.querySelector(
"select[name='bulk_action']",
);
this.tagAutoComplete = this.element.querySelector(".tag-autocomplete");
this.selectAcross = this.element.querySelector("label.select-across");
this.allCheckbox = this.element.querySelector(
".bulk-edit-checkbox.all input",
);
this.bookmarkCheckboxes = Array.from(
this.element.querySelectorAll(".bulk-edit-checkbox:not(.all) input"),
);
get bookmarkCheckboxes() {
return [
...this.element.querySelectorAll(
"[ld-bulk-edit-checkbox]:not([all]) input",
),
];
// Remove previous listeners if elements are the same
this.activeToggle.removeEventListener("click", this.onToggleActive);
this.actionSelect.removeEventListener("change", this.onActionSelected);
this.allCheckbox.removeEventListener("change", this.onToggleAll);
this.bookmarkCheckboxes.forEach((checkbox) => {
checkbox.removeEventListener("change", this.onToggleBookmark);
});
// Reset checkbox states
this.reset();
// Update total number of bookmarks
const totalHolder = this.element.querySelector("[data-bookmarks-total]");
const total = totalHolder?.dataset.bookmarksTotal || 0;
const totalSpan = this.selectAcross.querySelector("span.total");
totalSpan.textContent = total;
// Add new listeners
this.activeToggle.addEventListener("click", this.onToggleActive);
this.actionSelect.addEventListener("change", this.onActionSelected);
this.allCheckbox.addEventListener("change", this.onToggleAll);
this.bookmarkCheckboxes.forEach((checkbox) => {
checkbox.addEventListener("change", this.onToggleBookmark);
});
}
onToggleActive() {
@@ -81,16 +95,6 @@ class BulkEdit {
}
}
onListUpdated(event) {
// Reset checkbox states
this.reset();
// Update total number of bookmarks
const total = event.detail.bookmarksTotal;
const totalSpan = this.selectAcross.querySelector("span.total");
totalSpan.textContent = total;
}
updateSelectAcross(allChecked) {
if (allChecked) {
this.selectAcross.classList.remove("d-none");
@@ -109,33 +113,4 @@ class BulkEdit {
}
}
class BulkEditActiveToggle {
constructor(element) {
this.element = element;
element.addEventListener("click", this.onClick.bind(this));
}
onClick() {
this.element.dispatchEvent(
new CustomEvent("bulk-edit-toggle-active", { bubbles: true }),
);
}
}
class BulkEditCheckbox {
constructor(element) {
this.element = element;
element.addEventListener("change", this.onChange.bind(this));
}
onChange() {
const type = this.element.hasAttribute("all") ? "all" : "bookmark";
this.element.dispatchEvent(
new CustomEvent(`bulk-edit-toggle-${type}`, { bubbles: true }),
);
}
}
registerBehavior("ld-bulk-edit", BulkEdit);
registerBehavior("ld-bulk-edit-active-toggle", BulkEditActiveToggle);
registerBehavior("ld-bulk-edit-checkbox", BulkEditCheckbox);

View File

@@ -1,25 +1,29 @@
import { registerBehavior } from "./index";
import { Behavior, registerBehavior } from "./index";
class ConfirmButtonBehavior {
class ConfirmButtonBehavior extends Behavior {
constructor(element) {
const button = element;
button.dataset.type = button.type;
button.dataset.name = button.name;
button.dataset.value = button.value;
button.removeAttribute("type");
button.removeAttribute("name");
button.removeAttribute("value");
button.addEventListener("click", this.onClick.bind(this));
this.button = button;
super(element);
element.dataset.type = element.type;
element.dataset.name = element.name;
element.dataset.value = element.value;
element.removeAttribute("type");
element.removeAttribute("name");
element.removeAttribute("value");
element.addEventListener("click", this.onClick.bind(this));
}
destroy() {
Behavior.interacting = false;
}
onClick(event) {
event.preventDefault();
Behavior.interacting = true;
const container = document.createElement("span");
container.className = "confirmation";
const icon = this.button.getAttribute("confirm-icon");
const icon = this.element.getAttribute("ld-confirm-icon");
if (icon) {
const iconElement = document.createElementNS(
"http://www.w3.org/2000/svg",
@@ -31,38 +35,43 @@ class ConfirmButtonBehavior {
container.append(iconElement);
}
const question = this.button.getAttribute("confirm-question");
const question = this.element.getAttribute("ld-confirm-question");
if (question) {
const questionElement = document.createElement("span");
questionElement.innerText = question;
container.append(question);
}
const cancelButton = document.createElement(this.button.nodeName);
const buttonClasses = Array.from(this.element.classList.values())
.filter((cls) => cls.startsWith("btn"))
.join(" ");
const cancelButton = document.createElement(this.element.nodeName);
cancelButton.type = "button";
cancelButton.innerText = question ? "No" : "Cancel";
cancelButton.className = "btn btn-link btn-sm mr-1";
cancelButton.className = `${buttonClasses} mr-1`;
cancelButton.addEventListener("click", this.reset.bind(this));
const confirmButton = document.createElement(this.button.nodeName);
confirmButton.type = this.button.dataset.type;
confirmButton.name = this.button.dataset.name;
confirmButton.value = this.button.dataset.value;
const confirmButton = document.createElement(this.element.nodeName);
confirmButton.type = this.element.dataset.type;
confirmButton.name = this.element.dataset.name;
confirmButton.value = this.element.dataset.value;
confirmButton.innerText = question ? "Yes" : "Confirm";
confirmButton.className = "btn btn-link btn-sm";
confirmButton.className = buttonClasses;
confirmButton.addEventListener("click", this.reset.bind(this));
container.append(cancelButton, confirmButton);
this.container = container;
this.button.before(container);
this.button.classList.add("d-none");
this.element.before(container);
this.element.classList.add("d-none");
}
reset() {
setTimeout(() => {
Behavior.interacting = false;
this.container.remove();
this.button.classList.remove("d-none");
this.element.classList.remove("d-none");
});
}
}

View File

@@ -1,8 +1,8 @@
import { registerBehavior } from "./index";
import { Behavior, registerBehavior } from "./index";
class DropdownBehavior {
class DropdownBehavior extends Behavior {
constructor(element) {
this.element = element;
super(element);
this.opened = false;
this.onOutsideClick = this.onOutsideClick.bind(this);

View File

@@ -0,0 +1,48 @@
import { Behavior, fireEvents, registerBehavior, swap } from "./index";
class FetchBehavior extends Behavior {
constructor(element) {
super(element);
const eventName = element.getAttribute("ld-on");
const interval = parseInt(element.getAttribute("ld-interval")) * 1000;
this.onFetch = this.onFetch.bind(this);
this.onInterval = this.onInterval.bind(this);
element.addEventListener(eventName, this.onFetch);
if (interval) {
this.intervalId = setInterval(this.onInterval, interval);
}
}
destroy() {
if (this.intervalId) {
clearInterval(this.intervalId);
}
}
async onFetch(maybeEvent) {
if (maybeEvent) {
maybeEvent.preventDefault();
}
const url = this.element.getAttribute("ld-fetch");
const html = await fetch(url).then((response) => response.text());
const target = this.element.getAttribute("ld-target");
const select = this.element.getAttribute("ld-select");
swap(this.element, html, { target, select });
const events = this.element.getAttribute("ld-fire");
fireEvents(events);
}
onInterval() {
if (Behavior.interacting) {
return;
}
this.onFetch();
}
}
registerBehavior("ld-fetch", FetchBehavior);

View File

@@ -0,0 +1,64 @@
import { Behavior, fireEvents, registerBehavior } from "./index";
class FormBehavior extends Behavior {
constructor(element) {
super(element);
element.addEventListener("submit", this.onSubmit.bind(this));
}
async onSubmit(event) {
event.preventDefault();
const url = this.element.action;
const formData = new FormData(this.element);
if (event.submitter) {
formData.append(event.submitter.name, event.submitter.value);
}
await fetch(url, {
method: "POST",
body: formData,
redirect: "manual", // ignore redirect
});
const events = this.element.getAttribute("ld-fire");
if (fireEvents) {
fireEvents(events);
}
}
}
class AutoSubmitBehavior extends Behavior {
constructor(element) {
super(element);
element.addEventListener("change", () => {
const form = element.closest("form");
form.dispatchEvent(new Event("submit", { cancelable: true }));
});
}
}
class UploadButton extends Behavior {
constructor(element) {
super(element);
const fileInput = element.nextElementSibling;
element.addEventListener("click", () => {
fileInput.click();
});
fileInput.addEventListener("change", () => {
const form = fileInput.closest("form");
const event = new Event("submit", { cancelable: true });
event.submitter = element;
form.dispatchEvent(event);
});
}
}
registerBehavior("ld-form", FormBehavior);
registerBehavior("ld-auto-submit", AutoSubmitBehavior);
registerBehavior("ld-upload-button", UploadButton);

View File

@@ -1,7 +1,9 @@
import { registerBehavior } from "./index";
import { Behavior, registerBehavior } from "./index";
class GlobalShortcuts extends Behavior {
constructor(element) {
super(element);
class GlobalShortcuts {
constructor() {
document.addEventListener("keydown", this.onKeyDown.bind(this));
}

View File

@@ -1,4 +1,35 @@
const behaviorRegistry = {};
const debug = false;
const mutationObserver = new MutationObserver((mutations) => {
mutations.forEach((mutation) => {
mutation.removedNodes.forEach((node) => {
if (node instanceof HTMLElement && !node.isConnected) {
destroyBehaviors(node);
}
});
mutation.addedNodes.forEach((node) => {
if (node instanceof HTMLElement && node.isConnected) {
applyBehaviors(node);
}
});
});
});
mutationObserver.observe(document.body, {
childList: true,
subtree: true,
});
export class Behavior {
constructor(element) {
this.element = element;
}
destroy() {}
}
Behavior.interacting = false;
export function registerBehavior(name, behavior) {
behaviorRegistry[name] = behavior;
@@ -12,7 +43,14 @@ export function applyBehaviors(container, behaviorNames = null) {
behaviorNames.forEach((behaviorName) => {
const behavior = behaviorRegistry[behaviorName];
const elements = container.querySelectorAll(`[${behaviorName}]`);
const elements = Array.from(
container.querySelectorAll(`[${behaviorName}]`),
);
// Include the container element if it has the behavior
if (container.hasAttribute && container.hasAttribute(behaviorName)) {
elements.push(container);
}
elements.forEach((element) => {
element.__behaviors = element.__behaviors || [];
@@ -26,11 +64,82 @@ export function applyBehaviors(container, behaviorNames = null) {
const behaviorInstance = new behavior(element);
element.__behaviors.push(behaviorInstance);
if (debug) {
console.log(
`[Behavior] ${behaviorInstance.constructor.name} initialized`,
);
}
});
});
}
export function swap(element, html) {
element.innerHTML = html;
applyBehaviors(element);
export function destroyBehaviors(element) {
const behaviorNames = Object.keys(behaviorRegistry);
behaviorNames.forEach((behaviorName) => {
const elements = Array.from(element.querySelectorAll(`[${behaviorName}]`));
elements.push(element);
elements.forEach((element) => {
if (!element.__behaviors) {
return;
}
element.__behaviors.forEach((behavior) => {
behavior.destroy();
if (debug) {
console.log(`[Behavior] ${behavior.constructor.name} destroyed`);
}
});
delete element.__behaviors;
});
});
}
export function swap(element, html, options) {
const dom = new DOMParser().parseFromString(html, "text/html");
let targetElement = element;
let strategy = "innerHTML";
if (options.target) {
const parts = options.target.split("|");
targetElement =
parts[0] === "self" ? element : document.querySelector(parts[0]);
strategy = parts[1] || "innerHTML";
}
let contents = Array.from(dom.body.children);
if (options.select) {
contents = Array.from(dom.querySelectorAll(options.select));
}
switch (strategy) {
case "append":
targetElement.append(...contents);
break;
case "outerHTML":
targetElement.parentElement.replaceChild(contents[0], targetElement);
break;
case "innerHTML":
default:
Array.from(targetElement.children).forEach((child) => {
child.remove();
});
targetElement.append(...contents);
}
}
export function fireEvents(events) {
if (!events) {
return;
}
events.split(",").forEach((eventName) => {
const targets = Array.from(
document.querySelectorAll(`[ld-on='${eventName}']`),
);
targets.push(document);
targets.forEach((target) => {
target.dispatchEvent(new CustomEvent(eventName));
});
});
}

View File

@@ -1,64 +1,50 @@
import { registerBehavior } from "./index";
import { Behavior, registerBehavior } from "./index";
class ModalBehavior {
class ModalBehavior extends Behavior {
constructor(element) {
const toggle = element;
toggle.addEventListener("click", this.onToggleClick.bind(this));
this.toggle = toggle;
super(element);
this.onClose = this.onClose.bind(this);
this.onKeyDown = this.onKeyDown.bind(this);
const modalOverlay = element.querySelector(".modal-overlay");
const closeButton = element.querySelector("button.close");
modalOverlay.addEventListener("click", this.onClose);
closeButton.addEventListener("click", this.onClose);
document.addEventListener("keydown", this.onKeyDown);
}
onToggleClick() {
const contentSelector = this.toggle.getAttribute("modal-content");
const content = document.querySelector(contentSelector);
if (!content) {
destroy() {
document.removeEventListener("keydown", this.onKeyDown);
}
onKeyDown(event) {
// Skip if event occurred within an input element
const targetNodeName = event.target.nodeName;
const isInputTarget =
targetNodeName === "INPUT" ||
targetNodeName === "SELECT" ||
targetNodeName === "TEXTAREA";
if (isInputTarget) {
return;
}
// Create modal
const modal = document.createElement("div");
modal.classList.add("modal", "active");
modal.innerHTML = `
<div class="modal-overlay" aria-label="Close"></div>
<div class="modal-container">
<div class="modal-header d-flex justify-between align-center">
<div class="modal-title h5">Tags</div>
<button class="btn btn-link close">
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M18 6l-12 12"></path>
<path d="M6 6l12 12"></path>
</svg>
</button>
</div>
<div class="modal-body">
<div class="content"></div>
</div>
</div>
`;
// Teleport content element
const contentOwner = content.parentElement;
const contentContainer = modal.querySelector(".content");
contentContainer.append(content);
this.content = content;
this.contentOwner = contentOwner;
// Register close handlers
const modalOverlay = modal.querySelector(".modal-overlay");
const closeButton = modal.querySelector(".btn.close");
modalOverlay.addEventListener("click", this.onClose.bind(this));
closeButton.addEventListener("click", this.onClose.bind(this));
document.body.append(modal);
this.modal = modal;
if (event.key === "Escape") {
event.preventDefault();
this.onClose();
}
}
onClose() {
// Teleport content back
this.contentOwner.append(this.content);
// Remove modal
this.modal.remove();
document.removeEventListener("keydown", this.onKeyDown);
this.element.classList.add("closing");
this.element.addEventListener("animationend", (event) => {
if (event.animationName === "fade-out") {
this.element.remove();
}
});
}
}

View File

@@ -1,9 +1,10 @@
import { registerBehavior } from "./index";
import { Behavior, registerBehavior } from "./index";
import TagAutoCompleteComponent from "../components/TagAutocomplete.svelte";
import { ApiClient } from "../api";
class TagAutocomplete {
class TagAutocomplete extends Behavior {
constructor(element) {
super(element);
const wrapper = document.createElement("div");
const apiBaseUrl = document.documentElement.dataset.apiBaseUrl || "";
const apiClient = new ApiClient(apiBaseUrl);

View File

@@ -258,4 +258,4 @@
z-index: 2;
}
</style>
</style>

View File

@@ -22,7 +22,7 @@
async function init() {
// For now we cache all tags on load as the template did before
try {
tags = await apiClient.getTags({limit: 1000, offset: 0});
tags = await apiClient.getTags({limit: 5000, offset: 0});
tags.sort((left, right) => left.name.toLowerCase().localeCompare(right.name.toLowerCase()))
} catch (e) {
console.warn('TagAutocomplete: Error loading tag list');
@@ -150,19 +150,31 @@
display: block;
}
.form-autocomplete-input {
box-sizing: border-box;
height: var(--control-size);
min-height: var(--control-size);
padding: 0;
}
.form-autocomplete-input input {
width: 100%;
height: 100%;
border: none;
margin: 0;
}
.form-autocomplete.small .form-autocomplete-input {
height: 1.4rem;
min-height: 1.4rem;
padding: 0.05rem 0.3rem;
height: var(--control-size-sm);
min-height: var(--control-size-sm);
}
.form-autocomplete.small .form-autocomplete-input input {
margin: 0;
padding: 0;
font-size: 0.7rem;
padding: 0.05rem 0.3rem;
font-size: var(--font-size-sm);
}
.form-autocomplete.small .menu .menu-item {
font-size: 0.7rem;
font-size: var(--font-size-sm);
}
</style>

View File

@@ -1,16 +1,12 @@
import TagAutoComplete from "./components/TagAutocomplete.svelte";
import SearchAutoComplete from "./components/SearchAutoComplete.svelte";
import { ApiClient } from "./api";
import "./behaviors/bookmark-page";
import "./behaviors/bulk-edit";
import "./behaviors/confirm-button";
import "./behaviors/dropdown";
import "./behaviors/fetch";
import "./behaviors/form";
import "./behaviors/modal";
import "./behaviors/global-shortcuts";
import "./behaviors/tag-autocomplete";
export default {
ApiClient,
TagAutoComplete,
SearchAutoComplete,
};
export { default as TagAutoComplete } from "./components/TagAutocomplete.svelte";
export { default as SearchAutoComplete } from "./components/SearchAutoComplete.svelte";
export { ApiClient } from "./api";

View File

@@ -0,0 +1,31 @@
import sqlite3
import os
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = "Creates a backup of the linkding database"
def add_arguments(self, parser):
parser.add_argument("destination", type=str, help="Backup file destination")
def handle(self, *args, **options):
destination = options["destination"]
def progress(status, remaining, total):
self.stdout.write(f"Copied {total-remaining} of {total} pages...")
source_db = sqlite3.connect(os.path.join("data", "db.sqlite3"))
backup_db = sqlite3.connect(destination)
with backup_db:
source_db.backup(backup_db, pages=50, progress=progress)
backup_db.close()
source_db.close()
self.stdout.write(self.style.SUCCESS(f"Backup created at {destination}"))
self.stdout.write(
self.style.WARNING(
"This backup method is deprecated and may be removed in the future. Please use the full_backup command instead, which creates backup zip file with all contents of the data folder."
)
)

View File

@@ -1,15 +0,0 @@
from background_task.models import Task, CompletedTask
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = "Remove task locks and clear completed task history"
def handle(self, *args, **options):
# Remove task locks
# If the background task processor exited while executing tasks, these tasks would still be marked as locked,
# even though no process is working on them, and would prevent the task processor from picking the next task in
# the queue
Task.objects.all().update(locked_by=None, locked_at=None)
# Clear task history to prevent them from bloating the DB
CompletedTask.objects.all().delete()

View File

@@ -12,18 +12,20 @@ class Command(BaseCommand):
def handle(self, *args, **options):
User = get_user_model()
superuser_name = os.getenv('LD_SUPERUSER_NAME', None)
superuser_password = os.getenv('LD_SUPERUSER_PASSWORD', None)
superuser_name = os.getenv("LD_SUPERUSER_NAME", None)
superuser_password = os.getenv("LD_SUPERUSER_PASSWORD", None)
# Skip if option is undefined
if not superuser_name:
logger.info('Skip creating initial superuser, LD_SUPERUSER_NAME option is not defined')
logger.info(
"Skip creating initial superuser, LD_SUPERUSER_NAME option is not defined"
)
return
# Skip if user already exists
user_exists = User.objects.filter(username=superuser_name).exists()
if user_exists:
logger.info('Skip creating initial superuser, user already exists')
logger.info("Skip creating initial superuser, user already exists")
return
user = User(username=superuser_name, is_superuser=True, is_staff=True)
@@ -34,4 +36,4 @@ class Command(BaseCommand):
user.set_unusable_password()
user.save()
logger.info('Created initial superuser')
logger.info("Created initial superuser")

View File

@@ -14,11 +14,11 @@ class Command(BaseCommand):
if not settings.USE_SQLITE:
return
connection = connections['default']
connection = connections["default"]
with connection.cursor() as cursor:
cursor.execute("PRAGMA journal_mode")
current_mode = cursor.fetchone()[0]
logger.info(f'Current journal mode: {current_mode}')
if current_mode != 'wal':
logger.info(f"Current journal mode: {current_mode}")
if current_mode != "wal":
cursor.execute("PRAGMA journal_mode=wal;")
logger.info('Switched to WAL journal mode')
logger.info("Switched to WAL journal mode")

View File

@@ -6,13 +6,15 @@ class Command(BaseCommand):
help = "Creates an admin user non-interactively if it doesn't exist"
def add_arguments(self, parser):
parser.add_argument('--username', help="Admin's username")
parser.add_argument('--email', help="Admin's email")
parser.add_argument('--password', help="Admin's password")
parser.add_argument("--username", help="Admin's username")
parser.add_argument("--email", help="Admin's email")
parser.add_argument("--password", help="Admin's password")
def handle(self, *args, **options):
User = get_user_model()
if not User.objects.filter(username=options['username']).exists():
User.objects.create_superuser(username=options['username'],
email=options['email'],
password=options['password'])
if not User.objects.filter(username=options["username"]).exists():
User.objects.create_superuser(
username=options["username"],
email=options["email"],
password=options["password"],
)

View File

@@ -0,0 +1,75 @@
import sqlite3
import os
import tempfile
import zipfile
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = "Creates a backup of the linkding data folder"
def add_arguments(self, parser):
parser.add_argument("backup_file", type=str, help="Backup zip file destination")
def handle(self, *args, **options):
backup_file = options["backup_file"]
with zipfile.ZipFile(backup_file, "w", zipfile.ZIP_DEFLATED) as zip_file:
# Backup the database
self.stdout.write("Create database backup...")
with tempfile.TemporaryDirectory() as temp_dir:
backup_db_file = os.path.join(temp_dir, "db.sqlite3")
self.backup_database(backup_db_file)
zip_file.write(backup_db_file, "db.sqlite3")
# Backup the assets folder
if not os.path.exists(os.path.join("data", "assets")):
self.stdout.write(
self.style.WARNING("No assets folder found. Skipping...")
)
else:
self.stdout.write("Backup bookmark assets...")
assets_folder = os.path.join("data", "assets")
for root, _, files in os.walk(assets_folder):
for file in files:
file_path = os.path.join(root, file)
zip_file.write(file_path, os.path.join("assets", file))
# Backup the favicons folder
if not os.path.exists(os.path.join("data", "favicons")):
self.stdout.write(
self.style.WARNING("No favicons folder found. Skipping...")
)
else:
self.stdout.write("Backup bookmark favicons...")
favicons_folder = os.path.join("data", "favicons")
for root, _, files in os.walk(favicons_folder):
for file in files:
file_path = os.path.join(root, file)
zip_file.write(file_path, os.path.join("favicons", file))
# Backup the previews folder
if not os.path.exists(os.path.join("data", "previews")):
self.stdout.write(
self.style.WARNING("No previews folder found. Skipping...")
)
else:
self.stdout.write("Backup bookmark previews...")
previews_folder = os.path.join("data", "previews")
for root, _, files in os.walk(previews_folder):
for file in files:
file_path = os.path.join(root, file)
zip_file.write(file_path, os.path.join("previews", file))
self.stdout.write(self.style.SUCCESS(f"Backup created at {backup_file}"))
def backup_database(self, backup_db_file):
def progress(status, remaining, total):
self.stdout.write(f"Copied {total-remaining} of {total} pages...")
source_db = sqlite3.connect(os.path.join("data", "db.sqlite3"))
backup_db = sqlite3.connect(backup_db_file)
with backup_db:
source_db.backup(backup_db, pages=50, progress=progress)
backup_db.close()
source_db.close()

View File

@@ -0,0 +1,24 @@
import logging
import os
from django.core.management.base import BaseCommand
from django.core.management.utils import get_random_secret_key
logger = logging.getLogger(__name__)
class Command(BaseCommand):
help = "Generate secret key file if it does not exist"
def handle(self, *args, **options):
secret_key_file = os.path.join("data", "secretkey.txt")
if os.path.exists(secret_key_file):
logger.info(f"Secret key file already exists")
return
secret_key = get_random_secret_key()
with open(secret_key_file, "w") as f:
f.write(secret_key)
logger.info(f"Generated secret key file")

View File

@@ -5,15 +5,17 @@ from bookmarks.services.importer import import_netscape_html
class Command(BaseCommand):
help = 'Import Netscape HTML bookmark file'
help = "Import Netscape HTML bookmark file"
def add_arguments(self, parser):
parser.add_argument('file', type=str, help='Path to file')
parser.add_argument('user', type=str, help='Name of the user for which to import')
parser.add_argument("file", type=str, help="Path to file")
parser.add_argument(
"user", type=str, help="Name of the user for which to import"
)
def handle(self, *args, **kwargs):
filepath = kwargs['file']
username = kwargs['user']
filepath = kwargs["file"]
username = kwargs["user"]
with open(filepath) as html_file:
html = html_file.read()
user = User.objects.get(username=username)

View File

@@ -0,0 +1,75 @@
import json
import os
import sqlite3
import importlib
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = "Migrate tasks from django-background-tasks to Huey"
def handle(self, *args, **options):
db = sqlite3.connect(os.path.join("data", "db.sqlite3"))
# Check if background_task table exists
cursor = db.cursor()
cursor.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='background_task'"
)
row = cursor.fetchone()
if not row:
self.stdout.write(
"Legacy task table does not exist. Skipping task migration"
)
return
# Load legacy tasks
cursor.execute("SELECT id, task_name, task_params FROM background_task")
legacy_tasks = cursor.fetchall()
if len(legacy_tasks) == 0:
self.stdout.write("No legacy tasks found. Skipping task migration")
return
self.stdout.write(
f"Found {len(legacy_tasks)} legacy tasks. Migrating to Huey..."
)
# Migrate tasks to Huey
succeeded_tasks = []
for task in legacy_tasks:
task_id = task[0]
task_name = task[1]
task_params_json = task[2]
try:
task_params = json.loads(task_params_json)
function_params = task_params[0]
# Resolve task function
module_name, func_name = task_name.rsplit(".", 1)
module = importlib.import_module(module_name)
func = getattr(module, func_name)
# Call task function
func(*function_params)
succeeded_tasks.append(task_id)
except Exception:
self.stderr.write(f"Error migrating task [{task_id}] {task_name}")
self.stdout.write(f"Migrated {len(succeeded_tasks)} tasks successfully")
# Clean up
try:
placeholders = ", ".join("?" for _ in succeeded_tasks)
sql = f"DELETE FROM background_task WHERE id IN ({placeholders})"
cursor.execute(sql, succeeded_tasks)
db.commit()
self.stdout.write(
f"Deleted {len(succeeded_tasks)} migrated tasks from legacy table"
)
except Exception:
self.stderr.write("Error cleaning up legacy tasks")
cursor.close()
db.close()

View File

@@ -15,19 +15,36 @@ class Migration(migrations.Migration):
operations = [
migrations.CreateModel(
name='Bookmark',
name="Bookmark",
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('url', models.URLField()),
('title', models.CharField(max_length=512)),
('description', models.TextField()),
('website_title', models.CharField(blank=True, max_length=512, null=True)),
('website_description', models.TextField(blank=True, null=True)),
('unread', models.BooleanField(default=True)),
('date_added', models.DateTimeField()),
('date_modified', models.DateTimeField()),
('date_accessed', models.DateTimeField(blank=True, null=True)),
('owner', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("url", models.URLField()),
("title", models.CharField(max_length=512)),
("description", models.TextField()),
(
"website_title",
models.CharField(blank=True, max_length=512, null=True),
),
("website_description", models.TextField(blank=True, null=True)),
("unread", models.BooleanField(default=True)),
("date_added", models.DateTimeField()),
("date_modified", models.DateTimeField()),
("date_accessed", models.DateTimeField(blank=True, null=True)),
(
"owner",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
),
],
),
]

View File

@@ -9,22 +9,36 @@ class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('bookmarks', '0001_initial'),
("bookmarks", "0001_initial"),
]
operations = [
migrations.CreateModel(
name='Tag',
name="Tag",
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('name', models.CharField(max_length=64)),
('date_added', models.DateTimeField()),
('owner', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("name", models.CharField(max_length=64)),
("date_added", models.DateTimeField()),
(
"owner",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
),
],
),
migrations.AddField(
model_name='bookmark',
name='tags',
field=models.ManyToManyField(to='bookmarks.Tag'),
model_name="bookmark",
name="tags",
field=models.ManyToManyField(to="bookmarks.Tag"),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0002_auto_20190629_2303'),
("bookmarks", "0002_auto_20190629_2303"),
]
operations = [
migrations.AlterField(
model_name='bookmark',
name='url',
model_name="bookmark",
name="url",
field=models.URLField(max_length=2048),
),
]

View File

@@ -6,18 +6,18 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0003_auto_20200913_0656'),
("bookmarks", "0003_auto_20200913_0656"),
]
operations = [
migrations.AlterField(
model_name='bookmark',
name='description',
model_name="bookmark",
name="description",
field=models.TextField(blank=True),
),
migrations.AlterField(
model_name='bookmark',
name='title',
model_name="bookmark",
name="title",
field=models.CharField(blank=True, max_length=512),
),
]

View File

@@ -7,13 +7,16 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0004_auto_20200926_1028'),
("bookmarks", "0004_auto_20200926_1028"),
]
operations = [
migrations.AlterField(
model_name='bookmark',
name='url',
field=models.CharField(max_length=2048, validators=[bookmarks.validators.BookmarkURLValidator()]),
model_name="bookmark",
name="url",
field=models.CharField(
max_length=2048,
validators=[bookmarks.validators.BookmarkURLValidator()],
),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0005_auto_20210103_1212'),
("bookmarks", "0005_auto_20210103_1212"),
]
operations = [
migrations.AddField(
model_name='bookmark',
name='is_archived',
model_name="bookmark",
name="is_archived",
field=models.BooleanField(default=False),
),
]

View File

@@ -6,8 +6,8 @@ import django.db.models.deletion
def forwards(apps, schema_editor):
User = apps.get_model('auth', 'User')
UserProfile = apps.get_model('bookmarks', 'UserProfile')
User = apps.get_model("auth", "User")
UserProfile = apps.get_model("bookmarks", "UserProfile")
for user in User.objects.all():
try:
if user.profile:
@@ -24,19 +24,42 @@ def reverse(apps, schema_editor):
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('bookmarks', '0006_bookmark_is_archived'),
("bookmarks", "0006_bookmark_is_archived"),
]
operations = [
migrations.CreateModel(
name='UserProfile',
name="UserProfile",
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('theme',
models.CharField(choices=[('auto', 'Auto'), ('light', 'Light'), ('dark', 'Dark')], default='auto',
max_length=10)),
('user', models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, related_name='profile',
to=settings.AUTH_USER_MODEL)),
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
(
"theme",
models.CharField(
choices=[
("auto", "Auto"),
("light", "Light"),
("dark", "Dark"),
],
default="auto",
max_length=10,
),
),
(
"user",
models.OneToOneField(
on_delete=django.db.models.deletion.CASCADE,
related_name="profile",
to=settings.AUTH_USER_MODEL,
),
),
],
),
migrations.RunPython(forwards, reverse),

View File

@@ -6,13 +6,21 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0007_userprofile'),
("bookmarks", "0007_userprofile"),
]
operations = [
migrations.AddField(
model_name='userprofile',
name='bookmark_date_display',
field=models.CharField(choices=[('relative', 'Relative'), ('absolute', 'Absolute'), ('hidden', 'Hidden')], default='relative', max_length=10),
model_name="userprofile",
name="bookmark_date_display",
field=models.CharField(
choices=[
("relative", "Relative"),
("absolute", "Absolute"),
("hidden", "Hidden"),
],
default="relative",
max_length=10,
),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0008_userprofile_bookmark_date_display'),
("bookmarks", "0008_userprofile_bookmark_date_display"),
]
operations = [
migrations.AddField(
model_name='bookmark',
name='web_archive_snapshot_url',
model_name="bookmark",
name="web_archive_snapshot_url",
field=models.CharField(blank=True, max_length=2048),
),
]

View File

@@ -6,13 +6,17 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0009_bookmark_web_archive_snapshot_url'),
("bookmarks", "0009_bookmark_web_archive_snapshot_url"),
]
operations = [
migrations.AddField(
model_name='userprofile',
name='bookmark_link_target',
field=models.CharField(choices=[('_blank', 'New page'), ('_self', 'Same page')], default='_blank', max_length=10),
model_name="userprofile",
name="bookmark_link_target",
field=models.CharField(
choices=[("_blank", "New page"), ("_self", "Same page")],
default="_blank",
max_length=10,
),
),
]

View File

@@ -6,13 +6,17 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0010_userprofile_bookmark_link_target'),
("bookmarks", "0010_userprofile_bookmark_link_target"),
]
operations = [
migrations.AddField(
model_name='userprofile',
name='web_archive_integration',
field=models.CharField(choices=[('disabled', 'Disabled'), ('enabled', 'Enabled')], default='disabled', max_length=10),
model_name="userprofile",
name="web_archive_integration",
field=models.CharField(
choices=[("disabled", "Disabled"), ("enabled", "Enabled")],
default="disabled",
max_length=10,
),
),
]

View File

@@ -9,18 +9,32 @@ class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('bookmarks', '0011_userprofile_web_archive_integration'),
("bookmarks", "0011_userprofile_web_archive_integration"),
]
operations = [
migrations.CreateModel(
name='Toast',
name="Toast",
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('key', models.CharField(max_length=50)),
('message', models.TextField()),
('acknowledged', models.BooleanField(default=False)),
('owner', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("key", models.CharField(max_length=50)),
("message", models.TextField()),
("acknowledged", models.BooleanField(default=False)),
(
"owner",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
to=settings.AUTH_USER_MODEL,
),
),
],
),
]

View File

@@ -10,19 +10,21 @@ User = get_user_model()
def forwards(apps, schema_editor):
for user in User.objects.all():
toast = Toast(key='web_archive_opt_in_hint',
message='The Internet Archive Wayback Machine integration has been disabled by default. Check the Settings to re-enable it.',
owner=user)
toast = Toast(
key="web_archive_opt_in_hint",
message="The Internet Archive Wayback Machine integration has been disabled by default. Check the Settings to re-enable it.",
owner=user,
)
toast.save()
def reverse(apps, schema_editor):
Toast.objects.filter(key='web_archive_opt_in_hint').delete()
Toast.objects.filter(key="web_archive_opt_in_hint").delete()
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0012_toast'),
("bookmarks", "0012_toast"),
]
operations = [

View File

@@ -4,7 +4,7 @@ from django.db import migrations, models
def forwards(apps, schema_editor):
Bookmark = apps.get_model('bookmarks', 'Bookmark')
Bookmark = apps.get_model("bookmarks", "Bookmark")
Bookmark.objects.update(unread=False)
@@ -14,13 +14,13 @@ def reverse(apps, schema_editor):
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0013_web_archive_optin_toast'),
("bookmarks", "0013_web_archive_optin_toast"),
]
operations = [
migrations.AlterField(
model_name='bookmark',
name='unread',
model_name="bookmark",
name="unread",
field=models.BooleanField(default=False),
),
migrations.RunPython(forwards, reverse),

View File

@@ -9,16 +9,26 @@ class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('bookmarks', '0014_alter_bookmark_unread'),
("bookmarks", "0014_alter_bookmark_unread"),
]
operations = [
migrations.CreateModel(
name='FeedToken',
name="FeedToken",
fields=[
('key', models.CharField(max_length=40, primary_key=True, serialize=False)),
('created', models.DateTimeField(auto_now_add=True)),
('user', models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, related_name='feed_token', to=settings.AUTH_USER_MODEL)),
(
"key",
models.CharField(max_length=40, primary_key=True, serialize=False),
),
("created", models.DateTimeField(auto_now_add=True)),
(
"user",
models.OneToOneField(
on_delete=django.db.models.deletion.CASCADE,
related_name="feed_token",
to=settings.AUTH_USER_MODEL,
),
),
],
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0015_feedtoken'),
("bookmarks", "0015_feedtoken"),
]
operations = [
migrations.AddField(
model_name='bookmark',
name='shared',
model_name="bookmark",
name="shared",
field=models.BooleanField(default=False),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0016_bookmark_shared'),
("bookmarks", "0016_bookmark_shared"),
]
operations = [
migrations.AddField(
model_name='userprofile',
name='enable_sharing',
model_name="userprofile",
name="enable_sharing",
field=models.BooleanField(default=False),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0017_userprofile_enable_sharing'),
("bookmarks", "0017_userprofile_enable_sharing"),
]
operations = [
migrations.AddField(
model_name='bookmark',
name='favicon_file',
model_name="bookmark",
name="favicon_file",
field=models.CharField(blank=True, max_length=512),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0018_bookmark_favicon_file'),
("bookmarks", "0018_bookmark_favicon_file"),
]
operations = [
migrations.AddField(
model_name='userprofile',
name='enable_favicons',
model_name="userprofile",
name="enable_favicons",
field=models.BooleanField(default=False),
),
]

View File

@@ -6,13 +6,17 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0019_userprofile_enable_favicons'),
("bookmarks", "0019_userprofile_enable_favicons"),
]
operations = [
migrations.AddField(
model_name='userprofile',
name='tag_search',
field=models.CharField(choices=[('strict', 'Strict'), ('lax', 'Lax')], default='strict', max_length=10),
model_name="userprofile",
name="tag_search",
field=models.CharField(
choices=[("strict", "Strict"), ("lax", "Lax")],
default="strict",
max_length=10,
),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0020_userprofile_tag_search'),
("bookmarks", "0020_userprofile_tag_search"),
]
operations = [
migrations.AddField(
model_name='userprofile',
name='display_url',
model_name="userprofile",
name="display_url",
field=models.BooleanField(default=False),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0021_userprofile_display_url'),
("bookmarks", "0021_userprofile_display_url"),
]
operations = [
migrations.AddField(
model_name='bookmark',
name='notes',
model_name="bookmark",
name="notes",
field=models.TextField(blank=True),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0022_bookmark_notes'),
("bookmarks", "0022_bookmark_notes"),
]
operations = [
migrations.AddField(
model_name='userprofile',
name='permanent_notes',
model_name="userprofile",
name="permanent_notes",
field=models.BooleanField(default=False),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0023_userprofile_permanent_notes'),
("bookmarks", "0023_userprofile_permanent_notes"),
]
operations = [
migrations.AddField(
model_name='userprofile',
name='enable_public_sharing',
model_name="userprofile",
name="enable_public_sharing",
field=models.BooleanField(default=False),
),
]

View File

@@ -6,13 +6,13 @@ from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('bookmarks', '0024_userprofile_enable_public_sharing'),
("bookmarks", "0024_userprofile_enable_public_sharing"),
]
operations = [
migrations.AddField(
model_name='userprofile',
name='search_preferences',
model_name="userprofile",
name="search_preferences",
field=models.JSONField(default=dict),
),
]

View File

@@ -0,0 +1,18 @@
# Generated by Django 5.0.2 on 2024-03-16 23:05
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0025_userprofile_search_preferences"),
]
operations = [
migrations.AddField(
model_name="userprofile",
name="custom_css",
field=models.TextField(blank=True),
),
]

View File

@@ -0,0 +1,27 @@
# Generated by Django 5.0.2 on 2024-03-23 21:48
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0026_userprofile_custom_css"),
]
operations = [
migrations.AddField(
model_name="userprofile",
name="bookmark_description_display",
field=models.CharField(
choices=[("inline", "Inline"), ("separate", "Separate")],
default="inline",
max_length=10,
),
),
migrations.AddField(
model_name="userprofile",
name="bookmark_description_max_lines",
field=models.IntegerField(default=1),
),
]

View File

@@ -0,0 +1,33 @@
# Generated by Django 5.0.2 on 2024-03-29 20:05
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0027_userprofile_bookmark_description_display_and_more"),
]
operations = [
migrations.AddField(
model_name="userprofile",
name="display_archive_bookmark_action",
field=models.BooleanField(default=True),
),
migrations.AddField(
model_name="userprofile",
name="display_edit_bookmark_action",
field=models.BooleanField(default=True),
),
migrations.AddField(
model_name="userprofile",
name="display_remove_bookmark_action",
field=models.BooleanField(default=True),
),
migrations.AddField(
model_name="userprofile",
name="display_view_bookmark_action",
field=models.BooleanField(default=True),
),
]

View File

@@ -0,0 +1,34 @@
# Generated by Django 5.0.2 on 2024-03-29 21:25
from django.db import migrations
from django.contrib.auth import get_user_model
from bookmarks.models import Toast
User = get_user_model()
def forwards(apps, schema_editor):
for user in User.objects.all():
toast = Toast(
key="bookmark_list_actions_hint",
message="This version adds a new link to each bookmark to view details in a dialog. If you feel there is too much clutter you can now hide individual links in the settings.",
owner=user,
)
toast.save()
def reverse(apps, schema_editor):
Toast.objects.filter(key="bookmark_list_actions_hint").delete()
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0028_userprofile_display_archive_bookmark_action_and_more"),
]
operations = [
migrations.RunPython(forwards, reverse),
]

View File

@@ -0,0 +1,43 @@
# Generated by Django 5.0.2 on 2024-03-31 08:21
import django.db.models.deletion
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0029_bookmark_list_actions_toast"),
]
operations = [
migrations.CreateModel(
name="BookmarkAsset",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("date_created", models.DateTimeField(auto_now_add=True)),
("file", models.CharField(blank=True, max_length=2048)),
("file_size", models.IntegerField(null=True)),
("asset_type", models.CharField(max_length=64)),
("content_type", models.CharField(max_length=128)),
("display_name", models.CharField(blank=True, max_length=2048)),
("status", models.CharField(max_length=64)),
("gzip", models.BooleanField(default=False)),
(
"bookmark",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
to="bookmarks.bookmark",
),
),
],
),
]

View File

@@ -0,0 +1,18 @@
# Generated by Django 5.0.2 on 2024-04-01 10:29
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0030_bookmarkasset"),
]
operations = [
migrations.AddField(
model_name="userprofile",
name="enable_automatic_html_snapshots",
field=models.BooleanField(default=True),
),
]

View File

@@ -0,0 +1,34 @@
# Generated by Django 5.0.2 on 2024-04-01 12:17
from django.db import migrations
from django.contrib.auth import get_user_model
from bookmarks.models import Toast
User = get_user_model()
def forwards(apps, schema_editor):
for user in User.objects.all():
toast = Toast(
key="html_snapshots_hint",
message="This version adds a new feature for archiving snapshots of websites locally. To use it, you need to switch to a different Docker image. See the installation instructions on GitHub for details.",
owner=user,
)
toast.save()
def reverse(apps, schema_editor):
Toast.objects.filter(key="bookmark_list_actions_hint").delete()
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0031_userprofile_enable_automatic_html_snapshots"),
]
operations = [
migrations.RunPython(forwards, reverse),
]

View File

@@ -0,0 +1,18 @@
# Generated by Django 5.0.3 on 2024-04-17 19:27
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0032_html_snapshots_hint_toast"),
]
operations = [
migrations.AddField(
model_name="userprofile",
name="default_mark_unread",
field=models.BooleanField(default=False),
),
]

View File

@@ -0,0 +1,23 @@
# Generated by Django 5.0.3 on 2024-05-10 07:01
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0033_userprofile_default_mark_unread"),
]
operations = [
migrations.AddField(
model_name="bookmark",
name="preview_image_file",
field=models.CharField(blank=True, max_length=512),
),
migrations.AddField(
model_name="userprofile",
name="enable_preview_images",
field=models.BooleanField(default=False),
),
]

View File

@@ -0,0 +1,22 @@
# Generated by Django 5.0.3 on 2024-05-14 08:28
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0034_bookmark_preview_image_file_and_more"),
]
operations = [
migrations.AddField(
model_name="userprofile",
name="tag_grouping",
field=models.CharField(
choices=[("alphabetical", "Alphabetical"), ("disabled", "Disabled")],
default="alphabetical",
max_length=12,
),
),
]

View File

@@ -0,0 +1,18 @@
# Generated by Django 5.0.3 on 2024-05-17 07:09
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("bookmarks", "0035_userprofile_tag_grouping"),
]
operations = [
migrations.AddField(
model_name="userprofile",
name="auto_tagging_rules",
field=models.TextField(blank=True),
),
]

View File

@@ -1,18 +1,22 @@
import binascii
import logging
import os
from typing import List
import binascii
from django import forms
from django.conf import settings
from django.contrib.auth import get_user_model
from django.contrib.auth.models import User
from django.db import models
from django.db.models.signals import post_save
from django.db.models.signals import post_save, post_delete
from django.dispatch import receiver
from django.http import QueryDict
from bookmarks.utils import unique
from bookmarks.validators import BookmarkURLValidator
logger = logging.getLogger(__name__)
class Tag(models.Model):
name = models.CharField(max_length=64)
@@ -26,10 +30,10 @@ class Tag(models.Model):
def sanitize_tag_name(tag_name: str):
# strip leading/trailing spaces
# replace inner spaces with replacement char
return tag_name.strip().replace(' ', '-')
return tag_name.strip().replace(" ", "-")
def parse_tag_string(tag_string: str, delimiter: str = ','):
def parse_tag_string(tag_string: str, delimiter: str = ","):
if not tag_string:
return []
names = tag_string.strip().split(delimiter)
@@ -42,7 +46,7 @@ def parse_tag_string(tag_string: str, delimiter: str = ','):
return names
def build_tag_string(tag_names: List[str], delimiter: str = ','):
def build_tag_string(tag_names: List[str], delimiter: str = ","):
return delimiter.join(tag_names)
@@ -55,6 +59,7 @@ class Bookmark(models.Model):
website_description = models.TextField(blank=True, null=True)
web_archive_snapshot_url = models.CharField(max_length=2048, blank=True)
favicon_file = models.CharField(max_length=512, blank=True)
preview_image_file = models.CharField(max_length=512, blank=True)
unread = models.BooleanField(default=False)
is_archived = models.BooleanField(default=False)
shared = models.BooleanField(default=False)
@@ -82,7 +87,52 @@ class Bookmark(models.Model):
return [tag.name for tag in self.tags.all()]
def __str__(self):
return self.resolved_title + ' (' + self.url[:30] + '...)'
return self.resolved_title + " (" + self.url[:30] + "...)"
class BookmarkAsset(models.Model):
TYPE_SNAPSHOT = "snapshot"
TYPE_UPLOAD = "upload"
CONTENT_TYPE_HTML = "text/html"
STATUS_PENDING = "pending"
STATUS_COMPLETE = "complete"
STATUS_FAILURE = "failure"
bookmark = models.ForeignKey(Bookmark, on_delete=models.CASCADE)
date_created = models.DateTimeField(auto_now_add=True, null=False)
file = models.CharField(max_length=2048, blank=True, null=False)
file_size = models.IntegerField(null=True)
asset_type = models.CharField(max_length=64, blank=False, null=False)
content_type = models.CharField(max_length=128, blank=False, null=False)
display_name = models.CharField(max_length=2048, blank=True, null=False)
status = models.CharField(max_length=64, blank=False, null=False)
gzip = models.BooleanField(default=False, null=False)
def save(self, *args, **kwargs):
if self.file:
try:
file_path = os.path.join(settings.LD_ASSET_FOLDER, self.file)
if os.path.isfile(file_path):
self.file_size = os.path.getsize(file_path)
except Exception:
pass
super().save(*args, **kwargs)
def __str__(self):
return self.display_name or f"Bookmark Asset #{self.pk}"
@receiver(post_delete, sender=BookmarkAsset)
def bookmark_asset_deleted(sender, instance, **kwargs):
if instance.file:
filepath = os.path.join(settings.LD_ASSET_FOLDER, instance.file)
if os.path.isfile(filepath):
try:
os.remove(filepath)
except Exception as error:
logger.error(f"Failed to delete asset file: {filepath}", exc_info=error)
class BookmarkForm(forms.ModelForm):
@@ -90,15 +140,13 @@ class BookmarkForm(forms.ModelForm):
url = forms.CharField(validators=[BookmarkURLValidator()])
tag_string = forms.CharField(required=False)
# Do not require title and description in form as we fill these automatically if they are empty
title = forms.CharField(max_length=512,
required=False)
description = forms.CharField(required=False,
widget=forms.Textarea())
title = forms.CharField(max_length=512, required=False)
description = forms.CharField(required=False, widget=forms.Textarea())
# Include website title and description as hidden field as they only provide info when editing bookmarks
website_title = forms.CharField(max_length=512,
required=False, widget=forms.HiddenInput())
website_description = forms.CharField(required=False,
widget=forms.HiddenInput())
website_title = forms.CharField(
max_length=512, required=False, widget=forms.HiddenInput()
)
website_description = forms.CharField(required=False, widget=forms.HiddenInput())
unread = forms.BooleanField(required=False)
shared = forms.BooleanField(required=False)
# Hidden field that determines whether to close window/tab after saving the bookmark
@@ -107,16 +155,16 @@ class BookmarkForm(forms.ModelForm):
class Meta:
model = Bookmark
fields = [
'url',
'tag_string',
'title',
'description',
'notes',
'website_title',
'website_description',
'unread',
'shared',
'auto_close',
"url",
"tag_string",
"title",
"description",
"notes",
"website_title",
"website_description",
"unread",
"shared",
"auto_close",
]
@property
@@ -125,45 +173,47 @@ class BookmarkForm(forms.ModelForm):
class BookmarkSearch:
SORT_ADDED_ASC = 'added_asc'
SORT_ADDED_DESC = 'added_desc'
SORT_TITLE_ASC = 'title_asc'
SORT_TITLE_DESC = 'title_desc'
SORT_ADDED_ASC = "added_asc"
SORT_ADDED_DESC = "added_desc"
SORT_TITLE_ASC = "title_asc"
SORT_TITLE_DESC = "title_desc"
FILTER_SHARED_OFF = 'off'
FILTER_SHARED_SHARED = 'yes'
FILTER_SHARED_UNSHARED = 'no'
FILTER_SHARED_OFF = "off"
FILTER_SHARED_SHARED = "yes"
FILTER_SHARED_UNSHARED = "no"
FILTER_UNREAD_OFF = 'off'
FILTER_UNREAD_YES = 'yes'
FILTER_UNREAD_NO = 'no'
FILTER_UNREAD_OFF = "off"
FILTER_UNREAD_YES = "yes"
FILTER_UNREAD_NO = "no"
params = ['q', 'user', 'sort', 'shared', 'unread']
preferences = ['sort', 'shared', 'unread']
params = ["q", "user", "sort", "shared", "unread"]
preferences = ["sort", "shared", "unread"]
defaults = {
'q': '',
'user': '',
'sort': SORT_ADDED_DESC,
'shared': FILTER_SHARED_OFF,
'unread': FILTER_UNREAD_OFF,
"q": "",
"user": "",
"sort": SORT_ADDED_DESC,
"shared": FILTER_SHARED_OFF,
"unread": FILTER_UNREAD_OFF,
}
def __init__(self,
q: str = None,
user: str = None,
sort: str = None,
shared: str = None,
unread: str = None,
preferences: dict = None):
def __init__(
self,
q: str = None,
user: str = None,
sort: str = None,
shared: str = None,
unread: str = None,
preferences: dict = None,
):
if not preferences:
preferences = {}
self.defaults = {**BookmarkSearch.defaults, **preferences}
self.q = q or self.defaults['q']
self.user = user or self.defaults['user']
self.sort = sort or self.defaults['sort']
self.shared = shared or self.defaults['shared']
self.unread = unread or self.defaults['unread']
self.q = q or self.defaults["q"]
self.user = user or self.defaults["user"]
self.sort = sort or self.defaults["sort"]
self.shared = shared or self.defaults["shared"]
self.unread = unread or self.defaults["unread"]
def is_modified(self, param):
value = self.__dict__[param]
@@ -175,7 +225,11 @@ class BookmarkSearch:
@property
def modified_preferences(self):
return [preference for preference in self.preferences if self.is_modified(preference)]
return [
preference
for preference in self.preferences
if self.is_modified(preference)
]
@property
def has_modifications(self):
@@ -191,7 +245,9 @@ class BookmarkSearch:
@property
def preferences_dict(self):
return {preference: self.__dict__[preference] for preference in self.preferences}
return {
preference: self.__dict__[preference] for preference in self.preferences
}
@staticmethod
def from_request(query_dict: QueryDict, preferences: dict = None):
@@ -206,20 +262,20 @@ class BookmarkSearch:
class BookmarkSearchForm(forms.Form):
SORT_CHOICES = [
(BookmarkSearch.SORT_ADDED_ASC, 'Added ↑'),
(BookmarkSearch.SORT_ADDED_DESC, 'Added ↓'),
(BookmarkSearch.SORT_TITLE_ASC, 'Title ↑'),
(BookmarkSearch.SORT_TITLE_DESC, 'Title ↓'),
(BookmarkSearch.SORT_ADDED_ASC, "Added ↑"),
(BookmarkSearch.SORT_ADDED_DESC, "Added ↓"),
(BookmarkSearch.SORT_TITLE_ASC, "Title ↑"),
(BookmarkSearch.SORT_TITLE_DESC, "Title ↓"),
]
FILTER_SHARED_CHOICES = [
(BookmarkSearch.FILTER_SHARED_OFF, 'Off'),
(BookmarkSearch.FILTER_SHARED_SHARED, 'Shared'),
(BookmarkSearch.FILTER_SHARED_UNSHARED, 'Unshared'),
(BookmarkSearch.FILTER_SHARED_OFF, "Off"),
(BookmarkSearch.FILTER_SHARED_SHARED, "Shared"),
(BookmarkSearch.FILTER_SHARED_UNSHARED, "Unshared"),
]
FILTER_UNREAD_CHOICES = [
(BookmarkSearch.FILTER_UNREAD_OFF, 'Off'),
(BookmarkSearch.FILTER_UNREAD_YES, 'Unread'),
(BookmarkSearch.FILTER_UNREAD_NO, 'Read'),
(BookmarkSearch.FILTER_UNREAD_OFF, "Off"),
(BookmarkSearch.FILTER_UNREAD_YES, "Unread"),
(BookmarkSearch.FILTER_UNREAD_NO, "Read"),
]
q = forms.CharField()
@@ -228,7 +284,12 @@ class BookmarkSearchForm(forms.Form):
shared = forms.ChoiceField(choices=FILTER_SHARED_CHOICES, widget=forms.RadioSelect)
unread = forms.ChoiceField(choices=FILTER_UNREAD_CHOICES, widget=forms.RadioSelect)
def __init__(self, search: BookmarkSearch, editable_fields: List[str] = None, users: List[User] = None):
def __init__(
self,
search: BookmarkSearch,
editable_fields: List[str] = None,
users: List[User] = None,
):
super().__init__()
editable_fields = editable_fields or []
self.editable_fields = editable_fields
@@ -236,8 +297,8 @@ class BookmarkSearchForm(forms.Form):
# set choices for user field if users are provided
if users:
user_choices = [(user.username, user.username) for user in users]
user_choices.insert(0, ('', 'Everyone'))
self.fields['user'].choices = user_choices
user_choices.insert(0, ("", "Everyone"))
self.fields["user"].choices = user_choices
for param in search.params:
# set initial values for modified params
@@ -251,63 +312,142 @@ class BookmarkSearchForm(forms.Form):
class UserProfile(models.Model):
THEME_AUTO = 'auto'
THEME_LIGHT = 'light'
THEME_DARK = 'dark'
THEME_AUTO = "auto"
THEME_LIGHT = "light"
THEME_DARK = "dark"
THEME_CHOICES = [
(THEME_AUTO, 'Auto'),
(THEME_LIGHT, 'Light'),
(THEME_DARK, 'Dark'),
(THEME_AUTO, "Auto"),
(THEME_LIGHT, "Light"),
(THEME_DARK, "Dark"),
]
BOOKMARK_DATE_DISPLAY_RELATIVE = 'relative'
BOOKMARK_DATE_DISPLAY_ABSOLUTE = 'absolute'
BOOKMARK_DATE_DISPLAY_HIDDEN = 'hidden'
BOOKMARK_DATE_DISPLAY_RELATIVE = "relative"
BOOKMARK_DATE_DISPLAY_ABSOLUTE = "absolute"
BOOKMARK_DATE_DISPLAY_HIDDEN = "hidden"
BOOKMARK_DATE_DISPLAY_CHOICES = [
(BOOKMARK_DATE_DISPLAY_RELATIVE, 'Relative'),
(BOOKMARK_DATE_DISPLAY_ABSOLUTE, 'Absolute'),
(BOOKMARK_DATE_DISPLAY_HIDDEN, 'Hidden'),
(BOOKMARK_DATE_DISPLAY_RELATIVE, "Relative"),
(BOOKMARK_DATE_DISPLAY_ABSOLUTE, "Absolute"),
(BOOKMARK_DATE_DISPLAY_HIDDEN, "Hidden"),
]
BOOKMARK_LINK_TARGET_BLANK = '_blank'
BOOKMARK_LINK_TARGET_SELF = '_self'
BOOKMARK_DESCRIPTION_DISPLAY_INLINE = "inline"
BOOKMARK_DESCRIPTION_DISPLAY_SEPARATE = "separate"
BOOKMARK_DESCRIPTION_DISPLAY_CHOICES = [
(BOOKMARK_DESCRIPTION_DISPLAY_INLINE, "Inline"),
(BOOKMARK_DESCRIPTION_DISPLAY_SEPARATE, "Separate"),
]
BOOKMARK_LINK_TARGET_BLANK = "_blank"
BOOKMARK_LINK_TARGET_SELF = "_self"
BOOKMARK_LINK_TARGET_CHOICES = [
(BOOKMARK_LINK_TARGET_BLANK, 'New page'),
(BOOKMARK_LINK_TARGET_SELF, 'Same page'),
(BOOKMARK_LINK_TARGET_BLANK, "New page"),
(BOOKMARK_LINK_TARGET_SELF, "Same page"),
]
WEB_ARCHIVE_INTEGRATION_DISABLED = 'disabled'
WEB_ARCHIVE_INTEGRATION_ENABLED = 'enabled'
WEB_ARCHIVE_INTEGRATION_DISABLED = "disabled"
WEB_ARCHIVE_INTEGRATION_ENABLED = "enabled"
WEB_ARCHIVE_INTEGRATION_CHOICES = [
(WEB_ARCHIVE_INTEGRATION_DISABLED, 'Disabled'),
(WEB_ARCHIVE_INTEGRATION_ENABLED, 'Enabled'),
(WEB_ARCHIVE_INTEGRATION_DISABLED, "Disabled"),
(WEB_ARCHIVE_INTEGRATION_ENABLED, "Enabled"),
]
TAG_SEARCH_STRICT = 'strict'
TAG_SEARCH_LAX = 'lax'
TAG_SEARCH_STRICT = "strict"
TAG_SEARCH_LAX = "lax"
TAG_SEARCH_CHOICES = [
(TAG_SEARCH_STRICT, 'Strict'),
(TAG_SEARCH_LAX, 'Lax'),
(TAG_SEARCH_STRICT, "Strict"),
(TAG_SEARCH_LAX, "Lax"),
]
user = models.OneToOneField(get_user_model(), related_name='profile', on_delete=models.CASCADE)
theme = models.CharField(max_length=10, choices=THEME_CHOICES, blank=False, default=THEME_AUTO)
bookmark_date_display = models.CharField(max_length=10, choices=BOOKMARK_DATE_DISPLAY_CHOICES, blank=False,
default=BOOKMARK_DATE_DISPLAY_RELATIVE)
bookmark_link_target = models.CharField(max_length=10, choices=BOOKMARK_LINK_TARGET_CHOICES, blank=False,
default=BOOKMARK_LINK_TARGET_BLANK)
web_archive_integration = models.CharField(max_length=10, choices=WEB_ARCHIVE_INTEGRATION_CHOICES, blank=False,
default=WEB_ARCHIVE_INTEGRATION_DISABLED)
tag_search = models.CharField(max_length=10, choices=TAG_SEARCH_CHOICES, blank=False,
default=TAG_SEARCH_STRICT)
TAG_GROUPING_ALPHABETICAL = "alphabetical"
TAG_GROUPING_DISABLED = "disabled"
TAG_GROUPING_CHOICES = [
(TAG_GROUPING_ALPHABETICAL, "Alphabetical"),
(TAG_GROUPING_DISABLED, "Disabled"),
]
user = models.OneToOneField(
get_user_model(), related_name="profile", on_delete=models.CASCADE
)
theme = models.CharField(
max_length=10, choices=THEME_CHOICES, blank=False, default=THEME_AUTO
)
bookmark_date_display = models.CharField(
max_length=10,
choices=BOOKMARK_DATE_DISPLAY_CHOICES,
blank=False,
default=BOOKMARK_DATE_DISPLAY_RELATIVE,
)
bookmark_description_display = models.CharField(
max_length=10,
choices=BOOKMARK_DESCRIPTION_DISPLAY_CHOICES,
blank=False,
default=BOOKMARK_DESCRIPTION_DISPLAY_INLINE,
)
bookmark_description_max_lines = models.IntegerField(
null=False,
default=1,
)
bookmark_link_target = models.CharField(
max_length=10,
choices=BOOKMARK_LINK_TARGET_CHOICES,
blank=False,
default=BOOKMARK_LINK_TARGET_BLANK,
)
web_archive_integration = models.CharField(
max_length=10,
choices=WEB_ARCHIVE_INTEGRATION_CHOICES,
blank=False,
default=WEB_ARCHIVE_INTEGRATION_DISABLED,
)
tag_search = models.CharField(
max_length=10,
choices=TAG_SEARCH_CHOICES,
blank=False,
default=TAG_SEARCH_STRICT,
)
tag_grouping = models.CharField(
max_length=12,
choices=TAG_GROUPING_CHOICES,
blank=False,
default=TAG_GROUPING_ALPHABETICAL,
)
enable_sharing = models.BooleanField(default=False, null=False)
enable_public_sharing = models.BooleanField(default=False, null=False)
enable_favicons = models.BooleanField(default=False, null=False)
enable_preview_images = models.BooleanField(default=False, null=False)
display_url = models.BooleanField(default=False, null=False)
display_view_bookmark_action = models.BooleanField(default=True, null=False)
display_edit_bookmark_action = models.BooleanField(default=True, null=False)
display_archive_bookmark_action = models.BooleanField(default=True, null=False)
display_remove_bookmark_action = models.BooleanField(default=True, null=False)
permanent_notes = models.BooleanField(default=False, null=False)
custom_css = models.TextField(blank=True, null=False)
auto_tagging_rules = models.TextField(blank=True, null=False)
search_preferences = models.JSONField(default=dict, null=False)
enable_automatic_html_snapshots = models.BooleanField(default=True, null=False)
default_mark_unread = models.BooleanField(default=False, null=False)
class UserProfileForm(forms.ModelForm):
class Meta:
model = UserProfile
fields = ['theme', 'bookmark_date_display', 'bookmark_link_target', 'web_archive_integration', 'tag_search',
'enable_sharing', 'enable_public_sharing', 'enable_favicons', 'display_url', 'permanent_notes']
fields = [
"theme",
"bookmark_date_display",
"bookmark_description_display",
"bookmark_description_max_lines",
"bookmark_link_target",
"web_archive_integration",
"tag_search",
"tag_grouping",
"enable_sharing",
"enable_public_sharing",
"enable_favicons",
"enable_preview_images",
"enable_automatic_html_snapshots",
"display_url",
"display_view_bookmark_action",
"display_edit_bookmark_action",
"display_archive_bookmark_action",
"display_remove_bookmark_action",
"permanent_notes",
"default_mark_unread",
"custom_css",
"auto_tagging_rules",
]
@receiver(post_save, sender=get_user_model())
@@ -332,11 +472,13 @@ class FeedToken(models.Model):
"""
Adapted from authtoken.models.Token
"""
key = models.CharField(max_length=40, primary_key=True)
user = models.OneToOneField(get_user_model(),
related_name='feed_token',
on_delete=models.CASCADE,
)
user = models.OneToOneField(
get_user_model(),
related_name="feed_token",
on_delete=models.CASCADE,
)
created = models.DateTimeField(auto_now_add=True)
def save(self, *args, **kwargs):

View File

@@ -10,18 +10,24 @@ from bookmarks.models import Bookmark, BookmarkSearch, Tag, UserProfile
from bookmarks.utils import unique
def query_bookmarks(user: User, profile: UserProfile, search: BookmarkSearch) -> QuerySet:
return _base_bookmarks_query(user, profile, search) \
.filter(is_archived=False)
def query_bookmarks(
user: User, profile: UserProfile, search: BookmarkSearch
) -> QuerySet:
return _base_bookmarks_query(user, profile, search).filter(is_archived=False)
def query_archived_bookmarks(user: User, profile: UserProfile, search: BookmarkSearch) -> QuerySet:
return _base_bookmarks_query(user, profile, search) \
.filter(is_archived=True)
def query_archived_bookmarks(
user: User, profile: UserProfile, search: BookmarkSearch
) -> QuerySet:
return _base_bookmarks_query(user, profile, search).filter(is_archived=True)
def query_shared_bookmarks(user: Optional[User], profile: UserProfile, search: BookmarkSearch,
public_only: bool) -> QuerySet:
def query_shared_bookmarks(
user: Optional[User],
profile: UserProfile,
search: BookmarkSearch,
public_only: bool,
) -> QuerySet:
conditions = Q(shared=True) & Q(owner__profile__enable_sharing=True)
if public_only:
conditions = conditions & Q(owner__profile__enable_public_sharing=True)
@@ -29,7 +35,9 @@ def query_shared_bookmarks(user: Optional[User], profile: UserProfile, search: B
return _base_bookmarks_query(user, profile, search).filter(conditions)
def _base_bookmarks_query(user: Optional[User], profile: UserProfile, search: BookmarkSearch) -> QuerySet:
def _base_bookmarks_query(
user: Optional[User], profile: UserProfile, search: BookmarkSearch
) -> QuerySet:
query_set = Bookmark.objects
# Filter for user
@@ -40,34 +48,32 @@ def _base_bookmarks_query(user: Optional[User], profile: UserProfile, search: Bo
query = parse_query_string(search.q)
# Filter for search terms and tags
for term in query['search_terms']:
conditions = Q(title__icontains=term) \
| Q(description__icontains=term) \
| Q(notes__icontains=term) \
| Q(website_title__icontains=term) \
| Q(website_description__icontains=term) \
| Q(url__icontains=term)
for term in query["search_terms"]:
conditions = (
Q(title__icontains=term)
| Q(description__icontains=term)
| Q(notes__icontains=term)
| Q(website_title__icontains=term)
| Q(website_description__icontains=term)
| Q(url__icontains=term)
)
if profile.tag_search == UserProfile.TAG_SEARCH_LAX:
conditions = conditions | Exists(Bookmark.objects.filter(id=OuterRef('id'), tags__name__iexact=term))
conditions = conditions | Exists(
Bookmark.objects.filter(id=OuterRef("id"), tags__name__iexact=term)
)
query_set = query_set.filter(conditions)
for tag_name in query['tag_names']:
query_set = query_set.filter(
tags__name__iexact=tag_name
)
for tag_name in query["tag_names"]:
query_set = query_set.filter(tags__name__iexact=tag_name)
# Untagged bookmarks
if query['untagged']:
query_set = query_set.filter(
tags=None
)
if query["untagged"]:
query_set = query_set.filter(tags=None)
# Legacy unread bookmarks filter from query
if query['unread']:
query_set = query_set.filter(
unread=True
)
if query["unread"]:
query_set = query_set.filter(unread=True)
# Unread filter from bookmark search
if search.unread == BookmarkSearch.FILTER_UNREAD_YES:
@@ -83,29 +89,36 @@ def _base_bookmarks_query(user: Optional[User], profile: UserProfile, search: Bo
# Sort by date added
if search.sort == BookmarkSearch.SORT_ADDED_ASC:
query_set = query_set.order_by('date_added')
query_set = query_set.order_by("date_added")
elif search.sort == BookmarkSearch.SORT_ADDED_DESC:
query_set = query_set.order_by('-date_added')
query_set = query_set.order_by("-date_added")
# Sort by title
if search.sort == BookmarkSearch.SORT_TITLE_ASC or search.sort == BookmarkSearch.SORT_TITLE_DESC:
if (
search.sort == BookmarkSearch.SORT_TITLE_ASC
or search.sort == BookmarkSearch.SORT_TITLE_DESC
):
# For the title, the resolved_title logic from the Bookmark entity needs
# to be replicated as there is no corresponding database field
query_set = query_set.annotate(
effective_title=Case(
When(Q(title__isnull=False) & ~Q(title__exact=''), then=Lower('title')),
When(Q(website_title__isnull=False) & ~Q(website_title__exact=''), then=Lower('website_title')),
default=Lower('url'),
output_field=CharField()
))
When(Q(title__isnull=False) & ~Q(title__exact=""), then=Lower("title")),
When(
Q(website_title__isnull=False) & ~Q(website_title__exact=""),
then=Lower("website_title"),
),
default=Lower("url"),
output_field=CharField(),
)
)
# For SQLite, if the ICU extension is loaded, use the custom collation
# loaded into the connection. This results in an improved sort order for
# unicode characters (umlauts, etc.)
if settings.USE_SQLITE and settings.USE_SQLITE_ICU_EXTENSION:
order_field = RawSQL('effective_title COLLATE ICU', ())
order_field = RawSQL("effective_title COLLATE ICU", ())
else:
order_field = 'effective_title'
order_field = "effective_title"
if search.sort == BookmarkSearch.SORT_TITLE_ASC:
query_set = query_set.order_by(order_field)
@@ -115,7 +128,9 @@ def _base_bookmarks_query(user: Optional[User], profile: UserProfile, search: Bo
return query_set
def query_bookmark_tags(user: User, profile: UserProfile, search: BookmarkSearch) -> QuerySet:
def query_bookmark_tags(
user: User, profile: UserProfile, search: BookmarkSearch
) -> QuerySet:
bookmarks_query = query_bookmarks(user, profile, search)
query_set = Tag.objects.filter(bookmark__in=bookmarks_query)
@@ -123,7 +138,9 @@ def query_bookmark_tags(user: User, profile: UserProfile, search: BookmarkSearch
return query_set.distinct()
def query_archived_bookmark_tags(user: User, profile: UserProfile, search: BookmarkSearch) -> QuerySet:
def query_archived_bookmark_tags(
user: User, profile: UserProfile, search: BookmarkSearch
) -> QuerySet:
bookmarks_query = query_archived_bookmarks(user, profile, search)
query_set = Tag.objects.filter(bookmark__in=bookmarks_query)
@@ -131,8 +148,12 @@ def query_archived_bookmark_tags(user: User, profile: UserProfile, search: Bookm
return query_set.distinct()
def query_shared_bookmark_tags(user: Optional[User], profile: UserProfile, search: BookmarkSearch,
public_only: bool) -> QuerySet:
def query_shared_bookmark_tags(
user: Optional[User],
profile: UserProfile,
search: BookmarkSearch,
public_only: bool,
) -> QuerySet:
bookmarks_query = query_shared_bookmarks(user, profile, search, public_only)
query_set = Tag.objects.filter(bookmark__in=bookmarks_query)
@@ -140,7 +161,9 @@ def query_shared_bookmark_tags(user: Optional[User], profile: UserProfile, searc
return query_set.distinct()
def query_shared_bookmark_users(profile: UserProfile, search: BookmarkSearch, public_only: bool) -> QuerySet:
def query_shared_bookmark_users(
profile: UserProfile, search: BookmarkSearch, public_only: bool
) -> QuerySet:
bookmarks_query = query_shared_bookmarks(None, profile, search, public_only)
query_set = User.objects.filter(bookmark__in=bookmarks_query)
@@ -155,23 +178,23 @@ def get_user_tags(user: User):
def parse_query_string(query_string):
# Sanitize query params
if not query_string:
query_string = ''
query_string = ""
# Split query into search terms and tags
keywords = query_string.strip().split(' ')
keywords = query_string.strip().split(" ")
keywords = [word for word in keywords if word]
search_terms = [word for word in keywords if word[0] != '#' and word[0] != '!']
tag_names = [word[1:] for word in keywords if word[0] == '#']
search_terms = [word for word in keywords if word[0] != "#" and word[0] != "!"]
tag_names = [word[1:] for word in keywords if word[0] == "#"]
tag_names = unique(tag_names, str.lower)
# Special search commands
untagged = '!untagged' in keywords
unread = '!unread' in keywords
untagged = "!untagged" in keywords
unread = "!unread" in keywords
return {
'search_terms': search_terms,
'tag_names': tag_names,
'untagged': untagged,
'unread': unread,
"search_terms": search_terms,
"tag_names": tag_names,
"untagged": untagged,
"unread": unread,
}

View File

@@ -0,0 +1,70 @@
from urllib.parse import urlparse, parse_qs
import re
import idna
def get_tags(script: str, url: str):
parsed_url = urlparse(url.lower())
result = set()
for line in script.lower().split("\n"):
if "#" in line:
i = line.index("#")
line = line[:i]
parts = line.split()
if len(parts) < 2:
continue
domain_pattern = re.sub("^https?://", "", parts[0])
path_pattern = None
qs_pattern = None
if "/" in domain_pattern:
i = domain_pattern.index("/")
path_pattern = domain_pattern[i:]
domain_pattern = domain_pattern[:i]
if path_pattern and "?" in path_pattern:
i = path_pattern.index("?")
qs_pattern = path_pattern[i + 1 :]
path_pattern = path_pattern[:i]
if not _domains_matches(domain_pattern, parsed_url.netloc):
continue
if path_pattern and not _path_matches(path_pattern, parsed_url.path):
continue
if qs_pattern and not _qs_matches(qs_pattern, parsed_url.query):
continue
for tag in parts[1:]:
result.add(tag)
return result
def _path_matches(expected_path: str, actual_path: str) -> bool:
return actual_path.startswith(expected_path)
def _domains_matches(expected_domain: str, actual_domain: str) -> bool:
expected_domain = idna.encode(expected_domain)
actual_domain = idna.encode(actual_domain)
return actual_domain.endswith(expected_domain)
def _qs_matches(expected_qs: str, actual_qs: str) -> bool:
expected_qs = parse_qs(expected_qs, keep_blank_values=True)
actual_qs = parse_qs(actual_qs, keep_blank_values=True)
for key in expected_qs:
if key not in actual_qs:
return False
for value in expected_qs[key]:
if value != "" and value not in actual_qs[key]:
return False
return True

View File

@@ -1,17 +1,26 @@
import logging
import os
from typing import Union
from django.conf import settings
from django.contrib.auth.models import User
from django.core.files.uploadedfile import UploadedFile
from django.utils import timezone
from bookmarks.models import Bookmark, parse_tag_string
from bookmarks.services.tags import get_or_create_tags
from bookmarks.services import website_loader
from bookmarks.models import Bookmark, BookmarkAsset, parse_tag_string
from bookmarks.services import tasks
from bookmarks.services import website_loader
from bookmarks.services import auto_tagging
from bookmarks.services.tags import get_or_create_tags
logger = logging.getLogger(__name__)
def create_bookmark(bookmark: Bookmark, tag_string: str, current_user: User):
# If URL is already bookmarked, then update it
existing_bookmark: Bookmark = Bookmark.objects.filter(owner=current_user, url=bookmark.url).first()
existing_bookmark: Bookmark = Bookmark.objects.filter(
owner=current_user, url=bookmark.url
).first()
if existing_bookmark is not None:
_merge_bookmark_data(bookmark, existing_bookmark)
@@ -32,6 +41,11 @@ def create_bookmark(bookmark: Bookmark, tag_string: str, current_user: User):
tasks.create_web_archive_snapshot(current_user, bookmark, False)
# Load favicon
tasks.load_favicon(current_user, bookmark)
# Load preview image
tasks.load_preview_image(current_user, bookmark)
# Create HTML snapshot
if current_user.profile.enable_automatic_html_snapshots:
tasks.create_html_snapshot(bookmark)
return bookmark
@@ -47,6 +61,8 @@ def update_bookmark(bookmark: Bookmark, tag_string, current_user: User):
bookmark.save()
# Update favicon
tasks.load_favicon(current_user, bookmark)
# Update preview image
tasks.load_preview_image(current_user, bookmark)
if has_url_changed:
# Update web archive snapshot, if URL changed
@@ -67,9 +83,10 @@ def archive_bookmark(bookmark: Bookmark):
def archive_bookmarks(bookmark_ids: [Union[int, str]], current_user: User):
sanitized_bookmark_ids = _sanitize_id_list(bookmark_ids)
bookmarks = Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids)
bookmarks.update(is_archived=True, date_modified=timezone.now())
Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids).update(
is_archived=True, date_modified=timezone.now()
)
def unarchive_bookmark(bookmark: Bookmark):
@@ -81,70 +98,133 @@ def unarchive_bookmark(bookmark: Bookmark):
def unarchive_bookmarks(bookmark_ids: [Union[int, str]], current_user: User):
sanitized_bookmark_ids = _sanitize_id_list(bookmark_ids)
bookmarks = Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids)
bookmarks.update(is_archived=False, date_modified=timezone.now())
Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids).update(
is_archived=False, date_modified=timezone.now()
)
def delete_bookmarks(bookmark_ids: [Union[int, str]], current_user: User):
sanitized_bookmark_ids = _sanitize_id_list(bookmark_ids)
bookmarks = Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids)
bookmarks.delete()
Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids).delete()
def tag_bookmarks(bookmark_ids: [Union[int, str]], tag_string: str, current_user: User):
sanitized_bookmark_ids = _sanitize_id_list(bookmark_ids)
bookmarks = Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids)
owned_bookmark_ids = Bookmark.objects.filter(
owner=current_user, id__in=sanitized_bookmark_ids
).values_list("id", flat=True)
tag_names = parse_tag_string(tag_string)
tags = get_or_create_tags(tag_names, current_user)
for bookmark in bookmarks:
bookmark.tags.add(*tags)
bookmark.date_modified = timezone.now()
BookmarkToTagRelationShip = Bookmark.tags.through
relationships = []
for tag in tags:
for bookmark_id in owned_bookmark_ids:
relationships.append(
BookmarkToTagRelationShip(bookmark_id=bookmark_id, tag=tag)
)
Bookmark.objects.bulk_update(bookmarks, ['date_modified'])
# Insert all bookmark -> tag associations at once, should ignore errors if association already exists
BookmarkToTagRelationShip.objects.bulk_create(relationships, ignore_conflicts=True)
Bookmark.objects.filter(id__in=owned_bookmark_ids).update(
date_modified=timezone.now()
)
def untag_bookmarks(bookmark_ids: [Union[int, str]], tag_string: str, current_user: User):
def untag_bookmarks(
bookmark_ids: [Union[int, str]], tag_string: str, current_user: User
):
sanitized_bookmark_ids = _sanitize_id_list(bookmark_ids)
bookmarks = Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids)
owned_bookmark_ids = Bookmark.objects.filter(
owner=current_user, id__in=sanitized_bookmark_ids
).values_list("id", flat=True)
tag_names = parse_tag_string(tag_string)
tags = get_or_create_tags(tag_names, current_user)
for bookmark in bookmarks:
bookmark.tags.remove(*tags)
bookmark.date_modified = timezone.now()
BookmarkToTagRelationShip = Bookmark.tags.through
for tag in tags:
# Remove all bookmark -> tag associations for the owned bookmarks and the current tag
BookmarkToTagRelationShip.objects.filter(
bookmark_id__in=owned_bookmark_ids, tag=tag
).delete()
Bookmark.objects.bulk_update(bookmarks, ['date_modified'])
Bookmark.objects.filter(id__in=owned_bookmark_ids).update(
date_modified=timezone.now()
)
def mark_bookmarks_as_read(bookmark_ids: [Union[int, str]], current_user: User):
sanitized_bookmark_ids = _sanitize_id_list(bookmark_ids)
bookmarks = Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids)
bookmarks.update(unread=False, date_modified=timezone.now())
Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids).update(
unread=False, date_modified=timezone.now()
)
def mark_bookmarks_as_unread(bookmark_ids: [Union[int, str]], current_user: User):
sanitized_bookmark_ids = _sanitize_id_list(bookmark_ids)
bookmarks = Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids)
bookmarks.update(unread=True, date_modified=timezone.now())
Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids).update(
unread=True, date_modified=timezone.now()
)
def share_bookmarks(bookmark_ids: [Union[int, str]], current_user: User):
sanitized_bookmark_ids = _sanitize_id_list(bookmark_ids)
bookmarks = Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids)
bookmarks.update(shared=True, date_modified=timezone.now())
Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids).update(
shared=True, date_modified=timezone.now()
)
def unshare_bookmarks(bookmark_ids: [Union[int, str]], current_user: User):
sanitized_bookmark_ids = _sanitize_id_list(bookmark_ids)
bookmarks = Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids)
bookmarks.update(shared=False, date_modified=timezone.now())
Bookmark.objects.filter(owner=current_user, id__in=sanitized_bookmark_ids).update(
shared=False, date_modified=timezone.now()
)
def _generate_upload_asset_filename(asset: BookmarkAsset, filename: str):
formatted_datetime = asset.date_created.strftime("%Y-%m-%d_%H%M%S")
return f"{asset.asset_type}_{formatted_datetime}_{filename}"
def upload_asset(bookmark: Bookmark, upload_file: UploadedFile) -> BookmarkAsset:
asset = BookmarkAsset(
bookmark=bookmark,
asset_type=BookmarkAsset.TYPE_UPLOAD,
content_type=upload_file.content_type,
display_name=upload_file.name,
status=BookmarkAsset.STATUS_PENDING,
gzip=False,
)
asset.save()
try:
filename = _generate_upload_asset_filename(asset, upload_file.name)
filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
with open(filepath, "wb") as f:
for chunk in upload_file.chunks():
f.write(chunk)
asset.status = BookmarkAsset.STATUS_COMPLETE
asset.file = filename
asset.file_size = upload_file.size
logger.info(
f"Successfully uploaded asset file. bookmark={bookmark} file={upload_file.name}"
)
except Exception as e:
logger.error(
f"Failed to upload asset file. bookmark={bookmark} file={upload_file.name}",
exc_info=e,
)
asset.status = BookmarkAsset.STATUS_FAILURE
asset.save()
return asset
def _merge_bookmark_data(from_bookmark: Bookmark, to_bookmark: Bookmark):
@@ -163,6 +243,15 @@ def _update_website_metadata(bookmark: Bookmark):
def _update_bookmark_tags(bookmark: Bookmark, tag_string: str, user: User):
tag_names = parse_tag_string(tag_string)
if user.profile.auto_tagging_rules:
auto_tag_names = auto_tagging.get_tags(
user.profile.auto_tagging_rules, bookmark.url
)
for auto_tag_name in auto_tag_names:
if auto_tag_name not in tag_names:
tag_names.append(auto_tag_name)
tags = get_or_create_tags(tag_names, user)
bookmark.tags.set(tags)

View File

@@ -13,37 +13,41 @@ def export_netscape_html(bookmarks: List[Bookmark]):
[append_bookmark(doc, bookmark) for bookmark in bookmarks]
append_list_end(doc)
return '\n\r'.join(doc)
return "\n\r".join(doc)
def append_header(doc: BookmarkDocument):
doc.append('<!DOCTYPE NETSCAPE-Bookmark-file-1>')
doc.append("<!DOCTYPE NETSCAPE-Bookmark-file-1>")
doc.append('<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8">')
doc.append('<TITLE>Bookmarks</TITLE>')
doc.append('<H1>Bookmarks</H1>')
doc.append("<TITLE>Bookmarks</TITLE>")
doc.append("<H1>Bookmarks</H1>")
def append_list_start(doc: BookmarkDocument):
doc.append('<DL><p>')
doc.append("<DL><p>")
def append_bookmark(doc: BookmarkDocument, bookmark: Bookmark):
url = bookmark.url
title = html.escape(bookmark.resolved_title or '')
desc = html.escape(bookmark.resolved_description or '')
title = html.escape(bookmark.resolved_title or "")
desc = html.escape(bookmark.resolved_description or "")
if bookmark.notes:
desc += f'[linkding-notes]{html.escape(bookmark.notes)}[/linkding-notes]'
tags = ','.join(bookmark.tag_names)
toread = '1' if bookmark.unread else '0'
private = '0' if bookmark.shared else '1'
desc += f"[linkding-notes]{html.escape(bookmark.notes)}[/linkding-notes]"
tag_names = bookmark.tag_names
if bookmark.is_archived:
tag_names.append("linkding:archived")
tags = ",".join(tag_names)
toread = "1" if bookmark.unread else "0"
private = "0" if bookmark.shared else "1"
added = int(bookmark.date_added.timestamp())
doc.append(
f'<DT><A HREF="{url}" ADD_DATE="{added}" PRIVATE="{private}" TOREAD="{toread}" TAGS="{tags}">{title}</A>')
f'<DT><A HREF="{url}" ADD_DATE="{added}" PRIVATE="{private}" TOREAD="{toread}" TAGS="{tags}">{title}</A>'
)
if desc:
doc.append(f'<DD>{desc}')
doc.append(f"<DD>{desc}")
def append_list_end(doc: BookmarkDocument):
doc.append('</DL><p>')
doc.append("</DL><p>")

View File

@@ -15,7 +15,7 @@ logger = logging.getLogger(__name__)
# register mime type for .ico files, which is not included in the default
# mimetypes of the Docker image
mimetypes.add_type('image/x-icon', '.ico')
mimetypes.add_type("image/x-icon", ".ico")
def _ensure_favicon_folder():
@@ -23,16 +23,16 @@ def _ensure_favicon_folder():
def _url_to_filename(url: str) -> str:
return re.sub(r'\W+', '_', url)
return re.sub(r"\W+", "_", url)
def _get_url_parameters(url: str) -> dict:
parsed_uri = urlparse(url)
return {
# https://example.com/foo?bar -> https://example.com
'url': f'{parsed_uri.scheme}://{parsed_uri.hostname}',
"url": f"{parsed_uri.scheme}://{parsed_uri.hostname}",
# https://example.com/foo?bar -> example.com
'domain': parsed_uri.hostname,
"domain": parsed_uri.hostname,
}
@@ -63,21 +63,21 @@ def load_favicon(url: str) -> str:
# Create favicon folder if not exists
_ensure_favicon_folder()
# Use scheme+hostname as favicon filename to reuse icon for all pages on the same domain
favicon_name = _url_to_filename(url_parameters['url'])
favicon_name = _url_to_filename(url_parameters["url"])
favicon_file = _check_existing_favicon(favicon_name)
if not favicon_file:
# Load favicon from provider, save to file
favicon_url = settings.LD_FAVICON_PROVIDER.format(**url_parameters)
logger.debug(f'Loading favicon from: {favicon_url}')
logger.debug(f"Loading favicon from: {favicon_url}")
with requests.get(favicon_url, stream=True) as response:
content_type = response.headers['Content-Type']
content_type = response.headers["Content-Type"]
file_extension = mimetypes.guess_extension(content_type)
favicon_file = f'{favicon_name}{file_extension}'
favicon_file = f"{favicon_name}{file_extension}"
favicon_path = _get_favicon_path(favicon_file)
with open(favicon_path, 'wb') as file:
with open(favicon_path, "wb") as file:
for chunk in response.iter_content(chunk_size=8192):
file.write(chunk)
logger.debug(f'Saved favicon as: {favicon_path}')
logger.debug(f"Saved favicon as: {favicon_path}")
return favicon_file

View File

@@ -5,7 +5,7 @@ from typing import List
from django.contrib.auth.models import User
from django.utils import timezone
from bookmarks.models import Bookmark, Tag, parse_tag_string
from bookmarks.models import Bookmark, Tag
from bookmarks.services import tasks
from bookmarks.services.parser import parse, NetscapeBookmark
from bookmarks.utils import parse_timestamp
@@ -55,18 +55,20 @@ class TagCache:
self.cache[tag.name.lower()] = tag
def import_netscape_html(html: str, user: User, options: ImportOptions = ImportOptions()) -> ImportResult:
def import_netscape_html(
html: str, user: User, options: ImportOptions = ImportOptions()
) -> ImportResult:
result = ImportResult()
import_start = timezone.now()
try:
netscape_bookmarks = parse(html)
except:
logging.exception('Could not read bookmarks file.')
logging.exception("Could not read bookmarks file.")
raise
parse_end = timezone.now()
logger.debug(f'Parse duration: {parse_end - import_start}')
logger.debug(f"Parse duration: {parse_end - import_start}")
# Create and cache all tags beforehand
_create_missing_tags(netscape_bookmarks, user)
@@ -81,9 +83,11 @@ def import_netscape_html(html: str, user: User, options: ImportOptions = ImportO
tasks.schedule_bookmarks_without_snapshots(user)
# Load favicons for newly imported bookmarks
tasks.schedule_bookmarks_without_favicons(user)
# Load previews for newly imported bookmarks
tasks.schedule_bookmarks_without_previews(user)
end = timezone.now()
logger.debug(f'Import duration: {end - import_start}')
logger.debug(f"Import duration: {end - import_start}")
return result
@@ -93,8 +97,7 @@ def _create_missing_tags(netscape_bookmarks: List[NetscapeBookmark], user: User)
tags_to_create = []
for netscape_bookmark in netscape_bookmarks:
tag_names = parse_tag_string(netscape_bookmark.tag_string)
for tag_name in tag_names:
for tag_name in netscape_bookmark.tag_names:
tag = tag_cache.get(tag_name)
if not tag:
tag = Tag(name=tag_name, owner=user)
@@ -111,7 +114,7 @@ def _get_batches(items: List, batch_size: int):
num_items = len(items)
while offset < num_items:
batch = items[offset:min(offset + batch_size, num_items)]
batch = items[offset : min(offset + batch_size, num_items)]
if len(batch) > 0:
batches.append(batch)
offset = offset + batch_size
@@ -119,11 +122,13 @@ def _get_batches(items: List, batch_size: int):
return batches
def _import_batch(netscape_bookmarks: List[NetscapeBookmark],
user: User,
options: ImportOptions,
tag_cache: TagCache,
result: ImportResult):
def _import_batch(
netscape_bookmarks: List[NetscapeBookmark],
user: User,
options: ImportOptions,
tag_cache: TagCache,
result: ImportResult,
):
# Query existing bookmarks
batch_urls = [bookmark.href for bookmark in netscape_bookmarks]
existing_bookmarks = Bookmark.objects.filter(owner=user, url__in=batch_urls)
@@ -137,7 +142,13 @@ def _import_batch(netscape_bookmarks: List[NetscapeBookmark],
try:
# Lookup existing bookmark by URL, or create new bookmark if there is no bookmark for that URL yet
bookmark = next(
(bookmark for bookmark in existing_bookmarks if bookmark.url == netscape_bookmark.href), None)
(
bookmark
for bookmark in existing_bookmarks
if bookmark.url == netscape_bookmark.href
),
None,
)
if not bookmark:
bookmark = Bookmark(owner=user)
is_update = False
@@ -147,7 +158,7 @@ def _import_batch(netscape_bookmarks: List[NetscapeBookmark],
_copy_bookmark_data(netscape_bookmark, bookmark, options)
# Validate bookmark fields, exclude owner to prevent n+1 database query,
# also there is no specific validation on owner
bookmark.clean_fields(exclude=['owner'])
bookmark.clean_fields(exclude=["owner"])
# Schedule for update or insert
if is_update:
bookmarks_to_update.append(bookmark)
@@ -156,20 +167,25 @@ def _import_batch(netscape_bookmarks: List[NetscapeBookmark],
result.success = result.success + 1
except:
shortened_bookmark_tag_str = str(netscape_bookmark)[:100] + '...'
logging.exception('Error importing bookmark: ' + shortened_bookmark_tag_str)
shortened_bookmark_tag_str = str(netscape_bookmark)[:100] + "..."
logging.exception("Error importing bookmark: " + shortened_bookmark_tag_str)
result.failed = result.failed + 1
# Bulk update bookmarks in DB
Bookmark.objects.bulk_update(bookmarks_to_update, ['url',
'date_added',
'date_modified',
'unread',
'shared',
'title',
'description',
'notes',
'owner'])
Bookmark.objects.bulk_update(
bookmarks_to_update,
[
"url",
"date_added",
"date_modified",
"unread",
"shared",
"title",
"description",
"notes",
"owner",
],
)
# Bulk insert new bookmarks into DB
Bookmark.objects.bulk_create(bookmarks_to_create)
@@ -184,18 +200,24 @@ def _import_batch(netscape_bookmarks: List[NetscapeBookmark],
for netscape_bookmark in netscape_bookmarks:
# Lookup bookmark by URL again
bookmark = next(
(bookmark for bookmark in existing_bookmarks if bookmark.url == netscape_bookmark.href), None)
(
bookmark
for bookmark in existing_bookmarks
if bookmark.url == netscape_bookmark.href
),
None,
)
if not bookmark:
# Something is wrong, we should have just created this bookmark
shortened_bookmark_tag_str = str(netscape_bookmark)[:100] + '...'
shortened_bookmark_tag_str = str(netscape_bookmark)[:100] + "..."
logging.warning(
f'Failed to assign tags to the bookmark: {shortened_bookmark_tag_str}. Could not find bookmark by URL.')
f"Failed to assign tags to the bookmark: {shortened_bookmark_tag_str}. Could not find bookmark by URL."
)
continue
# Get tag models by string, schedule inserts for bookmark -> tag associations
tag_names = parse_tag_string(netscape_bookmark.tag_string)
tags = tag_cache.get_all(tag_names)
tags = tag_cache.get_all(netscape_bookmark.tag_names)
for tag in tags:
relationships.append(BookmarkToTagRelationShip(bookmark=bookmark, tag=tag))
@@ -203,7 +225,9 @@ def _import_batch(netscape_bookmarks: List[NetscapeBookmark],
BookmarkToTagRelationShip.objects.bulk_create(relationships, ignore_conflicts=True)
def _copy_bookmark_data(netscape_bookmark: NetscapeBookmark, bookmark: Bookmark, options: ImportOptions):
def _copy_bookmark_data(
netscape_bookmark: NetscapeBookmark, bookmark: Bookmark, options: ImportOptions
):
bookmark.url = netscape_bookmark.href
if netscape_bookmark.date_added:
bookmark.date_added = parse_timestamp(netscape_bookmark.date_added)
@@ -219,3 +243,5 @@ def _copy_bookmark_data(netscape_bookmark: NetscapeBookmark, bookmark: Bookmark,
bookmark.notes = netscape_bookmark.notes
if options.map_private_flag and not netscape_bookmark.private:
bookmark.shared = True
if netscape_bookmark.archived:
bookmark.is_archived = True

View File

@@ -0,0 +1,32 @@
import gzip
import shutil
import subprocess
import os
from django.conf import settings
class MonolithError(Exception):
pass
# Monolith isn't used at the moment, as the local snapshot implementation
# switched to single-file after the prototype. Keeping this around in case
# it turns out to be useful in the future.
def create_snapshot(url: str, filepath: str):
monolith_path = settings.LD_MONOLITH_PATH
monolith_options = settings.LD_MONOLITH_OPTIONS
temp_filepath = filepath + ".tmp"
try:
command = f"{monolith_path} '{url}' {monolith_options} -o {temp_filepath}"
subprocess.run(command, check=True, shell=True)
with open(temp_filepath, "rb") as raw_file, gzip.open(
filepath, "wb"
) as gz_file:
shutil.copyfileobj(raw_file, gz_file)
os.remove(temp_filepath)
except subprocess.CalledProcessError as error:
raise MonolithError(f"Failed to create snapshot: {error.stderr}")

View File

@@ -2,6 +2,8 @@ from dataclasses import dataclass
from html.parser import HTMLParser
from typing import Dict, List
from bookmarks.models import parse_tag_string
@dataclass
class NetscapeBookmark:
@@ -10,9 +12,10 @@ class NetscapeBookmark:
description: str
notes: str
date_added: str
tag_string: str
tag_names: List[str]
to_read: bool
private: bool
archived: bool
class BookmarkParser(HTMLParser):
@@ -22,29 +25,29 @@ class BookmarkParser(HTMLParser):
self.current_tag = None
self.bookmark = None
self.href = ''
self.add_date = ''
self.tags = ''
self.title = ''
self.description = ''
self.notes = ''
self.toread = ''
self.private = ''
self.href = ""
self.add_date = ""
self.tags = ""
self.title = ""
self.description = ""
self.notes = ""
self.toread = ""
self.private = ""
def handle_starttag(self, tag: str, attrs: list):
name = 'handle_start_' + tag.lower()
name = "handle_start_" + tag.lower()
if name in dir(self):
getattr(self, name)({k.lower(): v for k, v in attrs})
self.current_tag = tag
def handle_endtag(self, tag: str):
name = 'handle_end_' + tag.lower()
name = "handle_end_" + tag.lower()
if name in dir(self):
getattr(self, name)()
self.current_tag = None
def handle_data(self, data):
name = f'handle_{self.current_tag}_data'
name = f"handle_{self.current_tag}_data"
if name in dir(self):
getattr(self, name)(data)
@@ -56,16 +59,24 @@ class BookmarkParser(HTMLParser):
def handle_start_a(self, attrs: Dict[str, str]):
vars(self).update(attrs)
tag_names = parse_tag_string(self.tags)
archived = "linkding:archived" in self.tags
try:
tag_names.remove("linkding:archived")
except ValueError:
pass
self.bookmark = NetscapeBookmark(
href=self.href,
title='',
description='',
notes='',
title="",
description="",
notes="",
date_added=self.add_date,
tag_string=self.tags,
to_read=self.toread == '1',
tag_names=tag_names,
to_read=self.toread == "1",
# Mark as private by default, also when attribute is not specified
private=self.private != '0',
private=self.private != "0",
archived=archived,
)
def handle_a_data(self, data):
@@ -73,9 +84,9 @@ class BookmarkParser(HTMLParser):
def handle_dd_data(self, data):
desc = data.strip()
if '[linkding-notes]' in desc:
self.notes = desc.split('[linkding-notes]')[1].split('[/linkding-notes]')[0]
self.description = desc.split('[linkding-notes]')[0]
if "[linkding-notes]" in desc:
self.notes = desc.split("[linkding-notes]")[1].split("[/linkding-notes]")[0]
self.description = desc.split("[linkding-notes]")[0]
def add_bookmark(self):
if self.bookmark:
@@ -84,14 +95,14 @@ class BookmarkParser(HTMLParser):
self.bookmark.notes = self.notes
self.bookmarks.append(self.bookmark)
self.bookmark = None
self.href = ''
self.add_date = ''
self.tags = ''
self.title = ''
self.description = ''
self.notes = ''
self.toread = ''
self.private = ''
self.href = ""
self.add_date = ""
self.tags = ""
self.title = ""
self.description = ""
self.notes = ""
self.toread = ""
self.private = ""
def parse(html: str) -> List[NetscapeBookmark]:

View File

@@ -0,0 +1,88 @@
import logging
import mimetypes
import os.path
import hashlib
from pathlib import Path
import requests
from django.conf import settings
from bookmarks.services import website_loader
logger = logging.getLogger(__name__)
def _ensure_preview_folder():
Path(settings.LD_PREVIEW_FOLDER).mkdir(parents=True, exist_ok=True)
def _url_to_filename(preview_image: str) -> str:
return hashlib.md5(preview_image.encode()).hexdigest()
def _get_image_path(preview_image_file: str) -> Path:
return Path(os.path.join(settings.LD_PREVIEW_FOLDER, preview_image_file))
def load_preview_image(url: str) -> str | None:
_ensure_preview_folder()
metadata = website_loader.load_website_metadata(url)
if not metadata.preview_image:
logger.debug(f"Could not find preview image in metadata: {url}")
return None
image_url = metadata.preview_image
logger.debug(f"Loading preview image: {image_url}")
with requests.get(image_url, stream=True) as response:
if response.status_code < 200 or response.status_code >= 300:
logger.debug(
f"Bad response status code for preview image: {image_url} status_code={response.status_code}"
)
return None
if "Content-Length" not in response.headers:
logger.debug(f"Empty Content-Length for preview image: {image_url}")
return None
content_length = int(response.headers["Content-Length"])
if content_length > settings.LD_PREVIEW_MAX_SIZE:
logger.debug(
f"Content-Length exceeds LD_PREVIEW_MAX_SIZE: {image_url} length={content_length}"
)
return None
if "Content-Type" not in response.headers:
logger.debug(f"Empty Content-Type for preview image: {image_url}")
return None
content_type = response.headers["Content-Type"].split(";", 1)[0]
file_extension = mimetypes.guess_extension(content_type)
if file_extension not in settings.LD_PREVIEW_ALLOWED_EXTENSIONS:
logger.debug(
f"Unsupported Content-Type for preview image: {image_url} content_type={content_type}"
)
return None
preview_image_hash = _url_to_filename(url)
preview_image_file = f"{preview_image_hash}{file_extension}"
preview_image_path = _get_image_path(preview_image_file)
with open(preview_image_path, "wb") as file:
downloaded = 0
for chunk in response.iter_content(chunk_size=8192):
downloaded += len(chunk)
if downloaded > content_length:
logger.debug(
f"Content-Length mismatch for preview image: {image_url} length={content_length} downloaded={downloaded}"
)
file.close()
preview_image_path.unlink()
return None
file.write(chunk)
logger.debug(f"Saved preview image as: {preview_image_path}")
return preview_image_file

View File

@@ -0,0 +1,58 @@
import gzip
import logging
import os
import shlex
import shutil
import signal
import subprocess
from django.conf import settings
class SingeFileError(Exception):
pass
logger = logging.getLogger(__name__)
def create_snapshot(url: str, filepath: str):
singlefile_path = settings.LD_SINGLEFILE_PATH
# parse options to list of arguments
ublock_options = shlex.split(settings.LD_SINGLEFILE_UBLOCK_OPTIONS)
custom_options = shlex.split(settings.LD_SINGLEFILE_OPTIONS)
temp_filepath = filepath + ".tmp"
# concat lists
args = [singlefile_path] + ublock_options + custom_options + [url, temp_filepath]
try:
# Use start_new_session=True to create a new process group
process = subprocess.Popen(args, start_new_session=True)
process.wait(timeout=settings.LD_SINGLEFILE_TIMEOUT_SEC)
# check if the file was created
if not os.path.exists(temp_filepath):
raise SingeFileError("Failed to create snapshot")
with open(temp_filepath, "rb") as raw_file, gzip.open(
filepath, "wb"
) as gz_file:
shutil.copyfileobj(raw_file, gz_file)
os.remove(temp_filepath)
except subprocess.TimeoutExpired:
# First try to terminate properly
try:
logger.error(
"Timeout expired while creating snapshot. Terminating process..."
)
process.terminate()
process.wait(timeout=20)
raise SingeFileError("Timeout expired while creating snapshot")
except subprocess.TimeoutExpired:
# Kill the whole process group, which should also clean up any chromium
# processes spawned by single-file
logger.error("Timeout expired while terminating. Killing process...")
os.killpg(os.getpgid(process.pid), signal.SIGTERM)
raise SingeFileError("Timeout expired while creating snapshot")
except subprocess.CalledProcessError as error:
raise SingeFileError(f"Failed to create snapshot: {error.stderr}")

View File

@@ -13,7 +13,7 @@ logger = logging.getLogger(__name__)
def get_or_create_tags(tag_names: List[str], user: User):
tags = [get_or_create_tag(tag_name, user) for tag_name in tag_names]
return unique(tags, operator.attrgetter('id'))
return unique(tags, operator.attrgetter("id"))
def get_or_create_tag(name: str, user: User):

View File

@@ -1,25 +1,62 @@
import functools
import logging
import os
from typing import List
import waybackpy
from background_task import background
from background_task.models import Task
from django.conf import settings
from django.contrib.auth import get_user_model
from django.contrib.auth.models import User
from django.db.models import Q
from django.utils import timezone, formats
from huey import crontab
from huey.contrib.djhuey import HUEY as huey
from huey.exceptions import TaskLockedException
from waybackpy.exceptions import WaybackError, TooManyRequestsError, NoCDXRecordFound
import bookmarks.services.wayback
from bookmarks.models import Bookmark, UserProfile
from bookmarks.services import favicon_loader
from bookmarks.models import Bookmark, BookmarkAsset, UserProfile
from bookmarks.services import favicon_loader, singlefile, preview_image_loader
from bookmarks.services.website_loader import DEFAULT_USER_AGENT
logger = logging.getLogger(__name__)
# Create custom decorator for Huey tasks that implements exponential backoff
# Taken from: https://huey.readthedocs.io/en/latest/guide.html#tips-and-tricks
# Retry 1: 60
# Retry 2: 240
# Retry 3: 960
# Retry 4: 3840
# Retry 5: 15360
def task(retries=5, retry_delay=15, retry_backoff=4):
def deco(fn):
@functools.wraps(fn)
def inner(*args, **kwargs):
task = kwargs.pop("task")
try:
return fn(*args, **kwargs)
except TaskLockedException as exc:
# Task locks are currently only used as workaround to enforce
# running specific types of tasks (e.g. singlefile snapshots)
# sequentially. In that case don't reduce the number of retries.
task.retries = retries
raise exc
except Exception as exc:
task.retry_delay *= retry_backoff
raise exc
return huey.task(retries=retries, retry_delay=retry_delay, context=True)(inner)
return deco
def is_web_archive_integration_active(user: User) -> bool:
background_tasks_enabled = not settings.LD_DISABLE_BACKGROUND_TASKS
web_archive_integration_enabled = \
user.profile.web_archive_integration == UserProfile.WEB_ARCHIVE_INTEGRATION_ENABLED
web_archive_integration_enabled = (
user.profile.web_archive_integration
== UserProfile.WEB_ARCHIVE_INTEGRATION_ENABLED
)
return background_tasks_enabled and web_archive_integration_enabled
@@ -31,31 +68,39 @@ def create_web_archive_snapshot(user: User, bookmark: Bookmark, force_update: bo
def _load_newest_snapshot(bookmark: Bookmark):
try:
logger.info(f'Load existing snapshot for bookmark. url={bookmark.url}')
cdx_api = bookmarks.services.wayback.CustomWaybackMachineCDXServerAPI(bookmark.url)
logger.info(f"Load existing snapshot for bookmark. url={bookmark.url}")
cdx_api = bookmarks.services.wayback.CustomWaybackMachineCDXServerAPI(
bookmark.url
)
existing_snapshot = cdx_api.newest()
if existing_snapshot:
bookmark.web_archive_snapshot_url = existing_snapshot.archive_url
bookmark.save(update_fields=['web_archive_snapshot_url'])
logger.info(f'Using newest snapshot. url={bookmark.url} from={existing_snapshot.datetime_timestamp}')
bookmark.save(update_fields=["web_archive_snapshot_url"])
logger.info(
f"Using newest snapshot. url={bookmark.url} from={existing_snapshot.datetime_timestamp}"
)
except NoCDXRecordFound:
logger.info(f'Could not find any snapshots for bookmark. url={bookmark.url}')
logger.info(f"Could not find any snapshots for bookmark. url={bookmark.url}")
except WaybackError as error:
logger.error(f'Failed to load existing snapshot. url={bookmark.url}', exc_info=error)
logger.error(
f"Failed to load existing snapshot. url={bookmark.url}", exc_info=error
)
def _create_snapshot(bookmark: Bookmark):
logger.info(f'Create new snapshot for bookmark. url={bookmark.url}...')
archive = waybackpy.WaybackMachineSaveAPI(bookmark.url, DEFAULT_USER_AGENT, max_tries=1)
logger.info(f"Create new snapshot for bookmark. url={bookmark.url}...")
archive = waybackpy.WaybackMachineSaveAPI(
bookmark.url, DEFAULT_USER_AGENT, max_tries=1
)
archive.save()
bookmark.web_archive_snapshot_url = archive.archive_url
bookmark.save(update_fields=['web_archive_snapshot_url'])
logger.info(f'Successfully created new snapshot for bookmark:. url={bookmark.url}')
bookmark.save(update_fields=["web_archive_snapshot_url"])
logger.info(f"Successfully created new snapshot for bookmark:. url={bookmark.url}")
@background()
@task()
def _create_web_archive_snapshot_task(bookmark_id: int, force_update: bool):
try:
bookmark = Bookmark.objects.get(id=bookmark_id)
@@ -72,16 +117,19 @@ def _create_web_archive_snapshot_task(bookmark_id: int, force_update: bool):
return
except TooManyRequestsError:
logger.error(
f'Failed to create snapshot due to rate limiting, trying to load newest snapshot as fallback. url={bookmark.url}')
f"Failed to create snapshot due to rate limiting, trying to load newest snapshot as fallback. url={bookmark.url}"
)
except WaybackError as error:
logger.error(f'Failed to create snapshot, trying to load newest snapshot as fallback. url={bookmark.url}',
exc_info=error)
logger.error(
f"Failed to create snapshot, trying to load newest snapshot as fallback. url={bookmark.url}",
exc_info=error,
)
# Load the newest snapshot as fallback
_load_newest_snapshot(bookmark)
@background()
@task()
def _load_web_archive_snapshot_task(bookmark_id: int):
try:
bookmark = Bookmark.objects.get(id=bookmark_id)
@@ -99,11 +147,14 @@ def schedule_bookmarks_without_snapshots(user: User):
_schedule_bookmarks_without_snapshots_task(user.id)
@background()
@task()
def _schedule_bookmarks_without_snapshots_task(user_id: int):
user = get_user_model().objects.get(id=user_id)
bookmarks_without_snapshots = Bookmark.objects.filter(web_archive_snapshot_url__exact='', owner=user)
bookmarks_without_snapshots = Bookmark.objects.filter(
web_archive_snapshot_url__exact="", owner=user
)
# TODO: Implement bulk task creation
for bookmark in bookmarks_without_snapshots:
# To prevent rate limit errors from the Wayback API only try to load the latest snapshots instead of creating
# new ones when processing bookmarks in bulk
@@ -116,26 +167,34 @@ def is_favicon_feature_active(user: User) -> bool:
return background_tasks_enabled and user.profile.enable_favicons
def is_preview_feature_active(user: User) -> bool:
return (
user.profile.enable_preview_images and not settings.LD_DISABLE_BACKGROUND_TASKS
)
def load_favicon(user: User, bookmark: Bookmark):
if is_favicon_feature_active(user):
_load_favicon_task(bookmark.id)
@background()
@task()
def _load_favicon_task(bookmark_id: int):
try:
bookmark = Bookmark.objects.get(id=bookmark_id)
except Bookmark.DoesNotExist:
return
logger.info(f'Load favicon for bookmark. url={bookmark.url}')
logger.info(f"Load favicon for bookmark. url={bookmark.url}")
new_favicon_file = favicon_loader.load_favicon(bookmark.url)
if new_favicon_file != bookmark.favicon_file:
bookmark.favicon_file = new_favicon_file
bookmark.save(update_fields=['favicon_file'])
logger.info(f'Successfully updated favicon for bookmark. url={bookmark.url} icon={new_favicon_file}')
bookmark.save(update_fields=["favicon_file"])
logger.info(
f"Successfully updated favicon for bookmark. url={bookmark.url} icon={new_favicon_file}"
)
def schedule_bookmarks_without_favicons(user: User):
@@ -143,17 +202,15 @@ def schedule_bookmarks_without_favicons(user: User):
_schedule_bookmarks_without_favicons_task(user.id)
@background()
@task()
def _schedule_bookmarks_without_favicons_task(user_id: int):
user = get_user_model().objects.get(id=user_id)
bookmarks = Bookmark.objects.filter(favicon_file__exact='', owner=user)
tasks = []
bookmarks = Bookmark.objects.filter(favicon_file__exact="", owner=user)
# TODO: Implement bulk task creation
for bookmark in bookmarks:
task = Task.objects.new_task(task_name='bookmarks.services.tasks._load_favicon_task', args=(bookmark.id,))
tasks.append(task)
Task.objects.bulk_create(tasks)
_load_favicon_task(bookmark.id)
pass
def schedule_refresh_favicons(user: User):
@@ -161,14 +218,180 @@ def schedule_refresh_favicons(user: User):
_schedule_refresh_favicons_task(user.id)
@background()
@task()
def _schedule_refresh_favicons_task(user_id: int):
user = get_user_model().objects.get(id=user_id)
bookmarks = Bookmark.objects.filter(owner=user)
tasks = []
# TODO: Implement bulk task creation
for bookmark in bookmarks:
task = Task.objects.new_task(task_name='bookmarks.services.tasks._load_favicon_task', args=(bookmark.id,))
tasks.append(task)
_load_favicon_task(bookmark.id)
Task.objects.bulk_create(tasks)
def load_preview_image(user: User, bookmark: Bookmark):
if is_preview_feature_active(user):
_load_preview_image_task(bookmark.id)
@task()
def _load_preview_image_task(bookmark_id: int):
try:
bookmark = Bookmark.objects.get(id=bookmark_id)
except Bookmark.DoesNotExist:
return
logger.info(f"Load preview image for bookmark. url={bookmark.url}")
new_preview_image_file = preview_image_loader.load_preview_image(bookmark.url)
if new_preview_image_file != bookmark.preview_image_file:
bookmark.preview_image_file = new_preview_image_file or ""
bookmark.save(update_fields=["preview_image_file"])
logger.info(
f"Successfully updated preview image for bookmark. url={bookmark.url} preview_image_file={new_preview_image_file}"
)
def schedule_bookmarks_without_previews(user: User):
if is_preview_feature_active(user):
_schedule_bookmarks_without_previews_task(user.id)
@task()
def _schedule_bookmarks_without_previews_task(user_id: int):
user = get_user_model().objects.get(id=user_id)
bookmarks = Bookmark.objects.filter(
Q(preview_image_file__exact=""),
owner=user,
)
# TODO: Implement bulk task creation
for bookmark in bookmarks:
try:
_load_preview_image_task(bookmark.id)
except Exception as exc:
logging.exception(exc)
def is_html_snapshot_feature_active() -> bool:
return settings.LD_ENABLE_SNAPSHOTS and not settings.LD_DISABLE_BACKGROUND_TASKS
def create_html_snapshot(bookmark: Bookmark):
if not is_html_snapshot_feature_active():
return
asset = _create_snapshot_asset(bookmark)
asset.save()
def create_html_snapshots(bookmark_list: List[Bookmark]):
if not is_html_snapshot_feature_active():
return
assets_to_create = []
for bookmark in bookmark_list:
asset = _create_snapshot_asset(bookmark)
assets_to_create.append(asset)
BookmarkAsset.objects.bulk_create(assets_to_create)
MAX_SNAPSHOT_FILENAME_LENGTH = 192
def _create_snapshot_asset(bookmark: Bookmark) -> BookmarkAsset:
timestamp = formats.date_format(timezone.now(), "SHORT_DATE_FORMAT")
asset = BookmarkAsset(
bookmark=bookmark,
asset_type=BookmarkAsset.TYPE_SNAPSHOT,
content_type="text/html",
display_name=f"HTML snapshot from {timestamp}",
status=BookmarkAsset.STATUS_PENDING,
)
return asset
def _generate_snapshot_filename(asset: BookmarkAsset) -> str:
def sanitize_char(char):
if char.isalnum() or char in ("-", "_", "."):
return char
else:
return "_"
formatted_datetime = asset.date_created.strftime("%Y-%m-%d_%H%M%S")
sanitized_url = "".join(sanitize_char(char) for char in asset.bookmark.url)
# Calculate the length of the non-URL parts of the filename
non_url_length = len(f"{asset.asset_type}{formatted_datetime}__.html.gz")
# Calculate the maximum length for the URL part
max_url_length = MAX_SNAPSHOT_FILENAME_LENGTH - non_url_length
# Truncate the URL if necessary
sanitized_url = sanitized_url[:max_url_length]
return f"{asset.asset_type}_{formatted_datetime}_{sanitized_url}.html.gz"
# singe-file does not support running multiple instances in parallel, so we can
# not queue up multiple snapshot tasks at once. Instead, schedule a periodic
# task that grabs a number of pending assets and creates snapshots for them in
# sequence. The task uses a lock to ensure that a new task isn't scheduled
# before the previous one has finished.
@huey.periodic_task(crontab(minute="*"))
@huey.lock_task("schedule-html-snapshots-lock")
def _schedule_html_snapshots_task():
# Get five pending assets
assets = BookmarkAsset.objects.filter(status=BookmarkAsset.STATUS_PENDING).order_by(
"date_created"
)[:5]
for asset in assets:
_create_html_snapshot_task(asset.id)
def _create_html_snapshot_task(asset_id: int):
try:
asset = BookmarkAsset.objects.get(id=asset_id)
except BookmarkAsset.DoesNotExist:
return
logger.info(f"Create HTML snapshot for bookmark. url={asset.bookmark.url}")
try:
filename = _generate_snapshot_filename(asset)
filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
singlefile.create_snapshot(asset.bookmark.url, filepath)
asset.status = BookmarkAsset.STATUS_COMPLETE
asset.file = filename
asset.gzip = True
asset.save()
logger.info(
f"Successfully created HTML snapshot for bookmark. url={asset.bookmark.url}"
)
except Exception as error:
logger.error(
f"Failed to HTML snapshot for bookmark. url={asset.bookmark.url}",
exc_info=error,
)
asset.status = BookmarkAsset.STATUS_FAILURE
asset.save()
def create_missing_html_snapshots(user: User) -> int:
if not is_html_snapshot_feature_active():
return 0
bookmarks_without_snapshots = Bookmark.objects.filter(owner=user).exclude(
bookmarkasset__asset_type=BookmarkAsset.TYPE_SNAPSHOT,
bookmarkasset__status__in=[
BookmarkAsset.STATUS_PENDING,
BookmarkAsset.STATUS_COMPLETE,
],
)
bookmarks_without_snapshots |= Bookmark.objects.filter(owner=user).exclude(
bookmarkasset__asset_type=BookmarkAsset.TYPE_SNAPSHOT
)
create_html_snapshots(list(bookmarks_without_snapshots))
return bookmarks_without_snapshots.count()

View File

@@ -14,8 +14,10 @@ class CustomWaybackMachineCDXServerAPI(waybackpy.WaybackMachineCDXServerAPI):
def newest(self):
unix_timestamp = int(time.time())
self.closest = waybackpy.utils.unix_timestamp_to_wayback_timestamp(unix_timestamp)
self.sort = 'closest'
self.closest = waybackpy.utils.unix_timestamp_to_wayback_timestamp(
unix_timestamp
)
self.sort = "closest"
self.limit = -5
newest_snapshot = None
@@ -37,4 +39,4 @@ class CustomWaybackMachineCDXServerAPI(waybackpy.WaybackMachineCDXServerAPI):
super().add_payload(payload)
# Set fastLatest query param, as we are only using this API to get the latest snapshot and using fastLatest
# makes searching for latest snapshots faster
payload['fastLatest'] = 'true'
payload["fastLatest"] = "true"

View File

@@ -1,6 +1,7 @@
import logging
from dataclasses import dataclass
from functools import lru_cache
from urllib.parse import urljoin
import requests
from bs4 import BeautifulSoup
@@ -15,12 +16,14 @@ class WebsiteMetadata:
url: str
title: str
description: str
preview_image: str | None
def to_dict(self):
return {
'url': self.url,
'title': self.title,
'description': self.description,
"url": self.url,
"title": self.title,
"description": self.description,
"preview_image": self.preview_image,
}
@@ -30,23 +33,47 @@ class WebsiteMetadata:
def load_website_metadata(url: str):
title = None
description = None
preview_image = None
try:
start = timezone.now()
page_text = load_page(url)
end = timezone.now()
logger.debug(f'Load duration: {end - start}')
logger.debug(f"Load duration: {end - start}")
start = timezone.now()
soup = BeautifulSoup(page_text, 'html.parser')
soup = BeautifulSoup(page_text, "html.parser")
title = soup.title.string.strip() if soup.title is not None else None
description_tag = soup.find('meta', attrs={'name': 'description'})
description = description = description_tag['content'].strip() if description_tag and description_tag[
'content'] else None
description_tag = soup.find("meta", attrs={"name": "description"})
description = (
description_tag["content"].strip()
if description_tag and description_tag["content"]
else None
)
if not description:
description_tag = soup.find("meta", attrs={"property": "og:description"})
description = (
description_tag["content"].strip()
if description_tag and description_tag["content"]
else None
)
image_tag = soup.find("meta", attrs={"property": "og:image"})
preview_image = image_tag["content"].strip() if image_tag else None
if (
preview_image
and not preview_image.startswith("http://")
and not preview_image.startswith("https://")
):
preview_image = urljoin(url, preview_image)
end = timezone.now()
logger.debug(f'Parsing duration: {end - start}')
logger.debug(f"Parsing duration: {end - start}")
finally:
return WebsiteMetadata(url=url, title=title, description=description)
return WebsiteMetadata(
url=url, title=title, description=description, preview_image=preview_image
)
CHUNK_SIZE = 50 * 1024
@@ -68,30 +95,30 @@ def load_page(url: str):
else:
content = content + chunk
logger.debug(f'Loaded chunk (iteration={iteration}, total={size / 1024})')
logger.debug(f"Loaded chunk (iteration={iteration}, total={size / 1024})")
# Stop reading if we have parsed end of head tag
end_of_head = '</head>'.encode('utf-8')
end_of_head = "</head>".encode("utf-8")
if end_of_head in content:
logger.debug(f'Found closing head tag after {size} bytes')
logger.debug(f"Found closing head tag after {size} bytes")
content = content.split(end_of_head)[0] + end_of_head
break
# Stop reading if we exceed limit
if size > MAX_CONTENT_LIMIT:
logger.debug(f'Cancel reading document after {size} bytes')
logger.debug(f"Cancel reading document after {size} bytes")
break
if hasattr(r, '_content_consumed'):
logger.debug(f'Request consumed: {r._content_consumed}')
if hasattr(r, "_content_consumed"):
logger.debug(f"Request consumed: {r._content_consumed}")
# Use charset_normalizer to determine encoding that best matches the response content
# Several sites seem to specify the response encoding incorrectly, so we ignore it and use custom logic instead
# This is different from Response.text which does respect the encoding specified in the response first,
# before trying to determine one
results = from_bytes(content or '')
results = from_bytes(content or "")
return str(results.best())
DEFAULT_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.0.0 Safari/537.36'
DEFAULT_USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.0.0 Safari/537.36"
def fake_request_headers():

View File

@@ -15,9 +15,11 @@ def user_logged_in(sender, request, user, **kwargs):
def extend_sqlite(connection=None, **kwargs):
# Load ICU extension into Sqlite connection to support case-insensitive
# comparisons with unicode characters
if connection.vendor == 'sqlite' and settings.USE_SQLITE_ICU_EXTENSION:
if connection.vendor == "sqlite" and settings.USE_SQLITE_ICU_EXTENSION:
connection.connection.enable_load_extension(True)
connection.connection.load_extension(settings.SQLITE_ICU_EXTENSION_PATH.rstrip('.so'))
connection.connection.load_extension(
settings.SQLITE_ICU_EXTENSION_PATH.rstrip(".so")
)
with connection.cursor() as cursor:
# Load an ICU collation for case-insensitive ordering.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 2.5 KiB

Some files were not shown because too many files have changed in this diff Show More