Bump version

* Promoting singlefile timeout to env variable

* Promoting singlefile timeout to env variable

* add tests

* Add LD_SINGLEFILE_OPTIONS support

* add tests

---------

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>

.dockerignore

@@ -5,7 +5,6 @@
 !/bookmarks
 !/siteroot
 
-!/background-tasks-wrapper.sh
 !/bootstrap.sh
 !/LICENSE.txt
 !/manage.py

.github/workflows/main.yaml

@@ -24,7 +24,9 @@ jobs:
       - name: Install Node dependencies
         run: npm ci
       - name: Setup Python environment
-        run: pip install -r requirements.txt -r requirements.dev.txt
+        run: |
+          pip install -r requirements.txt -r requirements.dev.txt
+          mkdir data
       - name: Run tests
         run: python manage.py test bookmarks.tests
   e2e_tests:
@@ -47,6 +49,7 @@ jobs:
         run: |
           pip install -r requirements.txt -r requirements.dev.txt
           playwright install chromium
+          mkdir data
       - name: Run build
         run: |
           npm run build

CHANGELOG.md

@@ -1,5 +1,44 @@
 # Changelog
 
+## v1.27.1 (07/04/2024)
+
+### What's Changed
+* Fix HTML snapshot errors related to single-file-cli by @sissbruecker in https://github.com/sissbruecker/linkding/pull/683
+* Replace django-background-tasks with huey by @sissbruecker in https://github.com/sissbruecker/linkding/pull/657
+* Add Authelia OIDC example to docs by @hugo-vrijswijk in https://github.com/sissbruecker/linkding/pull/675
+
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.27.0...v1.27.1
+
+---
+
+## v1.27.0 (01/04/2024)
+
+### What's Changed
+* Archive snapshots of websites locally by @sissbruecker in https://github.com/sissbruecker/linkding/pull/672
+* Add Railway hosting option by @tianheg in https://github.com/sissbruecker/linkding/pull/661
+* Add how to for increasing the font size by @sissbruecker in https://github.com/sissbruecker/linkding/pull/667
+
+### New Contributors
+* @tianheg made their first contribution in https://github.com/sissbruecker/linkding/pull/661
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.26.0...v1.27.0
+
+---
+
+## v1.26.0 (30/03/2024)
+
+### What's Changed
+* Add option for showing bookmark description as separate block by @sissbruecker in https://github.com/sissbruecker/linkding/pull/663
+* Add bookmark details view by @sissbruecker in https://github.com/sissbruecker/linkding/pull/665
+* Make bookmark list actions configurable by @sissbruecker in https://github.com/sissbruecker/linkding/pull/666
+* Bump black from 24.1.1 to 24.3.0 by @dependabot in https://github.com/sissbruecker/linkding/pull/662
+
+
+**Full Changelog**: https://github.com/sissbruecker/linkding/compare/v1.25.0...v1.26.0
+
+---
+
 ## v1.25.0 (18/03/2024)
 
 ### What's Changed

README.md

@@ -33,16 +33,13 @@ The name comes from:
 **Feature Overview:**
 - Clean UI optimized for readability
 - Organize bookmarks with tags
-- Add notes using Markdown
-- Read it later functionality
-- Share bookmarks with other users
-- Bulk editing
+- Bulk editing, Markdown notes, read it later functionality
+- Share bookmarks with other users or guests
 - Automatically provides titles, descriptions and icons of bookmarked websites
-- Automatically creates snapshots of bookmarked websites on [the Internet Archive Wayback Machine](https://archive.org/web/)
+- Automatically archive websites, either as local HTML file or on Internet Archive
 - Import and export bookmarks in Netscape HTML format
 - Installable as a Progressive Web App (PWA)
 - Extensions for [Firefox](https://addons.mozilla.org/firefox/addon/linkding-extension/) and [Chrome](https://chrome.google.com/webstore/detail/linkding-extension/beakmhbijpdhipnjhnclmhgjlddhidpe), as well as a bookmarklet
-- Light and dark themes
 - SSO support via OIDC or authentication proxies
 - REST API for developing 3rd party apps
 - Admin panel for user self-service and raw data access
@@ -62,27 +59,45 @@ The Docker image is compatible with ARM platforms, so it can be run on a Raspber
 linkding uses an SQLite database by default.
 Alternatively linkding supports PostgreSQL, see the [database options](docs/Options.md#LD_DB_ENGINE) for more information.
 
-<details>
-
-<summary>🧪 Alpine-based image</summary>
-
-The default Docker image (`latest` tag) is based on a slim variant of Debian Linux.
-Alternatively, there is an image based on Alpine Linux (`latest-alpine` tag) which has a smaller size, resulting in a smaller download and less disk space required.
-The Alpine image is currently about 45 MB in compressed size, compared to about 130 MB for the Debian image.
-
-To use it, replace the `latest` tag with `latest-alpine`, either in the CLI command below when using Docker, or in the `docker-compose.yml` file when using docker-compose.
-
-> [!WARNING]
-> The image is currently considered experimental in order to gather feedback and iron out any issues.
-> Only use it if you are comfortable running experimental software or want to help out with testing.
-> While there should be no issues with creating new installations, there might be issues when migrating existing installations.
-> If you plan to migrate your existing installation, make sure to create proper [backups](https://github.com/sissbruecker/linkding/blob/master/docs/backup.md) first.
-
-</details>
-
 ###  Using Docker
 
-To install linkding using Docker you can just run the [latest image](https://hub.docker.com/repository/docker/sissbruecker/linkding) from Docker Hub:
+The Docker image comes in several variants. To use a different image than the default, replace `latest` with the desired tag in the commands below, or in the docker-compose file.
+
+<table>
+  <thead>
+    <tr>
+      <th>Tag</th>
+      <th>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><code>latest</code></td>
+      <td>Provides the basic functionality of linkding</td>
+    </tr>
+    <tr>
+      <td><code>latest-plus</code></td>
+      <td>
+        Includes feature for archiving websites as HTML snapshots
+        <ul>
+          <li>Significantly larger image size as it includes a Chromium installation</li>
+          <li>Requires more runtime memory to run Chromium</li>
+          <li>Requires more disk space for storing HTML snapshots</li>
+        </ul>            
+      </td>
+    </tr>
+    <tr>
+      <td><code>latest-alpine</code></td>
+      <td><code>latest</code>, but based on Alpine Linux. 🧪 Experimental</td>
+    </tr>    
+    <tr>
+      <td><code>latest-plus-alpine</code></td>
+      <td><code>latest-plus</code>, but based on Alpine Linux. 🧪 Experimental</td>
+    </tr>    
+  </tbody>
+</table>
+
+To install linkding using Docker you can just run the image from [Docker Hub](https://hub.docker.com/repository/docker/sissbruecker/linkding):
 ```shell
 docker run --name linkding -p 9090:9090 -v {host-data-folder}:/etc/linkding/data -d sissbruecker/linkding:latest
 ```
@@ -184,6 +199,7 @@ Self-hosting web applications still requires a lot of technical know-how and com
 - [linkding on fly.io](https://github.com/fspoettel/linkding-on-fly) - Guide for hosting a linkding installation on [fly.io](https://fly.io). By [fspoettel](https://github.com/fspoettel)
 - [PikaPods.com](https://www.pikapods.com/) - Managed hosting for linkding, EU and US regions available. [1-click setup link](https://www.pikapods.com/pods?run=linkding) ([Disclosure](#pikapods))
 - [CapRover](https://caprover.com/) - Linkding is included as a default one-click app
+- [linkding on railway.app](https://github.com/tianheg/linkding-on-railway) - Guide for hosting a linkding installation on [railway.app](https://railway.app/). By [tianheg](https://github.com/tianheg)
 
 ## Documentation
 

background-tasks-wrapper.sh

@@ -1,5 +0,0 @@
-#!/usr/bin/env bash
-# Wrapper script used by supervisord to first clear task locks before starting the background task processor
-
-python manage.py clean_tasks
-exec python manage.py process_tasks

bookmarks/admin.py

@@ -1,22 +1,96 @@
-from background_task.admin import TaskAdmin, CompletedTaskAdmin
-from background_task.models import Task, CompletedTask
 from django.contrib import admin, messages
 from django.contrib.admin import AdminSite
 from django.contrib.auth.admin import UserAdmin
 from django.contrib.auth.models import User
+from django.core.paginator import Paginator
 from django.db.models import Count, QuerySet
+from django.shortcuts import render
+from django.urls import path
 from django.utils.translation import ngettext, gettext
+from huey.contrib.djhuey import HUEY as huey
 from rest_framework.authtoken.admin import TokenAdmin
 from rest_framework.authtoken.models import TokenProxy
 
-from bookmarks.models import Bookmark, Tag, UserProfile, Toast, FeedToken
+from bookmarks.models import Bookmark, BookmarkAsset, Tag, UserProfile, Toast, FeedToken
 from bookmarks.services.bookmarks import archive_bookmark, unarchive_bookmark
 
 
+# Custom paginator to paginate through Huey tasks
+class TaskPaginator(Paginator):
+    def __init__(self):
+        super().__init__(self, 100)
+        self.task_count = huey.storage.queue_size()
+
+    @property
+    def count(self):
+        return self.task_count
+
+    def page(self, number):
+        limit = self.per_page
+        offset = (number - 1) * self.per_page
+        return self._get_page(
+            self.enqueued_items(limit, offset),
+            number,
+            self,
+        )
+
+    # Copied from Huey's SqliteStorage with some modifications to allow pagination
+    def enqueued_items(self, limit, offset):
+        to_bytes = lambda b: bytes(b) if not isinstance(b, bytes) else b
+        sql = "select data from task where queue=? order by priority desc, id limit ? offset ?"
+        params = (huey.storage.name, limit, offset)
+
+        serialized_tasks = [
+            to_bytes(i) for i, in huey.storage.sql(sql, params, results=True)
+        ]
+        return [huey.deserialize_task(task) for task in serialized_tasks]
+
+
+# Custom view to display Huey tasks in the admin
+def background_task_view(request):
+    page_number = int(request.GET.get("p", 1))
+    paginator = TaskPaginator()
+    page = paginator.get_page(page_number)
+    page_range = paginator.get_elided_page_range(page_number, on_each_side=2, on_ends=2)
+    context = {
+        **linkding_admin_site.each_context(request),
+        "title": "Background tasks",
+        "page": page,
+        "page_range": page_range,
+        "tasks": page.object_list,
+    }
+    return render(request, "admin/background_tasks.html", context)
+
+
 class LinkdingAdminSite(AdminSite):
     site_header = "linkding administration"
     site_title = "linkding Admin"
 
+    def get_urls(self):
+        urls = super().get_urls()
+        custom_urls = [
+            path("tasks/", background_task_view, name="background_tasks"),
+        ]
+        return custom_urls + urls
+
+    def get_app_list(self, request, app_label=None):
+        app_list = super().get_app_list(request, app_label)
+        app_list += [
+            {
+                "name": "Huey",
+                "app_label": "huey_app",
+                "models": [
+                    {
+                        "name": "Queued tasks",
+                        "object_name": "background_tasks",
+                        "admin_url": "/admin/tasks/",
+                        "view_only": True,
+                    }
+                ],
+            }
+        ]
+        return app_list
+
 
 class AdminBookmark(admin.ModelAdmin):
     list_display = ("resolved_title", "url", "is_archived", "owner", "date_added")
@@ -125,6 +199,15 @@ class AdminBookmark(admin.ModelAdmin):
         )
 
 
+class AdminBookmarkAsset(admin.ModelAdmin):
+    list_display = ("display_name", "date_created", "status")
+    search_fields = (
+        "display_name",
+        "file",
+    )
+    list_filter = ("status",)
+
+
 class AdminTag(admin.ModelAdmin):
     list_display = ("name", "bookmarks_count", "owner", "date_added")
     search_fields = ("name", "owner__username")
@@ -200,10 +283,9 @@ class AdminFeedToken(admin.ModelAdmin):
 
 linkding_admin_site = LinkdingAdminSite()
 linkding_admin_site.register(Bookmark, AdminBookmark)
+linkding_admin_site.register(BookmarkAsset, AdminBookmarkAsset)
 linkding_admin_site.register(Tag, AdminTag)
 linkding_admin_site.register(User, AdminCustomUser)
 linkding_admin_site.register(TokenProxy, TokenAdmin)
 linkding_admin_site.register(Toast, AdminToast)
 linkding_admin_site.register(FeedToken, AdminFeedToken)
-linkding_admin_site.register(Task, TaskAdmin)
-linkding_admin_site.register(CompletedTask, CompletedTaskAdmin)

bookmarks/frontend/behaviors/bookmark-details.js

@@ -1,38 +0,0 @@
-import { registerBehavior } from "./index";
-
-class BookmarkDetails {
-  constructor(element) {
-    this.form = element.querySelector(".status form");
-    if (!this.form) {
-      // Form may not exist if user does not own the bookmark
-      return;
-    }
-    this.form.addEventListener("submit", (event) => {
-      event.preventDefault();
-      this.submitForm();
-    });
-
-    const inputs = this.form.querySelectorAll("input");
-    inputs.forEach((input) => {
-      input.addEventListener("change", () => {
-        this.submitForm();
-      });
-    });
-  }
-
-  async submitForm() {
-    const url = this.form.action;
-    const formData = new FormData(this.form);
-
-    await fetch(url, {
-      method: "POST",
-      body: formData,
-      redirect: "manual", // ignore redirect
-    });
-
-    // Refresh bookmark page if it exists
-    document.dispatchEvent(new CustomEvent("bookmark-page-refresh"));
-  }
-}
-
-registerBehavior("ld-bookmark-details", BookmarkDetails);

bookmarks/frontend/behaviors/bookmark-page.js

@@ -1,4 +1,4 @@
-import { registerBehavior, swap } from "./index";
+import { registerBehavior, swapContent } from "./index";
 
 class BookmarkPage {
   constructor(element) {
@@ -37,8 +37,8 @@ class BookmarkPage {
       fetch(`${bookmarksUrl}${query}`).then((response) => response.text()),
       fetch(`${tagsUrl}${query}`).then((response) => response.text()),
     ]).then(([bookmarkListHtml, tagCloudHtml]) => {
-      swap(this.bookmarkList, bookmarkListHtml);
-      swap(this.tagCloud, tagCloudHtml);
+      swapContent(this.bookmarkList, bookmarkListHtml);
+      swapContent(this.tagCloud, tagCloudHtml);
 
       // Dispatch list updated event
       const listElement = this.bookmarkList.querySelector(

bookmarks/frontend/behaviors/form.js

@@ -0,0 +1,54 @@
+import { registerBehavior, swap } from "./index";
+
+class FormBehavior {
+  constructor(element) {
+    this.element = element;
+    element.addEventListener("submit", this.onFormSubmit.bind(this));
+  }
+
+  async onFormSubmit(event) {
+    event.preventDefault();
+
+    const url = this.element.action;
+    const formData = new FormData(this.element);
+    if (event.submitter) {
+      formData.append(event.submitter.name, event.submitter.value);
+    }
+
+    await fetch(url, {
+      method: "POST",
+      body: formData,
+      redirect: "manual", // ignore redirect
+    });
+
+    // Dispatch refresh events
+    const refreshEvents = this.element.getAttribute("refresh-events");
+    if (refreshEvents) {
+      refreshEvents.split(",").forEach((eventName) => {
+        document.dispatchEvent(new CustomEvent(eventName));
+      });
+    }
+
+    // Refresh form
+    await this.refresh();
+  }
+
+  async refresh() {
+    const refreshUrl = this.element.getAttribute("refresh-url");
+    const html = await fetch(refreshUrl).then((response) => response.text());
+    swap(this.element, html);
+  }
+}
+
+class FormAutoSubmitBehavior {
+  constructor(element) {
+    this.element = element;
+    this.element.addEventListener("change", () => {
+      const form = this.element.closest("form");
+      form.dispatchEvent(new Event("submit", { cancelable: true }));
+    });
+  }
+}
+
+registerBehavior("ld-form", FormBehavior);
+registerBehavior("ld-form-auto-submit", FormAutoSubmitBehavior);

bookmarks/frontend/behaviors/index.js

@@ -12,7 +12,14 @@ export function applyBehaviors(container, behaviorNames = null) {
 
   behaviorNames.forEach((behaviorName) => {
     const behavior = behaviorRegistry[behaviorName];
-    const elements = container.querySelectorAll(`[${behaviorName}]`);
+    const elements = Array.from(
+      container.querySelectorAll(`[${behaviorName}]`),
+    );
+
+    // Include the container element if it has the behavior
+    if (container.hasAttribute && container.hasAttribute(behaviorName)) {
+      elements.push(container);
+    }
 
     elements.forEach((element) => {
       element.__behaviors = element.__behaviors || [];
@@ -31,6 +38,13 @@ export function applyBehaviors(container, behaviorNames = null) {
 }
 
 export function swap(element, html) {
+  const dom = new DOMParser().parseFromString(html, "text/html");
+  const newElement = dom.body.firstChild;
+  element.replaceWith(newElement);
+  applyBehaviors(newElement);
+}
+
+export function swapContent(element, html) {
   element.innerHTML = html;
   applyBehaviors(element);
 }

bookmarks/frontend/components/TagAutocomplete.svelte

@@ -150,17 +150,27 @@
         display: block;
     }
 
+    .form-autocomplete-input {
+        box-sizing: border-box;
+        height: var(--control-size);
+        min-height: var(--control-size);
+        padding: 0;
+    }
+
+    .form-autocomplete-input input {
+        width: 100%;
+        height: 100%;
+        border: none;
+        margin: 0;
+    }
+
     .form-autocomplete.small .form-autocomplete-input {
         height: var(--control-size-sm);
         min-height: var(--control-size-sm);
-        padding: 0.05rem 0.3rem;
     }
 
     .form-autocomplete.small .form-autocomplete-input input {
-        width: 100%;
-        height: 100%;
-        margin: 0;
-        padding: 0;
+        padding: 0.05rem 0.3rem;
         font-size: var(--font-size-sm);
     }
 

bookmarks/frontend/index.js

@@ -1,8 +1,8 @@
-import "./behaviors/bookmark-details";
 import "./behaviors/bookmark-page";
 import "./behaviors/bulk-edit";
 import "./behaviors/confirm-button";
 import "./behaviors/dropdown";
+import "./behaviors/form";
 import "./behaviors/modal";
 import "./behaviors/global-shortcuts";
 import "./behaviors/tag-autocomplete";

bookmarks/management/commands/backup.py

@@ -24,3 +24,8 @@ class Command(BaseCommand):
         source_db.close()
 
         self.stdout.write(self.style.SUCCESS(f"Backup created at {destination}"))
+        self.stdout.write(
+            self.style.WARNING(
+                "This backup method is deprecated and may be removed in the future. Please use the full_backup command instead, which creates backup zip file with all contents of the data folder."
+            )
+        )

bookmarks/management/commands/clean_tasks.py

@@ -1,15 +0,0 @@
-from background_task.models import Task, CompletedTask
-from django.core.management.base import BaseCommand
-
-
-class Command(BaseCommand):
-    help = "Remove task locks and clear completed task history"
-
-    def handle(self, *args, **options):
-        # Remove task locks
-        # If the background task processor exited while executing tasks, these tasks would still be marked as locked,
-        # even though no process is working on them, and would prevent the task processor from picking the next task in
-        # the queue
-        Task.objects.all().update(locked_by=None, locked_at=None)
-        # Clear task history to prevent them from bloating the DB
-        CompletedTask.objects.all().delete()

bookmarks/management/commands/full_backup.py

@@ -0,0 +1,62 @@
+import sqlite3
+import os
+import tempfile
+import zipfile
+
+from django.core.management.base import BaseCommand
+
+
+class Command(BaseCommand):
+    help = "Creates a backup of the linkding data folder"
+
+    def add_arguments(self, parser):
+        parser.add_argument("backup_file", type=str, help="Backup zip file destination")
+
+    def handle(self, *args, **options):
+        backup_file = options["backup_file"]
+        with zipfile.ZipFile(backup_file, "w", zipfile.ZIP_DEFLATED) as zip_file:
+            # Backup the database
+            self.stdout.write("Create database backup...")
+            with tempfile.TemporaryDirectory() as temp_dir:
+                backup_db_file = os.path.join(temp_dir, "db.sqlite3")
+                self.backup_database(backup_db_file)
+                zip_file.write(backup_db_file, "db.sqlite3")
+
+            # Backup the assets folder
+            if not os.path.exists(os.path.join("data", "assets")):
+                self.stdout.write(
+                    self.style.WARNING("No assets folder found. Skipping...")
+                )
+            else:
+                self.stdout.write("Backup bookmark assets...")
+                assets_folder = os.path.join("data", "assets")
+                for root, _, files in os.walk(assets_folder):
+                    for file in files:
+                        file_path = os.path.join(root, file)
+                        zip_file.write(file_path, os.path.join("assets", file))
+
+            # Backup the favicons folder
+            if not os.path.exists(os.path.join("data", "favicons")):
+                self.stdout.write(
+                    self.style.WARNING("No favicons folder found. Skipping...")
+                )
+            else:
+                self.stdout.write("Backup bookmark favicons...")
+                favicons_folder = os.path.join("data", "favicons")
+                for root, _, files in os.walk(favicons_folder):
+                    for file in files:
+                        file_path = os.path.join(root, file)
+                        zip_file.write(file_path, os.path.join("favicons", file))
+
+        self.stdout.write(self.style.SUCCESS(f"Backup created at {backup_file}"))
+
+    def backup_database(self, backup_db_file):
+        def progress(status, remaining, total):
+            self.stdout.write(f"Copied {total-remaining} of {total} pages...")
+
+        source_db = sqlite3.connect(os.path.join("data", "db.sqlite3"))
+        backup_db = sqlite3.connect(backup_db_file)
+        with backup_db:
+            source_db.backup(backup_db, pages=50, progress=progress)
+        backup_db.close()
+        source_db.close()

bookmarks/management/commands/migrate_tasks.py

@@ -0,0 +1,75 @@
+import json
+import os
+import sqlite3
+import importlib
+
+from django.core.management.base import BaseCommand
+
+
+class Command(BaseCommand):
+    help = "Migrate tasks from django-background-tasks to Huey"
+
+    def handle(self, *args, **options):
+        db = sqlite3.connect(os.path.join("data", "db.sqlite3"))
+
+        # Check if background_task table exists
+        cursor = db.cursor()
+        cursor.execute(
+            "SELECT name FROM sqlite_master WHERE type='table' AND name='background_task'"
+        )
+        row = cursor.fetchone()
+        if not row:
+            self.stdout.write(
+                "Legacy task table does not exist. Skipping task migration"
+            )
+            return
+
+        # Load legacy tasks
+        cursor.execute("SELECT id, task_name, task_params FROM background_task")
+        legacy_tasks = cursor.fetchall()
+
+        if len(legacy_tasks) == 0:
+            self.stdout.write("No legacy tasks found. Skipping task migration")
+            return
+
+        self.stdout.write(
+            f"Found {len(legacy_tasks)} legacy tasks. Migrating to Huey..."
+        )
+
+        # Migrate tasks to Huey
+        succeeded_tasks = []
+        for task in legacy_tasks:
+            task_id = task[0]
+            task_name = task[1]
+            task_params_json = task[2]
+            try:
+                task_params = json.loads(task_params_json)
+                function_params = task_params[0]
+
+                # Resolve task function
+                module_name, func_name = task_name.rsplit(".", 1)
+                module = importlib.import_module(module_name)
+                func = getattr(module, func_name)
+
+                # Call task function
+                func(*function_params)
+                succeeded_tasks.append(task_id)
+            except Exception:
+                self.stderr.write(f"Error migrating task [{task_id}] {task_name}")
+
+        self.stdout.write(f"Migrated {len(succeeded_tasks)} tasks successfully")
+
+        # Clean up
+        try:
+            placeholders = ", ".join("?" for _ in succeeded_tasks)
+            sql = f"DELETE FROM background_task WHERE id IN ({placeholders})"
+            cursor.execute(sql, succeeded_tasks)
+            db.commit()
+            self.stdout.write(
+                f"Deleted {len(succeeded_tasks)} migrated tasks from legacy table"
+            )
+        except Exception:
+            self.stderr.write("Error cleaning up legacy tasks")
+
+        cursor.close()
+        db.close()

bookmarks/migrations/0030_bookmarkasset.py

@@ -0,0 +1,43 @@
+# Generated by Django 5.0.2 on 2024-03-31 08:21
+
+import django.db.models.deletion
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0029_bookmark_list_actions_toast"),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name="BookmarkAsset",
+            fields=[
+                (
+                    "id",
+                    models.AutoField(
+                        auto_created=True,
+                        primary_key=True,
+                        serialize=False,
+                        verbose_name="ID",
+                    ),
+                ),
+                ("date_created", models.DateTimeField(auto_now_add=True)),
+                ("file", models.CharField(blank=True, max_length=2048)),
+                ("file_size", models.IntegerField(null=True)),
+                ("asset_type", models.CharField(max_length=64)),
+                ("content_type", models.CharField(max_length=128)),
+                ("display_name", models.CharField(blank=True, max_length=2048)),
+                ("status", models.CharField(max_length=64)),
+                ("gzip", models.BooleanField(default=False)),
+                (
+                    "bookmark",
+                    models.ForeignKey(
+                        on_delete=django.db.models.deletion.CASCADE,
+                        to="bookmarks.bookmark",
+                    ),
+                ),
+            ],
+        ),
+    ]

bookmarks/migrations/0031_userprofile_enable_automatic_html_snapshots.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.0.2 on 2024-04-01 10:29
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0030_bookmarkasset"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="userprofile",
+            name="enable_automatic_html_snapshots",
+            field=models.BooleanField(default=True),
+        ),
+    ]

bookmarks/migrations/0032_html_snapshots_hint_toast.py

@@ -0,0 +1,34 @@
+# Generated by Django 5.0.2 on 2024-04-01 12:17
+
+from django.db import migrations
+from django.contrib.auth import get_user_model
+
+from bookmarks.models import Toast
+
+User = get_user_model()
+
+
+def forwards(apps, schema_editor):
+
+    for user in User.objects.all():
+        toast = Toast(
+            key="html_snapshots_hint",
+            message="This version adds a new feature for archiving snapshots of websites locally. To use it, you need to switch to a different Docker image. See the installation instructions on GitHub for details.",
+            owner=user,
+        )
+        toast.save()
+
+
+def reverse(apps, schema_editor):
+    Toast.objects.filter(key="bookmark_list_actions_hint").delete()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("bookmarks", "0031_userprofile_enable_automatic_html_snapshots"),
+    ]
+
+    operations = [
+        migrations.RunPython(forwards, reverse),
+    ]

bookmarks/models.py

@@ -1,18 +1,22 @@
-import binascii
+import logging
 import os
 from typing import List
 
+import binascii
 from django import forms
+from django.conf import settings
 from django.contrib.auth import get_user_model
 from django.contrib.auth.models import User
 from django.db import models
-from django.db.models.signals import post_save
+from django.db.models.signals import post_save, post_delete
 from django.dispatch import receiver
 from django.http import QueryDict
 
 from bookmarks.utils import unique
 from bookmarks.validators import BookmarkURLValidator
 
+logger = logging.getLogger(__name__)
+
 
 class Tag(models.Model):
     name = models.CharField(max_length=64)
@@ -85,6 +89,47 @@ class Bookmark(models.Model):
         return self.resolved_title + " (" + self.url[:30] + "...)"
 
 
+class BookmarkAsset(models.Model):
+    TYPE_SNAPSHOT = "snapshot"
+
+    CONTENT_TYPE_HTML = "text/html"
+
+    STATUS_PENDING = "pending"
+    STATUS_COMPLETE = "complete"
+    STATUS_FAILURE = "failure"
+
+    bookmark = models.ForeignKey(Bookmark, on_delete=models.CASCADE)
+    date_created = models.DateTimeField(auto_now_add=True, null=False)
+    file = models.CharField(max_length=2048, blank=True, null=False)
+    file_size = models.IntegerField(null=True)
+    asset_type = models.CharField(max_length=64, blank=False, null=False)
+    content_type = models.CharField(max_length=128, blank=False, null=False)
+    display_name = models.CharField(max_length=2048, blank=True, null=False)
+    status = models.CharField(max_length=64, blank=False, null=False)
+    gzip = models.BooleanField(default=False, null=False)
+
+    def save(self, *args, **kwargs):
+        if self.file:
+            try:
+                file_path = os.path.join(settings.LD_ASSET_FOLDER, self.file)
+                if os.path.isfile(file_path):
+                    self.file_size = os.path.getsize(file_path)
+            except Exception:
+                pass
+        super().save(*args, **kwargs)
+
+
+@receiver(post_delete, sender=BookmarkAsset)
+def bookmark_asset_deleted(sender, instance, **kwargs):
+    if instance.file:
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, instance.file)
+        if os.path.isfile(filepath):
+            try:
+                os.remove(filepath)
+            except Exception as error:
+                logger.error(f"Failed to delete asset file: {filepath}", exc_info=error)
+
+
 class BookmarkForm(forms.ModelForm):
     # Use URLField for URL
     url = forms.CharField(validators=[BookmarkURLValidator()])
@@ -353,6 +398,7 @@ class UserProfile(models.Model):
     permanent_notes = models.BooleanField(default=False, null=False)
     custom_css = models.TextField(blank=True, null=False)
     search_preferences = models.JSONField(default=dict, null=False)
+    enable_automatic_html_snapshots = models.BooleanField(default=True, null=False)
 
 
 class UserProfileForm(forms.ModelForm):
@@ -369,6 +415,7 @@ class UserProfileForm(forms.ModelForm):
             "enable_sharing",
             "enable_public_sharing",
             "enable_favicons",
+            "enable_automatic_html_snapshots",
             "display_url",
             "display_view_bookmark_action",
             "display_edit_bookmark_action",

bookmarks/services/bookmarks.py

@@ -34,6 +34,9 @@ def create_bookmark(bookmark: Bookmark, tag_string: str, current_user: User):
     tasks.create_web_archive_snapshot(current_user, bookmark, False)
     # Load favicon
     tasks.load_favicon(current_user, bookmark)
+    # Create HTML snapshot
+    if current_user.profile.enable_automatic_html_snapshots:
+        tasks.create_html_snapshot(bookmark)
 
     return bookmark
 

bookmarks/services/monolith.py

@@ -0,0 +1,32 @@
+import gzip
+import shutil
+import subprocess
+import os
+
+from django.conf import settings
+
+
+class MonolithError(Exception):
+    pass
+
+
+# Monolith isn't used at the moment, as the local snapshot implementation
+# switched to single-file after the prototype. Keeping this around in case
+# it turns out to be useful in the future.
+def create_snapshot(url: str, filepath: str):
+    monolith_path = settings.LD_MONOLITH_PATH
+    monolith_options = settings.LD_MONOLITH_OPTIONS
+    temp_filepath = filepath + ".tmp"
+
+    try:
+        command = f"{monolith_path} '{url}' {monolith_options} -o {temp_filepath}"
+        subprocess.run(command, check=True, shell=True)
+
+        with open(temp_filepath, "rb") as raw_file, gzip.open(
+            filepath, "wb"
+        ) as gz_file:
+            shutil.copyfileobj(raw_file, gz_file)
+
+        os.remove(temp_filepath)
+    except subprocess.CalledProcessError as error:
+        raise MonolithError(f"Failed to create snapshot: {error.stderr}")

bookmarks/services/singlefile.py

@@ -0,0 +1,57 @@
+import gzip
+import logging
+import os
+import shlex
+import shutil
+import signal
+import subprocess
+
+from django.conf import settings
+
+
+class SingeFileError(Exception):
+    pass
+
+
+logger = logging.getLogger(__name__)
+
+
+def create_snapshot(url: str, filepath: str):
+    singlefile_path = settings.LD_SINGLEFILE_PATH
+    # parse string to list of arguments
+    singlefile_options = shlex.split(settings.LD_SINGLEFILE_OPTIONS)
+    temp_filepath = filepath + ".tmp"
+    # concat lists
+    args = [singlefile_path] + singlefile_options + [url, temp_filepath]
+    try:
+        # Use start_new_session=True to create a new process group
+        process = subprocess.Popen(args, start_new_session=True)
+        process.wait(timeout=settings.LD_SINGLEFILE_TIMEOUT_SEC)
+
+        # check if the file was created
+        if not os.path.exists(temp_filepath):
+            raise SingeFileError("Failed to create snapshot")
+
+        with open(temp_filepath, "rb") as raw_file, gzip.open(
+            filepath, "wb"
+        ) as gz_file:
+            shutil.copyfileobj(raw_file, gz_file)
+
+        os.remove(temp_filepath)
+    except subprocess.TimeoutExpired:
+        # First try to terminate properly
+        try:
+            logger.error(
+                "Timeout expired while creating snapshot. Terminating process..."
+            )
+            process.terminate()
+            process.wait(timeout=20)
+            raise SingeFileError("Timeout expired while creating snapshot")
+        except subprocess.TimeoutExpired:
+            # Kill the whole process group, which should also clean up any chromium
+            # processes spawned by single-file
+            logger.error("Timeout expired while terminating. Killing process...")
+            os.killpg(os.getpgid(process.pid), signal.SIGTERM)
+            raise SingeFileError("Timeout expired while creating snapshot")
+    except subprocess.CalledProcessError as error:
+        raise SingeFileError(f"Failed to create snapshot: {error.stderr}")

bookmarks/services/tasks.py

@@ -1,21 +1,54 @@
+import functools
 import logging
+import os
 
 import waybackpy
-from background_task import background
-from background_task.models import Task
 from django.conf import settings
 from django.contrib.auth import get_user_model
 from django.contrib.auth.models import User
+from django.utils import timezone, formats
+from huey import crontab
+from huey.contrib.djhuey import HUEY as huey
+from huey.exceptions import TaskLockedException
 from waybackpy.exceptions import WaybackError, TooManyRequestsError, NoCDXRecordFound
 
 import bookmarks.services.wayback
-from bookmarks.models import Bookmark, UserProfile
-from bookmarks.services import favicon_loader
+from bookmarks.models import Bookmark, BookmarkAsset, UserProfile
+from bookmarks.services import favicon_loader, singlefile
 from bookmarks.services.website_loader import DEFAULT_USER_AGENT
 
 logger = logging.getLogger(__name__)
 
 
+# Create custom decorator for Huey tasks that implements exponential backoff
+# Taken from: https://huey.readthedocs.io/en/latest/guide.html#tips-and-tricks
+# Retry 1: 60
+# Retry 2: 240
+# Retry 3: 960
+# Retry 4: 3840
+# Retry 5: 15360
+def task(retries=5, retry_delay=15, retry_backoff=4):
+    def deco(fn):
+        @functools.wraps(fn)
+        def inner(*args, **kwargs):
+            task = kwargs.pop("task")
+            try:
+                return fn(*args, **kwargs)
+            except TaskLockedException as exc:
+                # Task locks are currently only used as workaround to enforce
+                # running specific types of tasks (e.g. singlefile snapshots)
+                # sequentially. In that case don't reduce the number of retries.
+                task.retries = retries
+                raise exc
+            except Exception as exc:
+                task.retry_delay *= retry_backoff
+                raise exc
+
+        return huey.task(retries=retries, retry_delay=retry_delay, context=True)(inner)
+
+    return deco
+
+
 def is_web_archive_integration_active(user: User) -> bool:
     background_tasks_enabled = not settings.LD_DISABLE_BACKGROUND_TASKS
     web_archive_integration_enabled = (
@@ -65,7 +98,7 @@ def _create_snapshot(bookmark: Bookmark):
     logger.info(f"Successfully created new snapshot for bookmark:. url={bookmark.url}")
 
 
-@background()
+@task()
 def _create_web_archive_snapshot_task(bookmark_id: int, force_update: bool):
     try:
         bookmark = Bookmark.objects.get(id=bookmark_id)
@@ -94,7 +127,7 @@ def _create_web_archive_snapshot_task(bookmark_id: int, force_update: bool):
     _load_newest_snapshot(bookmark)
 
 
-@background()
+@task()
 def _load_web_archive_snapshot_task(bookmark_id: int):
     try:
         bookmark = Bookmark.objects.get(id=bookmark_id)
@@ -112,13 +145,14 @@ def schedule_bookmarks_without_snapshots(user: User):
         _schedule_bookmarks_without_snapshots_task(user.id)
 
 
-@background()
+@task()
 def _schedule_bookmarks_without_snapshots_task(user_id: int):
     user = get_user_model().objects.get(id=user_id)
     bookmarks_without_snapshots = Bookmark.objects.filter(
         web_archive_snapshot_url__exact="", owner=user
     )
 
+    # TODO: Implement bulk task creation
     for bookmark in bookmarks_without_snapshots:
         # To prevent rate limit errors from the Wayback API only try to load the latest snapshots instead of creating
         # new ones when processing bookmarks in bulk
@@ -136,7 +170,7 @@ def load_favicon(user: User, bookmark: Bookmark):
         _load_favicon_task(bookmark.id)
 
 
-@background()
+@task()
 def _load_favicon_task(bookmark_id: int):
     try:
         bookmark = Bookmark.objects.get(id=bookmark_id)
@@ -160,19 +194,15 @@ def schedule_bookmarks_without_favicons(user: User):
         _schedule_bookmarks_without_favicons_task(user.id)
 
 
-@background()
+@task()
 def _schedule_bookmarks_without_favicons_task(user_id: int):
     user = get_user_model().objects.get(id=user_id)
     bookmarks = Bookmark.objects.filter(favicon_file__exact="", owner=user)
-    tasks = []
 
+    # TODO: Implement bulk task creation
     for bookmark in bookmarks:
-        task = Task.objects.new_task(
-            task_name="bookmarks.services.tasks._load_favicon_task", args=(bookmark.id,)
-        )
-        tasks.append(task)
-
-    Task.objects.bulk_create(tasks)
+        _load_favicon_task(bookmark.id)
+        pass
 
 
 def schedule_refresh_favicons(user: User):
@@ -180,16 +210,98 @@ def schedule_refresh_favicons(user: User):
         _schedule_refresh_favicons_task(user.id)
 
 
-@background()
+@task()
 def _schedule_refresh_favicons_task(user_id: int):
     user = get_user_model().objects.get(id=user_id)
     bookmarks = Bookmark.objects.filter(owner=user)
-    tasks = []
 
+    # TODO: Implement bulk task creation
     for bookmark in bookmarks:
-        task = Task.objects.new_task(
-            task_name="bookmarks.services.tasks._load_favicon_task", args=(bookmark.id,)
-        )
-        tasks.append(task)
+        _load_favicon_task(bookmark.id)
 
-    Task.objects.bulk_create(tasks)
+
+def is_html_snapshot_feature_active() -> bool:
+    return settings.LD_ENABLE_SNAPSHOTS and not settings.LD_DISABLE_BACKGROUND_TASKS
+
+
+def create_html_snapshot(bookmark: Bookmark):
+    if not is_html_snapshot_feature_active():
+        return
+
+    timestamp = formats.date_format(timezone.now(), "SHORT_DATE_FORMAT")
+    asset = BookmarkAsset(
+        bookmark=bookmark,
+        asset_type=BookmarkAsset.TYPE_SNAPSHOT,
+        content_type="text/html",
+        display_name=f"HTML snapshot from {timestamp}",
+        status=BookmarkAsset.STATUS_PENDING,
+    )
+    asset.save()
+
+
+MAX_SNAPSHOT_FILENAME_LENGTH = 192
+
+
+def _generate_snapshot_filename(asset: BookmarkAsset) -> str:
+    def sanitize_char(char):
+        if char.isalnum() or char in ("-", "_", "."):
+            return char
+        else:
+            return "_"
+
+    formatted_datetime = asset.date_created.strftime("%Y-%m-%d_%H%M%S")
+    sanitized_url = "".join(sanitize_char(char) for char in asset.bookmark.url)
+
+    # Calculate the length of the non-URL parts of the filename
+    non_url_length = len(f"{asset.asset_type}{formatted_datetime}__.html.gz")
+    # Calculate the maximum length for the URL part
+    max_url_length = MAX_SNAPSHOT_FILENAME_LENGTH - non_url_length
+    # Truncate the URL if necessary
+    sanitized_url = sanitized_url[:max_url_length]
+
+    return f"{asset.asset_type}_{formatted_datetime}_{sanitized_url}.html.gz"
+
+
+# singe-file does not support running multiple instances in parallel, so we can
+# not queue up multiple snapshot tasks at once. Instead, schedule a periodic
+# task that grabs a number of pending assets and creates snapshots for them in
+# sequence. The task uses a lock to ensure that a new task isn't scheduled
+# before the previous one has finished.
+@huey.periodic_task(crontab(minute="*"))
+@huey.lock_task("schedule-html-snapshots-lock")
+def _schedule_html_snapshots_task():
+    # Get five pending assets
+    assets = BookmarkAsset.objects.filter(status=BookmarkAsset.STATUS_PENDING).order_by(
+        "date_created"
+    )[:5]
+
+    for asset in assets:
+        _create_html_snapshot_task(asset.id)
+
+
+def _create_html_snapshot_task(asset_id: int):
+    try:
+        asset = BookmarkAsset.objects.get(id=asset_id)
+    except BookmarkAsset.DoesNotExist:
+        return
+
+    logger.info(f"Create HTML snapshot for bookmark. url={asset.bookmark.url}")
+
+    try:
+        filename = _generate_snapshot_filename(asset)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        singlefile.create_snapshot(asset.bookmark.url, filepath)
+        asset.status = BookmarkAsset.STATUS_COMPLETE
+        asset.file = filename
+        asset.gzip = True
+        asset.save()
+        logger.info(
+            f"Successfully created HTML snapshot for bookmark. url={asset.bookmark.url}"
+        )
+    except Exception as error:
+        logger.error(
+            f"Failed to HTML snapshot for bookmark. url={asset.bookmark.url}",
+            exc_info=error,
+        )
+        asset.status = BookmarkAsset.STATUS_FAILURE
+        asset.save()

bookmarks/styles/bookmark-details.scss

@@ -37,6 +37,53 @@
     margin-bottom: 0;
   }
 
+  .assets {
+    margin-top: $unit-2;
+  }
+
+  .assets .asset {
+    display: flex;
+    align-items: center;
+    gap: $unit-3;
+    padding: $unit-2 0;
+    border-top: $unit-o solid $border-color-light;
+  }
+
+  .assets .asset:last-child {
+    border-bottom: $unit-o solid $border-color-light;
+  }
+
+  .assets .asset-icon {
+    display: flex;
+    align-items: center;
+    justify-content: center;
+  }
+
+  .assets .asset-text {
+    flex: 1 1 0;
+  }
+
+  .assets .asset-text .filesize {
+    color: $gray-color;
+    margin-left: $unit-2;
+  }
+
+  .assets .asset-actions, .assets-actions {
+    display: flex;
+    gap: $unit-3;
+    align-items: center;
+  }
+
+  .assets .asset-actions .btn, .assets-actions .btn {
+    height: unset;
+    padding: 0;
+    border: none;
+  }
+
+  .assets-actions {
+    margin-top: $unit-2;
+  }
+
   .tags a {
     color: $alternative-color;
   }
@@ -46,7 +93,7 @@
     gap: $unit-2;
   }
 
-  .status form .form-group, .status form .form-switch {
+  .status .form-group, .status .form-switch {
     margin: 0;
   }
 

bookmarks/tasks.py

@@ -0,0 +1,2 @@
+# Expose task modules to Huey Django extension
+import bookmarks.services.tasks

bookmarks/templates/admin/background_tasks.html

@@ -0,0 +1,39 @@
+{% extends "admin/base_site.html" %}
+
+{% block content %}
+  <table style="width: 100%">
+    <thead>
+    <tr>
+      <th>ID</th>
+      <th>Name</th>
+      <th>Args</th>
+      <th>Retries</th>
+    </tr>
+    </thead>
+    <tbody>
+    {% for task in tasks %}
+      <tr>
+        <td>{{ task.id }}</td>
+        <td>{{ task.name }}</td>
+        <td>{{ task.args }}</td>
+        <td>{{ task.retries }}</td>
+      </tr>
+    {% endfor %}
+    </tbody>
+  </table>
+  <p class="paginator">
+    {% if page.paginator.num_pages > 1 %}
+      {% for page_number in page_range %}
+        {% if page_number == page.number %}
+          <span class="this-page">{{ page_number }}</span>
+        {% elif page_number == '…' %}
+          <span>…</span>
+        {% else %}
+          <a href="?p={{ page_number }}">{{ page_number }}</a>
+        {% endif %}
+      {% endfor %}
+      &nbsp;
+    {% endif %}
+    {{ page.paginator.count }} tasks
+  </p>
+{% endblock %}

bookmarks/templates/bookmarks/details.html

@@ -1,13 +1,13 @@
 {% extends 'bookmarks/layout.html' %}
 
 {% block content %}
-  <div ld-bookmark-details class="bookmark-details page">
-    {% if request.user == bookmark.owner %}
+  <div class="bookmark-details page">
+    {% if details.is_editable %}
       {% include 'bookmarks/details/actions.html' %}
     {% endif %}
     {% include 'bookmarks/details/title.html' %}
     <div>
-      {% include 'bookmarks/details/content.html' %}
+      {% include 'bookmarks/details/form.html' %}
     </div>
   </div>
 {% endblock %}

bookmarks/templates/bookmarks/details/actions.html

@@ -1,11 +1,14 @@
 <div class="actions">
   <div class="left-actions">
-    <a class="btn" href="{% url 'bookmarks:edit' bookmark.id %}?return_url={{ edit_return_url|urlencode }}">Edit</a>
+    <a class="btn"
+       href="{% url 'bookmarks:edit' details.bookmark.id %}?return_url={{ details.edit_return_url|urlencode }}">Edit</a>
   </div>
   <div class="right-actions">
-    <form action="{% url 'bookmarks:index.action' %}?return_url={{ delete_return_url|urlencode }}" method="post">
+    <form action="{% url 'bookmarks:index.action' %}?return_url={{ details.delete_return_url|urlencode }}"
+          method="post">
       {% csrf_token %}
-      <button ld-confirm-button type="submit" name="remove" value="{{ bookmark.id }}" class="btn btn-link text-error">
+      <button ld-confirm-button type="submit" name="remove" value="{{ details.bookmark.id }}"
+              class="btn btn-link text-error">
         Delete...
       </button>
     </form>

bookmarks/templates/bookmarks/details/asset_icon.html

@@ -0,0 +1,42 @@
+{% if asset.content_type == 'text/html' %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
+       stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M14 3v4a1 1 0 0 0 1 1h4"/>
+    <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/>
+    <path d="M2 21v-6"/>
+    <path d="M5 15v6"/>
+    <path d="M2 18h3"/>
+    <path d="M20 15v6h2"/>
+    <path d="M13 21v-6l2 3l2 -3v6"/>
+    <path d="M7.5 15h3"/>
+    <path d="M9 15v6"/>
+  </svg>
+{% elif asset.content_type == 'application/pdf' %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
+       stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M14 3v4a1 1 0 0 0 1 1h4"/>
+    <path d="M5 12v-7a2 2 0 0 1 2 -2h7l5 5v4"/>
+    <path d="M5 18h1.5a1.5 1.5 0 0 0 0 -3h-1.5v6"/>
+    <path d="M17 18h2"/>
+    <path d="M20 15h-3v6"/>
+    <path d="M11 15v6h1a2 2 0 0 0 2 -2v-2a2 2 0 0 0 -2 -2h-1z"/>
+  </svg>
+{% elif asset.content_type == 'image/png' or asset.content_type == 'image/jpeg' or asset.content_type == 'image.gif' %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none"
+       stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M15 8h.01"/>
+    <path d="M3 6a3 3 0 0 1 3 -3h12a3 3 0 0 1 3 3v12a3 3 0 0 1 -3 3h-12a3 3 0 0 1 -3 -3v-12z"/>
+    <path d="M3 16l5 -5c.928 -.893 2.072 -.893 3 0l5 5"/>
+    <path d="M14 14l1 -1c.928 -.893 2.072 -.893 3 0l3 3"/>
+  </svg>
+{% else %}
+  <svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor"
+       stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
+    <path stroke="none" d="M0 0h24v24H0z" fill="none"/>
+    <path d="M14 3v4a1 1 0 0 0 1 1h4"/>
+    <path d="M17 21h-10a2 2 0 0 1 -2 -2v-14a2 2 0 0 1 2 -2h7l5 5v11a2 2 0 0 1 -2 2z"/>
+  </svg>
+{% endif %}

bookmarks/templates/bookmarks/details/assets.html

@@ -0,0 +1,37 @@
+{% if details.assets %}
+  <div class="assets">
+    {% for asset in details.assets %}
+      <div class="asset" data-asset-id="{{ asset.id }}">
+        <div class="asset-icon {{ asset.icon_classes }}">
+          {% include 'bookmarks/details/asset_icon.html' %}
+        </div>
+        <div class="asset-text truncate {{ asset.text_classes }}">
+        <span>
+          {{ asset.display_name }}
+          {% if asset.status == 'pending' %}(queued){% endif %}
+          {% if asset.status == 'failure' %}(failed){% endif %}
+        </span>
+          {% if asset.file_size %}
+            <span class="filesize">{{ asset.file_size|filesizeformat }}</span>
+          {% endif %}
+        </div>
+        <div class="asset-actions">
+          {% if asset.file %}
+            <a class="btn btn-link" href="{% url 'bookmarks:assets.view' asset.id %}" target="_blank">View</a>
+          {% endif %}
+          {% if details.is_editable %}
+            <button ld-confirm-button type="submit" name="remove_asset" value="{{ asset.id }}" class="btn btn-link">
+              Remove
+            </button>
+          {% endif %}
+        </div>
+      </div>
+    {% endfor %}
+  </div>
+{% endif %}
+
+{% if details.is_editable %}
+  <div class="assets-actions">
+    <button type="submit" name="create_snapshot" class="btn btn-link">Create HTML snapshot</button>
+  </div>
+{% endif %}

bookmarks/templates/bookmarks/details/content.html

@@ -1,85 +0,0 @@
-{% load static %}
-{% load shared %}
-
-<div class="weblinks">
-  <a class="weblink" href="{{ bookmark.url }}" rel="noopener"
-     target="{{ request.user_profile.bookmark_link_target }}">
-    {% if bookmark.favicon_file and request.user_profile.enable_favicons %}
-      <img class="favicon" src="{% static bookmark.favicon_file %}" alt="">
-    {% endif %}
-    <span>{{ bookmark.url }}</span>
-  </a>
-  {% if bookmark.web_archive_snapshot_url %}
-    <a class="weblink" href="{{ bookmark.web_archive_snapshot_url }}"
-       target="{{ request.user_profile.bookmark_link_target }}">
-      {% if bookmark.favicon_file and request.user_profile.enable_favicons %}
-        <svg class="favicon" viewBox="0 0 76 86" xmlns="http://www.w3.org/2000/svg">
-          <path
-              d="m76 82v4h-76l.00080851-4zm-3-6v5h-70v-5zm-62.6696277-54 .8344146.4217275.4176066 6.7436084.4176065 10.9576581v10.5383496l-.4176065 13.1364492-.0694681 8.8498268-1.1825531.3523804h-4.17367003l-1.25202116-.3523804-.48627608-8.8498268-.41840503-13.0662957v-10.5375432l.41840503-11.028618.38167482-6.7798947.87034634-.3854412zm60.0004653 0 .8353798.4217275.4168913 6.7436084.4168913 10.9576581v10.5383496l-.4168913 13.1364492-.0686832 8.8498268-1.1835879.3523804h-4.1737047l-1.2522712-.3523804-.4879704-8.8498268-.4168913-13.0662957v-10.5375432l.4168913-11.028618.3833483-6.7798947.8697215-.3854412zm-42.000632 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2529447-.3523804-.4863246-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8688361-.3854412zm23 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2521462-.3523804-.4871231-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8696347-.3854412zm21.6697944-9v7h-70v-7zm-35.7200748-13 36.7200748 8.4088317-1.4720205 2.5911683h-70.32799254l-2.19998696-2.10140371z"
-              fill="currentColor" fill-rule="evenodd"/>
-        </svg>
-      {% endif %}
-      <span>View on Internet Archive</span>
-    </a>
-  {% endif %}
-</div>
-<dl class="grid columns-2 columns-sm-1 gap-0">
-  {% if request.user == bookmark.owner %}
-    <div class="status col-2">
-      <dt>Status</dt>
-      <dd class="d-flex" style="gap: .8rem">
-        <form action="{% url 'bookmarks:details' bookmark.id %}" method="post">
-          {% csrf_token %}
-          <div class="form-group">
-            <label class="form-switch">
-              <input type="checkbox" name="is_archived" {% if bookmark.is_archived %}checked{% endif %}>
-              <i class="form-icon"></i> Archived
-            </label>
-          </div>
-          <div class="form-group">
-            <label class="form-switch">
-              <input type="checkbox" name="unread" {% if bookmark.unread %}checked{% endif %}>
-              <i class="form-icon"></i> Unread
-            </label>
-          </div>
-          {% if request.user_profile.enable_sharing %}
-            <div class="form-group">
-              <label class="form-switch">
-                <input type="checkbox" name="shared" {% if bookmark.shared %}checked{% endif %}>
-                <i class="form-icon"></i> Shared
-              </label>
-            </div>
-          {% endif %}
-        </form>
-      </dd>
-    </div>
-  {% endif %}
-  {% if bookmark.tag_names %}
-    <div class="tags col-1">
-      <dt>Tags</dt>
-      <dd>
-        {% for tag_name in bookmark.tag_names %}
-          <a href="{% url 'bookmarks:index' %}?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
-        {% endfor %}
-      </dd>
-    </div>
-  {% endif %}
-  <div class="date-added col-1">
-    <dt>Date added</dt>
-    <dd>
-      <span>{{ bookmark.date_added }}</span>
-    </dd>
-  </div>
-  {% if bookmark.resolved_description %}
-    <div class="description col-2">
-      <dt>Description</dt>
-      <dd>{{ bookmark.resolved_description }}</dd>
-    </div>
-  {% endif %}
-  {% if bookmark.notes %}
-    <div class="notes col-2">
-      <dt>Notes</dt>
-      <dd class="markdown">{% markdown bookmark.notes %}</dd>
-    </div>
-  {% endif %}
-</dl>

bookmarks/templates/bookmarks/details/form.html

@@ -0,0 +1,99 @@
+{% load static %}
+{% load shared %}
+
+<form ld-form action="{% url 'bookmarks:details' details.bookmark.id %}"
+      refresh-url="{% url 'bookmarks:partials.details_form' details.bookmark.id %}"
+      refresh-events="bookmark-page-refresh"
+      method="post">
+  <div class="weblinks">
+    <a class="weblink" href="{{ details.bookmark.url }}" rel="noopener"
+       target="{{ details.profile.bookmark_link_target }}">
+      {% if details.show_link_icons %}
+        <img class="favicon" src="{% static details.bookmark.favicon_file %}" alt="">
+      {% endif %}
+      <span>{{ details.bookmark.url }}</span>
+    </a>
+    {% if details.bookmark.web_archive_snapshot_url %}
+      <a class="weblink" href="{{ details.bookmark.web_archive_snapshot_url }}"
+         target="{{ details.profile.bookmark_link_target }}">
+        {% if details.show_link_icons %}
+          <svg class="favicon" viewBox="0 0 76 86" xmlns="http://www.w3.org/2000/svg">
+            <path
+                d="m76 82v4h-76l.00080851-4zm-3-6v5h-70v-5zm-62.6696277-54 .8344146.4217275.4176066 6.7436084.4176065 10.9576581v10.5383496l-.4176065 13.1364492-.0694681 8.8498268-1.1825531.3523804h-4.17367003l-1.25202116-.3523804-.48627608-8.8498268-.41840503-13.0662957v-10.5375432l.41840503-11.028618.38167482-6.7798947.87034634-.3854412zm60.0004653 0 .8353798.4217275.4168913 6.7436084.4168913 10.9576581v10.5383496l-.4168913 13.1364492-.0686832 8.8498268-1.1835879.3523804h-4.1737047l-1.2522712-.3523804-.4879704-8.8498268-.4168913-13.0662957v-10.5375432l.4168913-11.028618.3833483-6.7798947.8697215-.3854412zm-42.000632 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2529447-.3523804-.4863246-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8688361-.3854412zm23 0 .8344979.4217275.4176483 6.7436084.4176482 10.9576581v10.5383496l-.4176482 13.1364492-.0686764 8.8498268-1.1834698.3523804h-4.1740866l-1.2521462-.3523804-.4871231-8.8498268-.4168497-13.0662957v-10.5375432l.4168497-11.028618.38331-6.7798947.8696347-.3854412zm21.6697944-9v7h-70v-7zm-35.7200748-13 36.7200748 8.4088317-1.4720205 2.5911683h-70.32799254l-2.19998696-2.10140371z"
+                fill="currentColor" fill-rule="evenodd"/>
+          </svg>
+        {% endif %}
+        <span>View on Internet Archive</span>
+      </a>
+    {% endif %}
+  </div>
+  <dl class="grid columns-2 columns-sm-1 gap-0">
+    {% if details.is_editable %}
+      <div class="status col-2">
+        <dt>Status</dt>
+        <dd class="d-flex" style="gap: .8rem">
+          {% csrf_token %}
+          <div class="form-group">
+            <label class="form-switch">
+              <input ld-form-auto-submit type="checkbox" name="is_archived"
+                     {% if details.bookmark.is_archived %}checked{% endif %}>
+              <i class="form-icon"></i> Archived
+            </label>
+          </div>
+          <div class="form-group">
+            <label class="form-switch">
+              <input ld-form-auto-submit type="checkbox" name="unread"
+                     {% if details.bookmark.unread %}checked{% endif %}>
+              <i class="form-icon"></i> Unread
+            </label>
+          </div>
+          {% if details.profile.enable_sharing %}
+            <div class="form-group">
+              <label class="form-switch">
+                <input ld-form-auto-submit type="checkbox" name="shared"
+                       {% if details.bookmark.shared %}checked{% endif %}>
+                <i class="form-icon"></i> Shared
+              </label>
+            </div>
+          {% endif %}
+        </dd>
+      </div>
+    {% endif %}
+    {% if details.show_files %}
+      <div class="files col-2">
+        <dt>Files</dt>
+        <dd>
+          {% include 'bookmarks/details/assets.html' %}
+        </dd>
+      </div>
+    {% endif %}
+    {% if details.bookmark.tag_names %}
+      <div class="tags col-1">
+        <dt>Tags</dt>
+        <dd>
+          {% for tag_name in details.bookmark.tag_names %}
+            <a href="{% url 'bookmarks:index' %}?{% add_tag_to_query tag_name %}">{{ tag_name|hash_tag }}</a>
+          {% endfor %}
+        </dd>
+      </div>
+    {% endif %}
+    <div class="date-added col-1">
+      <dt>Date added</dt>
+      <dd>
+        <span>{{ details.bookmark.date_added }}</span>
+      </dd>
+    </div>
+    {% if details.bookmark.resolved_description %}
+      <div class="description col-2">
+        <dt>Description</dt>
+        <dd>{{ details.bookmark.resolved_description }}</dd>
+      </div>
+    {% endif %}
+    {% if details.bookmark.notes %}
+      <div class="notes col-2">
+        <dt>Notes</dt>
+        <dd class="markdown">{% markdown details.bookmark.notes %}</dd>
+      </div>
+    {% endif %}
+  </dl>
+</form>

bookmarks/templates/bookmarks/details/title.html

@@ -1,3 +1,3 @@
 <h2>
-  {{ bookmark.resolved_title }}
+  {{ details.bookmark.resolved_title }}
 </h2>

bookmarks/templates/bookmarks/details_modal.html

@@ -1,4 +1,4 @@
-<div ld-bookmark-details class="modal active bookmark-details">
+<div class="modal active bookmark-details">
   <div class="modal-overlay" aria-label="Close"></div>
   <div class="modal-container">
     <div class="modal-header">
@@ -14,11 +14,11 @@
     </div>
     <div class="modal-body">
       <div class="content">
-        {% include 'bookmarks/details/content.html' %}
+        {% include 'bookmarks/details/form.html' %}
       </div>
     </div>
 
-    {% if request.user == bookmark.owner %}
+    {% if details.is_editable %}
       <div class="modal-footer">
         {% include 'bookmarks/details/actions.html' %}
       </div>

bookmarks/templates/settings/general.html

@@ -163,6 +163,18 @@
               href="{% url 'bookmarks:shared' %}">shared bookmarks page</a>.
           </div>
         </div>
+        {% if has_snapshot_support %}
+          <div class="form-group">
+            <label for="{{ form.enable_automatic_html_snapshots.id_for_label }}" class="form-checkbox">
+              {{ form.enable_automatic_html_snapshots }}
+              <i class="form-icon"></i> Automatically create HTML snapshots
+            </label>
+            <div class="form-input-hint">
+              Automatically creates HTML snapshots when adding bookmarks. Alternatively, when disabled, snapshots can be
+              created manually in the details view of a bookmark.
+            </div>
+          </div>
+        {% endif %}
         <div class="form-group">
           <details {% if form.custom_css.value %}open{% endif %}>
             <summary>Custom CSS</summary>

bookmarks/tests/helpers.py

@@ -1,6 +1,6 @@
 import random
 import logging
-import datetime
+from datetime import datetime
 from typing import List
 
 from bs4 import BeautifulSoup
@@ -10,7 +10,7 @@ from django.utils.crypto import get_random_string
 from rest_framework import status
 from rest_framework.test import APITestCase
 
-from bookmarks.models import Bookmark, Tag
+from bookmarks.models import Bookmark, BookmarkAsset, Tag
 
 
 class BookmarkFactoryMixin:
@@ -133,6 +133,38 @@ class BookmarkFactoryMixin:
     def get_numbered_bookmark(self, title: str):
         return Bookmark.objects.get(title=title)
 
+    def setup_asset(
+        self,
+        bookmark: Bookmark,
+        date_created: datetime = None,
+        file: str = None,
+        file_size: int = None,
+        asset_type: str = BookmarkAsset.TYPE_SNAPSHOT,
+        content_type: str = "image/html",
+        display_name: str = None,
+        status: str = BookmarkAsset.STATUS_COMPLETE,
+        gzip: bool = False,
+    ):
+        if date_created is None:
+            date_created = timezone.now()
+        if not file:
+            file = get_random_string(length=32)
+        if not display_name:
+            display_name = file
+        asset = BookmarkAsset(
+            bookmark=bookmark,
+            date_created=date_created,
+            file=file,
+            file_size=file_size,
+            asset_type=asset_type,
+            content_type=content_type,
+            display_name=display_name,
+            status=status,
+            gzip=gzip,
+        )
+        asset.save()
+        return asset
+
     def setup_tag(self, user: User = None, name: str = ""):
         if user is None:
             user = self.get_or_create_test_user()

bookmarks/tests/test_bookmark_asset_view.py

@@ -0,0 +1,125 @@
+import os
+
+from django.conf import settings
+from django.test import TestCase
+from django.urls import reverse
+
+from bookmarks.tests.helpers import (
+    BookmarkFactoryMixin,
+)
+
+
+class BookmarkAssetViewTestCase(TestCase, BookmarkFactoryMixin):
+    def setUp(self) -> None:
+        user = self.get_or_create_test_user()
+        self.client.force_login(user)
+
+    def tearDown(self):
+        temp_files = [
+            f for f in os.listdir(settings.LD_ASSET_FOLDER) if f.startswith("temp")
+        ]
+        for temp_file in temp_files:
+            os.remove(os.path.join(settings.LD_ASSET_FOLDER, temp_file))
+
+    def setup_asset_file(self, filename):
+        if not os.path.exists(settings.LD_ASSET_FOLDER):
+            os.makedirs(settings.LD_ASSET_FOLDER)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        with open(filepath, "w") as f:
+            f.write("test")
+
+    def setup_asset_with_file(self, bookmark):
+        filename = f"temp_{bookmark.id}.html.gzip"
+        self.setup_asset_file(filename)
+        asset = self.setup_asset(bookmark=bookmark, file=filename)
+        return asset
+
+    def test_view_access(self):
+        # own bookmark
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 200)
+
+        # other user's bookmark
+        other_user = self.setup_user()
+        bookmark = self.setup_bookmark(user=other_user)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing disabled
+        bookmark = self.setup_bookmark(user=other_user, shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # unshared, sharing enabled
+        profile = other_user.profile
+        profile.enable_sharing = True
+        profile.save()
+        bookmark = self.setup_bookmark(user=other_user, shared=False)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing enabled
+        bookmark = self.setup_bookmark(user=other_user, shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 200)
+
+    def test_view_access_guest_user(self):
+        self.client.logout()
+
+        # unshared, sharing disabled
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing disabled
+        bookmark = self.setup_bookmark(shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # unshared, sharing enabled
+        profile = self.get_or_create_test_user().profile
+        profile.enable_sharing = True
+        profile.save()
+        bookmark = self.setup_bookmark(shared=False)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, sharing enabled
+        bookmark = self.setup_bookmark(shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # unshared, public sharing enabled
+        profile.enable_public_sharing = True
+        profile.save()
+        bookmark = self.setup_bookmark(shared=False)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 404)
+
+        # shared, public sharing enabled
+        bookmark = self.setup_bookmark(shared=True)
+        asset = self.setup_asset_with_file(bookmark)
+
+        response = self.client.get(reverse("bookmarks:assets.view", args=[asset.id]))
+        self.assertEqual(response.status_code, 200)

bookmarks/tests/test_bookmark_assets.py

@@ -0,0 +1,89 @@
+import os
+
+from django.conf import settings
+from django.test import TestCase
+
+from bookmarks.tests.helpers import (
+    BookmarkFactoryMixin,
+)
+from bookmarks.services import bookmarks
+
+
+class BookmarkAssetsTestCase(TestCase, BookmarkFactoryMixin):
+    def tearDown(self):
+        temp_files = [
+            f for f in os.listdir(settings.LD_ASSET_FOLDER) if f.startswith("temp")
+        ]
+        for temp_file in temp_files:
+            os.remove(os.path.join(settings.LD_ASSET_FOLDER, temp_file))
+
+    def setup_asset_file(self, filename):
+        if not os.path.exists(settings.LD_ASSET_FOLDER):
+            os.makedirs(settings.LD_ASSET_FOLDER)
+        filepath = os.path.join(settings.LD_ASSET_FOLDER, filename)
+        with open(filepath, "w") as f:
+            f.write("test")
+
+    def setup_asset_with_file(self, bookmark):
+        filename = f"temp_{bookmark.id}.html.gzip"
+        self.setup_asset_file(filename)
+        asset = self.setup_asset(bookmark=bookmark, file=filename)
+        return asset
+
+    def test_delete_bookmark_deletes_asset_file(self):
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset_with_file(bookmark)
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset.file))
+        )
+
+        bookmark.delete()
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset.file))
+        )
+
+    def test_bulk_delete_bookmarks_deletes_asset_files(self):
+        bookmark1 = self.setup_bookmark()
+        asset1 = self.setup_asset_with_file(bookmark1)
+        bookmark2 = self.setup_bookmark()
+        asset2 = self.setup_asset_with_file(bookmark2)
+        bookmark3 = self.setup_bookmark()
+        asset3 = self.setup_asset_with_file(bookmark3)
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset1.file))
+        )
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset2.file))
+        )
+        self.assertTrue(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset3.file))
+        )
+
+        bookmarks.delete_bookmarks(
+            [bookmark1.id, bookmark2.id, bookmark3.id], self.get_or_create_test_user()
+        )
+
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset1.file))
+        )
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset2.file))
+        )
+        self.assertFalse(
+            os.path.exists(os.path.join(settings.LD_ASSET_FOLDER, asset3.file))
+        )
+
+    def test_save_updates_file_size(self):
+        # File does not exist initially
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark=bookmark, file="temp.html.gz")
+        self.assertIsNone(asset.file_size)
+
+        # Add file, save again
+        self.setup_asset_file(asset.file)
+        asset.save()
+        self.assertEqual(asset.file_size, 4)
+
+        # Create asset with initial file
+        asset = self.setup_asset(bookmark=bookmark, file="temp.html.gz")
+        self.assertEqual(asset.file_size, 4)

bookmarks/tests/test_bookmark_details_modal.py

@@ -1,8 +1,11 @@
-from django.test import TestCase
+from unittest.mock import patch
+
+from django.test import TestCase, override_settings
 from django.urls import reverse
 from django.utils import formats
 
-from bookmarks.models import UserProfile
+from bookmarks.models import BookmarkAsset, UserProfile
+from bookmarks.services import tasks
 from bookmarks.tests.helpers import BookmarkFactoryMixin, HtmlTestMixin
 
 
@@ -11,8 +14,15 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         user = self.get_or_create_test_user()
         self.client.force_login(user)
 
+    def get_view_name(self):
+        return "bookmarks:details_modal"
+
     def get_base_url(self, bookmark):
-        return reverse("bookmarks:details_modal", args=[bookmark.id])
+        return reverse(self.get_view_name(), args=[bookmark.id])
+
+    def get_details_form(self, soup, bookmark):
+        expected_url = reverse("bookmarks:details", args=[bookmark.id])
+        return soup.find("form", {"action": expected_url})
 
     def get_details(self, bookmark, return_url=""):
         url = self.get_base_url(bookmark)
@@ -35,43 +45,38 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
     def find_weblink(self, soup, url):
         return soup.find("a", {"class": "weblink", "href": url})
 
-    def test_access(self):
+    def find_asset(self, soup, asset):
+        return soup.find("div", {"data-asset-id": asset.id})
+
+    def details_route_access_test(self, view_name: str, shareable: bool):
         # own bookmark
         bookmark = self.setup_bookmark()
 
-        response = self.client.get(
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
         self.assertEqual(response.status_code, 200)
 
         # other user's bookmark
         other_user = self.setup_user()
         bookmark = self.setup_bookmark(user=other_user)
 
-        response = self.client.get(
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
         self.assertEqual(response.status_code, 404)
 
         # non-existent bookmark
-        response = self.client.get(reverse("bookmarks:details_modal", args=[9999]))
+        response = self.client.get(reverse(view_name, args=[9999]))
         self.assertEqual(response.status_code, 404)
 
         # guest user
         self.client.logout()
-        response = self.client.get(
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
-        self.assertEqual(response.status_code, 404)
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
+        self.assertEqual(response.status_code, 404 if shareable else 302)
 
-    def test_access_with_sharing(self):
+    def details_route_sharing_access_test(self, view_name: str, shareable: bool):
         # shared bookmark, sharing disabled
         other_user = self.setup_user()
         bookmark = self.setup_bookmark(shared=True, user=other_user)
 
-        response = self.client.get(
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
         self.assertEqual(response.status_code, 404)
 
         # shared bookmark, sharing enabled
@@ -79,26 +84,38 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         profile.enable_sharing = True
         profile.save()
 
-        response = self.client.get(
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
-        self.assertEqual(response.status_code, 200)
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
+        self.assertEqual(response.status_code, 200 if shareable else 404)
 
         # shared bookmark, guest user, no public sharing
         self.client.logout()
-        response = self.client.get(
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
-        self.assertEqual(response.status_code, 404)
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
+        self.assertEqual(response.status_code, 404 if shareable else 302)
 
         # shared bookmark, guest user, public sharing
         profile.enable_public_sharing = True
         profile.save()
 
-        response = self.client.get(
-            reverse("bookmarks:details_modal", args=[bookmark.id])
-        )
-        self.assertEqual(response.status_code, 200)
+        response = self.client.get(reverse(view_name, args=[bookmark.id]))
+        self.assertEqual(response.status_code, 200 if shareable else 302)
+
+    def test_access(self):
+        self.details_route_access_test(self.get_view_name(), True)
+
+    def test_access_with_sharing(self):
+        self.details_route_sharing_access_test(self.get_view_name(), True)
+
+    def test_form_partial_access(self):
+        # form partial is only used when submitting forms, which should be only
+        # accessible to the owner of the bookmark. As such assume it requires
+        # login.
+        self.details_route_access_test("bookmarks:partials.details_form", False)
+
+    def test_form_partial_access_with_sharing(self):
+        # form partial is only used when submitting forms, which should be only
+        # accessible to the owner of the bookmark. As such assume it requires
+        # login.
+        self.details_route_sharing_access_test("bookmarks:partials.details_form", False)
 
     def test_displays_title(self):
         # with title
@@ -246,9 +263,8 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         # renders form
         bookmark = self.setup_bookmark()
         soup = self.get_details(bookmark)
-        section = self.get_section(soup, "Status")
 
-        form = section.find("form")
+        form = self.get_details_form(soup, bookmark)
         self.assertIsNotNone(form)
         self.assertEqual(
             form["action"], reverse("bookmarks:details", args=[bookmark.id])
@@ -312,30 +328,21 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         bookmark = self.setup_bookmark()
         soup = self.get_details(bookmark)
         section = self.find_section(soup, "Status")
-        form_action = reverse("bookmarks:details", args=[bookmark.id])
-        form = soup.find("form", {"action": form_action})
         self.assertIsNotNone(section)
-        self.assertIsNotNone(form)
 
         # other user's bookmark
         other_user = self.setup_user(enable_sharing=True)
         bookmark = self.setup_bookmark(user=other_user, shared=True)
         soup = self.get_details(bookmark)
         section = self.find_section(soup, "Status")
-        form_action = reverse("bookmarks:details", args=[bookmark.id])
-        form = soup.find("form", {"action": form_action})
         self.assertIsNone(section)
-        self.assertIsNone(form)
 
         # guest user
         self.client.logout()
         bookmark = self.setup_bookmark(user=other_user, shared=True)
         soup = self.get_details(bookmark)
         section = self.find_section(soup, "Status")
-        form_action = reverse("bookmarks:details", args=[bookmark.id])
-        form = soup.find("form", {"action": form_action})
         self.assertIsNone(section)
-        self.assertIsNone(form)
 
     def test_status_update(self):
         bookmark = self.setup_bookmark()
@@ -560,3 +567,213 @@ class BookmarkDetailsModalTestCase(TestCase, BookmarkFactoryMixin, HtmlTestMixin
         delete_button = soup.find("button", {"type": "submit", "name": "remove"})
         self.assertIsNone(edit_link)
         self.assertIsNone(delete_button)
+
+    def test_assets_visibility_no_snapshot_support(self):
+        bookmark = self.setup_bookmark()
+
+        soup = self.get_details(bookmark)
+        section = self.find_section(soup, "Files")
+        self.assertIsNone(section)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_assets_visibility_with_snapshot_support(self):
+        bookmark = self.setup_bookmark()
+
+        soup = self.get_details(bookmark)
+        section = self.find_section(soup, "Files")
+        self.assertIsNotNone(section)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_list_visibility(self):
+        # no assets
+        bookmark = self.setup_bookmark()
+
+        soup = self.get_details(bookmark)
+        section = self.get_section(soup, "Files")
+        asset_list = section.find("div", {"class": "assets"})
+        self.assertIsNone(asset_list)
+
+        # with assets
+        bookmark = self.setup_bookmark()
+        self.setup_asset(bookmark)
+
+        soup = self.get_details(bookmark)
+        section = self.get_section(soup, "Files")
+        asset_list = section.find("div", {"class": "assets"})
+        self.assertIsNotNone(asset_list)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_list(self):
+        bookmark = self.setup_bookmark()
+        assets = [
+            self.setup_asset(bookmark),
+            self.setup_asset(bookmark),
+            self.setup_asset(bookmark),
+        ]
+
+        soup = self.get_details(bookmark)
+        section = self.get_section(soup, "Files")
+        asset_list = section.find("div", {"class": "assets"})
+
+        for asset in assets:
+            asset_item = self.find_asset(asset_list, asset)
+            self.assertIsNotNone(asset_item)
+
+            asset_icon = asset_item.select_one(".asset-icon svg")
+            self.assertIsNotNone(asset_icon)
+
+            asset_text = asset_item.select_one(".asset-text span")
+            self.assertIsNotNone(asset_text)
+            self.assertIn(asset.display_name, asset_text.text)
+
+            view_url = reverse("bookmarks:assets.view", args=[asset.id])
+            view_link = asset_item.find("a", {"href": view_url})
+            self.assertIsNotNone(view_link)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_without_file(self):
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark)
+        asset.file = ""
+        asset.save()
+
+        soup = self.get_details(bookmark)
+        asset_item = self.find_asset(soup, asset)
+        view_url = reverse("bookmarks:assets.view", args=[asset.id])
+        view_link = asset_item.find("a", {"href": view_url})
+        self.assertIsNone(view_link)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_status(self):
+        bookmark = self.setup_bookmark()
+        pending_asset = self.setup_asset(bookmark, status=BookmarkAsset.STATUS_PENDING)
+        failed_asset = self.setup_asset(bookmark, status=BookmarkAsset.STATUS_FAILURE)
+
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, pending_asset)
+        asset_text = asset_item.select_one(".asset-text span")
+        self.assertIn("(queued)", asset_text.text)
+
+        asset_item = self.find_asset(soup, failed_asset)
+        asset_text = asset_item.select_one(".asset-text span")
+        self.assertIn("(failed)", asset_text.text)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_file_size(self):
+        bookmark = self.setup_bookmark()
+        asset1 = self.setup_asset(bookmark, file_size=None)
+        asset2 = self.setup_asset(bookmark, file_size=54639)
+        asset3 = self.setup_asset(bookmark, file_size=11492020)
+
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset1)
+        asset_text = asset_item.select_one(".asset-text")
+        self.assertEqual(asset_text.text.strip(), asset1.display_name)
+
+        asset_item = self.find_asset(soup, asset2)
+        asset_text = asset_item.select_one(".asset-text")
+        self.assertIn("53.4\xa0KB", asset_text.text)
+
+        asset_item = self.find_asset(soup, asset3)
+        asset_text = asset_item.select_one(".asset-text")
+        self.assertIn("11.0\xa0MB", asset_text.text)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_asset_actions_visibility(self):
+        bookmark = self.setup_bookmark()
+
+        # with file
+        asset = self.setup_asset(bookmark)
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNotNone(view_link)
+        self.assertIsNotNone(delete_button)
+
+        # without file
+        asset.file = ""
+        asset.save()
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNone(view_link)
+        self.assertIsNotNone(delete_button)
+
+        # shared bookmark
+        other_user = self.setup_user(enable_sharing=True, enable_public_sharing=True)
+        bookmark = self.setup_bookmark(shared=True, user=other_user)
+        asset = self.setup_asset(bookmark)
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNotNone(view_link)
+        self.assertIsNone(delete_button)
+
+        # shared bookmark, guest user
+        self.client.logout()
+        soup = self.get_details(bookmark)
+
+        asset_item = self.find_asset(soup, asset)
+        view_link = asset_item.find("a", string="View")
+        delete_button = asset_item.find(
+            "button", {"type": "submit", "name": "remove_asset"}
+        )
+        self.assertIsNotNone(view_link)
+        self.assertIsNone(delete_button)
+
+    def test_remove_asset(self):
+        # remove asset
+        bookmark = self.setup_bookmark()
+        asset = self.setup_asset(bookmark)
+
+        response = self.client.post(
+            self.get_base_url(bookmark), {"remove_asset": asset.id}
+        )
+        self.assertEqual(response.status_code, 302)
+        self.assertFalse(BookmarkAsset.objects.filter(id=asset.id).exists())
+
+        # non-existent asset
+        response = self.client.post(self.get_base_url(bookmark), {"remove_asset": 9999})
+        self.assertEqual(response.status_code, 404)
+
+        # post without asset ID does not remove
+        asset = self.setup_asset(bookmark)
+        response = self.client.post(self.get_base_url(bookmark))
+        self.assertEqual(response.status_code, 302)
+        self.assertTrue(BookmarkAsset.objects.filter(id=asset.id).exists())
+
+        # guest user
+        asset = self.setup_asset(bookmark)
+        self.client.logout()
+        response = self.client.post(
+            self.get_base_url(bookmark), {"remove_asset": asset.id}
+        )
+        self.assertEqual(response.status_code, 404)
+        self.assertTrue(BookmarkAsset.objects.filter(id=asset.id).exists())
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_snapshot(self):
+        with patch.object(
+            tasks, "_create_html_snapshot_task"
+        ) as mock_create_html_snapshot_task:
+            bookmark = self.setup_bookmark()
+            response = self.client.post(
+                self.get_base_url(bookmark), {"create_snapshot": ""}
+            )
+            self.assertEqual(response.status_code, 302)
+
+            self.assertEqual(bookmark.bookmarkasset_set.count(), 1)

bookmarks/tests/test_bookmark_details_view.py

@@ -1,8 +1,6 @@
-from django.urls import reverse
-
 from bookmarks.tests.test_bookmark_details_modal import BookmarkDetailsModalTestCase
 
 
 class BookmarkDetailsViewTestCase(BookmarkDetailsModalTestCase):
-    def get_base_url(self, bookmark):
-        return reverse("bookmarks:details", args=[bookmark.id])
+    def get_view_name(self):
+        return "bookmarks:details"

bookmarks/tests/test_bookmarks_service.py

@@ -105,6 +105,24 @@ class BookmarkServiceTestCase(TestCase, BookmarkFactoryMixin):
 
             mock_load_favicon.assert_called_once_with(self.user, bookmark)
 
+    def test_create_should_load_html_snapshot(self):
+        with patch.object(tasks, "create_html_snapshot") as mock_create_html_snapshot:
+            bookmark_data = Bookmark(url="https://example.com")
+            bookmark = create_bookmark(bookmark_data, "tag1,tag2", self.user)
+
+            mock_create_html_snapshot.assert_called_once_with(bookmark)
+
+    def test_create_should_not_load_html_snapshot_when_setting_is_disabled(self):
+        profile = self.get_or_create_test_user().profile
+        profile.enable_automatic_html_snapshots = False
+        profile.save()
+
+        with patch.object(tasks, "create_html_snapshot") as mock_create_html_snapshot:
+            bookmark_data = Bookmark(url="https://example.com")
+            create_bookmark(bookmark_data, "tag1,tag2", self.user)
+
+            mock_create_html_snapshot.assert_not_called()
+
     def test_update_should_create_web_archive_snapshot_if_url_did_change(self):
         with patch.object(
             tasks, "create_web_archive_snapshot"
@@ -167,6 +185,14 @@ class BookmarkServiceTestCase(TestCase, BookmarkFactoryMixin):
 
             mock_load_favicon.assert_called_once_with(self.user, bookmark)
 
+    def test_update_should_not_create_html_snapshot(self):
+        with patch.object(tasks, "create_html_snapshot") as mock_create_html_snapshot:
+            bookmark = self.setup_bookmark()
+            bookmark.title = "updated title"
+            update_bookmark(bookmark, "tag1,tag2", self.user)
+
+            mock_create_html_snapshot.assert_not_called()
+
     def test_archive_bookmark(self):
         bookmark = Bookmark(
             url="https://example.com",

bookmarks/tests/test_bookmarks_tasks.py

@@ -1,19 +1,20 @@
 import datetime
+import os.path
 from dataclasses import dataclass
-from typing import Any
 from unittest import mock
 
 import waybackpy
-from background_task.models import Task
+from django.conf import settings
 from django.contrib.auth.models import User
 from django.test import TestCase, override_settings
+from huey.contrib.djhuey import HUEY as huey
 from waybackpy.exceptions import WaybackError
 
 import bookmarks.services.favicon_loader
 import bookmarks.services.wayback
-from bookmarks.models import UserProfile
-from bookmarks.services import tasks
-from bookmarks.tests.helpers import BookmarkFactoryMixin, disable_logging
+from bookmarks.models import BookmarkAsset, UserProfile
+from bookmarks.services import tasks, singlefile
+from bookmarks.tests.helpers import BookmarkFactoryMixin
 
 
 def create_wayback_machine_save_api_mock(
@@ -34,27 +35,45 @@ class MockCdxSnapshot:
     datetime_timestamp: datetime.datetime
 
 
-def create_cdx_server_api_mock(
-    archive_url: str | None = "https://example.com/newest_snapshot",
-    fail_loading_snapshot=False,
-):
-    mock_api = mock.Mock()
-
-    if fail_loading_snapshot:
-        mock_api.newest.side_effect = WaybackError
-    elif archive_url:
-        mock_api.newest.return_value = MockCdxSnapshot(
-            archive_url, datetime.datetime.now()
-        )
-    else:
-        mock_api.newest.return_value = None
-
-    return mock_api
-
-
 class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
 
     def setUp(self):
+        huey.immediate = True
+        huey.results = True
+        huey.store_none = True
+
+        self.mock_save_api = mock.Mock(
+            archive_url="https://example.com/created_snapshot"
+        )
+        self.mock_save_api_patcher = mock.patch.object(
+            waybackpy, "WaybackMachineSaveAPI", return_value=self.mock_save_api
+        )
+        self.mock_save_api_patcher.start()
+
+        self.mock_cdx_api = mock.Mock()
+        self.mock_cdx_api.newest.return_value = MockCdxSnapshot(
+            "https://example.com/newest_snapshot", datetime.datetime.now()
+        )
+        self.mock_cdx_api_patcher = mock.patch.object(
+            bookmarks.services.wayback,
+            "CustomWaybackMachineCDXServerAPI",
+            return_value=self.mock_cdx_api,
+        )
+        self.mock_cdx_api_patcher.start()
+
+        self.mock_load_favicon_patcher = mock.patch(
+            "bookmarks.services.favicon_loader.load_favicon"
+        )
+        self.mock_load_favicon = self.mock_load_favicon_patcher.start()
+        self.mock_load_favicon.return_value = "https_example_com.png"
+
+        self.mock_singlefile_create_snapshot_patcher = mock.patch(
+            "bookmarks.services.singlefile.create_snapshot",
+        )
+        self.mock_singlefile_create_snapshot = (
+            self.mock_singlefile_create_snapshot_patcher.start()
+        )
+
         user = self.get_or_create_test_user()
         user.profile.web_archive_integration = (
             UserProfile.WEB_ARCHIVE_INTEGRATION_ENABLED
@@ -62,157 +81,100 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         user.profile.enable_favicons = True
         user.profile.save()
 
-    @disable_logging
-    def run_pending_task(self, task_function: Any):
-        func = getattr(task_function, "task_function", None)
-        task = Task.objects.all()[0]
-        self.assertEqual(task_function.name, task.task_name)
-        args, kwargs = task.params()
-        func(*args, **kwargs)
-        task.delete()
+    def tearDown(self):
+        self.mock_save_api_patcher.stop()
+        self.mock_cdx_api_patcher.stop()
+        self.mock_load_favicon_patcher.stop()
+        self.mock_singlefile_create_snapshot_patcher.stop()
+        huey.storage.flush_results()
+        huey.immediate = False
 
-    @disable_logging
-    def run_all_pending_tasks(self, task_function: Any):
-        func = getattr(task_function, "task_function", None)
-        tasks = Task.objects.all()
-
-        for task in tasks:
-            self.assertEqual(task_function.name, task.task_name)
-            args, kwargs = task.params()
-            func(*args, **kwargs)
-            task.delete()
+    def executed_count(self):
+        return len(huey.all_results())
 
     def test_create_web_archive_snapshot_should_update_snapshot_url(self):
         bookmark = self.setup_bookmark()
-        mock_save_api = create_wayback_machine_save_api_mock()
 
-        with mock.patch.object(
-            waybackpy, "WaybackMachineSaveAPI", return_value=mock_save_api
-        ):
-            tasks.create_web_archive_snapshot(
-                self.get_or_create_test_user(), bookmark, False
-            )
-            self.run_pending_task(tasks._create_web_archive_snapshot_task)
-            bookmark.refresh_from_db()
+        tasks.create_web_archive_snapshot(
+            self.get_or_create_test_user(), bookmark, False
+        )
+        bookmark.refresh_from_db()
 
-            mock_save_api.save.assert_called_once()
-            self.assertEqual(
-                bookmark.web_archive_snapshot_url,
-                "https://example.com/created_snapshot",
-            )
+        self.mock_save_api.save.assert_called_once()
+        self.assertEqual(self.executed_count(), 1)
+        self.assertEqual(
+            bookmark.web_archive_snapshot_url,
+            "https://example.com/created_snapshot",
+        )
 
     def test_create_web_archive_snapshot_should_handle_missing_bookmark_id(self):
-        mock_save_api = create_wayback_machine_save_api_mock()
+        tasks._create_web_archive_snapshot_task(123, False)
 
-        with mock.patch.object(
-            waybackpy, "WaybackMachineSaveAPI", return_value=mock_save_api
-        ):
-            tasks._create_web_archive_snapshot_task(123, False)
-            self.run_pending_task(tasks._create_web_archive_snapshot_task)
-
-            mock_save_api.save.assert_not_called()
+        self.assertEqual(self.executed_count(), 1)
+        self.mock_save_api.save.assert_not_called()
 
     def test_create_web_archive_snapshot_should_skip_if_snapshot_exists(self):
         bookmark = self.setup_bookmark(web_archive_snapshot_url="https://example.com")
-        mock_save_api = create_wayback_machine_save_api_mock()
 
-        with mock.patch.object(
-            waybackpy, "WaybackMachineSaveAPI", return_value=mock_save_api
-        ):
-            tasks.create_web_archive_snapshot(
-                self.get_or_create_test_user(), bookmark, False
-            )
-            self.run_pending_task(tasks._create_web_archive_snapshot_task)
+        self.mock_save_api.create_web_archive_snapshot(
+            self.get_or_create_test_user(), bookmark, False
+        )
 
-            mock_save_api.assert_not_called()
+        self.assertEqual(self.executed_count(), 0)
+        self.mock_save_api.assert_not_called()
 
     def test_create_web_archive_snapshot_should_force_update_snapshot(self):
         bookmark = self.setup_bookmark(web_archive_snapshot_url="https://example.com")
-        mock_save_api = create_wayback_machine_save_api_mock(
-            archive_url="https://other.com"
+        self.mock_save_api.archive_url = "https://other.com"
+
+        tasks.create_web_archive_snapshot(
+            self.get_or_create_test_user(), bookmark, True
         )
+        bookmark.refresh_from_db()
 
-        with mock.patch.object(
-            waybackpy, "WaybackMachineSaveAPI", return_value=mock_save_api
-        ):
-            tasks.create_web_archive_snapshot(
-                self.get_or_create_test_user(), bookmark, True
-            )
-            self.run_pending_task(tasks._create_web_archive_snapshot_task)
-            bookmark.refresh_from_db()
-
-            self.assertEqual(bookmark.web_archive_snapshot_url, "https://other.com")
+        self.assertEqual(bookmark.web_archive_snapshot_url, "https://other.com")
 
     def test_create_web_archive_snapshot_should_use_newest_snapshot_as_fallback(self):
         bookmark = self.setup_bookmark()
-        mock_save_api = create_wayback_machine_save_api_mock(fail_on_save=True)
-        mock_cdx_api = create_cdx_server_api_mock()
+        self.mock_save_api.save.side_effect = WaybackError
 
-        with mock.patch.object(
-            waybackpy, "WaybackMachineSaveAPI", return_value=mock_save_api
-        ):
-            with mock.patch.object(
-                bookmarks.services.wayback,
-                "CustomWaybackMachineCDXServerAPI",
-                return_value=mock_cdx_api,
-            ):
-                tasks.create_web_archive_snapshot(
-                    self.get_or_create_test_user(), bookmark, False
-                )
-                self.run_pending_task(tasks._create_web_archive_snapshot_task)
+        tasks.create_web_archive_snapshot(
+            self.get_or_create_test_user(), bookmark, False
+        )
 
-                bookmark.refresh_from_db()
-                mock_cdx_api.newest.assert_called_once()
-                self.assertEqual(
-                    "https://example.com/newest_snapshot",
-                    bookmark.web_archive_snapshot_url,
-                )
+        bookmark.refresh_from_db()
+        self.mock_cdx_api.newest.assert_called_once()
+        self.assertEqual(
+            "https://example.com/newest_snapshot",
+            bookmark.web_archive_snapshot_url,
+        )
 
     def test_create_web_archive_snapshot_should_ignore_missing_newest_snapshot(self):
         bookmark = self.setup_bookmark()
-        mock_save_api = create_wayback_machine_save_api_mock(fail_on_save=True)
-        mock_cdx_api = create_cdx_server_api_mock(archive_url=None)
+        self.mock_save_api.save.side_effect = WaybackError
+        self.mock_cdx_api.newest.return_value = None
 
-        with mock.patch.object(
-            waybackpy, "WaybackMachineSaveAPI", return_value=mock_save_api
-        ):
-            with mock.patch.object(
-                bookmarks.services.wayback,
-                "CustomWaybackMachineCDXServerAPI",
-                return_value=mock_cdx_api,
-            ):
-                tasks.create_web_archive_snapshot(
-                    self.get_or_create_test_user(), bookmark, False
-                )
-                self.run_pending_task(tasks._create_web_archive_snapshot_task)
+        tasks.create_web_archive_snapshot(
+            self.get_or_create_test_user(), bookmark, False
+        )
 
-                bookmark.refresh_from_db()
-                self.assertEqual("", bookmark.web_archive_snapshot_url)
+        bookmark.refresh_from_db()
+        self.assertEqual("", bookmark.web_archive_snapshot_url)
 
     def test_create_web_archive_snapshot_should_ignore_newest_snapshot_errors(self):
         bookmark = self.setup_bookmark()
-        mock_save_api = create_wayback_machine_save_api_mock(fail_on_save=True)
-        mock_cdx_api = create_cdx_server_api_mock(fail_loading_snapshot=True)
+        self.mock_save_api.save.side_effect = WaybackError
+        self.mock_cdx_api.newest.side_effect = WaybackError
 
-        with mock.patch.object(
-            waybackpy, "WaybackMachineSaveAPI", return_value=mock_save_api
-        ):
-            with mock.patch.object(
-                bookmarks.services.wayback,
-                "CustomWaybackMachineCDXServerAPI",
-                return_value=mock_cdx_api,
-            ):
-                tasks.create_web_archive_snapshot(
-                    self.get_or_create_test_user(), bookmark, False
-                )
-                self.run_pending_task(tasks._create_web_archive_snapshot_task)
+        tasks.create_web_archive_snapshot(
+            self.get_or_create_test_user(), bookmark, False
+        )
 
-                bookmark.refresh_from_db()
-                self.assertEqual("", bookmark.web_archive_snapshot_url)
+        bookmark.refresh_from_db()
+        self.assertEqual("", bookmark.web_archive_snapshot_url)
 
     def test_create_web_archive_snapshot_should_not_save_stale_bookmark_data(self):
         bookmark = self.setup_bookmark()
-        mock_save_api = create_wayback_machine_save_api_mock()
 
         # update bookmark during API call to check that saving
         # the snapshot does not overwrite updated bookmark data
@@ -220,99 +182,64 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
             bookmark.title = "Updated title"
             bookmark.save()
 
-        mock_save_api.save.side_effect = mock_save_impl
+        self.mock_save_api.save.side_effect = mock_save_impl
 
-        with mock.patch.object(
-            waybackpy, "WaybackMachineSaveAPI", return_value=mock_save_api
-        ):
-            tasks.create_web_archive_snapshot(
-                self.get_or_create_test_user(), bookmark, False
-            )
-            self.run_pending_task(tasks._create_web_archive_snapshot_task)
-            bookmark.refresh_from_db()
+        tasks.create_web_archive_snapshot(
+            self.get_or_create_test_user(), bookmark, False
+        )
+        bookmark.refresh_from_db()
 
-            self.assertEqual(bookmark.title, "Updated title")
-            self.assertEqual(
-                "https://example.com/created_snapshot",
-                bookmark.web_archive_snapshot_url,
-            )
+        self.assertEqual(bookmark.title, "Updated title")
+        self.assertEqual(
+            "https://example.com/created_snapshot",
+            bookmark.web_archive_snapshot_url,
+        )
 
     def test_load_web_archive_snapshot_should_update_snapshot_url(self):
         bookmark = self.setup_bookmark()
-        mock_cdx_api = create_cdx_server_api_mock()
 
-        with mock.patch.object(
-            bookmarks.services.wayback,
-            "CustomWaybackMachineCDXServerAPI",
-            return_value=mock_cdx_api,
-        ):
-            tasks._load_web_archive_snapshot_task(bookmark.id)
-            self.run_pending_task(tasks._load_web_archive_snapshot_task)
+        tasks._load_web_archive_snapshot_task(bookmark.id)
 
-            bookmark.refresh_from_db()
-            mock_cdx_api.newest.assert_called_once()
-            self.assertEqual(
-                "https://example.com/newest_snapshot", bookmark.web_archive_snapshot_url
-            )
+        bookmark.refresh_from_db()
+
+        self.assertEqual(self.executed_count(), 1)
+        self.mock_cdx_api.newest.assert_called_once()
+        self.assertEqual(
+            "https://example.com/newest_snapshot", bookmark.web_archive_snapshot_url
+        )
 
     def test_load_web_archive_snapshot_should_handle_missing_bookmark_id(self):
-        mock_cdx_api = create_cdx_server_api_mock()
+        tasks._load_web_archive_snapshot_task(123)
 
-        with mock.patch.object(
-            bookmarks.services.wayback,
-            "CustomWaybackMachineCDXServerAPI",
-            return_value=mock_cdx_api,
-        ):
-            tasks._load_web_archive_snapshot_task(123)
-            self.run_pending_task(tasks._load_web_archive_snapshot_task)
-
-            mock_cdx_api.newest.assert_not_called()
+        self.assertEqual(self.executed_count(), 1)
+        self.mock_cdx_api.newest.assert_not_called()
 
     def test_load_web_archive_snapshot_should_skip_if_snapshot_exists(self):
         bookmark = self.setup_bookmark(web_archive_snapshot_url="https://example.com")
-        mock_cdx_api = create_cdx_server_api_mock()
 
-        with mock.patch.object(
-            bookmarks.services.wayback,
-            "CustomWaybackMachineCDXServerAPI",
-            return_value=mock_cdx_api,
-        ):
-            tasks._load_web_archive_snapshot_task(bookmark.id)
-            self.run_pending_task(tasks._load_web_archive_snapshot_task)
+        tasks._load_web_archive_snapshot_task(bookmark.id)
 
-            mock_cdx_api.newest.assert_not_called()
+        self.assertEqual(self.executed_count(), 1)
+        self.mock_cdx_api.newest.assert_not_called()
 
     def test_load_web_archive_snapshot_should_handle_missing_snapshot(self):
         bookmark = self.setup_bookmark()
-        mock_cdx_api = create_cdx_server_api_mock(archive_url=None)
+        self.mock_cdx_api.newest.return_value = None
 
-        with mock.patch.object(
-            bookmarks.services.wayback,
-            "CustomWaybackMachineCDXServerAPI",
-            return_value=mock_cdx_api,
-        ):
-            tasks._load_web_archive_snapshot_task(bookmark.id)
-            self.run_pending_task(tasks._load_web_archive_snapshot_task)
+        tasks._load_web_archive_snapshot_task(bookmark.id)
 
-            self.assertEqual("", bookmark.web_archive_snapshot_url)
+        self.assertEqual("", bookmark.web_archive_snapshot_url)
 
     def test_load_web_archive_snapshot_should_handle_wayback_errors(self):
         bookmark = self.setup_bookmark()
-        mock_cdx_api = create_cdx_server_api_mock(fail_loading_snapshot=True)
+        self.mock_cdx_api.newest.side_effect = WaybackError
 
-        with mock.patch.object(
-            bookmarks.services.wayback,
-            "CustomWaybackMachineCDXServerAPI",
-            return_value=mock_cdx_api,
-        ):
-            tasks._load_web_archive_snapshot_task(bookmark.id)
-            self.run_pending_task(tasks._load_web_archive_snapshot_task)
+        tasks._load_web_archive_snapshot_task(bookmark.id)
 
-            self.assertEqual("", bookmark.web_archive_snapshot_url)
+        self.assertEqual("", bookmark.web_archive_snapshot_url)
 
     def test_load_web_archive_snapshot_should_not_save_stale_bookmark_data(self):
         bookmark = self.setup_bookmark()
-        mock_cdx_api = create_cdx_server_api_mock()
 
         # update bookmark during API call to check that saving
         # the snapshot does not overwrite updated bookmark data
@@ -321,32 +248,26 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
             bookmark.save()
             return mock.DEFAULT
 
-        mock_cdx_api.newest.side_effect = mock_newest_impl
+        self.mock_cdx_api.newest.side_effect = mock_newest_impl
 
-        with mock.patch.object(
-            bookmarks.services.wayback,
-            "CustomWaybackMachineCDXServerAPI",
-            return_value=mock_cdx_api,
-        ):
-            tasks._load_web_archive_snapshot_task(bookmark.id)
-            self.run_pending_task(tasks._load_web_archive_snapshot_task)
-            bookmark.refresh_from_db()
+        tasks._load_web_archive_snapshot_task(bookmark.id)
+        bookmark.refresh_from_db()
 
-            self.assertEqual("Updated title", bookmark.title)
-            self.assertEqual(
-                "https://example.com/newest_snapshot", bookmark.web_archive_snapshot_url
-            )
+        self.assertEqual("Updated title", bookmark.title)
+        self.assertEqual(
+            "https://example.com/newest_snapshot", bookmark.web_archive_snapshot_url
+        )
 
     @override_settings(LD_DISABLE_BACKGROUND_TASKS=True)
     def test_create_web_archive_snapshot_should_not_run_when_background_tasks_are_disabled(
         self,
     ):
         bookmark = self.setup_bookmark()
+
         tasks.create_web_archive_snapshot(
             self.get_or_create_test_user(), bookmark, False
         )
-
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
 
     def test_create_web_archive_snapshot_should_not_run_when_web_archive_integration_is_disabled(
         self,
@@ -361,7 +282,7 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
             self.get_or_create_test_user(), bookmark, False
         )
 
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
 
     def test_schedule_bookmarks_without_snapshots_should_load_snapshot_for_all_bookmarks_without_snapshot(
         self,
@@ -375,16 +296,9 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         self.setup_bookmark(web_archive_snapshot_url="https://example.com")
 
         tasks.schedule_bookmarks_without_snapshots(user)
-        self.run_pending_task(tasks._schedule_bookmarks_without_snapshots_task)
 
-        task_list = Task.objects.all()
-        self.assertEqual(task_list.count(), 3)
-
-        for task in task_list:
-            self.assertEqual(
-                task.task_name,
-                "bookmarks.services.tasks._load_web_archive_snapshot_task",
-            )
+        self.assertEqual(self.executed_count(), 4)
+        self.assertEqual(self.mock_cdx_api.newest.call_count, 3)
 
     def test_schedule_bookmarks_without_snapshots_should_only_update_user_owned_bookmarks(
         self,
@@ -401,10 +315,8 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         self.setup_bookmark(user=other_user)
 
         tasks.schedule_bookmarks_without_snapshots(user)
-        self.run_pending_task(tasks._schedule_bookmarks_without_snapshots_task)
 
-        task_list = Task.objects.all()
-        self.assertEqual(task_list.count(), 3)
+        self.assertEqual(self.mock_cdx_api.newest.call_count, 3)
 
     @override_settings(LD_DISABLE_BACKGROUND_TASKS=True)
     def test_schedule_bookmarks_without_snapshots_should_not_run_when_background_tasks_are_disabled(
@@ -412,7 +324,7 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
     ):
         tasks.schedule_bookmarks_without_snapshots(self.user)
 
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
 
     def test_schedule_bookmarks_without_snapshots_should_not_run_when_web_archive_integration_is_disabled(
         self,
@@ -423,44 +335,32 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         self.user.profile.save()
         tasks.schedule_bookmarks_without_snapshots(self.user)
 
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
 
     def test_load_favicon_should_create_favicon_file(self):
         bookmark = self.setup_bookmark()
 
-        with mock.patch(
-            "bookmarks.services.favicon_loader.load_favicon"
-        ) as mock_load_favicon:
-            mock_load_favicon.return_value = "https_example_com.png"
+        tasks.load_favicon(self.get_or_create_test_user(), bookmark)
+        bookmark.refresh_from_db()
 
-            tasks.load_favicon(self.get_or_create_test_user(), bookmark)
-            self.run_pending_task(tasks._load_favicon_task)
-            bookmark.refresh_from_db()
-
-            self.assertEqual(bookmark.favicon_file, "https_example_com.png")
+        self.assertEqual(self.executed_count(), 1)
+        self.assertEqual(bookmark.favicon_file, "https_example_com.png")
 
     def test_load_favicon_should_update_favicon_file(self):
         bookmark = self.setup_bookmark(favicon_file="https_example_com.png")
 
-        with mock.patch(
-            "bookmarks.services.favicon_loader.load_favicon"
-        ) as mock_load_favicon:
-            mock_load_favicon.return_value = "https_example_updated_com.png"
-            tasks.load_favicon(self.get_or_create_test_user(), bookmark)
-            self.run_pending_task(tasks._load_favicon_task)
+        self.mock_load_favicon.return_value = "https_example_updated_com.png"
 
-            mock_load_favicon.assert_called()
-            bookmark.refresh_from_db()
-            self.assertEqual(bookmark.favicon_file, "https_example_updated_com.png")
+        tasks.load_favicon(self.get_or_create_test_user(), bookmark)
+
+        bookmark.refresh_from_db()
+        self.mock_load_favicon.assert_called_once()
+        self.assertEqual(bookmark.favicon_file, "https_example_updated_com.png")
 
     def test_load_favicon_should_handle_missing_bookmark(self):
-        with mock.patch(
-            "bookmarks.services.favicon_loader.load_favicon"
-        ) as mock_load_favicon:
-            tasks._load_favicon_task(123)
-            self.run_pending_task(tasks._load_favicon_task)
+        tasks._load_favicon_task(123)
 
-            mock_load_favicon.assert_not_called()
+        self.mock_load_favicon.assert_not_called()
 
     def test_load_favicon_should_not_save_stale_bookmark_data(self):
         bookmark = self.setup_bookmark()
@@ -472,24 +372,20 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
             bookmark.save()
             return "https_example_com.png"
 
-        with mock.patch(
-            "bookmarks.services.favicon_loader.load_favicon"
-        ) as mock_load_favicon:
-            mock_load_favicon.side_effect = mock_load_favicon_impl
+        self.mock_load_favicon.side_effect = mock_load_favicon_impl
 
-            tasks.load_favicon(self.get_or_create_test_user(), bookmark)
-            self.run_pending_task(tasks._load_favicon_task)
-            bookmark.refresh_from_db()
+        tasks.load_favicon(self.get_or_create_test_user(), bookmark)
+        bookmark.refresh_from_db()
 
-            self.assertEqual(bookmark.title, "Updated title")
-            self.assertEqual(bookmark.favicon_file, "https_example_com.png")
+        self.assertEqual(bookmark.title, "Updated title")
+        self.assertEqual(bookmark.favicon_file, "https_example_com.png")
 
     @override_settings(LD_DISABLE_BACKGROUND_TASKS=True)
     def test_load_favicon_should_not_run_when_background_tasks_are_disabled(self):
         bookmark = self.setup_bookmark()
         tasks.load_favicon(self.get_or_create_test_user(), bookmark)
 
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
 
     def test_load_favicon_should_not_run_when_favicon_feature_is_disabled(self):
         self.user.profile.enable_favicons = False
@@ -498,7 +394,7 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         bookmark = self.setup_bookmark()
         tasks.load_favicon(self.get_or_create_test_user(), bookmark)
 
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
 
     def test_schedule_bookmarks_without_favicons_should_load_favicon_for_all_bookmarks_without_favicon(
         self,
@@ -512,15 +408,9 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         self.setup_bookmark(favicon_file="https_example_com.png")
 
         tasks.schedule_bookmarks_without_favicons(user)
-        self.run_pending_task(tasks._schedule_bookmarks_without_favicons_task)
 
-        task_list = Task.objects.all()
-        self.assertEqual(task_list.count(), 3)
-
-        for task in task_list:
-            self.assertEqual(
-                task.task_name, "bookmarks.services.tasks._load_favicon_task"
-            )
+        self.assertEqual(self.executed_count(), 4)
+        self.assertEqual(self.mock_load_favicon.call_count, 3)
 
     def test_schedule_bookmarks_without_favicons_should_only_update_user_owned_bookmarks(
         self,
@@ -537,19 +427,17 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         self.setup_bookmark(user=other_user)
 
         tasks.schedule_bookmarks_without_favicons(user)
-        self.run_pending_task(tasks._schedule_bookmarks_without_favicons_task)
 
-        task_list = Task.objects.all()
-        self.assertEqual(task_list.count(), 3)
+        self.assertEqual(self.mock_load_favicon.call_count, 3)
 
     @override_settings(LD_DISABLE_BACKGROUND_TASKS=True)
     def test_schedule_bookmarks_without_favicons_should_not_run_when_background_tasks_are_disabled(
         self,
     ):
-        bookmark = self.setup_bookmark()
+        self.setup_bookmark()
         tasks.schedule_bookmarks_without_favicons(self.get_or_create_test_user())
 
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
 
     def test_schedule_bookmarks_without_favicons_should_not_run_when_favicon_feature_is_disabled(
         self,
@@ -557,10 +445,10 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         self.user.profile.enable_favicons = False
         self.user.profile.save()
 
-        bookmark = self.setup_bookmark()
+        self.setup_bookmark()
         tasks.schedule_bookmarks_without_favicons(self.get_or_create_test_user())
 
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
 
     def test_schedule_refresh_favicons_should_update_favicon_for_all_bookmarks(self):
         user = self.get_or_create_test_user()
@@ -572,15 +460,9 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         self.setup_bookmark(favicon_file="https_example_com.png")
 
         tasks.schedule_refresh_favicons(user)
-        self.run_pending_task(tasks._schedule_refresh_favicons_task)
 
-        task_list = Task.objects.all()
-        self.assertEqual(task_list.count(), 6)
-
-        for task in task_list:
-            self.assertEqual(
-                task.task_name, "bookmarks.services.tasks._load_favicon_task"
-            )
+        self.assertEqual(self.executed_count(), 7)
+        self.assertEqual(self.mock_load_favicon.call_count, 6)
 
     def test_schedule_refresh_favicons_should_only_update_user_owned_bookmarks(self):
         user = self.get_or_create_test_user()
@@ -595,10 +477,8 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         self.setup_bookmark(user=other_user)
 
         tasks.schedule_refresh_favicons(user)
-        self.run_pending_task(tasks._schedule_refresh_favicons_task)
 
-        task_list = Task.objects.all()
-        self.assertEqual(task_list.count(), 3)
+        self.assertEqual(self.mock_load_favicon.call_count, 3)
 
     @override_settings(LD_DISABLE_BACKGROUND_TASKS=True)
     def test_schedule_refresh_favicons_should_not_run_when_background_tasks_are_disabled(
@@ -607,14 +487,14 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         self.setup_bookmark()
         tasks.schedule_refresh_favicons(self.get_or_create_test_user())
 
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
 
     @override_settings(LD_ENABLE_REFRESH_FAVICONS=False)
     def test_schedule_refresh_favicons_should_not_run_when_refresh_is_disabled(self):
         self.setup_bookmark()
         tasks.schedule_refresh_favicons(self.get_or_create_test_user())
 
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
 
     def test_schedule_refresh_favicons_should_not_run_when_favicon_feature_is_disabled(
         self,
@@ -625,4 +505,109 @@ class BookmarkTasksTestCase(TestCase, BookmarkFactoryMixin):
         self.setup_bookmark()
         tasks.schedule_refresh_favicons(self.get_or_create_test_user())
 
-        self.assertEqual(Task.objects.count(), 0)
+        self.assertEqual(self.executed_count(), 0)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_html_snapshot_should_create_pending_asset(self):
+        bookmark = self.setup_bookmark()
+
+        # Mock the task function to avoid running it immediately
+        with mock.patch("bookmarks.services.tasks._create_html_snapshot_task"):
+            tasks.create_html_snapshot(bookmark)
+            self.assertEqual(BookmarkAsset.objects.count(), 1)
+
+            tasks.create_html_snapshot(bookmark)
+            self.assertEqual(BookmarkAsset.objects.count(), 2)
+
+            assets = BookmarkAsset.objects.filter(bookmark=bookmark)
+            for asset in assets:
+                self.assertEqual(asset.bookmark, bookmark)
+                self.assertEqual(asset.asset_type, BookmarkAsset.TYPE_SNAPSHOT)
+                self.assertEqual(asset.content_type, BookmarkAsset.CONTENT_TYPE_HTML)
+                self.assertIn("HTML snapshot", asset.display_name)
+                self.assertEqual(asset.status, BookmarkAsset.STATUS_PENDING)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_html_snapshot_should_update_file_info(self):
+        bookmark = self.setup_bookmark(url="https://example.com")
+
+        with mock.patch(
+            "bookmarks.services.tasks._generate_snapshot_filename"
+        ) as mock_generate:
+            expected_filename = "snapshot_2021-01-02_034455_https___example.com.html.gz"
+            mock_generate.return_value = expected_filename
+
+            tasks.create_html_snapshot(bookmark)
+            BookmarkAsset.objects.get(bookmark=bookmark)
+
+            # Run periodic task to process the snapshot
+            tasks._schedule_html_snapshots_task()
+
+            self.mock_singlefile_create_snapshot.assert_called_once_with(
+                "https://example.com",
+                os.path.join(
+                    settings.LD_ASSET_FOLDER,
+                    expected_filename,
+                ),
+            )
+
+            asset = BookmarkAsset.objects.get(bookmark=bookmark)
+            self.assertEqual(asset.status, BookmarkAsset.STATUS_COMPLETE)
+            self.assertEqual(asset.file, expected_filename)
+            self.assertTrue(asset.gzip)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_html_snapshot_truncate_filename(self):
+        # Create a bookmark with a very long URL
+        long_url = "http://" + "a" * 300 + ".com"
+        bookmark = self.setup_bookmark(url=long_url)
+
+        tasks.create_html_snapshot(bookmark)
+        BookmarkAsset.objects.get(bookmark=bookmark)
+
+        # Run periodic task to process the snapshot
+        tasks._schedule_html_snapshots_task()
+
+        asset = BookmarkAsset.objects.get(bookmark=bookmark)
+        self.assertEqual(len(asset.file), 192)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_html_snapshot_should_handle_error(self):
+        bookmark = self.setup_bookmark(url="https://example.com")
+
+        self.mock_singlefile_create_snapshot.side_effect = singlefile.SingeFileError(
+            "Error"
+        )
+        tasks.create_html_snapshot(bookmark)
+
+        # Run periodic task to process the snapshot
+        tasks._schedule_html_snapshots_task()
+
+        asset = BookmarkAsset.objects.get(bookmark=bookmark)
+        self.assertEqual(asset.status, BookmarkAsset.STATUS_FAILURE)
+        self.assertEqual(asset.file, "")
+        self.assertFalse(asset.gzip)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_create_html_snapshot_should_handle_missing_bookmark(self):
+        tasks._create_html_snapshot_task(123)
+
+        self.mock_singlefile_create_snapshot.assert_not_called()
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=False)
+    def test_create_html_snapshot_should_not_create_asset_when_single_file_is_disabled(
+        self,
+    ):
+        bookmark = self.setup_bookmark()
+        tasks.create_html_snapshot(bookmark)
+
+        self.assertEqual(BookmarkAsset.objects.count(), 0)
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True, LD_DISABLE_BACKGROUND_TASKS=True)
+    def test_create_html_snapshot_should_not_create_asset_when_background_tasks_are_disabled(
+        self,
+    ):
+        bookmark = self.setup_bookmark()
+        tasks.create_html_snapshot(bookmark)
+
+        self.assertEqual(BookmarkAsset.objects.count(), 0)

bookmarks/tests/test_monolith_service.py

@@ -0,0 +1,44 @@
+import gzip
+import os
+from unittest import mock
+import subprocess
+
+from django.test import TestCase
+
+from bookmarks.services import monolith
+
+
+class MonolithServiceTestCase(TestCase):
+    html_content = "<html><body><h1>Hello, World!</h1></body></html>"
+    html_filepath = "temp.html.gz"
+    temp_html_filepath = "temp.html.gz.tmp"
+
+    def tearDown(self):
+        if os.path.exists(self.html_filepath):
+            os.remove(self.html_filepath)
+        if os.path.exists(self.temp_html_filepath):
+            os.remove(self.temp_html_filepath)
+
+    def create_test_file(self, *args, **kwargs):
+        with open(self.temp_html_filepath, "w") as file:
+            file.write(self.html_content)
+
+    def test_create_snapshot(self):
+        with mock.patch("subprocess.run") as mock_run:
+            mock_run.side_effect = self.create_test_file
+
+            monolith.create_snapshot("http://example.com", self.html_filepath)
+
+            self.assertTrue(os.path.exists(self.html_filepath))
+            self.assertFalse(os.path.exists(self.temp_html_filepath))
+
+            with gzip.open(self.html_filepath, "rt") as file:
+                content = file.read()
+                self.assertEqual(content, self.html_content)
+
+    def test_create_snapshot_failure(self):
+        with mock.patch("subprocess.run") as mock_run:
+            mock_run.side_effect = subprocess.CalledProcessError(1, "command")
+
+            with self.assertRaises(monolith.MonolithError):
+                monolith.create_snapshot("http://example.com", self.html_filepath)

bookmarks/tests/test_oidc_support.py

@@ -49,3 +49,15 @@ class OidcSupportTest(TestCase):
             base_settings.AUTHENTICATION_BACKENDS,
         )
         del os.environ["LD_ENABLE_OIDC"]  # Remove the temporary environment variable
+
+    def test_default_settings(self):
+        os.environ["LD_ENABLE_OIDC"] = "True"
+        base_settings = importlib.import_module("siteroot.settings.base")
+        importlib.reload(base_settings)
+
+        self.assertEqual(
+            True,
+            base_settings.OIDC_VERIFY_SSL,
+        )
+
+        del os.environ["LD_ENABLE_OIDC"]

bookmarks/tests/test_settings_general_view.py

@@ -31,6 +31,7 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             "enable_sharing": False,
             "enable_public_sharing": False,
             "enable_favicons": False,
+            "enable_automatic_html_snapshots": True,
             "tag_search": UserProfile.TAG_SEARCH_STRICT,
             "display_url": False,
             "display_view_bookmark_action": True,
@@ -69,6 +70,7 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             "enable_sharing": True,
             "enable_public_sharing": True,
             "enable_favicons": True,
+            "enable_automatic_html_snapshots": False,
             "tag_search": UserProfile.TAG_SEARCH_LAX,
             "display_url": True,
             "display_view_bookmark_action": False,
@@ -110,6 +112,10 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
         self.assertEqual(
             self.user.profile.enable_favicons, form_data["enable_favicons"]
         )
+        self.assertEqual(
+            self.user.profile.enable_automatic_html_snapshots,
+            form_data["enable_automatic_html_snapshots"],
+        )
         self.assertEqual(self.user.profile.tag_search, form_data["tag_search"])
         self.assertEqual(self.user.profile.display_url, form_data["display_url"])
         self.assertEqual(
@@ -285,6 +291,35 @@ class SettingsGeneralViewTestCase(TestCase, BookmarkFactoryMixin):
             count=0,
         )
 
+    def test_automatic_html_snapshots_should_be_hidden_when_snapshots_not_supported(
+        self,
+    ):
+        response = self.client.get(reverse("bookmarks:settings.general"))
+        html = response.content.decode()
+
+        self.assertInHTML(
+            """
+            <input type="checkbox" name="enable_automatic_html_snapshots" id="id_enable_automatic_html_snapshots" checked="">
+            """,
+            html,
+            count=0,
+        )
+
+    @override_settings(LD_ENABLE_SNAPSHOTS=True)
+    def test_automatic_html_snapshots_should_be_visible_when_snapshots_supported(
+        self,
+    ):
+        response = self.client.get(reverse("bookmarks:settings.general"))
+        html = response.content.decode()
+
+        self.assertInHTML(
+            """
+            <input type="checkbox" name="enable_automatic_html_snapshots" id="id_enable_automatic_html_snapshots" checked="">
+            """,
+            html,
+            count=1,
+        )
+
     def test_about_shows_version_info(self):
         response = self.client.get(reverse("bookmarks:settings.general"))
         html = response.content.decode()

bookmarks/tests/test_singlefile_service.py

@@ -0,0 +1,111 @@
+import gzip
+import os
+import subprocess
+from unittest import mock
+
+from django.test import TestCase, override_settings
+
+from bookmarks.services import singlefile
+
+
+class SingleFileServiceTestCase(TestCase):
+    html_content = "<html><body><h1>Hello, World!</h1></body></html>"
+    html_filepath = "temp.html.gz"
+    temp_html_filepath = "temp.html.gz.tmp"
+
+    def tearDown(self):
+        if os.path.exists(self.html_filepath):
+            os.remove(self.html_filepath)
+        if os.path.exists(self.temp_html_filepath):
+            os.remove(self.temp_html_filepath)
+
+    def create_test_file(self, *args, **kwargs):
+        with open(self.temp_html_filepath, "w") as file:
+            file.write(self.html_content)
+
+    def test_create_snapshot(self):
+        mock_process = mock.Mock()
+        mock_process.wait.return_value = 0
+        self.create_test_file()
+
+        with mock.patch("subprocess.Popen", return_value=mock_process):
+            singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+            self.assertTrue(os.path.exists(self.html_filepath))
+            self.assertFalse(os.path.exists(self.temp_html_filepath))
+
+            with gzip.open(self.html_filepath, "rt") as file:
+                content = file.read()
+                self.assertEqual(content, self.html_content)
+
+    def test_create_snapshot_failure(self):
+        # subprocess fails - which it probably doesn't as single-file doesn't return exit codes
+        with mock.patch("subprocess.Popen") as mock_popen:
+            mock_popen.side_effect = subprocess.CalledProcessError(1, "command")
+
+            with self.assertRaises(singlefile.SingeFileError):
+                singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+        # so also check that it raises error if output file isn't created
+        with mock.patch("subprocess.Popen"):
+            with self.assertRaises(singlefile.SingeFileError):
+                singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+    def test_create_snapshot_empty_options(self):
+        mock_process = mock.Mock()
+        mock_process.wait.return_value = 0
+        self.create_test_file()
+
+        with mock.patch("subprocess.Popen") as mock_popen:
+            singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+            expected_args = [
+                "single-file",
+                "http://example.com",
+                self.html_filepath + ".tmp",
+            ]
+            mock_popen.assert_called_with(expected_args, start_new_session=True)
+
+    @override_settings(
+        LD_SINGLEFILE_OPTIONS='--some-option "some value" --another-option "another value" --third-option="third value"'
+    )
+    def test_create_snapshot_custom_options(self):
+        mock_process = mock.Mock()
+        mock_process.wait.return_value = 0
+        self.create_test_file()
+
+        with mock.patch("subprocess.Popen") as mock_popen:
+            singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+            expected_args = [
+                "single-file",
+                "--some-option",
+                "some value",
+                "--another-option",
+                "another value",
+                "--third-option=third value",
+                "http://example.com",
+                self.html_filepath + ".tmp",
+            ]
+            mock_popen.assert_called_with(expected_args, start_new_session=True)
+
+    def test_create_snapshot_default_timeout_setting(self):
+        mock_process = mock.Mock()
+        mock_process.wait.return_value = 0
+        self.create_test_file()
+
+        with mock.patch("subprocess.Popen", return_value=mock_process):
+            singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+            mock_process.wait.assert_called_with(timeout=60)
+
+    @override_settings(LD_SINGLEFILE_TIMEOUT_SEC=120)
+    def test_create_snapshot_custom_timeout_setting(self):
+        mock_process = mock.Mock()
+        mock_process.wait.return_value = 0
+        self.create_test_file()
+
+        with mock.patch("subprocess.Popen", return_value=mock_process):
+            singlefile.create_snapshot("http://example.com", self.html_filepath)
+
+            mock_process.wait.assert_called_with(timeout=120)

bookmarks/urls.py

@@ -44,6 +44,12 @@ urlpatterns = [
         views.bookmarks.details_modal,
         name="details_modal",
     ),
+    # Assets
+    path(
+        "assets/<int:asset_id>",
+        views.assets.view,
+        name="assets.view",
+    ),
     # Partials
     path(
         "bookmarks/partials/bookmark-list/active",
@@ -75,6 +81,11 @@ urlpatterns = [
         partials.shared_tag_cloud,
         name="partials.tag_cloud.shared",
     ),
+    path(
+        "bookmarks/partials/details-form/<int:bookmark_id>",
+        partials.details_form,
+        name="partials.details_form",
+    ),
     # Settings
     path("settings", views.settings.general, name="settings.index"),
     path("settings/general", views.settings.general, name="settings.general"),

bookmarks/views/__init__.py

@@ -1,3 +1,4 @@
+from .assets import *
 from .bookmarks import *
 from .settings import *
 from .toasts import *

bookmarks/views/assets.py

@@ -0,0 +1,43 @@
+import gzip
+import os
+
+from django.conf import settings
+from django.http import (
+    HttpResponse,
+    Http404,
+)
+
+from bookmarks.models import BookmarkAsset
+
+
+def view(request, asset_id: int):
+    try:
+        asset = BookmarkAsset.objects.get(pk=asset_id)
+    except BookmarkAsset.DoesNotExist:
+        raise Http404("Asset does not exist")
+
+    bookmark = asset.bookmark
+    is_owner = bookmark.owner == request.user
+    is_shared = (
+        request.user.is_authenticated
+        and bookmark.shared
+        and bookmark.owner.profile.enable_sharing
+    )
+    is_public_shared = bookmark.shared and bookmark.owner.profile.enable_public_sharing
+
+    if not is_owner and not is_shared and not is_public_shared:
+        raise Http404("Bookmark does not exist")
+
+    filepath = os.path.join(settings.LD_ASSET_FOLDER, asset.file)
+
+    if not os.path.exists(filepath):
+        raise Http404("Asset file does not exist")
+
+    if asset.gzip:
+        with gzip.open(filepath, "rb") as f:
+            content = f.read()
+    else:
+        with open(filepath, "rb") as f:
+            content = f.read()
+
+    return HttpResponse(content, content_type=asset.content_type)

bookmarks/views/bookmarks.py

@@ -12,7 +12,13 @@ from django.shortcuts import render
 from django.urls import reverse
 
 from bookmarks import queries
-from bookmarks.models import Bookmark, BookmarkForm, BookmarkSearch, build_tag_string
+from bookmarks.models import (
+    Bookmark,
+    BookmarkAsset,
+    BookmarkForm,
+    BookmarkSearch,
+    build_tag_string,
+)
 from bookmarks.services.bookmarks import (
     create_bookmark,
     update_bookmark,
@@ -28,6 +34,7 @@ from bookmarks.services.bookmarks import (
     share_bookmarks,
     unshare_bookmarks,
 )
+from bookmarks.services import tasks
 from bookmarks.utils import get_safe_return_url
 from bookmarks.views.partials import contexts
 
@@ -120,31 +127,39 @@ def _details(request, bookmark_id: int, template: str):
     if not is_owner and not is_shared and not is_public_shared:
         raise Http404("Bookmark does not exist")
 
-    edit_return_url = get_safe_return_url(
-        request.GET.get("return_url"), reverse("bookmarks:details", args=[bookmark_id])
-    )
-    delete_return_url = get_safe_return_url(
-        request.GET.get("return_url"), reverse("bookmarks:index")
-    )
-
-    # handles status actions form
     if request.method == "POST":
         if not is_owner:
             raise Http404("Bookmark does not exist")
-        bookmark.is_archived = request.POST.get("is_archived") == "on"
-        bookmark.unread = request.POST.get("unread") == "on"
-        bookmark.shared = request.POST.get("shared") == "on"
-        bookmark.save()
 
-        return HttpResponseRedirect(edit_return_url)
+        return_url = get_safe_return_url(
+            request.GET.get("return_url"),
+            reverse("bookmarks:details", args=[bookmark.id]),
+        )
+
+        if "remove_asset" in request.POST:
+            asset_id = request.POST["remove_asset"]
+            try:
+                asset = bookmark.bookmarkasset_set.get(pk=asset_id)
+            except BookmarkAsset.DoesNotExist:
+                raise Http404("Asset does not exist")
+            asset.delete()
+        if "create_snapshot" in request.POST:
+            tasks.create_html_snapshot(bookmark)
+        else:
+            bookmark.is_archived = request.POST.get("is_archived") == "on"
+            bookmark.unread = request.POST.get("unread") == "on"
+            bookmark.shared = request.POST.get("shared") == "on"
+            bookmark.save()
+
+        return HttpResponseRedirect(return_url)
+
+    details_context = contexts.BookmarkDetailsContext(request, bookmark)
 
     return render(
         request,
         template,
         {
-            "bookmark": bookmark,
-            "edit_return_url": edit_return_url,
-            "delete_return_url": delete_return_url,
+            "details": details_context,
         },
     )
 

bookmarks/views/partials/__init__.py

@@ -1,6 +1,8 @@
 from django.contrib.auth.decorators import login_required
+from django.http import Http404
 from django.shortcuts import render
 
+from bookmarks.models import Bookmark
 from bookmarks.views.partials import contexts
 
 
@@ -56,3 +58,15 @@ def shared_tag_cloud(request):
     tag_cloud_context = contexts.SharedTagCloudContext(request)
 
     return render(request, "bookmarks/tag_cloud.html", {"tag_cloud": tag_cloud_context})
+
+
+@login_required
+def details_form(request, bookmark_id: int):
+    try:
+        bookmark = Bookmark.objects.get(pk=bookmark_id, owner=request.user)
+    except Bookmark.DoesNotExist:
+        raise Http404("Bookmark does not exist")
+
+    details_context = contexts.BookmarkDetailsContext(request, bookmark)
+
+    return render(request, "bookmarks/details/form.html", {"details": details_context})

bookmarks/views/partials/contexts.py

@@ -6,11 +6,13 @@ from django.core.handlers.wsgi import WSGIRequest
 from django.core.paginator import Paginator
 from django.db import models
 from django.urls import reverse
+from django.conf import settings
 
 from bookmarks import queries
 from bookmarks import utils
 from bookmarks.models import (
     Bookmark,
+    BookmarkAsset,
     BookmarkSearch,
     User,
     UserProfile,
@@ -274,3 +276,55 @@ class SharedTagCloudContext(TagCloudContext):
         return queries.query_shared_bookmark_tags(
             user, self.request.user_profile, self.search, public_only
         )
+
+
+class BookmarkAssetItem:
+    def __init__(self, asset: BookmarkAsset):
+        self.asset = asset
+
+        self.id = asset.id
+        self.display_name = asset.display_name
+        self.content_type = asset.content_type
+        self.file = asset.file
+        self.file_size = asset.file_size
+        self.status = asset.status
+
+        icon_classes = []
+        text_classes = []
+        if asset.status == BookmarkAsset.STATUS_PENDING:
+            icon_classes.append("text-gray")
+            text_classes.append("text-gray")
+        elif asset.status == BookmarkAsset.STATUS_FAILURE:
+            icon_classes.append("text-error")
+            text_classes.append("text-error")
+        else:
+            icon_classes.append("text-primary")
+
+        self.icon_classes = " ".join(icon_classes)
+        self.text_classes = " ".join(text_classes)
+
+
+class BookmarkDetailsContext:
+    def __init__(self, request: WSGIRequest, bookmark: Bookmark):
+        user = request.user
+        user_profile = request.user_profile
+
+        self.edit_return_url = utils.get_safe_return_url(
+            request.GET.get("return_url"),
+            reverse("bookmarks:details", args=[bookmark.id]),
+        )
+        self.delete_return_url = utils.get_safe_return_url(
+            request.GET.get("return_url"), reverse("bookmarks:index")
+        )
+
+        self.bookmark = bookmark
+        self.profile = request.user_profile
+        self.is_editable = bookmark.owner == user
+        self.sharing_enabled = user_profile.enable_sharing
+        self.show_link_icons = user_profile.enable_favicons and bookmark.favicon_file
+        # For now hide files section if snapshots are not supported
+        self.show_files = settings.LD_ENABLE_SNAPSHOTS
+
+        self.assets = [
+            BookmarkAssetItem(asset) for asset in bookmark.bookmarkasset_set.all()
+        ]

bookmarks/views/settings.py

@@ -12,7 +12,7 @@ from django.shortcuts import render
 from django.urls import reverse
 from rest_framework.authtoken.models import Token
 
-from bookmarks.models import Bookmark, BookmarkSearch, UserProfileForm, FeedToken
+from bookmarks.models import Bookmark, UserProfileForm, FeedToken
 from bookmarks.services import exporter, tasks
 from bookmarks.services import importer
 from bookmarks.utils import app_version
@@ -24,6 +24,7 @@ logger = logging.getLogger(__name__)
 def general(request):
     profile_form = None
     enable_refresh_favicons = django_settings.LD_ENABLE_REFRESH_FAVICONS
+    has_snapshot_support = django_settings.LD_ENABLE_SNAPSHOTS
     update_profile_success_message = None
     refresh_favicons_success_message = None
     import_success_message = _find_message_with_tag(
@@ -53,6 +54,7 @@ def general(request):
         {
             "form": profile_form,
             "enable_refresh_favicons": enable_refresh_favicons,
+            "has_snapshot_support": has_snapshot_support,
             "update_profile_success_message": update_profile_success_message,
             "refresh_favicons_success_message": refresh_favicons_success_message,
             "import_success_message": import_success_message,

bootstrap.sh

@@ -7,6 +7,8 @@ LD_SERVER_PORT="${LD_SERVER_PORT:-9090}"
 mkdir -p data
 # Create favicon folder if it does not exist
 mkdir -p data/favicons
+# Create assets folder if it does not exist
+mkdir -p data/assets
 
 # Generate secret key file if it does not exist
 python manage.py generate_secret_key
@@ -16,6 +18,8 @@ python manage.py migrate
 python manage.py enable_wal
 # Create initial superuser if defined in options / environment variables
 python manage.py create_initial_superuser
+# Migrate legacy background tasks to Huey
+python manage.py migrate_tasks
 
 # Ensure the DB folder is owned by the right user
 chown -R www-data: /etc/linkding/data

docker/alpine.Dockerfile

@@ -31,7 +31,8 @@ RUN pip install -U pip && pip install -r requirements.txt -r requirements.dev.tx
 COPY . .
 COPY --from=node-build /etc/linkding .
 # run Django part of the build
-RUN python manage.py compilescss && \
+RUN mkdir data && \
+    python manage.py compilescss && \
     python manage.py collectstatic --ignore=*.scss && \
     python manage.py compilescss --delete-files
 
@@ -67,7 +68,7 @@ RUN wget https://www.sqlite.org/${SQLITE_RELEASE_YEAR}/sqlite-amalgamation-${SQL
     gcc -fPIC -shared icu.c `pkg-config --libs --cflags icu-uc icu-io` -o libicu.so
 
 
-FROM python:3.11.8-alpine3.19 AS final
+FROM python:3.11.8-alpine3.19 AS linkding
 # install runtime dependencies
 RUN apk update && apk add bash curl icu libpq mailcap libssl3
 # create www-data user and group
@@ -96,3 +97,12 @@ HEALTHCHECK --interval=30s --retries=3 --timeout=1s \
 CMD curl -f http://localhost:${LD_SERVER_PORT:-9090}/${LD_CONTEXT_PATH}health || exit 1
 
 CMD ["./bootstrap.sh"]
+
+
+FROM linkding AS linkding-plus
+# install node, chromium
+RUN apk update && apk add nodejs npm chromium
+# install single-file from fork for now, which contains several hotfixes
+RUN npm install -g https://github.com/sissbruecker/single-file-cli/tarball/f3730995a52f27d5041a1ad9e7528af4b6b4cf4b
+# enable snapshot support
+ENV LD_ENABLE_SNAPSHOTS=True

docker/default.Dockerfile

@@ -33,7 +33,8 @@ RUN pip install -U pip && pip install -r requirements.txt -r requirements.dev.tx
 COPY . .
 COPY --from=node-build /etc/linkding .
 # run Django part of the build
-RUN python manage.py compilescss && \
+RUN mkdir data && \
+    python manage.py compilescss && \
     python manage.py collectstatic --ignore=*.scss && \
     python manage.py compilescss --delete-files
 
@@ -69,7 +70,7 @@ RUN wget https://www.sqlite.org/${SQLITE_RELEASE_YEAR}/sqlite-amalgamation-${SQL
     gcc -fPIC -shared icu.c `pkg-config --libs --cflags icu-uc icu-io` -o libicu.so
 
 
-FROM python:3.11.8-slim-bookworm as final
+FROM python:3.11.8-slim-bookworm as linkding
 RUN apt-get update && apt-get -y install mime-support libpq-dev libicu-dev libssl3 curl
 WORKDIR /etc/linkding
 # copy prod dependencies
@@ -94,3 +95,17 @@ HEALTHCHECK --interval=30s --retries=3 --timeout=1s \
 CMD curl -f http://localhost:${LD_SERVER_PORT:-9090}/${LD_CONTEXT_PATH}health || exit 1
 
 CMD ["./bootstrap.sh"]
+
+FROM linkding AS linkding-plus
+# install chromium
+RUN apt-get update && apt-get -y install chromium
+# install node
+ENV NODE_MAJOR=20
+RUN apt-get install -y gnupg2 apt-transport-https ca-certificates && \
+    curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /usr/share/keyrings/nodesource.gpg && \
+    echo "deb [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list && \
+    apt-get update && apt-get install -y nodejs
+# install single-file from fork for now, which contains several hotfixes
+RUN npm install -g https://github.com/sissbruecker/single-file-cli/tarball/f3730995a52f27d5041a1ad9e7528af4b6b4cf4b
+# enable snapshot support
+ENV LD_ENABLE_SNAPSHOTS=True

docs/Options.md

@@ -118,6 +118,46 @@ The following options can be configured:
 - `OIDC_RP_CLIENT_SECRET` - The client secret of the application.
 - `OIDC_RP_SIGN_ALGO` - The algorithm the OIDC provider uses to sign ID tokens. Default is `RS256`.
 - `OIDC_USE_PKCE` - Whether to use PKCE for the OIDC flow. Default is `True`.
+- `OIDC_VERIFY_SSL` - Whether to verify the SSL certificate of the OIDC provider. Set to `False` if using self-signed certificates or custom certificate authority. Default is `True`.
+
+<details>
+
+<summary>Authelia Example</summary>
+
+#### Linkding Configuration
+
+```bash
+LD_ENABLE_OIDC=True
+OIDC_OP_AUTHORIZATION_ENDPOINT=https://auth.example.com/api/oidc/authorization
+OIDC_OP_TOKEN_ENDPOINT=https://auth.example.com/api/oidc/token
+OIDC_OP_USER_ENDPOINT=https://auth.example.com/api/oidc/userinfo
+OIDC_OP_JWKS_ENDPOINT=https://auth.example.com/jwks.json
+OIDC_RP_CLIENT_ID=linkding
+OIDC_RP_CLIENT_SECRET=myClientSecret
+```
+#### Authelia Configuration
+
+```yaml
+identity_providers:
+  oidc:
+    # --- more OIDC provider configuration ---
+
+    clients:
+      - id: linkding
+        description: Linkding
+        # docker run --rm authelia/authelia:latest authelia crypto rand --length 64 --charset alphanumeric
+        secret: myClientSecret
+        public: false
+        token_endpoint_auth_method: client_secret_post
+        scopes:
+          - openid
+          - email
+          - profile
+        redirect_uris:
+          - https://linkding.example.com/oidc/callback/
+```
+
+</details>
 
 ### `LD_CSRF_TRUSTED_ORIGINS`
 
@@ -206,3 +246,20 @@ See the default URL for how to insert the placeholder to the favicon provider UR
 
 Alternative favicon providers:
 - DuckDuckGo: `https://icons.duckduckgo.com/ip3/{domain}.ico`
+
+
+### `LD_SINGLEFILE_TIMEOUT_SEC`
+
+Values: `Float` | Default =  60.0
+
+When creating HTML archive snapshots, control the timeout for how long to wait for the snapshot to complete, in `seconds`.
+Defaults to 60 seconds; on lower-powered hardware you may need to increase this value. 
+
+### `LD_SINGLEFILE_OPTIONS`
+
+Values: `String` | Default = None
+
+When creating HTML archive snapshots, pass additional options to the `single-file` application that is used to create snapshots.
+See `single-file --help` for complete list of arguments, or browse source: https://github.com/gildas-lormeau/single-file-cli/blob/master/options.js
+
+Example: `LD_SINGLEFILE_OPTIONS=--user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:124.0) Gecko/20100101 Firefox/124.0"`

docs/backup.md

@@ -4,24 +4,56 @@ Linkding stores all data in the application's data folder.
 The full path to that folder in the Docker container is `/etc/linkding/data`.
 As described in the installation docs, you should mount the `/etc/linkding/data` folder to a folder on your host system.
 
-The data folder contains the following contents:
+The data folder contains the following contents that are relevant for backups:
 - `db.sqlite3` - the SQLite database
+- `assets` - folder that contains HTML snapshots of bookmarks
 - `favicons` - folder that contains downloaded favicons
 
 The following sections explain how to back up the individual contents.
 
-## Database
+## Full backup
 
-This section describes several methods on how to back up the contents of the SQLite database.
+linkding provides a CLI command to create a full backup of the data folder. This creates a zip file that contains backups of the database, assets, and favicons.
+
+> [!NOTE]
+> This method assumes that you are using the default SQLite database.
+> If you are using a different database, such as Postgres, you'll have to back up the database and other contents of the data folder manually.
+
+To create a full backup, execute the following command:
+```shell
+docker exec -it linkding python manage.py full_backup /etc/linkding/data/backup.zip
+```
+This creates a `backup.zip` file in the Docker container under `/etc/linkding/data`.
+
+To copy the backup file to your host system, execute the following command:
+```shell
+docker cp linkding:/etc/linkding/data/backup.zip backup.zip
+```
+This copies the backup file from the Docker container to the current folder on your host system.
+Now you can move that file to your backup location.
+
+To restore a backup: 
+- Extract the zip file in a folder of your new installation.
+- Rename the extracted folder to `data`.
+- When starting the Docker container, mount that folder to `/etc/linkding/data` as explained in the README.
+- Then start the Docker container.
+
+## Alternative backup methods
+
+If you can't use the full backup method, this section describes alternatives how to back up the individual contents of the data folder.
+
+### SQLite database backup
+
+linkding includes a CLI command for creating a backup copy of the database.
 
 > [!WARNING]
 > While the SQLite database is just a single file, it is not recommended to just copy that file.
 > This method is not transaction safe and may result in a [corrupted database](https://www.sqlite.org/howtocorrupt.html).
 > Use one of the backup methods described below.
 
-### Using the backup command
-
-linkding includes a CLI command for creating a backup copy of the database.
+> [!WARNING]
+> This method is deprecated and may be removed in the future.
+> Please use the full backup method described above.
 
 To create a backup, execute the following command:
 ```shell
@@ -38,12 +70,12 @@ Now you can move that file to your backup location.
 
 To restore the backup, just copy the backup file to the data folder of your new installation and rename it to `db.sqlite3`. Then start the Docker container.
 
-### Using the SQLite dump function
+### SQLite database SQL dump
 
 Requires [SQLite](https://www.sqlite.org/index.html) to be installed on your host system.
 
 With this method you create a plain text file with the SQL statements to recreate the SQLite database.
-To create a backup, execute the following command in the data folder:
+To create a backup, execute the following command in the data folder on your host system:
 ```shell
 sqlite3 db.sqlite3 .dump > backup.sql
 ```
@@ -56,8 +88,8 @@ Using git, you can commit the changes, followed by a git push to a remote reposi
 This is the least technical option to back up bookmarks, but has several limitations:
 - It does not export user profiles.
 - It only exports your own bookmarks, not those of other users.
-- It does not export archived bookmarks.
 - It does not export URLs of snapshots on the Internet Archive Wayback machine.
+- It does not export HTML snapshots of bookmarks. Even if you backup and restore the assets folder, the bookmarks will not be linked to the snapshots anymore.
 - It does not export favicons.
 
 Only use this method if you are fine with the above limitations.
@@ -70,7 +102,16 @@ To restore bookmarks, open the general settings on your new installation.
 In the Import section, click on the *Choose file* button to select the HTML file you downloaded before.
 Then click on the *Import* button to import the bookmarks.
 
-## Favicons
+### Assets
+
+If you are using the HTML snapshots feature, you should also do backups of the `assets` folder.
+It contains the HTML snapshots files of your bookmarks which are referenced from the database.
+
+To back up the assets, then you have to copy the `assets` folder to your backup location.
+
+To restore the assets, copy the `assets` folder back to the data folder of your new installation.
+
+### Favicons
 
 Doing a backup of the icons is optional, as they can be downloaded again.
 

docs/how-to.md

@@ -2,6 +2,13 @@
 
 Collection of tips and tricks around using linkding.
 
+**Table of Contents:**
+
+* [Using the bookmarklet on Android/Chrome](#using-the-bookmarklet-on-androidchrome)
+* [Using HTTP Shortcuts app on Android](#using-http-shortcuts-app-on-android)
+* [Create a share action on iOS for adding bookmarks to linkding](#create-a-share-action-on-ios-for-adding-bookmarks-to-linkding)
+* [Increase the font size](#increase-the-font-size)
+
 ## Using the bookmarklet on Android/Chrome
 
 This how-to explains the usage of the standard linkding bookmarklet on Android / Chrome. 
@@ -59,3 +66,27 @@ At the bottom of the share sheet there is a button for configuring share actions
 
 > [!NOTE]  
 > You can also check the [Community section](https://github.com/sissbruecker/linkding#community) for other pre-made shortcuts that you can use.
+
+## Increase the font size
+
+The font size can be adjusted globally by adding the following CSS to the custom CSS field in the settings:
+
+```css
+html {
+  --font-size: 0.75rem;
+  --font-size-sm: 0.7rem;
+  --font-size-lg: 0.9rem;
+}
+
+.bookmark-list {
+  line-height: 1.15rem;
+}
+
+.tag-cloud {
+  line-height: 1.15rem;
+}
+```
+
+You can adjust the `--font-size`, `--font-size-sm` and `--font-size-lg` variables to your liking.
+Note that increasing the font might also require you to adjust the line-height in certain places to better separate texts from each other.
+As an example, the above also increases the font size and line height of the bookmark list and tag cloud.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "linkding",
-  "version": "1.26.0",
+  "version": "1.28.0",
   "description": "",
   "main": "index.js",
   "scripts": {

requirements.in

@@ -5,8 +5,8 @@ Django
 django-registration
 django-sass-processor
 django-widget-tweaks
-django4-background-tasks
 djangorestframework
+huey
 Markdown
 mozilla-django-oidc
 psycopg2-binary

requirements.txt

@@ -39,10 +39,10 @@ django-sass-processor==1.4
     # via -r requirements.in
 django-widget-tweaks==1.5.0
     # via -r requirements.in
-django4-background-tasks==1.2.9
-    # via -r requirements.in
 djangorestframework==3.14.0
     # via -r requirements.in
+huey==2.5.0
+    # via -r requirements.in
 idna==3.6
     # via requests
 josepy==1.14.0
@@ -69,7 +69,6 @@ requests==2.31.0
 six==1.16.0
     # via
     #   bleach
-    #   django4-background-tasks
     #   python-dateutil
 soupsieve==2.5
     # via beautifulsoup4

scripts/build-docker.sh

@@ -2,14 +2,29 @@
 
 version=$(<version.txt)
 
-docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 \
+# Base image
+docker buildx build --target linkding --platform linux/amd64,linux/arm64,linux/arm/v7 \
   -f docker/default.Dockerfile \
   -t sissbruecker/linkding:latest \
   -t sissbruecker/linkding:$version \
   --push .
 
-docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 \
+docker buildx build --target linkding --platform linux/amd64,linux/arm64,linux/arm/v7 \
   -f docker/alpine.Dockerfile \
   -t sissbruecker/linkding:latest-alpine \
   -t sissbruecker/linkding:$version-alpine \
   --push .
+
+# Plus image with support for single-file snapshots
+# Needs checking if this works with ARMv7, excluded for now
+docker buildx build --target linkding-plus --platform linux/amd64,linux/arm64 \
+  -f docker/default.Dockerfile \
+  -t sissbruecker/linkding:latest-plus \
+  -t sissbruecker/linkding:$version-plus \
+  --push .
+
+docker buildx build --target linkding-plus --platform linux/amd64,linux/arm64 \
+  -f docker/alpine.Dockerfile \
+  -t sissbruecker/linkding:latest-plus-alpine \
+  -t sissbruecker/linkding:$version-plus-alpine \
+  --push .

siteroot/settings/base.py

@@ -42,7 +42,7 @@ INSTALLED_APPS = [
     "widget_tweaks",
     "rest_framework",
     "rest_framework.authtoken",
-    "background_task",
+    "huey.contrib.djhuey",
     "mozilla_django_oidc",
 ]
 
@@ -173,13 +173,27 @@ LD_DISABLE_BACKGROUND_TASKS = os.getenv("LD_DISABLE_BACKGROUND_TASKS", False) in
     "1",
 )
 
-# django-background-tasks
-MAX_ATTEMPTS = 5
-# How many tasks will run in parallel
-# We want to keep this low to prevent SQLite lock errors and in general not to consume too much resources on smaller
-# specced systems like Raspberries. Should be OK as tasks are not time critical.
-BACKGROUND_TASK_RUN_ASYNC = True
-BACKGROUND_TASK_ASYNC_THREADS = 2
+# Huey task queue
+HUEY = {
+    "huey_class": "huey.SqliteHuey",
+    "filename": os.path.join(BASE_DIR, "data", "tasks.sqlite3"),
+    "immediate": False,
+    "results": False,
+    "store_none": False,
+    "utc": True,
+    "consumer": {
+        "workers": 2,
+        "worker_type": "thread",
+        "initial_delay": 5,
+        "backoff": 1.15,
+        "max_delay": 10,
+        "scheduler_interval": 10,
+        "periodic": True,
+        "check_worker_health": True,
+        "health_check_interval": 10,
+    },
+}
+
 
 # Enable OICD support if configured
 LD_ENABLE_OIDC = os.getenv("LD_ENABLE_OIDC", False) in (True, "True", "1")
@@ -198,6 +212,7 @@ if LD_ENABLE_OIDC:
     OIDC_RP_CLIENT_SECRET = os.getenv("OIDC_RP_CLIENT_SECRET")
     OIDC_RP_SIGN_ALGO = os.getenv("OIDC_RP_SIGN_ALGO", "RS256")
     OIDC_USE_PKCE = os.getenv("OIDC_USE_PKCE", True) in (True, "True", "1")
+    OIDC_VERIFY_SSL = os.getenv("OIDC_VERIFY_SSL", True) in (True, "True", "1")
 
 # Enable authentication proxy support if configured
 LD_ENABLE_AUTH_PROXY = os.getenv("LD_ENABLE_AUTH_PROXY", False) in (True, "True", "1")
@@ -269,3 +284,21 @@ LD_ENABLE_REFRESH_FAVICONS = os.getenv("LD_ENABLE_REFRESH_FAVICONS", True) in (
     "True",
     "1",
 )
+
+# Asset / snapshot settings
+LD_ASSET_FOLDER = os.path.join(BASE_DIR, "data", "assets")
+
+LD_ENABLE_SNAPSHOTS = os.getenv("LD_ENABLE_SNAPSHOTS", False) in (
+    True,
+    "True",
+    "1",
+)
+LD_SINGLEFILE_PATH = os.getenv("LD_SINGLEFILE_PATH", "single-file")
+LD_SINGLEFILE_OPTIONS = os.getenv("LD_SINGLEFILE_OPTIONS", "")
+LD_SINGLEFILE_TIMEOUT_SEC = float(os.getenv("LD_SINGLEFILE_TIMEOUT_SEC", 60))
+
+# Monolith isn't used at the moment, as the local snapshot implementation
+# switched to single-file after the prototype. Keeping this around in case
+# it turns out to be useful in the future.
+LD_MONOLITH_PATH = os.getenv("LD_MONOLITH_PATH", "monolith")
+LD_MONOLITH_OPTIONS = os.getenv("LD_MONOLITH_OPTIONS", "-a -v -s")

siteroot/settings/dev.py

@@ -47,6 +47,11 @@ LOGGING = {
             "handlers": ["console"],
             "propagate": False,
         },
+        "huey": {  # Huey
+            "level": "INFO",
+            "handlers": ["console"],
+            "propagate": False,
+        },
     },
 }
 

siteroot/settings/prod.py

@@ -49,7 +49,12 @@ LOGGING = {
             "level": "INFO",
             "handlers": ["console"],
             "propagate": False,
-        }
+        },
+        "huey": {
+            "level": "INFO",
+            "handlers": ["console"],
+            "propagate": False,
+        },
     },
 }
 

supervisord.conf

@@ -4,8 +4,18 @@ loglevel=info
 
 [program:jobs]
 user=www-data
-command=sh background-tasks-wrapper.sh
+command=python manage.py run_huey -f
 stdout_logfile=background_tasks.log
 stdout_logfile_maxbytes=10MB
 stdout_logfile_backups=5
 redirect_stderr=true
+
+[unix_http_server]
+file=/var/run/supervisor.sock
+chmod=0700
+
+[rpcinterface:supervisor]
+supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
+
+[supervisorctl]
+serverurl=unix:///var/run/supervisor.sock

version.txt

@@ -1 +1 @@
-1.26.0
+1.28.0