Skip to content

Fix: generate video thumbnails locally when server provides none#81

Draft
beezly wants to merge 7 commits intoviktorstrate:mainfrom
beezly:fix/video-thumbnails
Draft

Fix: generate video thumbnails locally when server provides none#81
beezly wants to merge 7 commits intoviktorstrate:mainfrom
beezly:fix/video-thumbnails

Conversation

@beezly
Copy link
Contributor

@beezly beezly commented Mar 12, 2026

Summary

Improve video thumbnail handling in the timeline:

  • Blurhash fallback: Decode the blurhash string from event metadata as a lightweight placeholder when no server thumbnail exists. No download required — uses only the metadata already in the event.
  • Local thumbnail generation: Optionally download the video and extract the first frame using AVAssetImageGenerator. Gated behind a "Generate video thumbnails" toggle in Settings → Appearance (off by default, since it downloads the full video).
  • Refactored video download: Shared downloadVideo() helper reused by both playback and thumbnail generation, avoiding duplicate downloads.
  • thumbnailView computed property: Replaces inline MatrixImageView with a priority chain: server thumbnail → generated thumbnail → blurhash → grey placeholder.

Thumbnail priority

Source Download needed? When shown
Server thumbnail (thumbnailSource) No (fetched by MatrixImageView) Always, if available
Generated thumbnail Yes (full video) When enabled in settings
Blurhash No (event metadata) When no thumbnail and generation is off
Grey placeholder No Last resort

Depends on

@beezly beezly force-pushed the fix/video-thumbnails branch from 31947fb to d1d8e17 Compare March 15, 2026 12:20
@beezly beezly marked this pull request as ready for review March 15, 2026 12:28
Copy link
Owner

@viktorstrate viktorstrate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR, I've left some comments.

I'm wondering how big a fraction of homeservers generate thumbnails. Is it common for videos to not have thumbnails?

A complete sidenote: It would be nice to support blurhashes, but I guess most videos without thumbnails probably also lack a bluhash since they would likely be computed based on a thumbnail to begin with.

@beezly
Copy link
Contributor Author

beezly commented Mar 16, 2026

A complete sidenote: It would be nice to support blurhashes, but I guess most videos without thumbnails probably also lack a bluhash since they would likely be computed based on a thumbnail to begin with.

I've implemented blurhash in my "attachments" branch, so we could definitely use it.

@beezly
Copy link
Contributor Author

beezly commented Mar 16, 2026

I'm wondering how big a fraction of homeservers generate thumbnails. Is it common for videos to not have thumbnails?

When I was testing, I didn't have video thumbnails (that's why I ended up implementing it because they would all show as nil).

I think it should be possible to reuse some of this when it comes to handling attachments anyway, so it will probably be needed in the future for that.

beezly added 4 commits March 17, 2026 11:05
Prevent division by zero when image metadata reports zero or missing
dimensions. Without this guard, aspectRatio computes infinity or NaN,
which causes SwiftUI layout issues inside NSTableView cells.
…llback

- Guard against zero width/height in aspect ratio calculation (same as
  image fix).
- Reorder modifiers: .aspectRatio() before .frame(maxHeight:) so the
  aspect ratio is applied before the height constraint, matching the
  intended sizing behaviour.
- Add minHeight when thumbnailSource is nil. Without a thumbnail,
  MatrixImageView renders a Rectangle with no intrinsic size, causing
  .aspectRatio(.fit) to collapse the video to zero height.
Replace the conditional if/else if blocks for reactions and read
receipts with an always-present HStack that uses frame(height: 0)
and .clipped() when empty.

Conditional view branches (if !reactions.isEmpty / else if) change
the SwiftUI view tree structure when data loads asynchronously. Inside
an NSHostingView table cell, this triggers invalidateIntrinsicContentSize
during layout, contributing to an infinite constraint update loop that
crashes the app. Keeping the view tree stable avoids this trigger.
NSHostingView inside NSTableView with usesAutomaticRowHeights enters an
infinite layout loop: layout() flushes SwiftUI transactions, which
invalidate constraints, which re-enter layout. AppKit detects this and
crashes with NSGenericException.

Replace the entire auto-sizing mechanism with SelfSizingHostingView:

- sizingOptions = [] prevents the hosting view from participating in
  Auto Layout constraint solving.

- invalidateIntrinsicContentSize() is overridden to NOT call super
  (which would re-enter the constraint system) but instead schedule
  a coalesced async height update via DispatchQueue.main.async.

- measureHeight() uses a temporary NSHostingController.sizeThatFits()
  to properly measure content at the cell width, including text wrapping.

- heightOfRow returns cached heights (default 60px). noteHeightOfRows
  is called with animation duration 0 to avoid visible row resizing.

Also removes the old measurementHostingView and handleTableResize.
@beezly beezly marked this pull request as draft March 17, 2026 11:21
@beezly
Copy link
Contributor Author

beezly commented Mar 17, 2026

Paused whilst I fixed #84 and #85 as they were making my testing impossible!

@beezly beezly force-pushed the fix/video-thumbnails branch from d1d8e17 to 060c697 Compare March 17, 2026 11:53
… toggle

- Generate thumbnails using AVAssetImageGenerator when the server
  provides none and the user enables 'Generate video thumbnails' in
  settings (off by default, downloads the video)
- Decode blurhash from event metadata as a lightweight placeholder
  when no thumbnail or generated image is available (no download needed)
- Refactor video download into shared downloadVideo() helper, reused
  by both playback and thumbnail generation
- Add BlurHashDecoder.swift (MIT, based on woltapp/blurhash)
- Add settings toggle in Appearance preferences
@beezly beezly force-pushed the fix/video-thumbnails branch from 060c697 to 9f46a1d Compare March 17, 2026 12:00
@beezly
Copy link
Contributor Author

beezly commented Mar 17, 2026

I've added the "Generate video thumbnails" setting, with a help item explaining that it will download videos to do it. Also added the blurhash decode implementation from https://github.com/woltapp/blurhash - It's MIT licensed, so it should be fine to include so long as we keep the copyright notice.

Also implemented the other suggestion about sharing loadVideo.

@beezly
Copy link
Contributor Author

beezly commented Mar 17, 2026

I'll take this out of draft once 84/85 go in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants