Commit Graph

2498 Commits (perf/postgres-queue)

Author SHA1 Message Date
mertalev 7bcf9aa3a7
working `addJobs` 2025-04-30 21:54:44 -04:00
mertalev e773a7b7a1
use `addJobs` 2025-04-30 20:13:40 -04:00
mertalev f0c013f844
no queueing log 2025-04-30 18:21:35 -04:00
mertalev 2913a73456
no payload log 2025-04-30 18:18:52 -04:00
mertalev 262ef2a746
dummy jobs 2025-04-30 16:38:35 -04:00
Thomas Way 8c0c8a8d0e
feat: Use postgres as a queue
We've been keen to try this for a while as it means we can remove redis as a
dependency, which makes Immich easier to setup and run.

This replaces bullmq with a bespoke postgres queue. Jobs in the queue are
processed either immediately via triggers and notifications, or eventually if a
notification is missed.
2025-04-30 20:43:52 +01:00
Jason Rasmussen b845184c80
chore: remove old memory lane implementation (#18000) 2025-04-30 14:23:32 -04:00
Jason Rasmussen 1fde02ee1e
chore: remove unused types and code (#17999) 2025-04-30 13:41:23 -04:00
Jason Rasmussen 526c02297c
refactor: stream queue migration (#17997) 2025-04-30 16:23:13 +00:00
Alex 732b06eec8
refactor: stream for sidecar (#17995)
* refactor: stream for sidecar

* chore: make sql

---------

Co-authored-by: Jason Rasmussen <jason@rasm.me>
2025-04-30 10:53:51 -05:00
Jason Rasmussen be5cc2cdf5
refactor: stream detect faces (#17996) 2025-04-30 11:25:30 -04:00
Jason Rasmussen 094a41ac9a
chore: remove audit file report (#17994) 2025-04-30 11:17:23 -04:00
Jason Rasmussen 0c261ffbe2
fix: queue in batches (#17989) 2025-04-30 10:52:51 -04:00
Jason Rasmussen 8c5116bc1d
refactor: stream search duplicates (#17991) 2025-04-30 10:40:32 -04:00
Jason Rasmussen 2e8a286540
refactor: smart search queue (#17977) 2025-04-29 17:44:28 -04:00
Jason Rasmussen d89e88bb3f
feat: configure token endpoint auth method (#17968) 2025-04-29 15:17:48 -04:00
Thomas 3ce353393a
chore(server): don't insert embeddings if the model has changed (#17885)
* chore(server): don't insert embeddings if the model has changed

We're moving away from the heuristic of waiting for queues to complete. The job
which inserts embeddings can simply check if the model has changed before
inserting, rather than attempting to lock the queue.

* more robust dim size update

* use check constraint

* index command cleanup

* add create statement

* update medium test, create appropriate extension

* new line

* set dimension size when running on all assets

* why does it want braces smh

* take 2

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2025-04-29 14:23:01 -04:00
renovate[bot] 3d042cc7f1
fix(deps): update typescript-projects (#17961)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-29 13:00:37 +02:00
renovate[bot] fb52ac0f5b
chore(deps): update node.js to v22.15.0 (#17956)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-04-29 12:08:32 +02:00
Eli Gao 48bcbee6ed
feat(server): JXL previews from DNG 1.7+ (#17861)
* feat(server): JXL previews from RAW

* refactor(server): use var name assumedExtractedFormat for clarity

* test(server): fix existing media.extract() returning JPEG

* chore(openapi): regen

* style(server): lint

* fix(server): ignore undefined decode orientation

* fix(server): correct orientation assignment in media decode options

* test(server): unit tests of JXL-encoded DNG

* refactor(server): return buffer and format from mediaRepository.extract()

* chore(open-api): regen

* refactor

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2025-04-28 18:18:46 -04:00
Daniel Dietzler f621f8ef2c
refactor: more job queries (#17745) 2025-04-29 00:03:20 +02:00
Jason Rasmussen 1b5fc9c665
feat: notifications (#17701)
* feat: notifications

* UI works

* chore: pr feedback

* initial fetch and clear notification upon logging out

* fix: merge

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2025-04-28 10:36:14 -04:00
Jason Rasmussen ad272333db
refactor: user avatar color (#17753) 2025-04-28 08:54:51 -05:00
Zack Pollard 460d594791
feat: api response compression (#17878) 2025-04-28 08:54:11 -05:00
github-actions 02994883fe chore: version v1.132.3 2025-04-25 19:44:05 +00:00
github-actions 0d60be3d87 chore: version v1.132.2 2025-04-25 03:07:06 +00:00
Jason Rasmussen 1d610ad9cb
refactor: database connection parsing (#17852) 2025-04-24 12:58:29 -04:00
github-actions 37f5e6e2cb chore: version v1.132.1 2025-04-23 21:43:47 +00:00
Mert 6ce8a1deeb
fix(server): bump sharp (#17818)
* bump sharp

* test linking

* link in prod image too

* force global

* keep unnecessary libraries

* override sharp version

* revert dockerfile changes

* add node-gyp and napi

* dev dependency
2025-04-23 17:08:29 -04:00
github-actions f659ef4b7a chore: version v1.132.0 2025-04-23 16:44:47 +00:00
Zack Pollard d2f2f8d672
fix: retrieve version from lockfile and fallback to cli command (#17812) 2025-04-23 17:10:43 +01:00
Zack Pollard 19746a8685
fix: cache build versions (#17811) 2025-04-23 16:31:18 +01:00
Thomas 987e5ab76c
fix(server): start job workers after DB (#17806)
Job workers are currently started on app init, which means they are started
before the DB is initialised. This can be problematic if jobs which need to use
the DB start running before it's ready. It also means that swapping out the
queue implementation for something which uses the DB won't work.
2025-04-23 15:07:32 +00:00
Jason Rasmussen 1b5e981a45
fix: failing ci checks (#17810) 2025-04-23 10:59:54 -04:00
Tin Pecirep b7a0cf2470 feat: add oauth2 code verifier
* fix: ensure oauth state param matches before finishing oauth flow

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

* chore: upgrade openid-client to v6

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

* feat: use PKCE for oauth2 on supported clients

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

* feat: use state and PKCE in mobile app

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

* fix: remove obsolete oauth repository init

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

* fix: rewrite callback url if mobile redirect url is enabled

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

* fix: propagate oidc client error cause when oauth callback fails

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

* fix: adapt auth service tests to required state and PKCE params

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

* fix: update sdk types

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

* fix: adapt oauth e2e test to work with PKCE

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

* fix: allow insecure (http) oauth clients

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>

---------

Signed-off-by: Tin Pecirep <tin.pecirep@gmail.com>
Co-authored-by: Jason Rasmussen <jason@rasm.me>
2025-04-23 15:08:11 +01:00
renovate[bot] 0426b574fe
fix(deps): update typescript-projects (#17625)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Zack Pollard <zackpollard@ymail.com>
2025-04-23 11:45:38 +00:00
Mert 92ac1193e6
fix(server): queue android motion assets for transcoding (#17781) 2025-04-23 12:03:28 +01:00
Min Idzelis 2a95eccf6a
fix: vscode vitest ext - missing jsdom dev dependency (#17799) 2025-04-22 23:01:22 -04:00
renovate[bot] a8eec92da7
chore(deps): update dependency @types/node to ^22.14.1 (#17770)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
2025-04-22 10:18:44 +00:00
Bonne Eggleston fe8c5e8107
feat: add album start and end dates for storage template (#17188) 2025-04-21 19:54:33 -04:00
Jason Rasmussen 56a4aa9ffe
refactor: email repository (#17746) 2025-04-21 12:53:37 -04:00
Jason Rasmussen 488dc4efbd
refactor: notification-admin controller (#17748) 2025-04-21 10:49:26 -04:00
Daniel Dietzler dd1fcd5be5
chore: remove asset entity (#17703) 2025-04-18 21:39:56 +00:00
Daniel Dietzler 52ae06c119
refactor: remove album entity, update types (#17450) 2025-04-18 23:10:34 +02:00
Daniel Dietzler 854ea13d6a
chore: simplify asset getByIds (#17699) 2025-04-18 16:52:41 -04:00
Jason Rasmussen 160bb492a2
fix: skip initial kysely migration for existing installs (#17690)
* fix: skip initial kysely migration for existing installs

* Update docs/src/pages/errors.md

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2025-04-18 11:19:11 -05:00
Jason Rasmussen 6474a78b8b
feat: initial kysely migration file (#17678) 2025-04-17 17:38:47 -04:00
Jason Rasmussen e275f2d8b3
feat: add foreign key indexes (#17672) 2025-04-17 14:41:06 -04:00
Mert 242a559e0f
refactor: query for fetching faces and people of assets (#17661)
* use json instead of jsonb

* missing condition
2025-04-16 19:00:55 -04:00
Jonathan Jogenfors ed2b54527c
chore(server): don't check null dates (#17664) 2025-04-16 18:40:08 -04:00