Replies: 1 comment
|
Hi
|
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Summary
AsyncpgScheduleSource.startup()calls_update_schedules_on_startup(), which runs a fullDELETE FROM <table>and then re-inserts only the schedules extracted from task labels (extract_scheduled_tasks_from_broker()).This means any schedule added at runtime via
add_schedule()(e.g. a one-shottime=task) is permanently deleted on the nextstartup()of any process that holds this source — including thetaskiq schedulerCLI process on every restart/deploy.As a result, it seems impossible to use a single
AsyncpgScheduleSourcefor both:@broker.task(schedule=[...]), andadd_schedule().The whole point of a DB-backed source (vs
LabelScheduleSource) is persistence of dynamic schedules — butstartup()wipes them. I'd like to confirm whether this is intended, and what the recommended pattern is.Environment
taskiq-postgres(moduletaskiq_pg) == 0.7.0taskiq== 0.12.1asyncpg== 0.30.0Relevant source
In
taskiq_pg/asyncpg/schedule_source.py:DELETE_ALL_SCHEDULES_QUERY = "DELETE FROM {};"(full truncate).Minimal reproduction
Actual: after restart the dynamic schedule (
my_dynamic) is gone —startup()truncated the table and re-inserted only the label cron.Expected (or: desired): dynamically added schedules persist across restarts, while label crons are still synced. i.e. the DB source should not destroy rows it didn't create.
My real-world use case
A FastAPI app where:
source.add_schedule(ScheduledTask(time=order_expires_at, task_name="cancel_order_if_expired", ...))whenever a pending order is created, so the order is auto-cancelled at its expiry time.taskiq schedulerprocess runs the schedules and also serves a couple of static@broker.task(schedule=[...])crons.Every deploy of the scheduler (or the API) calls
startup()→ truncates the table → all pending order-cancellation schedules are lost.Questions
startup()intended?AsyncpgScheduleSourcefor persistent dynamic schedules alongside label-defined crons, without losing the dynamic ones on restart?startup()non-destructive — e.g. upsert label schedules by a stableschedule_idinstead ofDELETE FROM, or async_labels=Falseswitch? I'm happy to open a PR.Currently my only workaround is subclassing and overriding
startup()to create the pool + table without the truncate/label-sync, which works but bypasses the intended label-sync entirely.All reactions