Failure
- A signup request may create the account, send verification email, record analytics, and initialize data. Doing all of this before responding makes the user wait.
25:10 25:35
- Synchronous calls also couple signup reliability to an external email provider. If that provider is slow or down, signup becomes slow or fails.
25:35 26:07
Fix
- Complete only the work required for the immediate response, such as creating the account. Record deferrable work as a job in a queue, then respond to the user.
26:45 27:16
- A separate worker consumes jobs and performs slow tasks such as verification emails, password resets, notifications, and receipts.
27:16 27:47
- BullMQ is one queue option that runs on Redis, letting the same technology support sessions, caching, and background jobs.
28:13
sequenceDiagram
participant U as User
participant A as App server
participant Q as Queue
participant W as Worker
participant E as Email service
U->>A: Sign up
A->>A: Create account
A->>Q: Enqueue verification email
A-->>U: Signup complete
W->>Q: Take job
W->>E: Send email
Benefits
- Signup can return immediately and no longer fails merely because the email provider is temporarily unavailable. The queued job waits until the provider recovers.
28:42
- A short processing delay can be evidence of deliberate asynchronous design, as with delayed verification mail or a video marked "processing."
26:07 26:45
Costs and reliability rules
- Completion semantics change. "Done" may mean the system promised to do the work, not that the work finished.
29:11
- The queue and workers are additional infrastructure that must stay available. Jobs can fail, so workers need retries and a place to retain repeatedly failing jobs for human inspection.
29:11
- Enqueuing without monitoring completion can silently lose important work.
29:40