Compare commits

..

39 Commits

Author SHA1 Message Date
Bas
aa5a03b5b0 Merge pull request #57 from xtr-dev/dev
Fix TypeScript build error in jobScheduler.ts
2025-09-20 19:04:46 +02:00
8ee3ff5a7d Fix TypeScript build error in jobScheduler.ts
- Use static values for task and queue in logging instead of accessing job properties
- Properties 'task' and 'queue' don't exist on BaseJob type

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-20 19:02:38 +02:00
Bas
2220d83288 Merge pull request #56 from xtr-dev/dev
Dev
2025-09-20 19:01:01 +02:00
2f46dde532 Add configurable logger with PAYLOAD_MAILING_LOG_LEVEL support
- Created centralized logger utility using Payload's built-in logger system
- Added PAYLOAD_MAILING_LOG_LEVEL environment variable for log level configuration
- Replaced all console.log/error/warn calls with structured logger
- Added debug logging for immediate processing flow to help troubleshoot issues
- Improved logging context with specific prefixes (IMMEDIATE, PROCESSOR, JOB_SCHEDULER, etc.)
- Bumped version to 0.4.10

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-20 18:57:18 +02:00
02a9334bf4 Add npm version badge to README for improved visibility 2025-09-19 09:59:28 +02:00
Bas
de1ae636de Merge pull request #55 from xtr-dev/dev
Dev
2025-09-14 23:36:58 +02:00
ae38653466 Expand and clarify README documentation:
- Provide detailed examples of email template structure and rendering.
- Add guidance on job scheduling and direct email sending use cases.
- Enhance troubleshooting section with common issues and solutions.
- Introduce bulk operations, email monitoring, and query examples.
- Update plugin configuration requirements and clarify environment variables.

Improves overall usability and onboarding for developers.
2025-09-14 23:33:14 +02:00
fe8c4d194e Bump version to 0.4.9 and add comprehensive plugin configuration details to README. 2025-09-14 23:30:14 +02:00
0198821ff3 Update README for improved clarity and reduced redundancy. 2025-09-14 23:22:46 +02:00
Bas
5e0ed0a03a Merge pull request #52 from xtr-dev/dev
Dev
2025-09-14 22:05:08 +02:00
d661d2e13e Fix critical race conditions and error handling inconsistencies
Race Condition Fixes (jobScheduler.ts):
- Implement optimistic job creation with graceful fallback
- Minimize race condition window by trying create first, then check
- Add enhanced error detection for constraint violations
- Provide detailed error context for debugging data consistency issues

Error Handling Improvements (sendEmail.ts):
- Distinguish between POLLING_TIMEOUT vs JOB_NOT_FOUND errors
- Add specific error types for programmatic handling
- Provide actionable troubleshooting steps in error messages
- Include recovery instructions (processEmailById fallback)

Benefits:
- Eliminates the check-then-create race condition vulnerability
- Provides clear error classification for different failure modes
- Enables better monitoring and debugging of job scheduling issues
- Maintains robustness under high concurrency scenarios

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-14 21:57:52 +02:00
e4a16094d6 Eliminate code duplication in email sanitization
- Create centralized sanitization utilities in utils/helpers.ts
- Add sanitizeDisplayName() with configurable quote escaping
- Add sanitizeFromName() wrapper for consistent fromName handling
- Replace duplicated sanitization logic in sendEmail.ts (9 lines → 1 line)
- Replace duplicated sanitization logic in MailingService.ts (9 lines → 1 line)
- Export new utilities from main index for external use
- Maintain identical functionality while reducing maintenance overhead

Benefits:
- Single source of truth for email header sanitization
- Consistent security handling across all email components
- Easier to maintain and update sanitization logic
- Configurable quote escaping for different use cases

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-14 21:52:55 +02:00
8135ff61c2 Optimize polling performance and reduce memory usage
- Reduce polling attempts from 10 to 5 with 3-second timeout protection
- Optimize exponential backoff delays (25ms-400ms vs 50ms-2000ms)
- Remove memory-intensive unique keys from job creation
- Reduce ensureEmailJob retry attempts from 5 to 3
- Use gentler exponential backoff (1.5x vs 2x) capped at 200ms
- Rely on database constraints for duplicate prevention instead of memory keys

Performance improvements:
- Faster response times for immediate email sending
- Reduced memory bloat in job queue systems
- Better resource efficiency for high-volume scenarios

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-14 21:41:29 +02:00
e28ee6b358 Fix critical race conditions and performance issues
- Implement atomic check-and-create pattern in ensureEmailJob with exponential backoff
- Fix import mismatch by exporting processJobById from index.ts
- Enable database indexes for status+scheduledAt and priority+createdAt fields
- Standardize string conversion for consistent ID handling throughout codebase
- Fix TypeScript compilation errors in collection indexes and variable scope

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-14 21:35:27 +02:00
4680f3303e Fix race condition with robust exponential backoff polling
🛡️ Race Condition Fix:
- Replaced unreliable fixed timeout with exponential backoff polling
- Polls up to 10 times for job creation
- Delays: 50ms, 100ms, 200ms, 400ms, 800ms, 1600ms, 2000ms (capped)
- Total max wait time: ~7 seconds under extreme load

🎯 Benefits:
- Fast response under normal conditions (usually first attempt)
- Graceful degradation under heavy load
- Proper error messages after timeout
- Debug logging for troubleshooting (after 3rd attempt)
- No race conditions even under extreme concurrency

📊 Performance:
- Normal case: 0-50ms wait (immediate success)
- Under load: Progressive backoff prevents overwhelming
- Worst case: Clear timeout with actionable error message
- Total attempts: 10 (configurable if needed)

🔍 How it works:
1. Create email and trigger hooks
2. Poll for job with exponential backoff
3. Exit early on success (usually first check)
4. Log attempts for debugging if delayed
5. Clear error if job never appears
2025-09-14 21:23:03 +02:00
efc734689b Bump version to 0.4.8 2025-09-14 21:20:16 +02:00
95ab07d72b Simplify hook logic and improve concurrent update handling
🎯 Simplifications:
- Removed complex beforeChange hook - all logic now in afterChange
- Single clear decision point with 'shouldSkip' variable
- Document ID always available in afterChange
- Clearer comments explaining the logic flow

🛡️ Concurrent Update Protection:
- ensureEmailJob now handles race conditions properly
- Double-checks for jobs after creation failure
- Idempotent function safe for concurrent calls
- Better error handling and recovery

📊 Benefits:
- Much simpler hook logic (from ~70 lines to ~40 lines)
- Single source of truth (afterChange only)
- No complex hook interactions
- Clear skip conditions
- Concurrent update safety
- Better code readability

🔍 How it works:
1. Check skip conditions (not pending, has jobs, etc.)
2. Call ensureEmailJob (handles all complexity)
3. Update relationship if needed
4. Log errors but don't fail operations
2025-09-14 21:18:51 +02:00
640ea0818d Extract job scheduling logic into dedicated utility functions
♻️ Refactoring:
- Created new jobScheduler.ts utility module
- Extracted findExistingJobs() for duplicate detection
- Extracted ensureEmailJob() for job creation with duplicate prevention
- Extracted updateEmailJobRelationship() for relationship management

📦 Functions:
- findExistingJobs(): Queries for existing processing jobs by email ID
- ensureEmailJob(): Creates job only if none exists, returns job IDs
- updateEmailJobRelationship(): Updates email with job relationship

🎯 Benefits:
- Reusable functions for job management
- Single source of truth for job scheduling logic
- Cleaner, more testable code
- Exported utilities for external use
- Better separation of concerns

🔧 Updated:
- Emails collection hooks now use extracted functions
- Exports added to main index for public API
- Cleaner hook implementation with less duplication
2025-09-14 21:14:02 +02:00
6f3d0f56c5 Bump version to 0.4.7 2025-09-14 21:07:38 +02:00
4e96fbcd20 Simplify sendEmail to rely on hooks for job creation
🔄 Cleaner Architecture:
- sendEmail now just creates the email and lets hooks handle job creation
- Hooks automatically create and populate job relationship
- For processImmediately, retrieves job from relationship and runs it
- Removes duplicate job creation logic from sendEmail

📈 Benefits:
- Single source of truth for job creation (hooks)
- Consistent behavior across all email creation methods
- Simpler, more maintainable code
- Better separation of concerns

🔍 Flow:
1. sendEmail creates email document
2. Hooks auto-create job and populate relationship
3. If processImmediately, fetch job from relationship and run it
4. Return email with complete job relationship
2025-09-14 21:06:47 +02:00
2d270ca527 Improve job scheduling hooks to populate relationship immediately
 Enhanced Job Relationship Management:
- Use beforeChange to populate existing jobs in relationship field
- Use afterChange to create new jobs and add them to relationship
- Jobs now appear immediately in the relationship field
- Better handling of updates vs new document creation

🔄 Hook Flow:
1. beforeChange: Find existing jobs for updates and populate relationship
2. afterChange: Create missing jobs and update relationship field
3. Result: Jobs relationship is always populated correctly

📈 Benefits:
- Immediate job visibility in admin interface
- No reliance on dynamic filtering alone
- Proper relationship data in database
- Handles both new emails and status changes
- Prevents duplicate job creation
2025-09-14 21:03:01 +02:00
9a996a33e5 Add afterChange hook to auto-schedule jobs for pending emails
 Smart Job Scheduling:
- Automatically creates processing jobs for pending emails
- Prevents orphaned emails that bypass sendEmail() function
- Checks for existing jobs to avoid duplicates
- Respects scheduledAt for delayed sending
- Handles both create and update operations intelligently

🔍 Logic:
- Only triggers for emails with status 'pending'
- Skips if email was already pending (prevents duplicate jobs)
- Queries existing jobs to avoid creating duplicates
- Uses mailing config queue or defaults to 'default'
- Graceful error handling (logs but doesn't fail email operations)

📈 Benefits:
- Complete email processing coverage
- Works for emails created via admin interface
- Handles manual status changes back to pending
- Maintains scheduling for delayed emails
- Zero-configuration auto-recovery
2025-09-14 21:01:35 +02:00
Bas
060b1914b6 Merge pull request #46 from xtr-dev/dev
Dev
2025-09-14 20:45:26 +02:00
70fb79cca4 Add has-many relationship from emails to processing jobs
 New Feature:
- Add 'jobs' relationship field to emails collection
- Shows all PayloadCMS jobs associated with each email
- Read-only field with smart filtering by emailId
- Visible in admin interface for better email tracking

🔍 Benefits:
- Track job status and history for each email
- Debug processing issues more easily
- Monitor job queue performance per email
- Complete email processing visibility
2025-09-14 20:41:19 +02:00
f5e04d33ba Simplify error handling in processEmailJob
Remove unnecessary instanceof check since String() handles all types consistently.
2025-09-14 20:38:47 +02:00
27d504079a Fix critical error handling and race condition issues
🔴 Critical fixes:
- Fix race condition: processImmediately now properly fails if job creation fails
- Fix silent job failures: job creation failures now throw errors instead of warnings
- Ensure atomic operations: either email + job succeed together, or both fail

⚠️ Improvements:
- Simplify error handling in processEmailJob to be more consistent
- Add proper validation for missing PayloadCMS jobs configuration
- Make error messages more descriptive and actionable
2025-09-14 20:32:23 +02:00
b6ec55bc45 Bump version to 0.4.6 2025-09-14 20:26:59 +02:00
dcce3324ce Remove redundant blank lines in plugin initialization logic 2025-09-14 20:26:53 +02:00
f1f55d4444 Remove onReady callback from plugin
The onReady callback is no longer needed since the plugin no longer
schedules initial processing jobs during initialization.
2025-09-14 20:25:59 +02:00
b8950932f3 Remove unnecessary initial email processing job scheduling
Since sendEmail() now automatically creates individual jobs for each email,
the plugin no longer needs to schedule an initial batch processing job.
2025-09-14 20:24:45 +02:00
caa3686f1a Flatten email processing structure to individual jobs per email
BREAKING CHANGE: Replaced batch email processing with individual jobs per email

Changes:
- Remove sendEmailTask.ts - no longer needed as each email gets its own job
- Add processEmailJob.ts - handles individual email processing
- Update sendEmail() to automatically create individual job per email
- Add processImmediately option to sendEmail() for instant processing
- Add processJobById() utility to run specific jobs immediately
- Update job registration to use new individual job structure
- Update dev API routes to use new processImmediately pattern
- Fix all TypeScript compilation errors

Benefits:
- Better job queue visibility (one job per email)
- More granular control over individual email processing
- Easier job monitoring and failure tracking
- Maintains backward compatibility via processImmediately option
- Simpler job queue management

Migration:
- Replace sendEmailJob usage with sendEmail({ processImmediately: true })
- Individual emails now appear as separate jobs in queue
- Batch processing still available via processEmailsTask if needed

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-14 20:19:07 +02:00
Bas
d82b3f2276 Merge pull request #45 from xtr-dev/dev
Fix TypeScript compilation error in MailingService
2025-09-14 20:03:42 +02:00
08f017abed Bump version to 0.4.5 2025-09-14 20:03:34 +02:00
af9c5a1e1b Fix TypeScript compilation error in MailingService
- Replace unsafe BaseEmail type cast with proper type handling
- Fix error TS2352: Conversion of type 'JsonObject & TypeWithID' to type 'BaseEmail'
- Use targeted (email as any).attempts instead of full object cast
- Maintains functionality while resolving type safety issues

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-14 20:02:08 +02:00
Bas
05f4cd0d7c Merge pull request #44 from xtr-dev/dev
Dev
2025-09-14 20:00:46 +02:00
22190f38fd Fix critical type safety and validation issues
- Replace unsafe (payload as any).mailing with proper type checking
- Add validation for required fields (to, templateSlug/subject+html)
- Return proper 400 status codes for invalid requests
- Improve type safety without breaking existing functionality

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-14 19:56:19 +02:00
1ba770942d Bump version to 0.4.4 2025-09-14 19:53:53 +02:00
7f73fa5efc Add SQLite database files to gitignore
- Remove dev.db and dev/dev.db from git tracking
- Update .gitignore to exclude SQLite development databases
- Prevent local database files from being committed

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-14 19:49:28 +02:00
8993d20526 Migrate dev server from MongoDB to SQLite
- Replace mongooseAdapter with sqliteAdapter in payload config
- Update database configuration to use file:./dev.db
- Remove MongoDB memory database helper and references
- Simplify start script by removing verbose logging and MongoDB messaging
- Fix email processing with immediate send support and proper queue handling
- Restructure app with route groups for frontend/admin separation
- Add dashboard and test pages for email management
- Update API routes for improved email processing and testing

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-14 19:48:45 +02:00
29 changed files with 1506 additions and 1502 deletions

2
.gitignore vendored
View File

@@ -4,4 +4,6 @@ node_modules/
payload-docs payload-docs
dist/ dist/
/dev/payload.db /dev/payload.db
/dev/dev.db
dev.db
tsconfig.tsbuildinfo tsconfig.tsbuildinfo

1052
README.md

File diff suppressed because it is too large Load Diff

2
dev/.env.local Normal file
View File

@@ -0,0 +1,2 @@
USE_MEMORY_DB=true
PAYLOAD_SECRET=YOUR_SECRET_HERE

View File

@@ -0,0 +1,310 @@
'use client'
import { useState, useEffect } from 'react'
import Link from 'next/link'
interface EmailStats {
total: number
sent: number
pending: number
failed: number
processing: number
}
export default function HomePage() {
const [emailStats, setEmailStats] = useState<EmailStats>({
total: 0,
sent: 0,
pending: 0,
failed: 0,
processing: 0
})
const [loading, setLoading] = useState<boolean>(true)
useEffect(() => {
fetchEmailStats()
}, [])
const fetchEmailStats = async () => {
try {
const response = await fetch('/api/test-email')
const data = await response.json()
if (data.outbox?.emails) {
const emails = data.outbox.emails
const stats: EmailStats = {
total: emails.length,
sent: emails.filter((email: any) => email.status === 'sent').length,
pending: emails.filter((email: any) => email.status === 'pending').length,
failed: emails.filter((email: any) => email.status === 'failed').length,
processing: emails.filter((email: any) => email.status === 'processing').length
}
setEmailStats(stats)
}
} catch (error) {
console.error('Error fetching email statistics:', error)
} finally {
setLoading(false)
}
}
const StatCard = ({ label, value, color, description }: { label: string; value: number; color: string; description: string }) => (
<div style={{
backgroundColor: 'white',
border: '1px solid #e5e7eb',
borderRadius: '12px',
padding: '24px',
textAlign: 'center',
boxShadow: '0 1px 3px rgba(0, 0, 0, 0.1)',
}}>
<div style={{
fontSize: '3rem',
fontWeight: 'bold',
color: color,
marginBottom: '8px'
}}>
{value}
</div>
<div style={{
fontSize: '1.1rem',
fontWeight: '600',
color: '#374151',
marginBottom: '4px'
}}>
{label}
</div>
<div style={{
fontSize: '0.875rem',
color: '#6b7280'
}}>
{description}
</div>
</div>
)
return (
<div style={{
backgroundColor: '#f9fafb',
padding: '40px 20px',
minHeight: 'calc(100vh - 80px)'
}}>
<div style={{
maxWidth: '1200px',
margin: '0 auto'
}}>
{/* Header */}
<div style={{ textAlign: 'center', marginBottom: '48px' }}>
<h1 style={{
fontSize: '3rem',
fontWeight: 'bold',
color: '#1f2937',
marginBottom: '16px'
}}>
📧 PayloadCMS Mailing Plugin
</h1>
<p style={{
fontSize: '1.25rem',
color: '#6b7280',
marginBottom: '24px'
}}>
Development Dashboard
</p>
<div style={{ display: 'flex', gap: '16px', justifyContent: 'center', flexWrap: 'wrap' }}>
<Link
href="/admin"
style={{
backgroundColor: '#3b82f6',
color: 'white',
padding: '12px 24px',
borderRadius: '8px',
textDecoration: 'none',
fontWeight: '500',
transition: 'background-color 0.2s'
}}
>
📊 Admin Panel
</Link>
<Link
href="/mailing-test"
style={{
backgroundColor: '#10b981',
color: 'white',
padding: '12px 24px',
borderRadius: '8px',
textDecoration: 'none',
fontWeight: '500',
transition: 'background-color 0.2s'
}}
>
🧪 Test Interface
</Link>
</div>
</div>
{/* Email Statistics */}
<div style={{ marginBottom: '48px' }}>
<div style={{
display: 'flex',
justifyContent: 'space-between',
alignItems: 'center',
marginBottom: '24px'
}}>
<h2 style={{
fontSize: '2rem',
fontWeight: 'bold',
color: '#1f2937'
}}>
Email Statistics
</h2>
<button
onClick={fetchEmailStats}
disabled={loading}
style={{
backgroundColor: loading ? '#9ca3af' : '#6b7280',
color: 'white',
padding: '8px 16px',
borderRadius: '6px',
border: 'none',
cursor: loading ? 'not-allowed' : 'pointer',
fontWeight: '500'
}}
>
{loading ? 'Loading...' : 'Refresh'}
</button>
</div>
{loading ? (
<div style={{ textAlign: 'center', padding: '48px' }}>
<div style={{ color: '#6b7280', fontSize: '1.1rem' }}>Loading email statistics...</div>
</div>
) : (
<div style={{
display: 'grid',
gridTemplateColumns: 'repeat(auto-fit, minmax(250px, 1fr))',
gap: '24px'
}}>
<StatCard
label="Total Emails"
value={emailStats.total}
color="#1f2937"
description="All emails in the system"
/>
<StatCard
label="Successfully Sent"
value={emailStats.sent}
color="#10b981"
description="Delivered successfully"
/>
<StatCard
label="Pending"
value={emailStats.pending}
color="#f59e0b"
description="Waiting to be sent"
/>
<StatCard
label="Failed"
value={emailStats.failed}
color="#ef4444"
description="Failed to send"
/>
{emailStats.processing > 0 && (
<StatCard
label="Processing"
value={emailStats.processing}
color="#3b82f6"
description="Currently being sent"
/>
)}
</div>
)}
</div>
{/* Quick Actions */}
<div style={{
backgroundColor: 'white',
borderRadius: '12px',
padding: '32px',
border: '1px solid #e5e7eb',
boxShadow: '0 1px 3px rgba(0, 0, 0, 0.1)'
}}>
<h3 style={{
fontSize: '1.5rem',
fontWeight: 'bold',
color: '#1f2937',
marginBottom: '16px'
}}>
Quick Actions
</h3>
<div style={{
display: 'grid',
gridTemplateColumns: 'repeat(auto-fit, minmax(300px, 1fr))',
gap: '16px'
}}>
<div style={{ padding: '16px', backgroundColor: '#f9fafb', borderRadius: '8px' }}>
<h4 style={{ marginBottom: '8px', color: '#1f2937' }}>🎯 Test Email Sending</h4>
<p style={{ color: '#6b7280', marginBottom: '12px', fontSize: '0.9rem' }}>
Send test emails using templates with the interactive testing interface.
</p>
<Link
href="/mailing-test"
style={{
color: '#3b82f6',
textDecoration: 'none',
fontWeight: '500'
}}
>
Open Test Interface
</Link>
</div>
<div style={{ padding: '16px', backgroundColor: '#f9fafb', borderRadius: '8px' }}>
<h4 style={{ marginBottom: '8px', color: '#1f2937' }}>📝 Manage Templates</h4>
<p style={{ color: '#6b7280', marginBottom: '12px', fontSize: '0.9rem' }}>
Create and edit email templates in the Payload admin interface.
</p>
<Link
href="/admin/collections/email-templates"
style={{
color: '#3b82f6',
textDecoration: 'none',
fontWeight: '500'
}}
>
Manage Templates
</Link>
</div>
<div style={{ padding: '16px', backgroundColor: '#f9fafb', borderRadius: '8px' }}>
<h4 style={{ marginBottom: '8px', color: '#1f2937' }}>📬 Email Queue</h4>
<p style={{ color: '#6b7280', marginBottom: '12px', fontSize: '0.9rem' }}>
View and manage the email outbox and delivery status.
</p>
<Link
href="/admin/collections/emails"
style={{
color: '#3b82f6',
textDecoration: 'none',
fontWeight: '500'
}}
>
View Email Queue
</Link>
</div>
</div>
</div>
{/* Footer */}
<div style={{
textAlign: 'center',
marginTop: '48px',
padding: '24px',
color: '#6b7280',
fontSize: '0.875rem'
}}>
PayloadCMS Mailing Plugin Development Environment
</div>
</div>
</div>
)
}

View File

@@ -0,0 +1,24 @@
import type { Metadata } from 'next'
export const metadata: Metadata = {
title: 'PayloadCMS Mailing Plugin - Development',
description: 'Development environment for PayloadCMS Mailing Plugin',
}
export default function FrontendLayout({
children,
}: {
children: React.ReactNode
}) {
return (
<html lang="en">
<body style={{
margin: 0,
padding: 0,
fontFamily: '-apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif'
}}>
{children}
</body>
</html>
)
}

View File

@@ -33,6 +33,8 @@ export default function MailingTestPage() {
const [selectedTemplate, setSelectedTemplate] = useState<string>('') const [selectedTemplate, setSelectedTemplate] = useState<string>('')
const [toEmail, setToEmail] = useState<string>('test@example.com') const [toEmail, setToEmail] = useState<string>('test@example.com')
const [variables, setVariables] = useState<Record<string, any>>({}) const [variables, setVariables] = useState<Record<string, any>>({})
const [jsonVariables, setJsonVariables] = useState<string>('{}')
const [jsonError, setJsonError] = useState<string>('')
const [emailType, setEmailType] = useState<'send' | 'schedule'>('send') const [emailType, setEmailType] = useState<'send' | 'schedule'>('send')
const [scheduleDate, setScheduleDate] = useState<string>('') const [scheduleDate, setScheduleDate] = useState<string>('')
const [loading, setLoading] = useState<boolean>(false) const [loading, setLoading] = useState<boolean>(false)
@@ -58,6 +60,23 @@ export default function MailingTestPage() {
const template = templates.find(t => t.slug === templateSlug) const template = templates.find(t => t.slug === templateSlug)
if (template?.previewData) { if (template?.previewData) {
setVariables(template.previewData) setVariables(template.previewData)
setJsonVariables(JSON.stringify(template.previewData, null, 2))
} else {
setVariables({})
setJsonVariables('{}')
}
setJsonError('')
}
const handleJsonVariablesChange = (jsonString: string) => {
setJsonVariables(jsonString)
setJsonError('')
try {
const parsed = JSON.parse(jsonString)
setVariables(parsed)
} catch (error) {
setJsonError(error instanceof Error ? error.message : 'Invalid JSON')
} }
} }
@@ -67,6 +86,11 @@ export default function MailingTestPage() {
return return
} }
if (jsonError) {
setMessage('Please fix the JSON syntax error before sending')
return
}
setLoading(true) setLoading(true)
setMessage('') setMessage('')
@@ -88,7 +112,8 @@ export default function MailingTestPage() {
const result = await response.json() const result = await response.json()
if (result.success) { if (result.success) {
setMessage(`${result.message} (ID: ${result.emailId})`) const statusIcon = result.status === 'sent' ? '📧' : '📫'
setMessage(`${statusIcon} ${result.message} (ID: ${result.emailId})`)
fetchData() // Refresh email queue fetchData() // Refresh email queue
} else { } else {
setMessage(`❌ Error: ${result.error}`) setMessage(`❌ Error: ${result.error}`)
@@ -204,28 +229,43 @@ export default function MailingTestPage() {
</div> </div>
)} )}
{selectedTemplateData?.variables && ( {selectedTemplate && (
<div style={{ marginBottom: '15px' }}> <div style={{ marginBottom: '15px' }}>
<h3>Template Variables:</h3>
{selectedTemplateData.variables.map(variable => (
<div key={variable.name} style={{ marginBottom: '10px' }}>
<label style={{ display: 'block', marginBottom: '5px' }}> <label style={{ display: 'block', marginBottom: '5px' }}>
{variable.name} {variable.required && <span style={{ color: 'red' }}>*</span>} <strong>Template Variables (JSON):</strong>
{variable.description && <small style={{ color: '#666' }}> - {variable.description}</small>} {selectedTemplateData?.variables && (
<small style={{ color: '#666', marginLeft: '8px' }}>
Available variables: {selectedTemplateData.variables.map(v => v.name).join(', ')}
</small>
)}
</label> </label>
<input <textarea
type={variable.type === 'number' ? 'number' : variable.type === 'date' ? 'datetime-local' : 'text'} value={jsonVariables}
value={variables[variable.name] || ''} onChange={(e) => handleJsonVariablesChange(e.target.value)}
onChange={(e) => setVariables({ placeholder='{\n "firstName": "John",\n "siteName": "MyApp",\n "createdAt": "2023-01-01T00:00:00Z"\n}'
...variables, style={{
[variable.name]: variable.type === 'number' ? Number(e.target.value) : width: '100%',
variable.type === 'boolean' ? e.target.checked : height: '150px',
e.target.value padding: '8px',
})} borderRadius: '4px',
style={{ width: '100%', padding: '8px', borderRadius: '4px', border: '1px solid #ddd' }} border: jsonError ? '2px solid #dc3545' : '1px solid #ddd',
fontFamily: 'monaco, "Courier New", monospace',
fontSize: '13px',
resize: 'vertical'
}}
/> />
{jsonError && (
<div style={{
color: '#dc3545',
fontSize: '12px',
marginTop: '5px',
padding: '5px',
backgroundColor: '#f8d7da',
borderRadius: '4px'
}}>
Invalid JSON: {jsonError}
</div> </div>
))} )}
</div> </div>
)} )}

View File

@@ -5,16 +5,17 @@ export async function POST(request: Request) {
try { try {
const payload = await getPayload({ config }) const payload = await getPayload({ config })
// Queue the combined email queue processing job // Run jobs in the default queue (the plugin already schedules email processing on init)
const job = await payload.jobs.queue({ const results = await payload.jobs.run({
task: 'process-email-queue', queue: 'default',
input: {},
}) })
const processedCount = Array.isArray(results) ? results.length : (results ? 1 : 0)
return Response.json({ return Response.json({
success: true, success: true,
message: 'Email queue processing job queued successfully (will process both pending and failed emails)', message: `Email queue processing completed. Processed ${processedCount} jobs.`,
jobId: job.id, processedJobs: processedCount,
}) })
} catch (error) { } catch (error) {
console.error('Process emails error:', error) console.error('Process emails error:', error)

View File

@@ -8,6 +8,22 @@ export async function POST(request: Request) {
const body = await request.json() const body = await request.json()
const { type = 'send', templateSlug, to, variables, scheduledAt, subject, html, text } = body const { type = 'send', templateSlug, to, variables, scheduledAt, subject, html, text } = body
// Validate required fields
if (!to) {
return Response.json(
{ error: 'Recipient email address (to) is required' },
{ status: 400 }
)
}
// Validate email has either template or direct content
if (!templateSlug && (!subject || !html)) {
return Response.json(
{ error: 'Either templateSlug or both subject and html must be provided' },
{ status: 400 }
)
}
// Use the new sendEmail API // Use the new sendEmail API
const emailOptions: any = { const emailOptions: any = {
data: { data: {
@@ -39,12 +55,21 @@ export async function POST(request: Request) {
emailOptions.data.scheduledAt = scheduledAt ? new Date(scheduledAt) : new Date(Date.now() + 60000) emailOptions.data.scheduledAt = scheduledAt ? new Date(scheduledAt) : new Date(Date.now() + 60000)
} }
// Set processImmediately for "send now" type
const processImmediately = (type === 'send' && !scheduledAt)
emailOptions.processImmediately = processImmediately
const result = await sendEmail(payload, emailOptions) const result = await sendEmail(payload, emailOptions)
return Response.json({ return Response.json({
success: true, success: true,
emailId: result.id, emailId: result.id,
message: scheduledAt ? 'Email scheduled successfully' : 'Email queued successfully', message: processImmediately ? 'Email sent successfully' :
scheduledAt ? 'Email scheduled successfully' :
'Email queued successfully',
status: processImmediately ? 'sent' :
scheduledAt ? 'scheduled' :
'queued'
}) })
} catch (error) { } catch (error) {
console.error('Test email error:', error) console.error('Test email error:', error)
@@ -82,8 +107,8 @@ export async function GET() {
total: totalDocs, total: totalDocs,
}, },
mailing: { mailing: {
pluginActive: !!(payload as any).mailing, pluginActive: 'mailing' in payload && !!payload.mailing,
service: !!(payload as any).mailing?.service, service: 'mailing' in payload && payload.mailing && 'service' in payload.mailing && !!payload.mailing.service,
}, },
}) })
} catch (error) { } catch (error) {

11
dev/app/page.tsx Normal file
View File

@@ -0,0 +1,11 @@
import { Metadata } from 'next'
import {redirect} from "next/navigation.js"
export const metadata: Metadata = {
title: 'PayloadCMS Mailing Plugin - Development',
description: 'Development environment for PayloadCMS Mailing Plugin',
}
export default function HomePage() {
redirect('/dashboard')
}

View File

@@ -90,7 +90,7 @@ export interface Config {
'payload-migrations': PayloadMigrationsSelect<false> | PayloadMigrationsSelect<true>; 'payload-migrations': PayloadMigrationsSelect<false> | PayloadMigrationsSelect<true>;
}; };
db: { db: {
defaultIDType: string; defaultIDType: number;
}; };
globals: {}; globals: {};
globalsSelect: {}; globalsSelect: {};
@@ -100,7 +100,7 @@ export interface Config {
}; };
jobs: { jobs: {
tasks: { tasks: {
processEmails: ProcessEmailsJob; 'process-emails': ProcessEmailsTask;
'send-email': TaskSendEmail; 'send-email': TaskSendEmail;
inline: { inline: {
input: unknown; input: unknown;
@@ -133,7 +133,7 @@ export interface UserAuthOperations {
* via the `definition` "users". * via the `definition` "users".
*/ */
export interface User { export interface User {
id: string; id: number;
firstName?: string | null; firstName?: string | null;
lastName?: string | null; lastName?: string | null;
updatedAt: string; updatedAt: string;
@@ -159,7 +159,7 @@ export interface User {
* via the `definition` "posts". * via the `definition` "posts".
*/ */
export interface Post { export interface Post {
id: string; id: number;
updatedAt: string; updatedAt: string;
createdAt: string; createdAt: string;
} }
@@ -168,7 +168,7 @@ export interface Post {
* via the `definition` "media". * via the `definition` "media".
*/ */
export interface Media { export interface Media {
id: string; id: number;
updatedAt: string; updatedAt: string;
createdAt: string; createdAt: string;
url?: string | null; url?: string | null;
@@ -186,7 +186,7 @@ export interface Media {
* via the `definition` "email-templates". * via the `definition` "email-templates".
*/ */
export interface EmailTemplate { export interface EmailTemplate {
id: string; id: number;
/** /**
* A descriptive name for this email template * A descriptive name for this email template
*/ */
@@ -227,11 +227,11 @@ export interface EmailTemplate {
* via the `definition` "emails". * via the `definition` "emails".
*/ */
export interface Email { export interface Email {
id: string; id: number;
/** /**
* Email template used (optional if custom content provided) * Email template used (optional if custom content provided)
*/ */
template?: (string | null) | EmailTemplate; template?: (number | null) | EmailTemplate;
/** /**
* Recipient email addresses * Recipient email addresses
*/ */
@@ -316,7 +316,7 @@ export interface Email {
* via the `definition` "payload-jobs". * via the `definition` "payload-jobs".
*/ */
export interface PayloadJob { export interface PayloadJob {
id: string; id: number;
/** /**
* Input data provided to the job * Input data provided to the job
*/ */
@@ -363,7 +363,7 @@ export interface PayloadJob {
| { | {
executedAt: string; executedAt: string;
completedAt: string; completedAt: string;
taskSlug: 'inline' | 'processEmails' | 'send-email'; taskSlug: 'inline' | 'process-emails' | 'send-email';
taskID: string; taskID: string;
input?: input?:
| { | {
@@ -396,7 +396,7 @@ export interface PayloadJob {
id?: string | null; id?: string | null;
}[] }[]
| null; | null;
taskSlug?: ('inline' | 'processEmails' | 'send-email') | null; taskSlug?: ('inline' | 'process-emails' | 'send-email') | null;
queue?: string | null; queue?: string | null;
waitUntil?: string | null; waitUntil?: string | null;
processing?: boolean | null; processing?: boolean | null;
@@ -408,36 +408,36 @@ export interface PayloadJob {
* via the `definition` "payload-locked-documents". * via the `definition` "payload-locked-documents".
*/ */
export interface PayloadLockedDocument { export interface PayloadLockedDocument {
id: string; id: number;
document?: document?:
| ({ | ({
relationTo: 'users'; relationTo: 'users';
value: string | User; value: number | User;
} | null) } | null)
| ({ | ({
relationTo: 'posts'; relationTo: 'posts';
value: string | Post; value: number | Post;
} | null) } | null)
| ({ | ({
relationTo: 'media'; relationTo: 'media';
value: string | Media; value: number | Media;
} | null) } | null)
| ({ | ({
relationTo: 'email-templates'; relationTo: 'email-templates';
value: string | EmailTemplate; value: number | EmailTemplate;
} | null) } | null)
| ({ | ({
relationTo: 'emails'; relationTo: 'emails';
value: string | Email; value: number | Email;
} | null) } | null)
| ({ | ({
relationTo: 'payload-jobs'; relationTo: 'payload-jobs';
value: string | PayloadJob; value: number | PayloadJob;
} | null); } | null);
globalSlug?: string | null; globalSlug?: string | null;
user: { user: {
relationTo: 'users'; relationTo: 'users';
value: string | User; value: number | User;
}; };
updatedAt: string; updatedAt: string;
createdAt: string; createdAt: string;
@@ -447,10 +447,10 @@ export interface PayloadLockedDocument {
* via the `definition` "payload-preferences". * via the `definition` "payload-preferences".
*/ */
export interface PayloadPreference { export interface PayloadPreference {
id: string; id: number;
user: { user: {
relationTo: 'users'; relationTo: 'users';
value: string | User; value: number | User;
}; };
key?: string | null; key?: string | null;
value?: value?:
@@ -470,7 +470,7 @@ export interface PayloadPreference {
* via the `definition` "payload-migrations". * via the `definition` "payload-migrations".
*/ */
export interface PayloadMigration { export interface PayloadMigration {
id: string; id: number;
name?: string | null; name?: string | null;
batch?: number | null; batch?: number | null;
updatedAt: string; updatedAt: string;
@@ -628,9 +628,9 @@ export interface PayloadMigrationsSelect<T extends boolean = true> {
} }
/** /**
* This interface was referenced by `Config`'s JSON-Schema * This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "ProcessEmailsJob". * via the `definition` "ProcessEmailsTask".
*/ */
export interface ProcessEmailsJob { export interface ProcessEmailsTask {
input?: unknown; input?: unknown;
output?: unknown; output?: unknown;
} }
@@ -640,6 +640,10 @@ export interface ProcessEmailsJob {
*/ */
export interface TaskSendEmail { export interface TaskSendEmail {
input: { input: {
/**
* Process and send the email immediately instead of waiting for the queue processor
*/
processImmediately?: boolean | null;
/** /**
* Use a template (leave empty for direct email) * Use a template (leave empty for direct email)
*/ */

View File

@@ -1,4 +1,4 @@
import { mongooseAdapter } from '@payloadcms/db-mongodb' import { sqliteAdapter } from '@payloadcms/db-sqlite'
import { lexicalEditor } from '@payloadcms/richtext-lexical' import { lexicalEditor } from '@payloadcms/richtext-lexical'
import { import {
FixedToolbarFeature, FixedToolbarFeature,
@@ -6,7 +6,6 @@ import {
HorizontalRuleFeature, HorizontalRuleFeature,
InlineToolbarFeature, InlineToolbarFeature,
} from '@payloadcms/richtext-lexical' } from '@payloadcms/richtext-lexical'
import { MongoMemoryReplSet } from 'mongodb-memory-server'
import path from 'path' import path from 'path'
import { buildConfig } from 'payload' import { buildConfig } from 'payload'
import sharp from 'sharp' import sharp from 'sharp'
@@ -24,36 +23,7 @@ if (!process.env.ROOT_DIR) {
process.env.ROOT_DIR = dirname process.env.ROOT_DIR = dirname
} }
const buildConfigWithMemoryDB = async () => { export default buildConfig({
// Use in-memory MongoDB for development and testing
if (process.env.NODE_ENV === 'test' || process.env.USE_MEMORY_DB === 'true' || !process.env.DATABASE_URI) {
console.log('🚀 Starting MongoDB in-memory database...')
const memoryDB = await MongoMemoryReplSet.create({
replSet: {
count: 1, // Single instance for dev (faster startup)
dbName: process.env.NODE_ENV === 'test' ? 'payloadmemory' : 'payload-mailing-dev',
storageEngine: 'wiredTiger',
},
})
const uri = `${memoryDB.getUri()}&retryWrites=true`
process.env.DATABASE_URI = uri
console.log('✅ MongoDB in-memory database started')
console.log(`📊 Database URI: ${uri.replace(/mongodb:\/\/[^@]*@/, 'mongodb://***@')}`)
// Graceful shutdown
process.on('SIGINT', async () => {
console.log('🛑 Stopping MongoDB in-memory database...')
await memoryDB.stop()
process.exit(0)
})
} else {
console.log(`🔗 Using external MongoDB: ${process.env.DATABASE_URI?.replace(/mongodb:\/\/[^@]*@/, 'mongodb://***@')}`)
}
return buildConfig({
admin: { admin: {
importMap: { importMap: {
baseDir: path.resolve(dirname), baseDir: path.resolve(dirname),
@@ -122,22 +92,36 @@ const buildConfigWithMemoryDB = async () => {
}, },
}, },
], ],
db: mongooseAdapter({ db: sqliteAdapter({
ensureIndexes: true, client: {
url: process.env.DATABASE_URI || '', url: process.env.DATABASE_URI || 'file:./dev.db',
},
}), }),
editor: lexicalEditor(), editor: lexicalEditor(),
email: testEmailAdapter, email: testEmailAdapter,
onInit: async (payload) => { onInit: async (payload) => {
await seed(payload) await seed(payload)
}, },
jobs: {
jobsCollectionOverrides: c => {
if (c.defaultJobsCollection.admin) c.defaultJobsCollection.admin.hidden = false
return c.defaultJobsCollection
},
autoRun: [
{
cron: '*/1 * * * *', // every minute
limit: 10, // limit jobs to process each run
queue: 'default', // name of the queue
},
],
},
plugins: [ plugins: [
mailingPlugin({ mailingPlugin({
defaultFrom: 'noreply@test.com', defaultFrom: 'noreply@test.com',
initOrder: 'after', initOrder: 'after',
retryAttempts: 3, retryAttempts: 3,
retryDelay: 60000, // 1 minute for dev retryDelay: 60000, // 1 minute for dev
queue: 'email-queue', queue: 'default',
// Example: Collection overrides for customization // Example: Collection overrides for customization
// Uncomment and modify as needed for your use case // Uncomment and modify as needed for your use case
@@ -286,6 +270,3 @@ const buildConfigWithMemoryDB = async () => {
outputFile: path.resolve(dirname, 'payload-types.ts'), outputFile: path.resolve(dirname, 'payload-types.ts'),
}, },
}) })
}
export default buildConfigWithMemoryDB()

View File

@@ -3,26 +3,14 @@
// Development startup script for PayloadCMS Mailing Plugin // Development startup script for PayloadCMS Mailing Plugin
// This ensures proper environment setup and provides helpful information // This ensures proper environment setup and provides helpful information
console.log('🚀 PayloadCMS Mailing Plugin - Development Mode')
console.log('=' .repeat(50))
// Set development environment // Set development environment
process.env.NODE_ENV = process.env.NODE_ENV || 'development' process.env.NODE_ENV = process.env.NODE_ENV || 'development'
// Enable in-memory MongoDB by default for development // Set default SQLite database for development
if (!process.env.DATABASE_URI) { if (!process.env.DATABASE_URI) {
process.env.USE_MEMORY_DB = 'true' process.env.DATABASE_URI = 'file:./dev.db'
console.log('📦 Using in-memory MongoDB (no installation required)')
} else {
console.log(`🔗 Using external MongoDB: ${process.env.DATABASE_URI}`)
} }
console.log('')
console.log('🔧 Starting development server...')
console.log('📧 Mailing plugin configured with test transport')
console.log('🎯 Test interface will be available at: /mailing-test')
console.log('')
// Import and start Next.js // Import and start Next.js
import('next/dist/cli/next-dev.js') import('next/dist/cli/next-dev.js')
.then(({ nextDev }) => { .then(({ nextDev }) => {
@@ -35,11 +23,9 @@ import('next/dist/cli/next-dev.js')
// Handle graceful shutdown // Handle graceful shutdown
process.on('SIGTERM', () => { process.on('SIGTERM', () => {
console.log('\n🛑 Shutting down development server...')
process.exit(0) process.exit(0)
}) })
process.on('SIGINT', () => { process.on('SIGINT', () => {
console.log('\n🛑 Shutting down development server...')
process.exit(0) process.exit(0)
}) })

View File

@@ -1,6 +1,6 @@
{ {
"name": "@xtr-dev/payload-mailing", "name": "@xtr-dev/payload-mailing",
"version": "0.4.3", "version": "0.4.10",
"description": "Template-based email system with scheduling and job processing for PayloadCMS", "description": "Template-based email system with scheduling and job processing for PayloadCMS",
"type": "module", "type": "module",
"main": "dist/index.js", "main": "dist/index.js",
@@ -23,9 +23,6 @@
"dev:generate-importmap": "npm run dev:payload generate:importmap", "dev:generate-importmap": "npm run dev:payload generate:importmap",
"dev:generate-types": "npm run dev:payload generate:types", "dev:generate-types": "npm run dev:payload generate:types",
"dev:payload": "cross-env PAYLOAD_CONFIG_PATH=./dev/payload.config.ts payload", "dev:payload": "cross-env PAYLOAD_CONFIG_PATH=./dev/payload.config.ts payload",
"payload": "cross-env NODE_OPTIONS=--no-deprecation payload",
"generate:importmap": "npm run payload generate:importmap",
"generate:types": "npm run payload generate:types",
"lint": "eslint", "lint": "eslint",
"lint:fix": "eslint ./src --fix", "lint:fix": "eslint ./src --fix",
"prepublishOnly": "npm run clean && npm run build", "prepublishOnly": "npm run clean && npm run build",

View File

@@ -1,31 +0,0 @@
/**
* This config is only used to generate types.
*/
import { BaseDatabaseAdapter, buildConfig, Payload} from 'payload'
import Emails from "./src/collections/Emails.js"
import {createEmailTemplatesCollection} from "./src/collections/EmailTemplates.js"
import path from "path"
import { fileURLToPath } from 'url'
const __filename = fileURLToPath(import.meta.url)
const __dirname = path.dirname(__filename)
export default buildConfig({
collections: [
Emails,
createEmailTemplatesCollection()
],
db: {
allowIDOnCreate: undefined,
defaultIDType: 'number',
init: function (args: { payload: Payload; }): BaseDatabaseAdapter {
throw new Error('Function not implemented.');
},
name: undefined
},
secret: '',
typescript: {
outputFile: path.resolve(__dirname, 'src/payload-types.ts'),
}
});

View File

@@ -1,10 +1,12 @@
import type { CollectionConfig } from 'payload' import type { CollectionConfig } from 'payload'
import { findExistingJobs, ensureEmailJob, updateEmailJobRelationship } from '../utils/jobScheduler.js'
import { createContextLogger } from '../utils/logger.js'
const Emails: CollectionConfig = { const Emails: CollectionConfig = {
slug: 'emails', slug: 'emails',
admin: { admin: {
useAsTitle: 'subject', useAsTitle: 'subject',
defaultColumns: ['subject', 'to', 'status', 'scheduledAt', 'sentAt'], defaultColumns: ['subject', 'to', 'status', 'jobs', 'scheduledAt', 'sentAt'],
group: 'Mailing', group: 'Mailing',
description: 'Email delivery and status tracking', description: 'Email delivery and status tracking',
}, },
@@ -164,22 +166,76 @@ const Emails: CollectionConfig = {
description: 'Email priority (1=highest, 10=lowest)', description: 'Email priority (1=highest, 10=lowest)',
}, },
}, },
{
name: 'jobs',
type: 'relationship',
relationTo: 'payload-jobs',
hasMany: true,
admin: {
description: 'Processing jobs associated with this email',
allowCreate: false,
readOnly: true,
},
filterOptions: ({ id }) => {
return {
'input.emailId': {
equals: id,
},
}
},
},
], ],
hooks: {
// Simple approach: Only use afterChange hook for job management
// This avoids complex interaction between hooks and ensures document ID is always available
afterChange: [
async ({ doc, previousDoc, req, operation }) => {
// Skip if:
// 1. Email is not pending status
// 2. Jobs are not configured
// 3. Email already has jobs (unless status just changed to pending)
const shouldSkip =
doc.status !== 'pending' ||
!req.payload.jobs ||
(doc.jobs?.length > 0 && previousDoc?.status === 'pending')
if (shouldSkip) {
return
}
try {
// Ensure a job exists for this email
// This function handles:
// - Checking for existing jobs (duplicate prevention)
// - Creating new job if needed
// - Returning all job IDs
const result = await ensureEmailJob(req.payload, doc.id, {
scheduledAt: doc.scheduledAt,
})
// Update the email's job relationship if we have jobs
// This handles both new jobs and existing jobs that weren't in the relationship
if (result.jobIds.length > 0) {
await updateEmailJobRelationship(req.payload, doc.id, result.jobIds, 'emails')
}
} catch (error) {
// Log error but don't throw - we don't want to fail the email operation
const logger = createContextLogger(req.payload, 'EMAILS_HOOK')
logger.error(`Failed to ensure job for email ${doc.id}:`, error)
}
}
]
},
timestamps: true, timestamps: true,
// indexes: [ indexes: [
// { {
// fields: { fields: ['status', 'scheduledAt'],
// status: 1, },
// scheduledAt: 1, {
// }, fields: ['priority', 'createdAt'],
// }, },
// { ],
// fields: {
// priority: -1,
// createdAt: 1,
// },
// },
// ],
} }
export default Emails export default Emails

View File

@@ -11,9 +11,9 @@ export { MailingService } from './services/MailingService.js'
export { default as EmailTemplates, createEmailTemplatesCollection } from './collections/EmailTemplates.js' export { default as EmailTemplates, createEmailTemplatesCollection } from './collections/EmailTemplates.js'
export { default as Emails } from './collections/Emails.js' export { default as Emails } from './collections/Emails.js'
// Jobs (includes the send email task) // Jobs (includes the individual email processing job)
export { mailingJobs, sendEmailJob } from './jobs/index.js' export { mailingJobs } from './jobs/index.js'
export type { SendEmailTaskInput } from './jobs/sendEmailTask.js' export type { ProcessEmailJobInput } from './jobs/processEmailJob.js'
// Main email sending function // Main email sending function
export { sendEmail, type SendEmailOptions } from './sendEmail.js' export { sendEmail, type SendEmailOptions } from './sendEmail.js'
@@ -26,7 +26,12 @@ export {
processEmails, processEmails,
retryFailedEmails, retryFailedEmails,
parseAndValidateEmails, parseAndValidateEmails,
sanitizeDisplayName,
sanitizeFromName,
} from './utils/helpers.js' } from './utils/helpers.js'
// Email processing utilities // Email processing utilities
export { processEmailById, processAllEmails } from './utils/emailProcessor.js' export { processEmailById, processJobById, processAllEmails } from './utils/emailProcessor.js'
// Job scheduling utilities
export { findExistingJobs, ensureEmailJob, updateEmailJobRelationship } from './utils/jobScheduler.js'

View File

@@ -1,14 +1,16 @@
import { processEmailsJob } from './processEmailsTask.js' import { processEmailsJob } from './processEmailsTask.js'
import { sendEmailJob } from './sendEmailTask.js' import { processEmailJob } from './processEmailJob.js'
/** /**
* All mailing-related jobs that get registered with Payload * All mailing-related jobs that get registered with Payload
*
* Note: The sendEmailJob has been removed as each email now gets its own individual processEmailJob
*/ */
export const mailingJobs = [ export const mailingJobs = [
processEmailsJob, processEmailsJob, // Kept for backward compatibility and batch processing if needed
sendEmailJob, processEmailJob, // New individual email processing job
] ]
// Re-export everything from individual job files // Re-export everything from individual job files
export * from './processEmailsTask.js' export * from './processEmailsTask.js'
export * from './sendEmailTask.js' export * from './processEmailJob.js'

View File

@@ -0,0 +1,72 @@
import type { PayloadRequest } from 'payload'
import { processEmailById } from '../utils/emailProcessor.js'
/**
* Data passed to the individual email processing job
*/
export interface ProcessEmailJobInput {
/**
* The ID of the email to process
*/
emailId: string | number
}
/**
* Job definition for processing a single email
* This replaces the batch processing approach with individual email jobs
*/
export const processEmailJob = {
slug: 'process-email',
label: 'Process Individual Email',
inputSchema: [
{
name: 'emailId',
type: 'text' as const,
required: true,
label: 'Email ID',
admin: {
description: 'The ID of the email to process and send'
}
}
],
outputSchema: [
{
name: 'success',
type: 'checkbox' as const
},
{
name: 'emailId',
type: 'text' as const
},
{
name: 'status',
type: 'text' as const
}
],
handler: async ({ input, req }: { input: ProcessEmailJobInput; req: PayloadRequest }) => {
const payload = (req as any).payload
const { emailId } = input
if (!emailId) {
throw new Error('Email ID is required for processing')
}
try {
// Process the individual email
await processEmailById(payload, String(emailId))
return {
output: {
success: true,
emailId: String(emailId),
status: 'sent',
message: `Email ${emailId} processed successfully`
}
}
} catch (error) {
throw new Error(`Failed to process email ${emailId}: ${String(error)}`)
}
}
}
export default processEmailJob

View File

@@ -1,5 +1,6 @@
import type { PayloadRequest, Payload } from 'payload' import type { PayloadRequest, Payload } from 'payload'
import { processAllEmails } from '../utils/emailProcessor.js' import { processAllEmails } from '../utils/emailProcessor.js'
import { createContextLogger } from '../utils/logger.js'
/** /**
* Data passed to the process emails task * Data passed to the process emails task
@@ -19,17 +20,8 @@ export const processEmailsTaskHandler = async (
const { req } = context const { req } = context
const payload = (req as any).payload const payload = (req as any).payload
try {
console.log('🔄 Processing email queue (pending + failed emails)...')
// Use the shared email processing logic // Use the shared email processing logic
await processAllEmails(payload) await processAllEmails(payload)
console.log('✅ Email queue processing completed successfully')
} catch (error) {
console.error('❌ Email queue processing failed:', error)
throw error
}
} }
/** /**
@@ -76,7 +68,8 @@ export const scheduleEmailsJob = async (
delay?: number delay?: number
) => { ) => {
if (!payload.jobs) { if (!payload.jobs) {
console.warn('PayloadCMS jobs not configured - emails will not be processed automatically') const logger = createContextLogger(payload, 'SCHEDULER')
logger.warn('PayloadCMS jobs not configured - emails will not be processed automatically')
return return
} }
@@ -88,6 +81,7 @@ export const scheduleEmailsJob = async (
waitUntil: delay ? new Date(Date.now() + delay) : undefined, waitUntil: delay ? new Date(Date.now() + delay) : undefined,
} as any) } as any)
} catch (error) { } catch (error) {
console.error('Failed to schedule email processing job:', error) const logger = createContextLogger(payload, 'SCHEDULER')
logger.error('Failed to schedule email processing job:', error)
} }
} }

View File

@@ -1,256 +0,0 @@
import { sendEmail } from '../sendEmail.js'
import { BaseEmailDocument } from '../types/index.js'
import { processEmailById } from '../utils/emailProcessor.js'
export interface SendEmailTaskInput {
// Template mode fields
templateSlug?: string
variables?: Record<string, any>
// Direct email mode fields
subject?: string
html?: string
text?: string
// Common fields
to: string | string[]
cc?: string | string[]
bcc?: string | string[]
from?: string
fromName?: string
replyTo?: string
scheduledAt?: string | Date // ISO date string or Date object
priority?: number
processImmediately?: boolean // If true, process the email immediately instead of waiting for the queue
// Allow any additional fields that users might have in their email collection
[key: string]: any
}
/**
* Transforms task input into sendEmail options by separating template and data fields
*/
function transformTaskInputToSendEmailOptions(taskInput: SendEmailTaskInput) {
const sendEmailOptions: any = {
data: {}
}
// If using template mode, set template options
if (taskInput.templateSlug) {
sendEmailOptions.template = {
slug: taskInput.templateSlug,
variables: taskInput.variables || {}
}
}
// Standard email fields that should be copied to data
const standardFields = ['to', 'cc', 'bcc', 'from', 'fromName', 'replyTo', 'subject', 'html', 'text', 'scheduledAt', 'priority']
// Fields that should not be copied to data
const excludedFields = ['templateSlug', 'variables', 'processImmediately']
// Copy standard fields to data
standardFields.forEach(field => {
if (taskInput[field] !== undefined) {
sendEmailOptions.data[field] = taskInput[field]
}
})
// Copy any additional custom fields that aren't excluded or standard fields
Object.keys(taskInput).forEach(key => {
if (!excludedFields.includes(key) && !standardFields.includes(key)) {
sendEmailOptions.data[key] = taskInput[key]
}
})
return sendEmailOptions
}
/**
* Job definition for sending emails
* Can be used through Payload's job queue system to send emails programmatically
*/
export const sendEmailJob = {
slug: 'send-email',
label: 'Send Email',
inputSchema: [
{
name: 'processImmediately',
type: 'checkbox' as const,
label: 'Process Immediately',
defaultValue: false,
admin: {
description: 'Process and send the email immediately instead of waiting for the queue processor'
}
},
{
name: 'templateSlug',
type: 'text' as const,
label: 'Template Slug',
admin: {
description: 'Use a template (leave empty for direct email)',
condition: (data: any) => !data.subject && !data.html
}
},
{
name: 'variables',
type: 'json' as const,
label: 'Template Variables',
admin: {
description: 'JSON object with variables for template rendering',
condition: (data: any) => Boolean(data.templateSlug)
}
},
{
name: 'subject',
type: 'text' as const,
label: 'Subject',
admin: {
description: 'Email subject (required if not using template)',
condition: (data: any) => !data.templateSlug
}
},
{
name: 'html',
type: 'textarea' as const,
label: 'HTML Content',
admin: {
description: 'HTML email content (required if not using template)',
condition: (data: any) => !data.templateSlug
}
},
{
name: 'text',
type: 'textarea' as const,
label: 'Text Content',
admin: {
description: 'Plain text email content (optional)',
condition: (data: any) => !data.templateSlug
}
},
{
name: 'to',
type: 'text' as const,
required: true,
label: 'To (Email Recipients)',
admin: {
description: 'Comma-separated list of email addresses'
}
},
{
name: 'cc',
type: 'text' as const,
label: 'CC (Carbon Copy)',
admin: {
description: 'Optional comma-separated list of CC email addresses'
}
},
{
name: 'bcc',
type: 'text' as const,
label: 'BCC (Blind Carbon Copy)',
admin: {
description: 'Optional comma-separated list of BCC email addresses'
}
},
{
name: 'from',
type: 'text' as const,
label: 'From Email',
admin: {
description: 'Optional sender email address (uses default if not provided)'
}
},
{
name: 'fromName',
type: 'text' as const,
label: 'From Name',
admin: {
description: 'Optional sender display name (e.g., "John Doe")'
}
},
{
name: 'replyTo',
type: 'text' as const,
label: 'Reply To',
admin: {
description: 'Optional reply-to email address'
}
},
{
name: 'scheduledAt',
type: 'date' as const,
label: 'Schedule For',
admin: {
description: 'Optional date/time to schedule email for future delivery',
condition: (data: any) => !data.processImmediately
}
},
{
name: 'priority',
type: 'number' as const,
label: 'Priority',
min: 1,
max: 10,
defaultValue: 5,
admin: {
description: 'Email priority (1 = highest, 10 = lowest)'
}
}
],
outputSchema: [
{
name: 'id',
type: 'text' as const
}
],
handler: async ({ input, payload }: any) => {
// Cast input to our expected type
const taskInput = input as SendEmailTaskInput
const shouldProcessImmediately = taskInput.processImmediately || false
try {
// Transform task input into sendEmail options using helper function
const sendEmailOptions = transformTaskInputToSendEmailOptions(taskInput)
// Use the sendEmail helper to create the email
const email = await sendEmail<BaseEmailDocument>(payload, sendEmailOptions)
// If processImmediately is true, process the email now
if (shouldProcessImmediately) {
console.log(`⚡ Processing email ${email.id} immediately...`)
await processEmailById(payload, String(email.id))
console.log(`✅ Email ${email.id} processed and sent immediately`)
return {
output: {
success: true,
id: email.id,
status: 'sent',
processedImmediately: true
}
}
}
return {
output: {
success: true,
id: email.id,
status: 'queued',
processedImmediately: false
}
}
} catch (error) {
// Re-throw Error instances to preserve stack trace and error context
if (error instanceof Error) {
throw error
} else {
// Only wrap non-Error values
throw new Error(`Failed to process email: ${String(error)}`)
}
}
}
}
export default sendEmailJob

View File

@@ -1,436 +0,0 @@
/* tslint:disable */
/* eslint-disable */
/**
* This file was automatically generated by Payload.
* DO NOT MODIFY IT BY HAND. Instead, modify your source Payload config,
* and re-run `payload generate:types` to regenerate this file.
*/
/**
* Supported timezones in IANA format.
*
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "supportedTimezones".
*/
export type SupportedTimezones =
| 'Pacific/Midway'
| 'Pacific/Niue'
| 'Pacific/Honolulu'
| 'Pacific/Rarotonga'
| 'America/Anchorage'
| 'Pacific/Gambier'
| 'America/Los_Angeles'
| 'America/Tijuana'
| 'America/Denver'
| 'America/Phoenix'
| 'America/Chicago'
| 'America/Guatemala'
| 'America/New_York'
| 'America/Bogota'
| 'America/Caracas'
| 'America/Santiago'
| 'America/Buenos_Aires'
| 'America/Sao_Paulo'
| 'Atlantic/South_Georgia'
| 'Atlantic/Azores'
| 'Atlantic/Cape_Verde'
| 'Europe/London'
| 'Europe/Berlin'
| 'Africa/Lagos'
| 'Europe/Athens'
| 'Africa/Cairo'
| 'Europe/Moscow'
| 'Asia/Riyadh'
| 'Asia/Dubai'
| 'Asia/Baku'
| 'Asia/Karachi'
| 'Asia/Tashkent'
| 'Asia/Calcutta'
| 'Asia/Dhaka'
| 'Asia/Almaty'
| 'Asia/Jakarta'
| 'Asia/Bangkok'
| 'Asia/Shanghai'
| 'Asia/Singapore'
| 'Asia/Tokyo'
| 'Asia/Seoul'
| 'Australia/Brisbane'
| 'Australia/Sydney'
| 'Pacific/Guam'
| 'Pacific/Noumea'
| 'Pacific/Auckland'
| 'Pacific/Fiji';
export interface Config {
auth: {
users: UserAuthOperations;
};
blocks: {};
collections: {
emails: Email;
'email-templates': EmailTemplate;
users: User;
'payload-locked-documents': PayloadLockedDocument;
'payload-preferences': PayloadPreference;
'payload-migrations': PayloadMigration;
};
collectionsJoins: {};
collectionsSelect: {
emails: EmailsSelect<false> | EmailsSelect<true>;
'email-templates': EmailTemplatesSelect<false> | EmailTemplatesSelect<true>;
users: UsersSelect<false> | UsersSelect<true>;
'payload-locked-documents': PayloadLockedDocumentsSelect<false> | PayloadLockedDocumentsSelect<true>;
'payload-preferences': PayloadPreferencesSelect<false> | PayloadPreferencesSelect<true>;
'payload-migrations': PayloadMigrationsSelect<false> | PayloadMigrationsSelect<true>;
};
db: {
defaultIDType: number;
};
globals: {};
globalsSelect: {};
locale: null;
user: User & {
collection: 'users';
};
jobs: {
tasks: unknown;
workflows: unknown;
};
}
export interface UserAuthOperations {
forgotPassword: {
email: string;
password: string;
};
login: {
email: string;
password: string;
};
registerFirstUser: {
email: string;
password: string;
};
unlock: {
email: string;
password: string;
};
}
/**
* Email delivery and status tracking
*
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "emails".
*/
export interface Email {
id: number;
/**
* Email template used (optional if custom content provided)
*/
template?: (number | null) | EmailTemplate;
/**
* Recipient email addresses
*/
to: string[];
/**
* CC email addresses
*/
cc?: string[] | null;
/**
* BCC email addresses
*/
bcc?: string[] | null;
/**
* Sender email address (optional, uses default if not provided)
*/
from?: string | null;
/**
* Sender display name (optional, e.g., "John Doe" for "John Doe <john@example.com>")
*/
fromName?: string | null;
/**
* Reply-to email address
*/
replyTo?: string | null;
/**
* Email subject line
*/
subject: string;
/**
* Rendered HTML content of the email
*/
html: string;
/**
* Plain text version of the email
*/
text?: string | null;
/**
* Template variables used to render this email
*/
variables?:
| {
[k: string]: unknown;
}
| unknown[]
| string
| number
| boolean
| null;
/**
* When this email should be sent (leave empty for immediate)
*/
scheduledAt?: string | null;
/**
* When this email was actually sent
*/
sentAt?: string | null;
/**
* Current status of this email
*/
status: 'pending' | 'processing' | 'sent' | 'failed';
/**
* Number of send attempts made
*/
attempts?: number | null;
/**
* When the last send attempt was made
*/
lastAttemptAt?: string | null;
/**
* Last error message if send failed
*/
error?: string | null;
/**
* Email priority (1=highest, 10=lowest)
*/
priority?: number | null;
updatedAt: string;
createdAt: string;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "email-templates".
*/
export interface EmailTemplate {
id: number;
/**
* A descriptive name for this email template
*/
name: string;
/**
* Unique identifier for this template (e.g., "welcome-email", "password-reset")
*/
slug: string;
/**
* Email subject line. You can use Handlebars variables like {{firstName}} or {{siteName}}.
*/
subject: string;
/**
* Email content with rich text formatting. Supports Handlebars variables like {{firstName}} and helpers like {{formatDate createdAt "long"}}. Content is converted to HTML and plain text automatically.
*/
content: {
root: {
type: string;
children: {
type: string;
version: number;
[k: string]: unknown;
}[];
direction: ('ltr' | 'rtl') | null;
format: 'left' | 'start' | 'center' | 'right' | 'end' | 'justify' | '';
indent: number;
version: number;
};
[k: string]: unknown;
};
updatedAt: string;
createdAt: string;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "users".
*/
export interface User {
id: number;
updatedAt: string;
createdAt: string;
email: string;
resetPasswordToken?: string | null;
resetPasswordExpiration?: string | null;
salt?: string | null;
hash?: string | null;
loginAttempts?: number | null;
lockUntil?: string | null;
sessions?:
| {
id: string;
createdAt?: string | null;
expiresAt: string;
}[]
| null;
password?: string | null;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "payload-locked-documents".
*/
export interface PayloadLockedDocument {
id: number;
document?:
| ({
relationTo: 'emails';
value: number | Email;
} | null)
| ({
relationTo: 'email-templates';
value: number | EmailTemplate;
} | null)
| ({
relationTo: 'users';
value: number | User;
} | null);
globalSlug?: string | null;
user: {
relationTo: 'users';
value: number | User;
};
updatedAt: string;
createdAt: string;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "payload-preferences".
*/
export interface PayloadPreference {
id: number;
user: {
relationTo: 'users';
value: number | User;
};
key?: string | null;
value?:
| {
[k: string]: unknown;
}
| unknown[]
| string
| number
| boolean
| null;
updatedAt: string;
createdAt: string;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "payload-migrations".
*/
export interface PayloadMigration {
id: number;
name?: string | null;
batch?: number | null;
updatedAt: string;
createdAt: string;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "emails_select".
*/
export interface EmailsSelect<T extends boolean = true> {
template?: T;
to?: T;
cc?: T;
bcc?: T;
from?: T;
fromName?: T;
replyTo?: T;
subject?: T;
html?: T;
text?: T;
variables?: T;
scheduledAt?: T;
sentAt?: T;
status?: T;
attempts?: T;
lastAttemptAt?: T;
error?: T;
priority?: T;
updatedAt?: T;
createdAt?: T;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "email-templates_select".
*/
export interface EmailTemplatesSelect<T extends boolean = true> {
name?: T;
slug?: T;
subject?: T;
content?: T;
updatedAt?: T;
createdAt?: T;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "users_select".
*/
export interface UsersSelect<T extends boolean = true> {
updatedAt?: T;
createdAt?: T;
email?: T;
resetPasswordToken?: T;
resetPasswordExpiration?: T;
salt?: T;
hash?: T;
loginAttempts?: T;
lockUntil?: T;
sessions?:
| T
| {
id?: T;
createdAt?: T;
expiresAt?: T;
};
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "payload-locked-documents_select".
*/
export interface PayloadLockedDocumentsSelect<T extends boolean = true> {
document?: T;
globalSlug?: T;
user?: T;
updatedAt?: T;
createdAt?: T;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "payload-preferences_select".
*/
export interface PayloadPreferencesSelect<T extends boolean = true> {
user?: T;
key?: T;
value?: T;
updatedAt?: T;
createdAt?: T;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "payload-migrations_select".
*/
export interface PayloadMigrationsSelect<T extends boolean = true> {
name?: T;
batch?: T;
updatedAt?: T;
createdAt?: T;
}
/**
* This interface was referenced by `Config`'s JSON-Schema
* via the `definition` "auth".
*/
export interface Auth {
[k: string]: unknown;
}
declare module 'payload' {
export interface GeneratedTypes extends Config {}
}

View File

@@ -3,7 +3,7 @@ import { MailingPluginConfig, MailingContext } from './types/index.js'
import { MailingService } from './services/MailingService.js' import { MailingService } from './services/MailingService.js'
import { createEmailTemplatesCollection } from './collections/EmailTemplates.js' import { createEmailTemplatesCollection } from './collections/EmailTemplates.js'
import Emails from './collections/Emails.js' import Emails from './collections/Emails.js'
import { mailingJobs, scheduleEmailsJob } from './jobs/index.js' import { mailingJobs } from './jobs/index.js'
export const mailingPlugin = (pluginConfig: MailingPluginConfig) => (config: Config): Config => { export const mailingPlugin = (pluginConfig: MailingPluginConfig) => (config: Config): Config => {
@@ -106,18 +106,6 @@ export const mailingPlugin = (pluginConfig: MailingPluginConfig) => (config: Con
}, },
} as MailingContext } as MailingContext
// Schedule the initial email processing job
try {
await scheduleEmailsJob(payload, queueName, 60000) // Schedule in 1 minute
} catch (error) {
console.error('Failed to schedule email processing job:', error)
}
// Call onReady callback if provided
if (pluginConfig.onReady) {
await pluginConfig.onReady(payload)
}
if (pluginConfig.initOrder !== 'after' && config.onInit) { if (pluginConfig.initOrder !== 'after' && config.onInit) {
await config.onInit(payload) await config.onInit(payload)
} }

View File

@@ -1,6 +1,8 @@
import { Payload } from 'payload' import { Payload } from 'payload'
import { getMailing, renderTemplate, parseAndValidateEmails } from './utils/helpers.js' import { getMailing, renderTemplate, parseAndValidateEmails, sanitizeFromName } from './utils/helpers.js'
import { BaseEmailDocument } from './types/index.js' import { BaseEmailDocument } from './types/index.js'
import { processJobById } from './utils/emailProcessor.js'
import { createContextLogger } from './utils/logger.js'
// Options for sending emails // Options for sending emails
export interface SendEmailOptions<T extends BaseEmailDocument = BaseEmailDocument> { export interface SendEmailOptions<T extends BaseEmailDocument = BaseEmailDocument> {
@@ -13,6 +15,8 @@ export interface SendEmailOptions<T extends BaseEmailDocument = BaseEmailDocumen
data?: Partial<T> data?: Partial<T>
// Common options // Common options
collectionSlug?: string // defaults to 'emails' collectionSlug?: string // defaults to 'emails'
processImmediately?: boolean // if true, creates job and processes it immediately
queue?: string // queue name for the job, defaults to mailing config queue
} }
/** /**
@@ -39,8 +43,8 @@ export const sendEmail = async <TEmail extends BaseEmailDocument = BaseEmailDocu
payload: Payload, payload: Payload,
options: SendEmailOptions<TEmail> options: SendEmailOptions<TEmail>
): Promise<TEmail> => { ): Promise<TEmail> => {
const mailing = getMailing(payload) const mailingConfig = getMailing(payload)
const collectionSlug = options.collectionSlug || mailing.collections.emails || 'emails' const collectionSlug = options.collectionSlug || mailingConfig.collections.emails || 'emails'
let emailData: Partial<TEmail> = { ...options.data } as Partial<TEmail> let emailData: Partial<TEmail> = { ...options.data } as Partial<TEmail>
@@ -101,15 +105,7 @@ export const sendEmail = async <TEmail extends BaseEmailDocument = BaseEmailDocu
} }
// Sanitize fromName to prevent header injection // Sanitize fromName to prevent header injection
if (emailData.fromName) { emailData.fromName = sanitizeFromName(emailData.fromName as string)
emailData.fromName = emailData.fromName
.trim()
// Remove/replace newlines and carriage returns to prevent header injection
.replace(/[\r\n]/g, ' ')
// Remove control characters (except space and printable characters)
.replace(/[\x00-\x1F\x7F-\x9F]/g, '')
// Note: We don't escape quotes here as that's handled in MailingService
}
// Normalize Date objects to ISO strings for consistent database storage // Normalize Date objects to ISO strings for consistent database storage
if (emailData.scheduledAt instanceof Date) { if (emailData.scheduledAt instanceof Date) {
@@ -129,6 +125,7 @@ export const sendEmail = async <TEmail extends BaseEmailDocument = BaseEmailDocu
} }
// Create the email in the collection with proper typing // Create the email in the collection with proper typing
// The hooks will automatically create and populate the job relationship
const email = await payload.create({ const email = await payload.create({
collection: collectionSlug, collection: collectionSlug,
data: emailData data: emailData
@@ -139,6 +136,93 @@ export const sendEmail = async <TEmail extends BaseEmailDocument = BaseEmailDocu
throw new Error('Failed to create email: invalid response from database') throw new Error('Failed to create email: invalid response from database')
} }
// If processImmediately is true, get the job from the relationship and process it now
if (options.processImmediately) {
const logger = createContextLogger(payload, 'IMMEDIATE')
logger.debug(`Starting immediate processing for email ${email.id}`)
if (!payload.jobs) {
throw new Error('PayloadCMS jobs not configured - cannot process email immediately')
}
// Poll for the job with optimized backoff and timeout protection
// This handles the async nature of hooks and ensures we wait for job creation
const maxAttempts = 5 // Reduced from 10 to minimize delay
const initialDelay = 25 // Reduced from 50ms for faster response
const maxTotalTime = 3000 // 3 second total timeout
const startTime = Date.now()
let jobId: string | undefined
logger.debug(`Polling for job creation (max ${maxAttempts} attempts, ${maxTotalTime}ms timeout)`)
for (let attempt = 0; attempt < maxAttempts; attempt++) {
// Check total timeout before continuing
if (Date.now() - startTime > maxTotalTime) {
throw new Error(
`Job polling timed out after ${maxTotalTime}ms for email ${email.id}. ` +
`The auto-scheduling may have failed or is taking longer than expected.`
)
}
// Calculate delay with exponential backoff (25ms, 50ms, 100ms, 200ms, 400ms)
// Cap at 400ms per attempt for better responsiveness
const delay = Math.min(initialDelay * Math.pow(2, attempt), 400)
if (attempt > 0) {
await new Promise(resolve => setTimeout(resolve, delay))
}
// Refetch the email to check for jobs
const emailWithJobs = await payload.findByID({
collection: collectionSlug,
id: email.id,
})
logger.debug(`Attempt ${attempt + 1}/${maxAttempts}: Found ${emailWithJobs.jobs?.length || 0} jobs for email ${email.id}`)
if (emailWithJobs.jobs && emailWithJobs.jobs.length > 0) {
// Job found! Get the first job ID (should only be one for a new email)
jobId = Array.isArray(emailWithJobs.jobs)
? String(emailWithJobs.jobs[0])
: String(emailWithJobs.jobs)
logger.info(`Found job ID: ${jobId}`)
break
}
// Log on later attempts to help with debugging (reduced threshold)
if (attempt >= 1) {
logger.debug(`Waiting for job creation for email ${email.id}, attempt ${attempt + 1}/${maxAttempts}`)
}
}
if (!jobId) {
// Distinguish between different failure scenarios for better error handling
const timeoutMsg = Date.now() - startTime >= maxTotalTime
const errorType = timeoutMsg ? 'POLLING_TIMEOUT' : 'JOB_NOT_FOUND'
const baseMessage = timeoutMsg
? `Job polling timed out after ${maxTotalTime}ms for email ${email.id}`
: `No processing job found for email ${email.id} after ${maxAttempts} attempts (${Date.now() - startTime}ms)`
throw new Error(
`${errorType}: ${baseMessage}. ` +
`This indicates the email was created but job auto-scheduling failed. ` +
`The email exists in the database but immediate processing cannot proceed. ` +
`You may need to: 1) Check job queue configuration, 2) Verify database hooks are working, ` +
`3) Process the email later using processEmailById('${email.id}').`
)
}
logger.info(`Starting job execution for job ${jobId}`)
try {
await processJobById(payload, jobId)
logger.info(`Successfully processed email ${email.id} immediately`)
} catch (error) {
logger.error(`Failed to process email ${email.id} immediately:`, error)
throw new Error(`Failed to process email ${email.id} immediately: ${String(error)}`)
}
}
return email as TEmail return email as TEmail
} }

View File

@@ -7,6 +7,7 @@ import {
BaseEmail, BaseEmailTemplate, BaseEmailDocument, BaseEmailTemplateDocument BaseEmail, BaseEmailTemplate, BaseEmailDocument, BaseEmailTemplateDocument
} from '../types/index.js' } from '../types/index.js'
import { serializeRichTextToHTML, serializeRichTextToText } from '../utils/richTextSerializer.js' import { serializeRichTextToHTML, serializeRichTextToText } from '../utils/richTextSerializer.js'
import { sanitizeDisplayName } from '../utils/helpers.js'
export class MailingService implements IMailingService { export class MailingService implements IMailingService {
public payload: Payload public payload: Payload
@@ -44,17 +45,10 @@ export class MailingService implements IMailingService {
/** /**
* Sanitizes a display name for use in email headers to prevent header injection * Sanitizes a display name for use in email headers to prevent header injection
* and ensure proper formatting * Uses the centralized sanitization utility with quote escaping for headers
*/ */
private sanitizeDisplayName(name: string): string { private sanitizeDisplayName(name: string): string {
return name return sanitizeDisplayName(name, true) // escapeQuotes = true for email headers
.trim()
// Remove/replace newlines and carriage returns to prevent header injection
.replace(/[\r\n]/g, ' ')
// Remove control characters (except space and printable characters)
.replace(/[\x00-\x1F\x7F-\x9F]/g, '')
// Escape quotes to prevent malformed headers
.replace(/"/g, '\\"')
} }
/** /**
@@ -321,9 +315,9 @@ export class MailingService implements IMailingService {
const email = await this.payload.findByID({ const email = await this.payload.findByID({
collection: this.emailsCollection as any, collection: this.emailsCollection as any,
id: emailId, id: emailId,
}) as BaseEmail })
const newAttempts = (email.attempts || 0) + 1 const newAttempts = ((email as any).attempts || 0) + 1
await this.payload.update({ await this.payload.update({
collection: this.emailsCollection as any, collection: this.emailsCollection as any,

View File

@@ -76,7 +76,6 @@ export interface MailingPluginConfig {
templateEngine?: TemplateEngine templateEngine?: TemplateEngine
richTextEditor?: RichTextField['editor'] richTextEditor?: RichTextField['editor']
beforeSend?: BeforeSendHook beforeSend?: BeforeSendHook
onReady?: (payload: any) => Promise<void>
initOrder?: 'before' | 'after' initOrder?: 'before' | 'after'
} }

View File

@@ -1,4 +1,5 @@
import type { Payload } from 'payload' import type { Payload } from 'payload'
import { createContextLogger } from './logger.js'
/** /**
* Processes a single email by ID using the mailing service * Processes a single email by ID using the mailing service
@@ -29,6 +30,39 @@ export async function processEmailById(payload: Payload, emailId: string): Promi
await mailingContext.service.processEmailItem(emailId) await mailingContext.service.processEmailItem(emailId)
} }
/**
* Processes a job immediately by finding and executing it
* @param payload Payload instance
* @param jobId The ID of the job to run immediately
* @returns Promise that resolves when job is processed
*/
export async function processJobById(payload: Payload, jobId: string): Promise<void> {
const logger = createContextLogger(payload, 'PROCESSOR')
logger.debug(`Starting processJobById for job ${jobId}`)
if (!payload.jobs) {
throw new Error('PayloadCMS jobs not configured - cannot process job immediately')
}
try {
logger.debug(`Running job ${jobId} with payload.jobs.run()`)
// Run a specific job by its ID (using where clause to find the job)
const result = await payload.jobs.run({
where: {
id: {
equals: jobId
}
}
})
logger.info(`Job ${jobId} execution completed`, { result })
} catch (error) {
logger.error(`Job ${jobId} execution failed:`, error)
throw new Error(`Failed to process job ${jobId}: ${String(error)}`)
}
}
/** /**
* Processes all pending and failed emails using the mailing service * Processes all pending and failed emails using the mailing service
* @param payload Payload instance * @param payload Payload instance

View File

@@ -36,6 +36,44 @@ export const parseAndValidateEmails = (emails: string | string[] | null | undefi
return emailList return emailList
} }
/**
* Sanitize display names to prevent email header injection
* Removes newlines, carriage returns, and control characters
* @param displayName - The display name to sanitize
* @param escapeQuotes - Whether to escape quotes (for email headers)
* @returns Sanitized display name
*/
export const sanitizeDisplayName = (displayName: string, escapeQuotes = false): string => {
if (!displayName) return displayName
let sanitized = displayName
.trim()
// Remove/replace newlines and carriage returns to prevent header injection
.replace(/[\r\n]/g, ' ')
// Remove control characters (except space and printable characters)
.replace(/[\x00-\x1F\x7F-\x9F]/g, '')
// Escape quotes if needed (for email headers)
if (escapeQuotes) {
sanitized = sanitized.replace(/"/g, '\\"')
}
return sanitized
}
/**
* Sanitize and validate fromName for emails
* Wrapper around sanitizeDisplayName for consistent fromName handling
* @param fromName - The fromName to sanitize
* @returns Sanitized fromName or undefined if empty after sanitization
*/
export const sanitizeFromName = (fromName: string | null | undefined): string | undefined => {
if (!fromName) return undefined
const sanitized = sanitizeDisplayName(fromName, false)
return sanitized.length > 0 ? sanitized : undefined
}
export const getMailing = (payload: Payload) => { export const getMailing = (payload: Payload) => {
const mailing = (payload as any).mailing const mailing = (payload as any).mailing
if (!mailing) { if (!mailing) {

160
src/utils/jobScheduler.ts Normal file
View File

@@ -0,0 +1,160 @@
import type { Payload } from 'payload'
import { createContextLogger } from './logger.js'
/**
* Finds existing processing jobs for an email
*/
export async function findExistingJobs(
payload: Payload,
emailId: string | number
): Promise<{ docs: any[], totalDocs: number }> {
return await payload.find({
collection: 'payload-jobs',
where: {
'input.emailId': {
equals: String(emailId),
},
task: {
equals: 'process-email',
},
},
limit: 10,
})
}
/**
* Ensures a processing job exists for an email
* Creates one if it doesn't exist, or returns existing job IDs
*
* This function is idempotent and safe for concurrent calls:
* - Uses atomic check-and-create pattern with retry logic
* - Multiple concurrent calls will only create one job
* - Database-level uniqueness prevents duplicate jobs
* - Race conditions are handled with exponential backoff retry
*/
export async function ensureEmailJob(
payload: Payload,
emailId: string | number,
options?: {
scheduledAt?: string | Date
queueName?: string
}
): Promise<{ jobIds: (string | number)[], created: boolean }> {
if (!payload.jobs) {
throw new Error('PayloadCMS jobs not configured - cannot create email job')
}
const normalizedEmailId = String(emailId)
const mailingContext = (payload as any).mailing
const queueName = options?.queueName || mailingContext?.config?.queue || 'default'
const logger = createContextLogger(payload, 'JOB_SCHEDULER')
logger.debug(`Ensuring job for email ${normalizedEmailId}`)
logger.debug(`Queue: ${queueName}, scheduledAt: ${options?.scheduledAt || 'immediate'}`)
// First, optimistically try to create the job
// If it fails due to uniqueness constraint, then check for existing jobs
// This approach minimizes the race condition window
try {
logger.debug(`Attempting to create new job for email ${normalizedEmailId}`)
// Attempt to create job - rely on database constraints for duplicate prevention
const job = await payload.jobs.queue({
queue: queueName,
task: 'process-email',
input: {
emailId: normalizedEmailId
},
waitUntil: options?.scheduledAt ? new Date(options.scheduledAt) : undefined
})
logger.info(`Auto-scheduled processing job ${job.id} for email ${normalizedEmailId}`)
logger.debug(`Job details`, {
jobId: job.id,
emailId: normalizedEmailId,
scheduledAt: options?.scheduledAt || 'immediate',
task: 'process-email',
queue: queueName
})
return {
jobIds: [job.id],
created: true
}
} catch (createError) {
logger.warn(`Job creation failed for email ${normalizedEmailId}: ${String(createError)}`)
// Job creation failed - likely due to duplicate constraint or system issue
// Check if duplicate jobs exist (handles race condition where another process created job)
const existingJobs = await findExistingJobs(payload, normalizedEmailId)
logger.debug(`Found ${existingJobs.totalDocs} existing jobs after creation failure`)
if (existingJobs.totalDocs > 0) {
// Found existing jobs - return them (race condition handled successfully)
logger.info(`Using existing jobs for email ${normalizedEmailId}: ${existingJobs.docs.map(j => j.id).join(', ')}`)
return {
jobIds: existingJobs.docs.map(job => job.id),
created: false
}
}
// No existing jobs found - this is a genuine error
// Enhanced error context for better debugging
const errorMessage = String(createError)
const isLikelyUniqueConstraint = errorMessage.toLowerCase().includes('duplicate') ||
errorMessage.toLowerCase().includes('unique') ||
errorMessage.toLowerCase().includes('constraint')
if (isLikelyUniqueConstraint) {
// This should not happen if our check above worked, but provide a clear error
logger.error(`Unique constraint violation but no existing jobs found for email ${normalizedEmailId}`)
throw new Error(
`Database uniqueness constraint violation for email ${normalizedEmailId}, but no existing jobs found. ` +
`This indicates a potential data consistency issue. Original error: ${errorMessage}`
)
}
// Non-constraint related error
logger.error(`Non-constraint job creation error for email ${normalizedEmailId}: ${errorMessage}`)
throw new Error(`Failed to create job for email ${normalizedEmailId}: ${errorMessage}`)
}
}
/**
* Updates an email document to include job IDs in the relationship field
*/
export async function updateEmailJobRelationship(
payload: Payload,
emailId: string | number,
jobIds: (string | number)[],
collectionSlug: string = 'emails'
): Promise<void> {
try {
const normalizedEmailId = String(emailId)
const normalizedJobIds = jobIds.map(id => String(id))
// Get current jobs to avoid overwriting
const currentEmail = await payload.findByID({
collection: collectionSlug,
id: normalizedEmailId,
})
const currentJobs = (currentEmail.jobs || []).map((job: any) => String(job))
const allJobs = [...new Set([...currentJobs, ...normalizedJobIds])] // Deduplicate with normalized strings
await payload.update({
collection: collectionSlug,
id: normalizedEmailId,
data: {
jobs: allJobs
}
})
} catch (error) {
const normalizedEmailId = String(emailId)
const logger = createContextLogger(payload, 'JOB_SCHEDULER')
logger.error(`Failed to update email ${normalizedEmailId} with job relationship:`, error)
throw error
}
}

48
src/utils/logger.ts Normal file
View File

@@ -0,0 +1,48 @@
import type { Payload } from 'payload'
let pluginLogger: any = null
/**
* Get or create the plugin logger instance
* Uses PAYLOAD_MAILING_LOG_LEVEL environment variable to configure log level
* Defaults to 'info' if not set
*/
export function getPluginLogger(payload: Payload) {
if (!pluginLogger && payload.logger) {
const logLevel = process.env.PAYLOAD_MAILING_LOG_LEVEL || 'info'
pluginLogger = payload.logger.child({
level: logLevel,
plugin: '@xtr-dev/payload-mailing'
})
// Log the configured log level on first initialization
pluginLogger.info(`Logger initialized with level: ${logLevel}`)
}
// Fallback to console if logger not available (shouldn't happen in normal operation)
if (!pluginLogger) {
return {
debug: (...args: any[]) => console.log('[MAILING DEBUG]', ...args),
info: (...args: any[]) => console.log('[MAILING INFO]', ...args),
warn: (...args: any[]) => console.warn('[MAILING WARN]', ...args),
error: (...args: any[]) => console.error('[MAILING ERROR]', ...args),
}
}
return pluginLogger
}
/**
* Create a context-specific logger for a particular operation
*/
export function createContextLogger(payload: Payload, context: string) {
const logger = getPluginLogger(payload)
return {
debug: (message: string, ...args: any[]) => logger.debug(`[${context}] ${message}`, ...args),
info: (message: string, ...args: any[]) => logger.info(`[${context}] ${message}`, ...args),
warn: (message: string, ...args: any[]) => logger.warn(`[${context}] ${message}`, ...args),
error: (message: string, ...args: any[]) => logger.error(`[${context}] ${message}`, ...args),
}
}