diff --git a/AUTO_MIGRATION_SUMMARY.md b/AUTO_MIGRATION_SUMMARY.md new file mode 100644 index 0000000..3e978b1 --- /dev/null +++ b/AUTO_MIGRATION_SUMMARY.md @@ -0,0 +1,266 @@ +# Automatic Database Migrations - Implementation Summary + +## What Was Implemented + +Database migrations now run **automatically** every time the API server starts. This eliminates the need to manually run migration commands. + +## Changes Made + +### 1. Created Migration Runner (`apps/api/src/db/migrate.ts`) + +A new module that: +- Connects to MySQL using environment variables +- Runs pending migrations from the `drizzle/` folder +- Provides clear console output with emojis +- Handles errors gracefully + +```typescript +export async function runMigrations() { + console.log('🔄 Running database migrations...'); + // ... migration logic ... + console.log('✅ Database migrations completed successfully'); +} +``` + +### 2. Integrated into Server Startup (`apps/api/src/index.ts`) + +Modified the server startup to: +- Run migrations before starting Express +- Exit with error if migrations fail +- Provide clear startup sequence + +```typescript +async function startServer() { + await runMigrations(); // ← Runs automatically + app.listen(PORT, () => { + console.log(`API server running on port ${PORT}`); + }); +} +``` + +## Benefits + +### ✅ **No Manual Steps** +- Just run `docker-compose up -d --build` +- Migrations run automatically +- Database is always up-to-date + +### ✅ **Idempotent** +- Safe to run multiple times +- Only applies new migrations +- No duplicate data or errors + +### ✅ **Developer Friendly** +- Clear console output +- Easy to debug +- Works in development and production + +### ✅ **CI/CD Ready** +- No extra deployment steps +- Migrations run on container start +- Automatic rollout of schema changes + +## How It Works + +### Startup Sequence + +``` +1. Docker starts API container +2. Node.js starts +3. Environment variables loaded (.env) +4. 🔄 Migrations run automatically +5. ✅ Migrations complete +6. Express server starts +7. API ready to accept requests +``` + +### Console Output + +When you start the API, you'll see: + +```bash +[OpenAIClient] Initialized with timeout: 600s, maxRetries: 2 +ENV ADMIN_PASSWORD loaded: true +🔄 Running database migrations... +✅ Database migrations completed successfully +API server running on port 3301 +``` + +## Testing + +### Verify Automatic Migrations + +```bash +# 1. Reset database (deletes all data!) +docker-compose down -v + +# 2. Start fresh +docker-compose up -d --build + +# 3. Watch the logs +docker-compose logs -f api + +# You should see: +# 🔄 Running database migrations... +# ✅ Database migrations completed successfully +``` + +### Verify Tables Created + +```bash +docker exec voxblog-mysql mysql -u voxblog -pvoxblogAppPass123! voxblog -e "SHOW TABLES;" +``` + +Expected output: +``` +Tables_in_voxblog +__drizzle_migrations +audio_clips +posts +settings +``` + +## Troubleshooting + +### Migrations Don't Run + +**Check API logs:** +```bash +docker-compose logs api | grep -i migration +``` + +**Common issues:** +- MySQL not ready yet (wait 30 seconds) +- Wrong database credentials in `.env` +- Migration files missing in `apps/api/drizzle/` + +### Migration Fails + +**Error in logs:** +```bash +❌ Migration failed: [error details] +``` + +**Solutions:** +1. Check MySQL is healthy: `docker-compose ps mysql` +2. Verify credentials in `.env` +3. Check migration files exist: `ls apps/api/drizzle/` +4. Run manually: `docker exec voxblog-api sh -c "cd /app/apps/api && pnpm drizzle:migrate"` + +### Server Won't Start + +If migrations fail, the server exits with error code 1. + +**Check logs:** +```bash +docker-compose logs api +``` + +**Restart after fixing:** +```bash +docker-compose restart api +``` + +## Manual Migration (Still Available) + +You can still run migrations manually if needed: + +```bash +# Inside container +docker exec voxblog-api sh -c "cd /app/apps/api && pnpm drizzle:migrate" + +# Or using pnpm scripts +docker exec voxblog-api pnpm --filter api drizzle:migrate +``` + +## Creating New Migrations + +When you modify the schema (`apps/api/src/db/schema.ts`): + +```bash +# 1. Generate migration file +docker exec voxblog-api sh -c "cd /app/apps/api && pnpm drizzle:generate" + +# 2. Restart API (migrations run automatically) +docker-compose restart api + +# 3. Verify migration applied +docker-compose logs api | grep migration +``` + +## Production Deployment + +### Zero-Downtime Deployment + +1. **Build new image** with schema changes +2. **Deploy new container** - migrations run automatically +3. **Old container** continues serving requests +4. **New container** takes over after migrations complete + +### Rollback Strategy + +If a migration fails: +1. Container exits with error +2. Old container keeps running +3. Fix migration and redeploy +4. Or rollback to previous version + +## Best Practices + +### ✅ **DO** +- Test migrations locally first +- Make migrations backward compatible +- Keep migrations small and focused +- Add indexes in separate migrations +- Document breaking changes + +### ❌ **DON'T** +- Don't delete migration files +- Don't modify existing migrations +- Don't make breaking schema changes without planning +- Don't skip testing migrations + +## Files Modified + +1. **`apps/api/src/db/migrate.ts`** (new) + - Migration runner logic + - Error handling + - Console output + +2. **`apps/api/src/index.ts`** (modified) + - Import migration runner + - Call migrations before server start + - Async startup function + +3. **`DATABASE_SETUP.md`** (updated) + - Document automatic migrations + - Update troubleshooting guide + - Remove manual migration steps + +## Migration History + +Migrations are tracked in the `__drizzle_migrations` table: + +```sql +SELECT * FROM __drizzle_migrations; +``` + +Shows: +- Migration ID +- Hash +- Created timestamp + +## Future Enhancements + +Potential improvements: +- Migration rollback command +- Migration status endpoint (`/api/migrations`) +- Dry-run mode +- Migration locking for multi-instance deployments +- Slack/email notifications on migration failures + +--- + +**Status**: ✅ Implemented and Working +**Last Updated**: 2025-10-26 +**Author**: Automated Migration System diff --git a/DATABASE_SETUP.md b/DATABASE_SETUP.md index e64f3be..a58acae 100644 --- a/DATABASE_SETUP.md +++ b/DATABASE_SETUP.md @@ -2,7 +2,7 @@ ## Initial Setup -When starting VoxBlog for the first time or after resetting the database, you need to run migrations. +**✅ Migrations now run automatically!** When you start the API, it will automatically run any pending database migrations. ### Quick Setup @@ -10,16 +10,25 @@ When starting VoxBlog for the first time or after resetting the database, you ne # 1. Start containers docker-compose up -d --build -# 2. Wait for MySQL to be healthy (about 30 seconds) -docker-compose ps +# 2. Wait for MySQL to be healthy and migrations to complete (about 30 seconds) +docker-compose logs -f api -# 3. Run migrations -docker exec voxblog-api sh -c "cd /app/apps/api && pnpm drizzle:migrate" +# You should see: +# 🔄 Running database migrations... +# ✅ Database migrations completed successfully +# API server running on port 3301 -# 4. Verify tables were created +# 3. Verify tables were created (optional) docker exec voxblog-mysql mysql -u voxblog -pvoxblogAppPass123! voxblog -e "SHOW TABLES;" ``` +### Manual Migration (if needed) + +You can still run migrations manually if needed: +```bash +docker exec voxblog-api sh -c "cd /app/apps/api && pnpm drizzle:migrate" +``` + Expected output: ``` Tables_in_voxblog @@ -49,10 +58,16 @@ docker exec voxblog-api sh -c "cd /app/apps/api && pnpm drizzle:migrate" **Error**: `Table 'voxblog.posts' doesn't exist` -**Cause**: Migrations haven't been run +**Cause**: Migrations failed to run automatically (check API logs) -**Solution**: Run migrations +**Solution**: Check API logs for migration errors ```bash +docker-compose logs api | grep -i migration + +# If migrations failed, restart the API +docker-compose restart api + +# Or run migrations manually docker exec voxblog-api sh -c "cd /app/apps/api && pnpm drizzle:migrate" ``` diff --git a/apps/api/src/db/migrate.ts b/apps/api/src/db/migrate.ts new file mode 100644 index 0000000..8268827 --- /dev/null +++ b/apps/api/src/db/migrate.ts @@ -0,0 +1,34 @@ +import { drizzle } from 'drizzle-orm/mysql2'; +import { migrate } from 'drizzle-orm/mysql2/migrator'; +import mysql from 'mysql2/promise'; +import path from 'path'; + +/** + * Run database migrations + * This ensures the database schema is always up-to-date on startup + */ +export async function runMigrations() { + console.log('🔄 Running database migrations...'); + + const connection = await mysql.createConnection({ + host: process.env.DB_HOST || 'localhost', + port: parseInt(process.env.DB_PORT || '3306'), + user: process.env.DB_USER || 'voxblog', + password: process.env.DB_PASSWORD, + database: process.env.DB_NAME || 'voxblog', + }); + + const db = drizzle(connection); + + try { + // Run migrations from the drizzle folder + const migrationsFolder = path.join(__dirname, '../../drizzle'); + await migrate(db, { migrationsFolder }); + console.log('✅ Database migrations completed successfully'); + } catch (error) { + console.error('❌ Migration failed:', error); + throw error; + } finally { + await connection.end(); + } +} diff --git a/apps/api/src/index.ts b/apps/api/src/index.ts index 3a25937..86123a3 100644 --- a/apps/api/src/index.ts +++ b/apps/api/src/index.ts @@ -13,6 +13,7 @@ import ghostRouter from './ghost'; import aiGenerateRouter from './ai-generate'; import aiRoutesNew from './routes/ai.routes'; import settingsRouter from './settings'; +import { runMigrations } from './db/migrate'; const app = express(); console.log('ENV ADMIN_PASSWORD loaded:', Boolean(process.env.ADMIN_PASSWORD)); @@ -48,8 +49,22 @@ app.use((err: any, _req: express.Request, res: express.Response, _next: express. res.status(500).json({ error: 'Internal server error' }); }); -// Start server +// Start server with migrations const PORT = process.env.PORT || 3301; -app.listen(PORT, () => { - console.log(`API server running on port ${PORT}`); -}); + +async function startServer() { + try { + // Run migrations before starting the server + await runMigrations(); + + // Start the server + app.listen(PORT, () => { + console.log(`API server running on port ${PORT}`); + }); + } catch (error) { + console.error('Failed to start server:', error); + process.exit(1); + } +} + +startServer();