endery/voxblog
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: endery:197fd69ce3248422361237244404fb23d2ab72ab
Branches Tags
endery:main
..
compare: endery:3896f8cad71c336f4b35203e725bade149a4fb17
Branches Tags
endery:main

No commits in common. "197fd69ce3248422361237244404fb23d2ab72ab" and "3896f8cad71c336f4b35203e725bade149a4fb17" have entirely different histories.
197fd69ce3 ... 3896f8cad7

5 changed files with 24 additions and 533 deletions
Show Stats Expand all files Collapse all files

188
apps/admin/STREAMING_FIX.md
View File

@ -1,188 +0,0 @@
# Streaming Content Accumulation Fix
## Problem
The streaming content was showing individual tokens instead of the accumulated HTML article because of a **stale closure** issue.
### What Was Happening
```typescript
// ❌ WRONG - Stale closure
onContent: (data) => {
onSetStreamingContent(streamingContent + data.delta);
// streamingContent is captured from the initial render
// It never updates in this callback!
}
```
**Result**: Each delta was added to the INITIAL empty string, not the accumulated content.
```
Delta 1: "" + "TypeScript" = "TypeScript"
Delta 2: "" + " is" = " is" // Lost "TypeScript"!
Delta 3: "" + " a" = " a" // Lost everything!
```
## Solution
Use a **ref** to track the accumulated content outside the closure:
```typescript
// ✅ CORRECT - Ref accumulation
const contentBufferRef = useRef<string>('');
onContent: (data) => {
contentBufferRef.current += data.delta;
onSetStreamingContent(contentBufferRef.current);
}
```
**Result**: Each delta is properly accumulated.
```
Delta 1: "" + "TypeScript" = "TypeScript"
Delta 2: "TypeScript" + " is" = "TypeScript is"
Delta 3: "TypeScript is" + " a" = "TypeScript is a"
```
## Additional Improvements
### 1. **Auto-Scroll**
```typescript
const streamingBoxRef = useRef<HTMLDivElement>(null);
useEffect(() => {
if (isGenerating && streamingContent && streamingBoxRef.current) {
streamingBoxRef.current.scrollTop = streamingBoxRef.current.scrollHeight;
}
}, [streamingContent, isGenerating]);
```
Automatically scrolls to show new content as it arrives.
### 2. **Smooth Scrolling**
```typescript
sx={{
scrollBehavior: 'smooth',
// ...
}}
```
Smooth animation when auto-scrolling.
### 3. **Better HTML Styling**
```typescript
'& h2, & h3': { mt: 2, mb: 1 },
'& p': { mb: 1 },
'& ul, & ol': { pl: 3, mb: 1 },
'& li': { mb: 0.5 },
```
Proper spacing for HTML elements.
## Technical Explanation
### Closure Problem
In JavaScript, when you create a callback function, it "closes over" the variables from its surrounding scope. Those variables are captured at the time the function is created.
```typescript
const [content, setContent] = useState('');
// This callback is created once when component mounts
const callback = () => {
console.log(content); // Always logs the INITIAL value!
};
```
### Why Refs Work
Refs are **mutable objects** that persist across renders:
```typescript
const ref = useRef('');
// This always reads the CURRENT value
const callback = () => {
console.log(ref.current); // Gets the latest value!
};
```
## Before vs After
### Before (Broken)
```
User sees:
"TypeScript"
" is"
" a"
"powerful"
...individual tokens, no accumulation
```
### After (Fixed)
```
User sees:
"TypeScript"
"TypeScript is"
"TypeScript is a"
"TypeScript is a powerful"
...proper accumulation, rendered as HTML
```
## Files Changed
```
✅ apps/admin/src/components/steps/StepGenerate.tsx
- Added contentBufferRef for accumulation
- Added streamingBoxRef for auto-scroll
- Added useEffect for auto-scroll
- Fixed onContent callback to use ref
- Added smooth scroll behavior
```
## Testing
1. Start generation
2. Watch the "Live Generation" box
3. Should see:
- ✅ Proper HTML rendering (headings, paragraphs, lists)
- ✅ Content accumulating (not individual tokens)
- ✅ Auto-scroll to bottom
- ✅ Smooth animations
- ✅ Pulsing blue border
## Common React Pitfalls
This is a classic React pitfall that catches many developers:
### ❌ Don't Do This
```typescript
const [value, setValue] = useState(0);
setInterval(() => {
setValue(value + 1); // Always adds to initial value!
}, 1000);
```
### ✅ Do This Instead
```typescript
const [value, setValue] = useState(0);
setInterval(() => {
setValue(prev => prev + 1); // Functional update
}, 1000);
// OR use a ref
const valueRef = useRef(0);
setInterval(() => {
valueRef.current += 1;
setValue(valueRef.current);
}, 1000);
```
## Conclusion
The fix ensures that streaming content properly accumulates and displays as formatted HTML, providing a smooth, professional user experience with auto-scrolling and proper rendering.
**Status**: ✅ Fixed and working!

263
apps/admin/STREAMING_PERSISTENCE.md
View File

@ -1,263 +0,0 @@
# Streaming Persistence Across Navigation
## Problem Solved
**Before**: When navigating away from the Generate step during AI generation, the streaming would stop and all progress would be lost.
**After**: Streaming continues in the background even when you navigate to other steps. Content is preserved and available when you return.
## How It Works
### State Management Architecture
```
usePostEditor Hook (Persistent)
↓
├─ isGenerating: boolean
├─ streamingContent: string
├─ tokenCount: number
└─ generationError: string
↓
EditorShell (Parent)
↓
StepGenerate (Child)
```
### Key Changes
#### 1. **Lifted State to Hook** (`usePostEditor.ts`)
```typescript
// Streaming state (persisted across navigation)
const [isGenerating, setIsGenerating] = useState(false);
const [streamingContent, setStreamingContent] = useState('');
const [tokenCount, setTokenCount] = useState(0);
const [generationError, setGenerationError] = useState<string>('');
```
#### 2. **Passed Through EditorShell** (`EditorShell.tsx`)
```typescript
<StepGenerate
// ... other props
isGenerating={isGenerating}
streamingContent={streamingContent}
tokenCount={tokenCount}
generationError={generationError}
onSetIsGenerating={setIsGenerating}
onSetStreamingContent={setStreamingContent}
onSetTokenCount={setTokenCount}
onSetGenerationError={setGenerationError}
/>
```
#### 3. **Used in Component** (`StepGenerate.tsx`)
```typescript
// No longer local state - uses props from hook
const {
isGenerating,
streamingContent,
tokenCount,
generationError,
onSetIsGenerating,
onSetStreamingContent,
onSetTokenCount,
onSetGenerationError
} = props;
```
## User Experience
### Scenario 1: Navigate During Streaming
1. **Start generation** on Generate step
2. **See content streaming** in real-time
3. **Navigate to Assets step** to check something
4. **Generation continues** in background
5. **Return to Generate step**
6. **See completed content** or ongoing stream
### Scenario 2: Check Other Steps While Generating
```
Step 0: Assets ← Navigate here
Step 1: AI Prompt ← Or here
Step 2: Generate ← Streaming continues here
Step 3: Edit ← Or here
Step 4: Metadata
Step 5: Publish
```
**Result**: Generation keeps running, content preserved
### Scenario 3: Long Generation
- Start 2000-word article generation (~60 seconds)
- Navigate to Edit step to prepare
- Navigate to Metadata to plan tags
- Return to Generate step
- Content is complete and ready!
## Technical Benefits
### 1. **State Persistence**
- State lives in `usePostEditor` hook
- Hook persists across step navigation
- Only unmounts when leaving entire editor
### 2. **Background Processing**
- Streaming API continues regardless of UI
- Server-Sent Events connection stays open
- Content accumulates in hook state
### 3. **No Data Loss**
- Partial content preserved if navigation occurs
- Token count maintained
- Error state preserved
- Can resume viewing at any time
### 4. **Better UX**
- Don't have to wait on Generate step
- Can multitask while AI generates
- No accidental cancellation
- Flexible workflow
## Implementation Details
### State Flow During Streaming
```
1. User clicks "Generate Draft"
↓
2. onSetIsGenerating(true) in hook
↓
3. Stream starts, chunks arrive
↓
4. onSetStreamingContent(content + delta)
↓
5. User navigates to another step
↓
6. StepGenerate unmounts (local state lost)
BUT hook state persists!
↓
7. Stream continues, updating hook state
↓
8. User returns to Generate step
↓
9. StepGenerate remounts with current hook state
↓
10. Sees current streaming content or final result
```
### Memory Management
- **Hook state**: ~few KB for content string
- **Streaming connection**: Maintained by browser
- **Cleanup**: Automatic when leaving editor
- **No memory leaks**: State cleared on unmount
## Edge Cases Handled
### 1. **Navigation During Stream**
✅ Stream continues
✅ Content preserved
✅ Can return anytime
### 2. **Error During Stream**
✅ Error state preserved
✅ Visible when returning
✅ Can retry generation
### 3. **Multiple Generations**
✅ Previous content cleared on new generation
✅ State reset properly
✅ No conflicts
### 4. **Browser Refresh**
❌ Stream lost (expected - SSE connection closed)
✅ Last saved draft preserved in database
✅ Can regenerate if needed
## Comparison
### Before (Local State)
```typescript
// In StepGenerate.tsx
const [isGenerating, setIsGenerating] = useState(false);
const [streamingContent, setStreamingContent] = useState('');
// ❌ Lost on navigation
// ❌ Stream stops
// ❌ Progress lost
```
### After (Hook State)
```typescript
// In usePostEditor.ts
const [isGenerating, setIsGenerating] = useState(false);
const [streamingContent, setStreamingContent] = useState('');
// ✅ Persists across navigation
// ✅ Stream continues
// ✅ Progress preserved
```
## Testing
### Test Case 1: Basic Persistence
1. Start generation
2. Wait 5 seconds (partial content)
3. Navigate to Assets
4. Navigate back to Generate
5. **Expected**: See partial content, stream continuing
### Test Case 2: Complete During Navigation
1. Start generation
2. Navigate away immediately
3. Wait 60 seconds
4. Navigate back to Generate
5. **Expected**: See complete content
### Test Case 3: Error Handling
1. Disconnect network
2. Start generation
3. Navigate away
4. Navigate back
5. **Expected**: See error message
## Future Enhancements
### 1. **Visual Indicator**
Show streaming status in step navigation:
```
Step 2: Generate ⚡ (Streaming...)
```
### 2. **Notification on Complete**
Toast notification when generation completes while on another step:
```
✅ Article generation complete! (2,456 tokens)
```
### 3. **Progress in Sidebar**
Show live progress in sidebar:
```
┌─────────────────────┐
│ AI Generation │
│ ▓▓▓▓▓▓▓░░░░░░░░░ │
│ 1,234 tokens │
└─────────────────────┘
```
### 4. **Pause/Resume**
Add ability to pause streaming:
```typescript
const [isPaused, setIsPaused] = useState(false);
// Pause SSE consumption, resume later
```
## Conclusion
The streaming persistence feature provides a seamless, flexible workflow where users can multitask during long AI generations without losing progress. The implementation is clean, using React's built-in state management patterns and requiring minimal changes to the existing codebase.
**Status**: ✅ Fully implemented and tested
**Impact**: Significantly improved UX for long-running generations
**Complexity**: Low (simple state lifting pattern)

16
apps/admin/src/components/EditorShell.tsx
View File

@ -33,10 +33,6 @@ export default function EditorShell({ onLogout: _onLogout, initialPostId, onBack
generatedDraft, generatedDraft,
imagePlaceholders, imagePlaceholders,
generationSources, generationSources,
isGenerating,
streamingContent,
tokenCount,
generationError,
// setters // setters
setDraft, setDraft,
setMeta, setMeta,
@ -47,10 +43,6 @@ export default function EditorShell({ onLogout: _onLogout, initialPostId, onBack
setGeneratedDraft, setGeneratedDraft,
setImagePlaceholders, setImagePlaceholders,
setGenerationSources, setGenerationSources,
setIsGenerating,
setStreamingContent,
setTokenCount,
setGenerationError,
// actions // actions
savePost, savePost,
deletePost, deletePost,
@ -180,14 +172,6 @@ export default function EditorShell({ onLogout: _onLogout, initialPostId, onBack
setGenerationSources(sources); setGenerationSources(sources);
void savePost({ generationSources: sources }); void savePost({ generationSources: sources });
}} }}
isGenerating={isGenerating}
streamingContent={streamingContent}
tokenCount={tokenCount}
generationError={generationError}
onSetIsGenerating={setIsGenerating}
onSetStreamingContent={setStreamingContent}
onSetTokenCount={setTokenCount}
onSetGenerationError={setGenerationError}
/> />
</StepContainer> </StepContainer>
)} )}

76
apps/admin/src/components/steps/StepGenerate.tsx
View File

@ -1,4 +1,4 @@
import { useState, useRef, useEffect } from 'react'; import { useState } from 'react';
import { Box, Stack, TextField, Typography, Button, Alert, CircularProgress, FormControlLabel, Checkbox, Link, LinearProgress } from '@mui/material'; import { Box, Stack, TextField, Typography, Button, Alert, CircularProgress, FormControlLabel, Checkbox, Link, LinearProgress } from '@mui/material';
import SelectedImages from './SelectedImages'; import SelectedImages from './SelectedImages';
import CollapsibleSection from './CollapsibleSection'; import CollapsibleSection from './CollapsibleSection';
@ -21,14 +21,6 @@ export default function StepGenerate({
onGeneratedDraft, onGeneratedDraft,
onImagePlaceholders, onImagePlaceholders,
onGenerationSources, onGenerationSources,
isGenerating,
streamingContent,
tokenCount,
generationError,
onSetIsGenerating,
onSetStreamingContent,
onSetTokenCount,
onSetGenerationError,
}: { }: {
postClips: Clip[]; postClips: Clip[];
genImageKeys: string[]; genImageKeys: string[];
@ -43,27 +35,13 @@ export default function StepGenerate({
onGeneratedDraft: (content: string) => void; onGeneratedDraft: (content: string) => void;
onImagePlaceholders: (placeholders: string[]) => void; onImagePlaceholders: (placeholders: string[]) => void;
onGenerationSources: (sources: Array<{ title: string; url: string }>) => void; onGenerationSources: (sources: Array<{ title: string; url: string }>) => void;
isGenerating: boolean;
streamingContent: string;
tokenCount: number;
generationError: string;
onSetIsGenerating: (v: boolean) => void;
onSetStreamingContent: (v: string) => void;
onSetTokenCount: (v: number) => void;
onSetGenerationError: (v: string) => void;
}) { }) {
const [generating, setGenerating] = useState(false);
const [error, setError] = useState<string>('');
const [useWebSearch, setUseWebSearch] = useState(false); const [useWebSearch, setUseWebSearch] = useState(false);
const [streamingContent, setStreamingContent] = useState('');
const [tokenCount, setTokenCount] = useState(0);
const [useStreaming, setUseStreaming] = useState(true); const [useStreaming, setUseStreaming] = useState(true);
const streamingBoxRef = useRef<HTMLDivElement>(null);
const contentBufferRef = useRef<string>('');
// Auto-scroll to bottom when streaming content updates
useEffect(() => {
if (isGenerating && streamingContent && streamingBoxRef.current) {
streamingBoxRef.current.scrollTop = streamingBoxRef.current.scrollHeight;
}
}, [streamingContent, isGenerating]);
return ( return (
<Box sx={{ display: 'grid', gap: 2 }}> <Box sx={{ display: 'grid', gap: 2 }}>
<StepHeader <StepHeader
@ -163,14 +141,13 @@ export default function StepGenerate({
size="large" size="large"
onClick={async () => { onClick={async () => {
if (!promptText.trim()) { if (!promptText.trim()) {
onSetGenerationError('Please provide an AI prompt'); setError('Please provide an AI prompt');
return; return;
} }
onSetIsGenerating(true); setGenerating(true);
onSetGenerationError(''); setError('');
onSetStreamingContent(''); setStreamingContent('');
onSetTokenCount(0); setTokenCount(0);
contentBufferRef.current = ''; // Reset buffer
try { try {
const transcriptions = postClips const transcriptions = postClips
@ -196,22 +173,20 @@ export default function StepGenerate({
console.log('Stream started:', data.requestId); console.log('Stream started:', data.requestId);
}, },
onContent: (data) => { onContent: (data) => {
// Accumulate in ref to avoid stale closure setStreamingContent(prev => prev + data.delta);
contentBufferRef.current += data.delta; setTokenCount(data.tokenCount);
onSetStreamingContent(contentBufferRef.current);
onSetTokenCount(data.tokenCount);
}, },
onDone: (data) => { onDone: (data) => {
console.log('Stream complete:', data.elapsedMs, 'ms'); console.log('Stream complete:', data.elapsedMs, 'ms');
onGeneratedDraft(data.content); onGeneratedDraft(data.content);
onImagePlaceholders(data.imagePlaceholders); onImagePlaceholders(data.imagePlaceholders);
onGenerationSources([]); onGenerationSources([]);
onSetStreamingContent(''); setStreamingContent('');
onSetIsGenerating(false); setGenerating(false);
}, },
onError: (data) => { onError: (data) => {
onSetGenerationError(data.error); setError(data.error);
onSetIsGenerating(false); setGenerating(false);
}, },
}); });
} else { } else {
@ -220,17 +195,17 @@ export default function StepGenerate({
onGeneratedDraft(result.content); onGeneratedDraft(result.content);
onImagePlaceholders(result.imagePlaceholders); onImagePlaceholders(result.imagePlaceholders);
onGenerationSources(result.sources || []); onGenerationSources(result.sources || []);
onSetIsGenerating(false); setGenerating(false);
} }
} catch (err: any) { } catch (err: any) {
onSetGenerationError(err?.message || 'Generation failed'); setError(err?.message || 'Generation failed');
onSetIsGenerating(false); setGenerating(false);
} }
}} }}
disabled={isGenerating || !promptText.trim()} disabled={generating || !promptText.trim()}
fullWidth fullWidth
> >
{isGenerating ? ( {generating ? (
<> <>
<CircularProgress size={20} sx={{ mr: 1 }} /> <CircularProgress size={20} sx={{ mr: 1 }} />
{useStreaming ? `Streaming... (${tokenCount} tokens)` : 'Generating Draft...'} {useStreaming ? `Streaming... (${tokenCount} tokens)` : 'Generating Draft...'}
@ -241,8 +216,8 @@ export default function StepGenerate({
'Generate Draft' 'Generate Draft'
)} )}
</Button> </Button>
{generationError && <Alert severity="error" sx={{ mt: 1 }}>{generationError}</Alert>} {error && <Alert severity="error" sx={{ mt: 1 }}>{error}</Alert>}
{isGenerating && useStreaming && ( {generating && useStreaming && (
<Box sx={{ mt: 2 }}> <Box sx={{ mt: 2 }}>
<LinearProgress /> <LinearProgress />
<Typography variant="caption" sx={{ color: 'text.secondary', mt: 0.5, display: 'block' }}> <Typography variant="caption" sx={{ color: 'text.secondary', mt: 0.5, display: 'block' }}>
@ -253,10 +228,9 @@ export default function StepGenerate({
</Box> </Box>
{/* Streaming Content Display (while generating) */} {/* Streaming Content Display (while generating) */}
{isGenerating && useStreaming && streamingContent && ( {generating && useStreaming && streamingContent && (
<CollapsibleSection title="Live Generation" defaultCollapsed={false}> <CollapsibleSection title="Live Generation" defaultCollapsed={false}>
<Box <Box
ref={streamingBoxRef}
sx={{ sx={{
p: 2, p: 2,
border: '2px solid', border: '2px solid',
@ -265,11 +239,9 @@ export default function StepGenerate({
bgcolor: 'background.paper', bgcolor: 'background.paper',
maxHeight: '500px', maxHeight: '500px',
overflowY: 'auto', overflowY: 'auto',
scrollBehavior: 'smooth',
'& h2, & h3': { mt: 2, mb: 1 }, '& h2, & h3': { mt: 2, mb: 1 },
'& p': { mb: 1 }, '& p': { mb: 1 },
'& ul, & ol': { pl: 3, mb: 1 }, '& ul, & ol': { pl: 3, mb: 1 },
'& li': { mb: 0.5 },
animation: 'pulse 2s ease-in-out infinite', animation: 'pulse 2s ease-in-out infinite',
'@keyframes pulse': { '@keyframes pulse': {
'0%, 100%': { borderColor: 'primary.main' }, '0%, 100%': { borderColor: 'primary.main' },

14
apps/admin/src/hooks/usePostEditor.ts
View File

@ -22,12 +22,6 @@ export function usePostEditor(initialPostId?: string | null) {
const [generationSources, setGenerationSources] = useState<Array<{ title: string; url: string }>>([]); const [generationSources, setGenerationSources] = useState<Array<{ title: string; url: string }>>([]);
const autoSaveTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null); const autoSaveTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
// Streaming state (persisted across navigation)
const [isGenerating, setIsGenerating] = useState(false);
const [streamingContent, setStreamingContent] = useState('');
const [tokenCount, setTokenCount] = useState(0);
const [generationError, setGenerationError] = useState<string>('');
useEffect(() => { useEffect(() => {
const savedId = initialPostId || localStorage.getItem('voxblog_post_id'); const savedId = initialPostId || localStorage.getItem('voxblog_post_id');
const savedLocal = localStorage.getItem('voxblog_draft'); const savedLocal = localStorage.getItem('voxblog_draft');
@ -199,10 +193,6 @@ export function usePostEditor(initialPostId?: string | null) {
generatedDraft, generatedDraft,
imagePlaceholders, imagePlaceholders,
generationSources, generationSources,
isGenerating,
streamingContent,
tokenCount,
generationError,
// setters // setters
setDraft, setDraft,
setMeta, setMeta,
@ -214,10 +204,6 @@ export function usePostEditor(initialPostId?: string | null) {
setGeneratedDraft, setGeneratedDraft,
setImagePlaceholders, setImagePlaceholders,
setGenerationSources, setGenerationSources,
setIsGenerating,
setStreamingContent,
setTokenCount,
setGenerationError,
// actions // actions
savePost, savePost,
deletePost, deletePost,