Vision analysis addresses the critical gap that text-only scraping
misses ~40% of clinic website information (founding year in banners,
doctor photos, certification marks, social icons in images).
Sprint 0 adds: Firecrawl screenshot → Gemini Vision → structured
data extraction for founding year, doctors, certifications, services,
social icons, floating buttons, brand colors, slogans.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Added A4 parallel Firecrawl call with actions: [wait 3s, scrape]
to execute JavaScript and extract social button href URLs from
header/footer. This is the most reliable source — most Korean
clinics have Facebook/Instagram/YouTube/Blog icons in their nav.
Results merged as Source 3 (buttonHandles) alongside HTML links,
JSON extraction, and API searches.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WP-1: YouTube channel ID regex {20,} → {22} (exactly 24 chars)
WP-2: Naver Place category filtering in enrich-channels (성형/피부)
WP-3: Google Maps stores mapsUrl separately from clinicWebsite
WP-4: Naver Blog separates officialBlogUrl from search results
WP-5: 강남언니 rawRating + normalized rating (≤5 → ×2), Firecrawl
prompt explicitly states "out of 10, NOT out of 5"
WP-6: Perplexity model centralized in _shared/config.ts (env override)
WP-7: Apify Instagram timeout 30s → 45s
Frontend: transformReport uses mapsUrl and officialBlogUrl when available
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comprehensive audit of discover→collect→generate pipeline found:
- 16 silent failures, 8 data quality issues, 0 error recovery, 6 API issues
- Organized into 4 sprints (15 WPs, ~11h total)
- Each WP has file locations, changes, and verification criteria
- Checkbox format for progress tracking
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Google Maps: was using gm.website (clinic's own site) → now always
generates maps.google.com/search URL
Naver Blog: was linking to first search result post (random personal
blog) → now links to Naver blog search results page
Naver Place: np.link was the clinic's own website, not Naver Place →
now generates map.naver.com search URL. Also fixed collect-channel-data
to search with "성형외과" suffix and match by category (성형/피부) to
avoid same-name dental clinics.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Facebook pageName now links to facebook.com/{url} with ExternalLink icon.
OtherChannels: moved ExternalLink from right-end to inline with channel
name, matching the Instagram/YouTube pattern.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
staffCount is the number of registered doctors from 강남언니, not total
staff. Label changed from 의료진(medical staff) to 전문의(specialists)
to accurately reflect the data source. Diagnosis message also updated.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
YouTubeAudit: handle links to youtube.com/@{handle} with ExternalLink icon
InstagramAudit: handle links to instagram.com/{handle} with ExternalLink icon
ClinicSnapshot: domain is now clickable link, phone is tel: link
OtherChannels: Google Maps generates search URL, Naver Blog links to
first blog post or search results (previously empty string)
transformReport: fills missing URL fields for Google Maps and Naver Blog
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Includes every prompt in production across 3 pipeline phases:
- Phase 1 (discover): 7 data sources with exact prompts
- Phase 2 (collect): 5 API calls + 4 market analysis queries
- Phase 3 (generate): full report generation prompt template
Added prompt engineering learnings:
- Short prompts outperform long ones on Perplexity sonar
- sonar > sonar-pro for channel search
- English clinic name in parentheses improves international results
- Verify strategy: keep unverified handles as candidates
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Instagram HEAD requests often fail (rate limiting, blocking) causing
valid handles to be dropped. Now all discovered handles are kept
(verified or not) and Apify attempts collection on all of them.
Apify's own scraper validates existence more reliably than HEAD requests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split queries performed worse. The proven working pattern is:
- Single query with Korean+English clinic name
- "검색해서 찾아줘. 검색 결과에서 발견된 계정을 모두 알려줘" phrasing
- All channels in one request
- English name in parentheses helps Perplexity find international accounts
Tested: "그랜드성형외과 (Grand Plastic Surgery)" → finds Instagram,
YouTube, Facebook, TikTok, Naver Blog all in one call.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Single mega-query returns empty results. Split into:
B4a. Instagram + YouTube (most important, focused search)
B4b. Facebook + TikTok + Naver Blog + Kakao
B4c. 강남언니 + review platforms
Each query is short and focused — matches the proven pattern of
2-5 keyword searches that Perplexity handles well.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Long system prompt caused sonar-pro to return empty results.
Reverted to sonar model with short, proven prompt pattern that
matches the user's successful manual test.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaced simple "find handles" prompt with comprehensive research agent:
- Model: sonar → sonar-pro (advanced multi-step web search)
- System prompt: full research methodology with 2-3 keyword searches,
URL fetching, quantitative data extraction
- Output: structured JSON with channels (handles + follower counts +
subscriber counts) + platforms (강남언니 rating, reviews)
- Research results saved to scrape_data.onlinePresenceResearch for
downstream use in collect-channel-data and generate-report
Added _shared/researchPrompt.ts with prompt template + builder.
Updated agent documentation in doc/.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
B4 Perplexity: rewrote from narrow "find social accounts" to broad
"Online Presence 종합 분석" — finds Instagram, YouTube, Facebook,
TikTok, Naver, Kakao, 강남언니, 바비톡 in one query.
B5 Apify Instagram: generates handle candidates from clinic name
(english name, domain, _official, _ps, _clinic variants) and directly
checks each via Apify instagram-profile-scraper. Finds accounts that
web search misses.
Removed redundant B4b (platform presence) — now merged into B4.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
API results may contain null, numbers, or objects instead of strings.
Now coerces all values to strings before processing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaced Perplexity-only approach with 5 parallel direct API searches:
B1. YouTube Data API: search?type=channel&q={clinicName} → find channel
B2a. Naver Blog API: search blog.json → find official Naver blog
B2b. Naver Web API: search webkr.json → find Instagram/YouTube/Facebook URLs
B3. Firecrawl Search: web search → extract social URLs from results
B4. Perplexity: supplement — catch what direct APIs missed
All 5 sources run in parallel after Stage A (Firecrawl scrape for clinicName).
Results merged + deduplicated + verified. Perplexity is now a fallback,
not the primary source.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Perplexity prompts changed from "find verified accounts" (returns all
null) to "search and report what you find" (returns actual handles).
Added clinicName resolution: Firecrawl Korean → English → Perplexity
URL-to-name lookup → domain fallback.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously Firecrawl and Perplexity ran in parallel, so Perplexity
received raw URL instead of clinic name → poor search results.
Now:
Stage A: Firecrawl scrape+map (parallel) → extract clinicName from HTML
Stage B: Perplexity searches using extracted clinicName → finds Instagram,
YouTube, Facebook handles that Firecrawl HTML parsing missed
Stage C: Merge 3 sources + verify all handles
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
discover-channels: extractHandle('youtube') now detects UC* channel IDs
and returns them without @ prefix (previously @UC... caused verify fail)
verifyHandles: verifyYouTube uses cleanHandle for UC* check, requests
part=id,snippet for richer data
collect-channel-data: if channelId missing but handle present, resolves
via forHandle/forUsername lookup or direct UC* detection before skipping
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Halved blob movement (30px→15px, 8vw→4vw) and reduced scale
(1.1→1.05). Using translate3d forces GPU compositing layer,
preventing main-thread layout recalculation that causes jitter.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Added will-change:transform, contain:layout style, and
backface-visibility:hidden to all blob animations. This promotes
blobs to their own GPU compositing layer, preventing them from
triggering main-thread reflow/repaint that causes page shaking.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Animated blobs with translate+scale overflow their containers,
causing horizontal scrollbar to flicker and page to jitter.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Uses scrollbar-width:thin + 6px webkit scrollbar to minimize
layout shift from the default ~15px scrollbar. Keeps original
design intact while reducing visual asymmetry.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reverts to the original design with full-color button (no disabled
opacity), original background gradients, and original blob decorations.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
discover-channels: new extractHandle() validates each handle belongs to
its platform (rejects hospital-internal URLs like /idtube/view being
treated as YouTube). Extracts handles from full URLs correctly.
collect-channel-data: explicit Record<string,unknown> typing for DB JSON
fields — fixes TypeScript property access on VerifiedChannels from DB.
verifyHandles: fix TikTok double-URL concatenation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All layout changes (blob modifications, overflow-x/clip, scrollbar-gutter)
reverted to the original version that was working correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removes scrollbar-gutter:stable which pushed content left.
overlay scrollbar floats over content with zero layout impact.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Modules.tsx: wrap blobs in overflow-clip container, use max-w instead
of min-w to prevent viewport overflow
- index.css: add #root { overflow-x:clip; max-width:100vw } as final
safety net — no child element can expand beyond viewport
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
overflow-x:hidden doesn't clip CSS blur() filter radius, allowing
blurred blobs to still cause horizontal scrollbar. overflow-x:clip
fully contains all rendered pixels including blur.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Loading steps back to English (Scanning website, Collecting data, etc.)
- Removed verified channels panel from loading screen
- Fixed blank report page: detect empty report JSON from DB and show
appropriate error message instead of rendering empty components
- Navigation state: only pass if report+metadata exist
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Animated blur blobs and absolute-positioned elements were overflowing
the viewport, creating a horizontal scrollbar that shifted all content.
Added overflow-x: hidden to html and body.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
buildKpiDashboard now reads channelEnrichment (real API data from
Phase 2) with fallback to channelAnalysis (AI-generated). YouTube
subscribers, Instagram followers, 강남언니 rating/reviews all use
verified data when available. Fixed || ?? operator precedence.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously AI-provided kpiTargets (often only 3-4 items) would
completely replace our channel-based KPI generation. Now we always
build the full set (YouTube, Instagram, Naver, 강남언니, Google Maps,
cross-platform) and merge AI extras that don't overlap.
Also adds 강남언니 평점/리뷰, 네이버 플레이스 평점 as standard KPIs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restructured the entire analysis pipeline from AI-guessing social
handles to deterministic 3-phase discovery + collection + generation.
Phase 1 (discover-channels): 3-source channel discovery
- Firecrawl scrape: extract social links from HTML
- Perplexity search: find handles via web search
- URL regex parsing: deterministic link extraction
- Handle verification: HEAD requests + YouTube API
- DB: creates row with verified_channels + scrape_data
Phase 2 (collect-channel-data): 9 parallel data collectors
- Instagram (Apify), YouTube (Data API v3), Facebook (Apify)
- 강남언니 (Firecrawl), Naver Blog + Place (Naver API)
- Google Maps (Apify), Market analysis (Perplexity 4x parallel)
- DB: stores ALL raw data in channel_data column
Phase 3 (generate-report): AI report from real data
- Reads channel_data + analysis_data from DB
- Builds channel summary with real metrics
- AI generates report using only verified data
- V1 backwards compatibility preserved (url-based flow)
Supporting changes:
- DB migration: status, verified_channels, channel_data columns
- _shared/extractSocialLinks.ts: regex-based social link parser
- _shared/verifyHandles.ts: multi-platform handle verifier
- AnalysisLoadingPage: real 3-phase progress + channel panel
- useReport: channel_data column support + V2 enrichment merge
- 강남언니 rating: auto-correct 5→10 scale + search fallback
- KPIDashboard: navigate() instead of <a href>
- Loading text: 20-30초 → 1-2분
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Naver Blog search: collect blog post results for clinic name (total count + top 10 posts)
- Naver Place search: collect place info (name, category, address, telephone)
- Multi-account Instagram: AI prompt requests all IG accounts (국내/해외)
- enrich-channels: process multiple IG handles with fallback per handle
- transformReport: merge multiple IG accounts into instagramAudit.accounts[]
- generate-report: socialHandles.instagram now array of handles
- Hero/CTA: transition-all → transition-shadow for instant click response
- Hero/CTA: disabled state when URL is empty (opacity-50 + cursor-not-allowed)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- enrich-channels: add 강남언니 scraping module (search + structured JSON extraction)
- Collects: rating/10, reviews, doctors with ratings, procedures, certifications
- transformReport: merge 강남언니 data into clinicSnapshot + otherChannels
- Updates lead doctor info, certifications, and review counts from real data
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- enrich-channels: Instagram fallback — auto-try _ps, .ps, _clinic suffixes when <100 followers
- enrich-channels: YouTube URL normalization via normalizeYouTubeChannel (handles /c/, /user/, @handle)
- enrich-channels: Google Maps multi-query search for better hit rate
- generate-report: AI-found social handles prioritized over Firecrawl scrape
- generate-report: Added socialMedia field to AI prompt for accurate handle discovery
- normalizeHandles: Added normalizeYouTubeChannel for /c/, /user/, /channel/, @handle URLs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add creatomateVideoGen.ts service with polling-based async rendering
- Replace video stub (setTimeout) with actual Creatomate API calls
- Add video preview (<video> tag) and MP4 download support
- Build programmatic source (branded slideshow) without pre-built templates
- Error handling: auth, rate limit, render failure → Korean messages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Haewon Kam