SEO/AEO 키워드 온톨로지 서비스 초기 구현

발행된 사이트에 업체별 SEO/AEO 키워드를 제공하는 서비스.

- PostgreSQL 16 + pgvector/ltree/pg_trgm 단일 스토어
  (정확·의미·계층 조회를 한 엔진에서 처리)
- 키워드는 전역 사전 + merchant_keyword 연결 테이블 구조
- 4단계 계단식 중복제거: 금칙어 → normalized 완전일치 →
  pg_trgm → 코사인 유사도, 걸린 표기는 aliases[] 로 흡수
- BullMQ 생성 큐 (발행 즉시 / 일 1회 크론 / 성과 기반)
- OpenAI Structured Outputs + mock provider
  (API 키 없이 로컬 전 구간 동작)
- 서빙 API: /v1/sites/:id/seo, /aeo, /performance, /keywords/search
- docs/architecture.html 설계 도식

JSON-LD 조립과 o2o-site-AEO 연동은 후속 작업.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
hbyang 2026-09-09 11:56:34 +09:00
commit 209a381091
37 changed files with 8121 additions and 0 deletions

29
.env.example Normal file
View File

@ -0,0 +1,29 @@
# --- server ---
PORT=3100
# --- postgres (docker-compose 기본값) ---
DATABASE_URL=postgres://ontology:ontology@localhost:55432/ontology
# --- redis (BullMQ) ---
REDIS_HOST=localhost
REDIS_PORT=56379
# --- LLM ---
# mock : API 키 없이 로컬에서 전체 파이프라인 동작 (기본값)
# openai : 실제 OpenAI 호출
LLM_PROVIDER=mock
OPENAI_API_KEY=
OPENAI_MODEL=gpt-4.1-mini
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
# --- 생성/중복제거 튜닝 ---
# 코사인 유사도가 이 값 이상이면 기존 키워드에 alias 로 흡수
DEDUP_COSINE_THRESHOLD=0.92
# trigram 유사도 사전 필터
DEDUP_TRIGRAM_THRESHOLD=0.6
# 벡터 비교 대상 상위 후보 수
DEDUP_CANDIDATE_LIMIT=20
# 1회 생성 요청당 키워드 목표 개수
GENERATION_TARGET_KEYWORDS=15
# 주기 리프레시 간격(일)
REFRESH_INTERVAL_DAYS=30

5
.gitignore vendored Normal file
View File

@ -0,0 +1,5 @@
node_modules/
dist/
.env
*.tsbuildinfo
.DS_Store

133
README.md Normal file
View File

@ -0,0 +1,133 @@
# o2o-site-ontology
o2o-site-AEO 가 발행한 사이트에 **업체별 SEO/AEO 키워드**를 제공하는 온톨로지 서비스.
- 주기적으로 LLM 에게 업체 정보를 주고 키워드·태그·Q&A 를 생성
- **4단계 계단식 중복제거**로 전역 키워드 사전을 오염 없이 유지
- 발행 사이트는 REST 로 SEO/AEO payload 만 받아 쓴다 (JSON-LD 조립은 후속 단계)
## 빠른 시작 (로컬)
```bash
npm install
cp .env.example .env # 기본값은 LLM_PROVIDER=mock — API 키 불필요
npm run db:up # postgres(pgvector) + redis
npm run db:migrate
npm run db:seed # 업종/지역 계층 + 데모 업체 3곳
npm start # http://localhost:3100
npm run smoke # (다른 터미널) 엔드투엔드 점검
```
`npm run db:reset` 은 볼륨까지 지우고 migrate + seed 를 다시 돌린다.
### 실제 OpenAI 로 전환
```bash
# .env
LLM_PROVIDER=openai
OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-4.1-mini
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
```
`LLM_PROVIDER=mock` 은 문자 bigram 해싱 임베딩을 쓴다. 랜덤이 아니라 **비슷한 문자열이면
비슷한 벡터**가 나오므로 중복제거 파이프라인 검증에는 충분하지만, 의미 기반 중복
(`강남 미용실` ↔ `강남 헤어샵`) 은 실제 임베딩 모델에서만 잡힌다.
## 데이터 모델
| 테이블 | 역할 |
|---|---|
| `industry` / `region` | `ltree` 업종·지역 계층. 상위 노드 키워드 상속의 기반 |
| `merchant` | 업체. `external_id` 가 o2o-site-AEO 의 사이트 ID |
| `keyword` | **전역** 키워드 사전. `normalized` 유니크, `aliases[]`, `embedding vector(1536)` |
| `merchant_keyword` | 업체 ↔ 키워드 연결. `relevance` / `status` / `impressions` / `ctr` |
| `qa_pair` | AEO 용 질문-답변 쌍 |
| `generation_run` | 생성 감사 로그 (프롬프트 버전·토큰·통계) |
키워드는 업체에 복제하지 않고 전역 사전 + 연결 테이블로 둔다. 그래야 임베딩이 하나만
저장되고, `강남 미용실` 을 쓰는 업체가 100곳이어도 중복제거가 성립한다.
## 중복제거 4단계
값비싼 벡터 비교를 마지막에 두고, 후보 집합 안에서만 수행한다.
| 단계 | 방법 | 걸러내는 것 |
|---|---|---|
| 0 | 금칙어 필터 | `최고`, `1위`, `100%` 등 과장광고 |
| 1 | `normalized` 완전 일치 (공백·구두점 제거) | `강남 뿌리 염색` = `강남 뿌리염색` |
| 2 | `pg_trgm` 유사도 ≥ 0.6 | `강남 뿌리염색약` → `강남 뿌리염색` |
| 3 | 코사인 유사도 ≥ 0.92 | `강남 미용실` ↔ `강남 헤어샵` (의미 중복) |
| 4 | 신규 등록 | 위에 안 걸리면 새 키워드 |
1~3 단계에서 매칭되면 원래 표기는 버리지 않고 기존 키워드의 `aliases[]` 로 흡수한다
(롱테일 검색어 보존 + 성과 피드백 매칭에 사용).
임계값은 `.env` 의 `DEDUP_COSINE_THRESHOLD` / `DEDUP_TRIGRAM_THRESHOLD` 로 조정.
## API
| 메서드 | 경로 | 용도 |
|---|---|---|
| `GET` | `/health` | 헬스체크 |
| `POST` | `/v1/merchants/publish` | **사이트 발행 웹훅** — 업체 upsert + 생성 예약 (`sync:true` 면 동기 실행) |
| `POST` | `/v1/merchants/:id/generate?sync=true` | 수동 재생성 |
| `GET` | `/v1/merchants` `/v1/merchants/:id` | 조회 |
| `GET` | `/v1/sites/:id/seo?limit=20` | **발행 사이트가 렌더링 시 호출** — title/description/keywords/tags |
| `GET` | `/v1/sites/:id/aeo?limit=10` | 답변엔진용 topics/FAQ/structuredDataHints |
| `POST` | `/v1/keywords/search` | 의미 기반 키워드 검색 (어드민) |
| `POST` | `/v1/sites/:id/performance` | Search Console·유입 로그 피드백 → 저성과 키워드 강등 |
`:id` 는 `external_id` 또는 내부 UUID 둘 다 받는다.
### 발행 웹훅 예시
```bash
curl -X POST http://localhost:3100/v1/merchants/publish \
-H 'content-type: application/json' \
-d '{
"externalId": "site-1003",
"name": "강남 뷰티랩",
"industryId": "beauty.hair",
"regionId": "kr.seoul.gangnam",
"description": "강남 미용실. 염색 전문.",
"profile": { "services": ["뿌리염색", "여성펌"], "features": ["주차가능"] }
}'
```
### 서빙 예시
```bash
curl 'http://localhost:3100/v1/sites/site-1001/seo?limit=8'
```
```json
{
"title": "레브살롱 | 강남 미용실",
"description": "강남역 3번 출구 앞 프라이빗 헤어살롱. ... 정보를 확인하세요.",
"keywords": ["레브살롱", "강남 미용실", "강남 남자 커트", "..."],
"tags": [{ "keyword": "강남 미용실", "intent": "local", "relevance": 0.95, "aliases": ["강남미용실"] }]
}
```
## 생성 주기
- **발행 즉시** — `/v1/merchants/publish` 가 BullMQ 에 적재 (60초 dedupe 창)
- **주기 리프레시** — 매일 03:00 크론이 `REFRESH_INTERVAL_DAYS`(기본 30일) 지난 업체를 적재
- **성과 기반** — 노출 100회 이상 & CTR < 0.2% 인 키워드는 `demoted` 로 강등, 다음 사이클에서 대체
프롬프트에는 해당 업체와 같은 업종의 기존 키워드 목록을 넣어 **중복 후보 생성 자체를 줄인다.**
그래도 남는 중복만 위 4단계가 처리한다.
## 남은 작업
- [ ] JSON-LD (`LocalBusiness` / `FAQPage` / `Service`) 조립 — `structuredDataHints` 를 그대로 매핑
- [ ] `/llms.txt` 서빙
- [ ] 업종 `ltree` 상위 노드 키워드 상속 (`source: 'inherited'`)
- [ ] Redis 응답 캐시 (서빙은 읽기 99%)
- [ ] Search Console API 연동 (현재는 `/performance` 수동 주입)
- [ ] 어드민 UI
## 아키텍처 도식
`docs/architecture.html` 을 브라우저로 열면 전체 흐름·중복제거 단계·데이터 모델을 볼 수 있다.

33
docker-compose.yml Normal file
View File

@ -0,0 +1,33 @@
services:
postgres:
image: pgvector/pgvector:pg16
container_name: ontology-postgres
restart: unless-stopped
environment:
POSTGRES_USER: ontology
POSTGRES_PASSWORD: ontology
POSTGRES_DB: ontology
ports:
- '55432:5432'
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ['CMD-SHELL', 'pg_isready -U ontology -d ontology']
interval: 5s
timeout: 3s
retries: 20
redis:
image: redis:7-alpine
container_name: ontology-redis
restart: unless-stopped
ports:
- '56379:6379'
healthcheck:
test: ['CMD', 'redis-cli', 'ping']
interval: 5s
timeout: 3s
retries: 20
volumes:
pgdata:

714
docs/architecture.html Normal file
View File

@ -0,0 +1,714 @@
<title>키워드 온톨로지 설계</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Gowun+Batang:wght@400;700&family=IBM+Plex+Mono:wght@400;500;600&family=IBM+Plex+Sans+KR:wght@300;400;500;600;700&display=swap">
<style>
:root {
--bg: #f4f6f5;
--surface: #ffffff;
--surface-2: #eceff0;
--ink: #101819;
--ink-soft: #3d4c4e;
--muted: #63757a;
--line: #d5dcdb;
--line-soft: #e4e9e8;
--accent: #0d6a60;
--accent-bg: #dff0ec;
--warn: #8a5a06;
--warn-bg: #f6ead2;
--stop: #9d3a30;
--stop-bg: #f6e0dc;
--shadow: 0 1px 2px rgba(16,24,25,.05), 0 8px 24px -16px rgba(16,24,25,.35);
--display: 'Gowun Batang', 'Apple SD Gothic Neo', serif;
--body: 'IBM Plex Sans KR', 'Apple SD Gothic Neo', -apple-system, sans-serif;
--mono: 'IBM Plex Mono', 'SFMono-Regular', ui-monospace, monospace;
}
@media (prefers-color-scheme: dark) {
:root:not([data-theme="light"]) {
--bg: #0d1213;
--surface: #141b1c;
--surface-2: #1b2425;
--ink: #e7edeb;
--ink-soft: #c2cecd;
--muted: #8d9d9f;
--line: #263130;
--line-soft: #1e2728;
--accent: #56c2b1;
--accent-bg: #12312e;
--warn: #d7a34a;
--warn-bg: #33270f;
--stop: #e28a80;
--stop-bg: #37201d;
--shadow: 0 1px 2px rgba(0,0,0,.4), 0 8px 24px -16px rgba(0,0,0,.8);
}
}
:root[data-theme="dark"] {
--bg: #0d1213;
--surface: #141b1c;
--surface-2: #1b2425;
--ink: #e7edeb;
--ink-soft: #c2cecd;
--muted: #8d9d9f;
--line: #263130;
--line-soft: #1e2728;
--accent: #56c2b1;
--accent-bg: #12312e;
--warn: #d7a34a;
--warn-bg: #33270f;
--stop: #e28a80;
--stop-bg: #37201d;
--shadow: 0 1px 2px rgba(0,0,0,.4), 0 8px 24px -16px rgba(0,0,0,.8);
}
* { box-sizing: border-box; }
body {
margin: 0;
background: var(--bg);
color: var(--ink);
font-family: var(--body);
font-weight: 400;
line-height: 1.7;
-webkit-font-smoothing: antialiased;
}
.wrap { max-width: 1240px; margin: 0 auto; padding: 56px 32px 96px; }
.col { max-width: 760px; }
/* ---------- masthead ---------- */
.masthead { border-bottom: 1px solid var(--line); padding-bottom: 28px; margin-bottom: 44px; }
.eyebrow {
font-family: var(--mono); font-size: 11px; font-weight: 500;
letter-spacing: .14em; text-transform: uppercase; color: var(--accent);
margin: 0 0 14px;
}
h1 {
font-family: var(--display); font-weight: 700;
font-size: clamp(30px, 4.4vw, 46px); line-height: 1.18; letter-spacing: -.01em;
margin: 0 0 16px; text-wrap: balance;
}
.standfirst { font-size: 17px; color: var(--ink-soft); margin: 0; max-width: 62ch; font-weight: 300; }
.meta {
display: flex; flex-wrap: wrap; gap: 8px; margin-top: 22px;
font-family: var(--mono); font-size: 11.5px; color: var(--muted);
}
.meta span {
border: 1px solid var(--line); border-radius: 3px;
padding: 3px 9px; background: var(--surface);
}
/* ---------- sections ---------- */
section { margin-top: 64px; }
h2 {
font-family: var(--display); font-weight: 700;
font-size: 25px; line-height: 1.3; margin: 0 0 6px; letter-spacing: -.005em;
}
.lede { color: var(--muted); margin: 0 0 26px; max-width: 66ch; font-size: 15px; }
h3 {
font-size: 15px; font-weight: 600; margin: 34px 0 10px;
letter-spacing: .01em;
}
p { margin: 0 0 14px; max-width: 68ch; }
strong { font-weight: 600; }
code {
font-family: var(--mono); font-size: .875em;
background: var(--surface-2); padding: 1px 5px; border-radius: 3px;
color: var(--ink-soft);
}
/* ---------- figures ---------- */
figure { margin: 0 0 8px; }
.fig {
background: var(--surface); border: 1px solid var(--line);
border-radius: 6px; box-shadow: var(--shadow);
padding: 26px 22px 18px; margin: 8px 0 0;
}
.fig-scroll { overflow-x: auto; }
.fig svg { display: block; min-width: 720px; max-width: 100%; height: auto; color: var(--ink); }
figcaption {
font-size: 13px; color: var(--muted); margin-top: 16px;
padding-top: 14px; border-top: 1px solid var(--line-soft); max-width: 78ch;
}
/* ---------- tables ---------- */
.tbl-wrap { overflow-x: auto; margin: 20px 0 8px; }
table { border-collapse: collapse; width: 100%; font-size: 14px; min-width: 520px; }
th, td { text-align: left; padding: 11px 14px; border-bottom: 1px solid var(--line-soft); vertical-align: top; }
thead th {
font-family: var(--mono); font-size: 11px; font-weight: 600;
letter-spacing: .1em; text-transform: uppercase; color: var(--muted);
border-bottom: 1px solid var(--line);
}
tbody tr:last-child td { border-bottom: none; }
td.mono, th.mono { font-family: var(--mono); font-size: 12.5px; }
.num { font-variant-numeric: tabular-nums; }
/* ---------- callout ---------- */
.verdict {
background: var(--accent-bg); border-left: 3px solid var(--accent);
padding: 18px 22px; border-radius: 0 5px 5px 0; margin: 24px 0;
}
.verdict p { margin: 0; max-width: none; }
.verdict p + p { margin-top: 10px; }
/* ---------- stage list (진짜 순서가 있는 것에만) ---------- */
ol.stages { list-style: none; counter-reset: s -1; padding: 0; margin: 20px 0 8px; }
ol.stages li {
counter-increment: s; display: grid;
grid-template-columns: 34px 1fr; gap: 16px;
padding: 14px 0; border-bottom: 1px solid var(--line-soft);
}
ol.stages li:last-child { border-bottom: none; }
ol.stages li::before {
content: counter(s);
font-family: var(--mono); font-size: 12px; font-weight: 600;
color: var(--accent); border: 1px solid var(--line);
border-radius: 3px; height: 26px; display: grid; place-items: center;
background: var(--surface);
}
ol.stages b { display: block; font-weight: 600; font-size: 14.5px; }
ol.stages span { font-size: 13.5px; color: var(--muted); }
ul.plain { padding-left: 20px; margin: 12px 0; }
ul.plain li { margin-bottom: 7px; max-width: 68ch; }
pre {
background: var(--surface); border: 1px solid var(--line); border-radius: 5px;
padding: 16px 18px; overflow-x: auto; font-family: var(--mono);
font-size: 12.5px; line-height: 1.75; margin: 16px 0; color: var(--ink-soft);
}
pre b { color: var(--accent); font-weight: 500; }
.pill {
display: inline-block; font-family: var(--mono); font-size: 11px;
padding: 2px 7px; border-radius: 3px; letter-spacing: .02em;
}
.pill-go { background: var(--accent-bg); color: var(--accent); }
.pill-warn { background: var(--warn-bg); color: var(--warn); }
.pill-stop { background: var(--stop-bg); color: var(--stop); }
footer {
margin-top: 76px; padding-top: 22px; border-top: 1px solid var(--line);
font-size: 13px; color: var(--muted);
}
a { color: var(--accent); }
a:focus-visible, summary:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
</style>
<div class="wrap">
<header class="masthead col">
<p class="eyebrow">o2o-site-ontology</p>
<h1>발행 사이트에 붙는<br>SEO/AEO 키워드 온톨로지</h1>
<p class="standfirst">
업체 사이트를 발행하면 그 업체에 맞는 검색 키워드·태그·질문답변이 따라붙어야 한다.
LLM 이 주기적으로 후보를 만들고, 4단계 중복제거가 전역 키워드 사전을 깨끗하게 유지하고,
발행된 사이트는 REST 로 완성된 payload 만 받아 쓴다.
</p>
<div class="meta">
<span>PostgreSQL 16 + pgvector</span>
<span>NestJS</span>
<span>BullMQ</span>
<span>OpenAI Structured Outputs</span>
</div>
</header>
<!-- ======================================================= 1 -->
<section>
<div class="col">
<h2>일반 DB 냐 벡터 DB 냐</h2>
<p class="lede">둘 중 하나를 고르는 문제가 아니다. 이 서비스는 성격이 다른 세 종류의 조회를 동시에 요구한다.</p>
</div>
<div class="tbl-wrap col">
<table>
<thead>
<tr><th>조회 유형</th><th>실제 질의</th><th>필요한 것</th></tr>
</thead>
<tbody>
<tr>
<td>정확 조회</td>
<td>업체 A 의 활성 키워드 20개</td>
<td class="mono">B-tree / 관계형 조인</td>
</tr>
<tr>
<td>의미 조회</td>
<td>이 후보가 기존 키워드와 의미상 겹치는가</td>
<td class="mono">vector (HNSW)</td>
</tr>
<tr>
<td>관계 탐색</td>
<td>업종 트리 상위에서 물려받을 공통 키워드</td>
<td class="mono">ltree 계층 / recursive CTE</td>
</tr>
</tbody>
</table>
</div>
<div class="verdict col">
<p><strong>결론 — PostgreSQL 하나로 시작한다.</strong>
<code>pgvector</code> + <code>ltree</code> + <code>pg_trgm</code> + <code>JSONB</code> 로 세 가지가 모두 한 엔진 안에서 해결되고,
무엇보다 <em>키워드 조회에는 항상 "어느 업체의"라는 조인이 따라붙는다.</em></p>
<p>전용 벡터 DB 를 지금 분리하면 매 요청이 2-hop 이 되고 정합성을 따로 관리해야 한다.
벡터 행이 1천만 건을 넘거나 ANN 지연이 실제로 문제가 되는 시점에 Qdrant 로 떼어내도 늦지 않다.
Neo4j 도 같은 논리 — 고정 깊이 상속이면 <code>ltree</code> 로 충분하다.</p>
</div>
</section>
<!-- ======================================================= 2 -->
<section>
<div class="col">
<h2>전체 흐름</h2>
<p class="lede">생성은 큐 뒤에서 비동기로, 서빙은 DB 읽기만으로. 두 경로가 만나는 지점은 Postgres 한 곳뿐이다.</p>
</div>
<figure>
<div class="fig fig-scroll">
<svg viewBox="0 0 1160 500" role="img"
aria-label="트리거가 BullMQ 큐에 적재되고, 생성 워커가 OpenAI 를 호출해 후보 키워드를 만들고, 4단계 중복제거를 거쳐 PostgreSQL 에 저장되며, 서빙 API 가 발행 사이트에 SEO/AEO payload 를 내려주고, 유입 성과가 다시 트리거로 돌아오는 순환 구조">
<defs>
<marker id="a1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
<path d="M0,0 L10,5 L0,10 z" fill="currentColor"/>
</marker>
<marker id="a1acc" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
<path d="M0,0 L10,5 L0,10 z" fill="var(--accent)"/>
</marker>
</defs>
<!-- boxes -->
<g stroke="currentColor" stroke-width="1" fill="var(--surface-2)" opacity="1">
<rect x="24" y="64" width="180" height="88" rx="4"/>
<rect x="252" y="64" width="180" height="88" rx="4"/>
<rect x="480" y="64" width="180" height="88" rx="4"/>
<rect x="708" y="248" width="180" height="88" rx="4"/>
<rect x="252" y="248" width="180" height="88" rx="4"/>
<rect x="252" y="400" width="400" height="60" rx="4"/>
</g>
<rect x="708" y="64" width="180" height="88" rx="4" fill="var(--warn-bg)" stroke="var(--warn)" stroke-width="1.5"/>
<rect x="936" y="48" width="200" height="120" rx="4" fill="var(--accent-bg)" stroke="var(--accent)" stroke-width="1.5"/>
<!-- labels -->
<g font-family="IBM Plex Sans KR, sans-serif" fill="currentColor">
<text x="40" y="88" font-size="13" font-weight="600">트리거</text>
<text x="40" y="110" font-size="11" opacity=".75">사이트 발행 — 즉시</text>
<text x="40" y="127" font-size="11" opacity=".75">크론 03:00 — 30일 경과</text>
<text x="40" y="144" font-size="11" opacity=".75">성과 저조 — 재생성</text>
<text x="268" y="88" font-size="13" font-weight="600">BullMQ 큐</text>
<text x="268" y="110" font-size="11" opacity=".75">60초 dedupe 창</text>
<text x="268" y="127" font-size="11" opacity=".75">재시도 3회 · 지수 백오프</text>
<text x="268" y="144" font-size="11" opacity=".75">동시성 2</text>
<text x="496" y="88" font-size="13" font-weight="600">생성 워커</text>
<text x="496" y="110" font-size="11" opacity=".75">OpenAI · gpt-4.1-mini</text>
<text x="496" y="127" font-size="11" opacity=".75">Structured Outputs</text>
<text x="496" y="144" font-size="11" opacity=".75">임베딩 배치 1회</text>
<text x="724" y="88" font-size="13" font-weight="600" fill="var(--warn)">중복제거 4단계</text>
<text x="724" y="110" font-size="11" fill="var(--warn)" opacity=".9">해시 → trigram → 벡터</text>
<text x="724" y="127" font-size="11" fill="var(--warn)" opacity=".9">미일치만 신규 등록</text>
<text x="724" y="144" font-size="11" fill="var(--warn)" opacity=".9">나머지는 alias 흡수</text>
<text x="952" y="76" font-size="13" font-weight="600" fill="var(--accent)">PostgreSQL 16</text>
<text x="952" y="98" font-size="11" fill="var(--accent)" opacity=".9">pgvector · ltree · pg_trgm</text>
<text x="952" y="120" font-size="11" fill="var(--accent)" opacity=".9">keyword (전역 사전)</text>
<text x="952" y="137" font-size="11" fill="var(--accent)" opacity=".9">merchant_keyword</text>
<text x="952" y="154" font-size="11" fill="var(--accent)" opacity=".9">qa_pair · generation_run</text>
<text x="724" y="272" font-size="13" font-weight="600">Serving API</text>
<text x="724" y="294" font-size="11" opacity=".75">GET /v1/sites/:id/seo</text>
<text x="724" y="311" font-size="11" opacity=".75">GET /v1/sites/:id/aeo</text>
<text x="724" y="328" font-size="11" opacity=".75">읽기 99% · 캐시 대상</text>
<text x="268" y="272" font-size="13" font-weight="600">발행된 사이트</text>
<text x="268" y="294" font-size="11" opacity=".75">o2o-site-AEO</text>
<text x="268" y="311" font-size="11" opacity=".75">렌더링 시 호출</text>
<text x="268" y="426" font-size="13" font-weight="600">성과 수집</text>
<text x="268" y="447" font-size="11" opacity=".75">Search Console · 네이버 서치어드바이저 · 유입 로그</text>
</g>
<!-- flow arrows -->
<g stroke="currentColor" stroke-width="1.4" fill="none" marker-end="url(#a1)">
<line x1="204" y1="108" x2="244" y2="108"/>
<line x1="432" y1="108" x2="472" y2="108"/>
<line x1="660" y1="108" x2="700" y2="108"/>
<line x1="888" y1="108" x2="928" y2="108"/>
<path d="M1036 168 L1036 292 L896 292"/>
<line x1="708" y1="292" x2="440" y2="292"/>
<line x1="342" y1="336" x2="342" y2="392"/>
<path d="M252 430 L114 430 L114 160"/>
</g>
<!-- prompt feedback (dashed, accent) -->
<g stroke="var(--accent)" stroke-width="1.4" fill="none" stroke-dasharray="5 4" marker-end="url(#a1acc)">
<path d="M1036 48 L1036 24 L570 24 L570 56"/>
</g>
<!-- arrow labels -->
<g font-family="IBM Plex Mono, monospace" font-size="10.5" fill="currentColor" opacity=".7">
<text x="224" y="100" text-anchor="middle">적재</text>
<text x="452" y="100" text-anchor="middle">job</text>
<text x="680" y="100" text-anchor="middle">후보</text>
<text x="908" y="100" text-anchor="middle">write</text>
<text x="1046" y="230">읽기</text>
<text x="574" y="284" text-anchor="middle">SEO / AEO payload</text>
<text x="352" y="368">노출 · 클릭</text>
<text x="124" y="212">CTR &lt; 0.2% → 강등</text>
</g>
<text x="570" y="16" text-anchor="middle" font-family="IBM Plex Mono, monospace"
font-size="10.5" fill="var(--accent)">기존 키워드 주입 — 중복 후보 생성 자체를 억제</text>
</svg>
</div>
<figcaption>
점선 화살표가 이 설계의 핵심이다. 프롬프트에 해당 업종의 기존 키워드를 넣어 중복 후보가 <em>만들어지기 전에</em> 줄이고,
그래도 남는 것만 중복제거 단계가 처리한다. 생성 경로(위)와 서빙 경로(아래)는 Postgres 에서만 만나므로
OpenAI 가 느리거나 죽어도 발행된 사이트의 응답에는 영향이 없다.
</figcaption>
</figure>
</section>
<!-- ======================================================= 3 -->
<section>
<div class="col">
<h2>중복제거 4단계</h2>
<p class="lede">
값싼 판정을 먼저, 비싼 판정을 나중에. 벡터 비교는 후보 20건 안에서만 일어나므로 전수 비교가 발생하지 않는다.
</p>
</div>
<figure>
<div class="fig fig-scroll">
<svg viewBox="0 0 1000 500" role="img"
aria-label="LLM 후보 키워드가 금칙어 필터, 정규화 완전 일치, trigram 유사도, 코사인 유사도 순으로 통과하며 각 단계에서 탈락한 것은 차단되거나 기존 키워드의 alias 로 흡수되고, 전부 통과한 것만 새 키워드로 등록된다">
<defs>
<marker id="a2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
<path d="M0,0 L10,5 L0,10 z" fill="currentColor"/>
</marker>
<marker id="a2w" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
<path d="M0,0 L10,5 L0,10 z" fill="var(--warn)"/>
</marker>
<marker id="a2s" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
<path d="M0,0 L10,5 L0,10 z" fill="var(--stop)"/>
</marker>
<marker id="a2acc" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
<path d="M0,0 L10,5 L0,10 z" fill="var(--accent)"/>
</marker>
</defs>
<text x="440" y="26" text-anchor="middle" font-family="IBM Plex Sans KR, sans-serif"
font-size="12.5" font-weight="600" fill="currentColor">LLM 후보 키워드</text>
<line x1="440" y1="34" x2="440" y2="54" stroke="currentColor" stroke-width="1.4" marker-end="url(#a2)"/>
<!-- stage spine -->
<g stroke="currentColor" stroke-width="1" fill="var(--surface-2)">
<rect x="280" y="60" width="320" height="58" rx="4"/>
<rect x="280" y="150" width="320" height="58" rx="4"/>
<rect x="280" y="240" width="320" height="58" rx="4"/>
<rect x="280" y="330" width="320" height="58" rx="4"/>
</g>
<rect x="280" y="420" width="320" height="58" rx="4" fill="var(--accent-bg)" stroke="var(--accent)" stroke-width="1.5"/>
<g font-family="IBM Plex Sans KR, sans-serif" fill="currentColor">
<text x="298" y="84" font-size="13" font-weight="600">0 · 금칙어 필터</text>
<text x="298" y="104" font-size="11" opacity=".75">최고 · 1위 · 100% · 완치</text>
<text x="298" y="174" font-size="13" font-weight="600">1 · normalized 완전 일치</text>
<text x="298" y="194" font-size="11" opacity=".75">NFKC · 소문자 · 구두점/공백 제거</text>
<text x="298" y="264" font-size="13" font-weight="600">2 · pg_trgm 유사도 ≥ 0.6</text>
<text x="298" y="284" font-size="11" opacity=".75">표기 변형 · 오타</text>
<text x="298" y="354" font-size="13" font-weight="600">3 · 코사인 유사도 ≥ 0.92</text>
<text x="298" y="374" font-size="11" opacity=".75">의미 중복 — 후보 20건 안에서만</text>
<text x="298" y="444" font-size="13" font-weight="600" fill="var(--accent)">4 · 새 키워드로 INSERT</text>
<text x="298" y="464" font-size="11" fill="var(--accent)" opacity=".9">embedding 저장 · usage_count 1</text>
</g>
<!-- pass-down arrows -->
<g stroke="currentColor" stroke-width="1.4" fill="none" marker-end="url(#a2)">
<line x1="440" y1="118" x2="440" y2="144"/>
<line x1="440" y1="208" x2="440" y2="234"/>
<line x1="440" y1="298" x2="440" y2="324"/>
<line x1="440" y1="388" x2="440" y2="414"/>
</g>
<g font-family="IBM Plex Mono, monospace" font-size="10" fill="currentColor" opacity=".6">
<text x="450" y="137">미일치</text>
<text x="450" y="227">미일치</text>
<text x="450" y="317">미일치</text>
<text x="450" y="407">미일치</text>
</g>
<!-- cost annotations (left) -->
<g font-family="IBM Plex Mono, monospace" font-size="10" fill="currentColor" opacity=".55" text-anchor="end">
<text x="262" y="93">비용 0</text>
<text x="262" y="183">B-tree 1회</text>
<text x="262" y="273">GIN trgm</text>
<text x="262" y="363">HNSW top-20</text>
<text x="262" y="453">INSERT</text>
</g>
<!-- exits -->
<rect x="672" y="66" width="304" height="46" rx="4" fill="var(--stop-bg)" stroke="var(--stop)" stroke-width="1.2"/>
<line x1="600" y1="89" x2="664" y2="89" stroke="var(--stop)" stroke-width="1.4" marker-end="url(#a2s)"/>
<text x="688" y="84" font-family="IBM Plex Sans KR, sans-serif" font-size="12" font-weight="600" fill="var(--stop)">차단 — 저장하지 않음</text>
<text x="688" y="102" font-family="IBM Plex Mono, monospace" font-size="10.5" fill="var(--stop)" opacity=".9">rejected_banned</text>
<g>
<rect x="672" y="156" width="304" height="46" rx="4" fill="var(--warn-bg)" stroke="var(--warn)" stroke-width="1.2"/>
<rect x="672" y="246" width="304" height="46" rx="4" fill="var(--warn-bg)" stroke="var(--warn)" stroke-width="1.2"/>
<rect x="672" y="336" width="304" height="46" rx="4" fill="var(--warn-bg)" stroke="var(--warn)" stroke-width="1.2"/>
</g>
<g stroke="var(--warn)" stroke-width="1.4" marker-end="url(#a2w)">
<line x1="600" y1="179" x2="664" y2="179"/>
<line x1="600" y1="269" x2="664" y2="269"/>
<line x1="600" y1="359" x2="664" y2="359"/>
</g>
<g font-family="IBM Plex Sans KR, sans-serif" fill="var(--warn)">
<text x="688" y="174" font-size="12" font-weight="600">기존 키워드에 alias 흡수</text>
<text x="688" y="192" font-size="10.5" font-family="IBM Plex Mono, monospace" opacity=".9">강남 뿌리 염색 → 강남 뿌리염색</text>
<text x="688" y="264" font-size="12" font-weight="600">기존 키워드에 alias 흡수</text>
<text x="688" y="282" font-size="10.5" font-family="IBM Plex Mono, monospace" opacity=".9">강남 뿌리염색약 → 강남 뿌리염색 (0.67)</text>
<text x="688" y="354" font-size="12" font-weight="600">기존 키워드에 alias 흡수</text>
<text x="688" y="372" font-size="10.5" font-family="IBM Plex Mono, monospace" opacity=".9">강남 헤어샵 → 강남 미용실 (0.94)</text>
</g>
<!-- 모든 경로가 합류하는 지점 -->
<rect x="672" y="420" width="304" height="52" rx="4"
fill="none" stroke="currentColor" stroke-width="1.2" stroke-dasharray="5 4" opacity=".8"/>
<line x1="824" y1="382" x2="824" y2="414" stroke="var(--warn)" stroke-width="1.4"
fill="none" marker-end="url(#a2w)"/>
<line x1="600" y1="446" x2="664" y2="446" stroke="var(--accent)" stroke-width="1.4"
fill="none" marker-end="url(#a2acc)"/>
<text x="688" y="443" font-family="IBM Plex Sans KR, sans-serif" font-size="12" font-weight="600"
fill="currentColor">어느 경로든 업체에는 연결된다</text>
<text x="688" y="462" font-family="IBM Plex Mono, monospace" font-size="10.5"
fill="currentColor" opacity=".7">merchant_keyword · relevance · status</text>
</svg>
</div>
<figcaption>
1~3 단계에서 걸린 표기는 버리지 않고 기존 키워드의 <code>aliases[]</code> 에 흡수한다.
롱테일 검색어를 잃지 않으면서 사전은 한 행으로 유지되고, 나중에 Search Console 이
<code>강남 뿌리염색약</code> 으로 성과를 보고해도 같은 키워드에 매칭된다.
</figcaption>
</figure>
<div class="col">
<h3>실제 로컬 실행 결과</h3>
<p>같은 지역·업종 업체를 순서대로 발행했을 때 <code>npm run smoke</code> 출력이다.</p>
</div>
<pre>1. 레브살롱 (첫 업체) 후보 19 → <b>신규 19</b> / 중복 0
2. 헤어랩 강남점 후보 19 → <b>신규 4</b> / 중복(정확 15, 표기 0, 의미 0)
3. 강남 뷰티랩 후보 16 → <b>신규 3</b> / 중복(정확 12, 표기 1, 의미 0)
matched_exact 강남 뿌리 염색 (sim=1.000 → '강남 뿌리염색')
matched_trigram 강남 뿌리염색약 (sim=0.667 → '강남 뿌리염색')
matched_exact 강남미용실추천 (sim=1.000 → '강남 미용실 추천')</pre>
<div class="col">
<p style="font-size:13.5px;color:var(--muted)">
<span class="pill pill-warn">참고</span>
위 수치는 <code>LLM_PROVIDER=mock</code> 기준이다. mock 임베딩은 문자 bigram 해싱이라 표기 유사도만 잡는다.
의미 중복(<code>강남 미용실</code> ↔ <code>강남 헤어샵</code>)은 실제 <code>text-embedding-3-small</code> 로 전환해야 3단계가 발동한다.
</p>
</div>
</section>
<!-- ======================================================= 4 -->
<section>
<div class="col">
<h2>데이터 모델</h2>
<p class="lede">
키워드를 업체에 복제하지 않는 것이 이 스키마의 전부다. 복제하는 순간 중복제거 자체가 성립하지 않는다.
</p>
</div>
<figure>
<div class="fig fig-scroll">
<svg viewBox="0 0 1000 420" role="img"
aria-label="industry 와 region 계층이 keyword 를 분류하고, merchant 는 merchant_keyword 연결 테이블을 통해 전역 keyword 사전을 참조하며, qa_pair 는 merchant 에 직접 매달린다">
<defs>
<marker id="a3" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
<path d="M0,0 L10,5 L0,10 z" fill="currentColor"/>
</marker>
</defs>
<g stroke="currentColor" stroke-width="1" fill="var(--surface-2)">
<rect x="24" y="32" width="190" height="62" rx="4"/>
<rect x="24" y="116" width="190" height="62" rx="4"/>
<rect x="24" y="224" width="190" height="104" rx="4"/>
<rect x="380" y="224" width="230" height="104" rx="4"/>
<rect x="720" y="250" width="250" height="90" rx="4"/>
</g>
<rect x="720" y="32" width="250" height="158" rx="4" fill="var(--accent-bg)" stroke="var(--accent)" stroke-width="1.5"/>
<g font-family="IBM Plex Mono, monospace" fill="currentColor">
<text x="40" y="56" font-size="12.5" font-weight="600">industry</text>
<text x="40" y="76" font-size="10.5" opacity=".7">path ltree · beauty.hair</text>
<text x="40" y="140" font-size="12.5" font-weight="600">region</text>
<text x="40" y="160" font-size="10.5" opacity=".7">path ltree · kr.seoul.gangnam</text>
<text x="40" y="250" font-size="12.5" font-weight="600">merchant</text>
<text x="40" y="270" font-size="10.5" opacity=".7">external_id ← 사이트 ID</text>
<text x="40" y="288" font-size="10.5" opacity=".7">description · profile jsonb</text>
<text x="40" y="306" font-size="10.5" opacity=".7">last_generated_at</text>
<text x="396" y="250" font-size="12.5" font-weight="600">merchant_keyword</text>
<text x="396" y="270" font-size="10.5" opacity=".7">relevance · status · source</text>
<text x="396" y="288" font-size="10.5" opacity=".7">impressions · clicks · ctr</text>
<text x="396" y="306" font-size="10.5" opacity=".7">PK (merchant_id, keyword_id)</text>
<text x="736" y="56" font-size="12.5" font-weight="600" fill="var(--accent)">keyword — 전역 사전</text>
<text x="736" y="80" font-size="10.5" fill="var(--accent)" opacity=".9">canonical · 표시용</text>
<text x="736" y="98" font-size="10.5" fill="var(--accent)" opacity=".9">normalized UNIQUE · 판정용</text>
<text x="736" y="116" font-size="10.5" fill="var(--accent)" opacity=".9">aliases text[] · 흡수된 표기</text>
<text x="736" y="134" font-size="10.5" fill="var(--accent)" opacity=".9">embedding vector(1536) HNSW</text>
<text x="736" y="152" font-size="10.5" fill="var(--accent)" opacity=".9">intent · locale</text>
<text x="736" y="170" font-size="10.5" fill="var(--accent)" opacity=".9">usage_count</text>
<text x="736" y="274" font-size="12.5" font-weight="600">qa_pair</text>
<text x="736" y="294" font-size="10.5" opacity=".7">question · answer</text>
<text x="736" y="312" font-size="10.5" opacity=".7">normalized_question UNIQUE</text>
<text x="736" y="330" font-size="10.5" opacity=".7">embedding vector(1536)</text>
</g>
<g stroke="currentColor" stroke-width="1.3" fill="none" marker-end="url(#a3)">
<line x1="214" y1="63" x2="712" y2="63"/>
<line x1="214" y1="147" x2="712" y2="147"/>
<line x1="214" y1="276" x2="372" y2="276"/>
<path d="M610 262 L666 262 L666 111 L712 111"/>
<path d="M119 328 L119 380 L845 380 L845 348"/>
</g>
<g font-family="IBM Plex Mono, monospace" font-size="10.5" fill="currentColor" opacity=".65">
<text x="463" y="56" text-anchor="middle">업종 분류</text>
<text x="463" y="140" text-anchor="middle">지역 분류</text>
<text x="293" y="269" text-anchor="middle">1 : N</text>
<text x="672" y="205">N : 1</text>
<text x="482" y="373" text-anchor="middle">1 : N</text>
</g>
</svg>
</div>
<figcaption>
<code>강남 미용실</code> 을 100개 업체가 쓰더라도 <code>keyword</code> 에는 행이 하나, 임베딩도 하나뿐이다.
업체별 관련도·성과는 전부 <code>merchant_keyword</code> 가 들고 있으므로 사전을 오염시키지 않고
업체마다 다른 순위를 낼 수 있다.
</figcaption>
</figure>
</section>
<!-- ======================================================= 5 -->
<section>
<div class="col">
<h2>API</h2>
<p class="lede">
<code>:id</code> 는 o2o-site-AEO 의 <code>external_id</code> 와 내부 UUID 를 모두 받는다.
연동 쪽에서 ID 매핑 테이블을 따로 들 필요가 없다.
</p>
</div>
<div class="tbl-wrap">
<table>
<thead>
<tr><th style="width:78px">메서드</th><th style="width:300px">경로</th><th>용도</th></tr>
</thead>
<tbody>
<tr><td class="mono">GET</td><td class="mono">/health</td><td>헬스체크 · 현재 LLM provider 확인</td></tr>
<tr><td class="mono">POST</td><td class="mono">/v1/merchants/publish</td><td><strong>사이트 발행 웹훅.</strong> 업체 upsert 후 생성 작업 적재. <code>sync:true</code> 면 동기 실행</td></tr>
<tr><td class="mono">POST</td><td class="mono">/v1/merchants/:id/generate</td><td>수동 재생성. <code>?sync=true</code> 로 결과를 즉시 확인</td></tr>
<tr><td class="mono">GET</td><td class="mono">/v1/sites/:id/seo</td><td><strong>발행 사이트가 렌더링 시 호출.</strong> title · description · keywords · tags(alias 포함)</td></tr>
<tr><td class="mono">GET</td><td class="mono">/v1/sites/:id/aeo</td><td>답변엔진용 topics · FAQ · structuredDataHints</td></tr>
<tr><td class="mono">POST</td><td class="mono">/v1/keywords/search</td><td>어드민 — 자연어 질의로 키워드 사전 벡터 검색</td></tr>
<tr><td class="mono">POST</td><td class="mono">/v1/sites/:id/performance</td><td>노출·클릭 주입 → CTR 갱신 → 저성과 키워드 강등</td></tr>
</tbody>
</table>
</div>
<div class="col">
<h3>SEO 응답</h3>
</div>
<pre>$ curl 'http://localhost:3100/v1/sites/site-1001/seo?limit=8'
{
"title": "레브살롱 | 강남 미용실",
"description": "강남역 3번 출구 앞 프라이빗 헤어살롱. … 정보를 확인하세요.",
"keywords": ["레브살롱", "강남 미용실", "강남 남자 커트", "강남 두피 클리닉", …],
"tags": [
{ "keyword": "강남 미용실", "intent": "local", "relevance": 0.95,
"aliases": ["강남미용실"] }
]
}</pre>
<div class="col">
<h3>AEO 응답</h3>
<p>
SEO 가 키워드라면 AEO 는 <strong>질문-답변 쌍과 구조화 데이터</strong>다. AI 검색 크롤러가 인용하는 것은 이쪽이다.
<code>structuredDataHints</code> 는 후속 단계에서 <code>LocalBusiness</code> / <code>FAQPage</code> JSON-LD 로 그대로 매핑되도록
필드를 미리 맞춰 두었다.
</p>
</div>
<pre>{
"topics": ["강남 미용실", "강남 남자 커트", "강남 여성 펌"],
"faqs": [
{ "question": "레브살롱은(는) 어디에 있나요?",
"answer": "레브살롱은(는) 강남에 위치한 미용실입니다." }
],
"structuredDataHints": {
"type": "LocalBusiness", "name": "레브살롱",
"areaServed": "강남", "category": "미용실"
}
}</pre>
</section>
<!-- ======================================================= 6 -->
<section>
<div class="col">
<h2>기술 선택</h2>
</div>
<div class="tbl-wrap">
<table>
<thead><tr><th style="width:130px">레이어</th><th style="width:250px">선택</th><th>이유</th></tr></thead>
<tbody>
<tr><td>런타임</td><td class="mono">NestJS · TypeScript</td><td>o2o-site-AEO 와 payload 타입을 공유할 수 있다</td></tr>
<tr><td>DB</td><td class="mono">PostgreSQL 16 + pgvector<br>+ ltree + pg_trgm</td><td>정확 · 의미 · 계층 조회 3-in-1</td></tr>
<tr><td>DB 접근</td><td class="mono">postgres.js (raw SQL)</td><td>벡터 연산자 <code>&lt;=&gt;</code> 와 <code>ltree</code> 는 어차피 raw SQL. ORM 을 얹으면 우회 코드가 더 는다</td></tr>
<tr><td>큐 · 스케줄</td><td class="mono">BullMQ + Redis</td><td>60초 dedupe 창, 지수 백오프 재시도, 크론이 전부 내장</td></tr>
<tr><td>LLM</td><td class="mono">OpenAI Structured Outputs<br>text-embedding-3-small</td><td>JSON Schema 강제 — 자유 텍스트 파싱은 반드시 깨진다</td></tr>
<tr><td>관측</td><td class="mono">generation_run 테이블</td><td>프롬프트 버전 · 토큰 · 단계별 통계를 행으로 남긴다</td></tr>
</tbody>
</table>
</div>
<div class="col">
<h3>로컬 실행</h3>
</div>
<pre>npm install
cp .env.example .env <b># 기본 LLM_PROVIDER=mock — API 키 불필요</b>
npm run db:up <b># postgres(pgvector) + redis</b>
npm run db:migrate &amp;&amp; npm run db:seed
npm start <b># http://localhost:3100</b>
npm run smoke <b># 다른 터미널 — 엔드투엔드 점검</b></pre>
</section>
<!-- ======================================================= 7 -->
<section>
<div class="col">
<h2>남은 작업</h2>
<p class="lede">연동에 필요한 API 표면은 이미 고정되어 있다. 아래는 그 뒤에서 채워 넣는 것들이다.</p>
<ul class="plain">
<li><span class="pill pill-go">next</span> JSON-LD 조립 — <code>structuredDataHints</code> → <code>LocalBusiness</code> / <code>FAQPage</code> / <code>Service</code></li>
<li><span class="pill pill-go">next</span> <code>/llms.txt</code> 서빙 — AI 검색 크롤러 진입점</li>
<li><span class="pill pill-warn">later</span> 업종 <code>ltree</code> 상위 노드 키워드 상속 (<code>source: 'inherited'</code>)</li>
<li><span class="pill pill-warn">later</span> Redis 응답 캐시 — 서빙은 읽기 99%, TTL 1시간 + 발행 이벤트 무효화</li>
<li><span class="pill pill-warn">later</span> Search Console API 직접 연동 (지금은 <code>/performance</code> 수동 주입)</li>
<li><span class="pill pill-warn">later</span> 키워드 승인 · 차단 어드민 UI</li>
</ul>
</div>
</section>
<footer class="col">
o2o-site-ontology · 설계 문서 · 코드와 함께 <code>docs/architecture.html</code> 에 보관
</footer>
</div>

124
drizzle/0000_init.sql Normal file
View File

@ -0,0 +1,124 @@
-- o2o-site-ontology : initial schema
-- pgvector(유사도) + ltree(업종/지역 계층) + pg_trgm(표기 변형) 3-in-1
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS ltree;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
-- ---------------------------------------------------------------- 분류 계층
CREATE TABLE IF NOT EXISTS industry (
id text PRIMARY KEY,
path ltree NOT NULL UNIQUE,
name text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS industry_path_gist ON industry USING gist (path);
CREATE TABLE IF NOT EXISTS region (
id text PRIMARY KEY,
path ltree NOT NULL UNIQUE,
name text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS region_path_gist ON region USING gist (path);
-- ---------------------------------------------------------------- 업체
CREATE TABLE IF NOT EXISTS merchant (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
external_id text NOT NULL UNIQUE, -- o2o-site-AEO 의 사이트/업체 ID
name text NOT NULL,
industry_id text REFERENCES industry(id),
region_id text REFERENCES region(id),
description text NOT NULL DEFAULT '',
profile jsonb NOT NULL DEFAULT '{}'::jsonb,
site_url text,
last_generated_at timestamptz,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS merchant_industry_idx ON merchant (industry_id);
CREATE INDEX IF NOT EXISTS merchant_stale_idx ON merchant (last_generated_at NULLS FIRST);
-- ---------------------------------------------------------------- 전역 키워드 사전
DO $$ BEGIN
CREATE TYPE keyword_intent AS ENUM
('informational','navigational','transactional','local','brand');
EXCEPTION WHEN duplicate_object THEN NULL; END $$;
CREATE TABLE IF NOT EXISTS keyword (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
canonical text NOT NULL, -- 화면 노출용 대표 표기
normalized text NOT NULL, -- 중복 판정용 정규화 표기
locale text NOT NULL DEFAULT 'ko-KR',
aliases text[] NOT NULL DEFAULT '{}', -- 흡수된 표기 변형 (롱테일 확보)
intent keyword_intent NOT NULL DEFAULT 'informational',
industry_id text REFERENCES industry(id),
region_id text REFERENCES region(id),
embedding vector(1536),
usage_count integer NOT NULL DEFAULT 0, -- 몇 개 업체가 쓰는가
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT keyword_normalized_locale_uq UNIQUE (normalized, locale)
);
CREATE INDEX IF NOT EXISTS keyword_embedding_hnsw
ON keyword USING hnsw (embedding vector_cosine_ops);
CREATE INDEX IF NOT EXISTS keyword_normalized_trgm
ON keyword USING gin (normalized gin_trgm_ops);
CREATE INDEX IF NOT EXISTS keyword_industry_idx ON keyword (industry_id);
-- ---------------------------------------------------------------- 업체 <-> 키워드
DO $$ BEGIN
CREATE TYPE merchant_keyword_status AS ENUM
('candidate','active','demoted','blocked');
EXCEPTION WHEN duplicate_object THEN NULL; END $$;
CREATE TABLE IF NOT EXISTS merchant_keyword (
merchant_id uuid NOT NULL REFERENCES merchant(id) ON DELETE CASCADE,
keyword_id uuid NOT NULL REFERENCES keyword(id) ON DELETE CASCADE,
relevance real NOT NULL DEFAULT 0,
source text NOT NULL DEFAULT 'llm', -- llm | manual | inherited
status merchant_keyword_status NOT NULL DEFAULT 'candidate',
rationale text,
impressions bigint NOT NULL DEFAULT 0,
clicks bigint NOT NULL DEFAULT 0,
ctr real NOT NULL DEFAULT 0,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (merchant_id, keyword_id)
);
CREATE INDEX IF NOT EXISTS merchant_keyword_serving_idx
ON merchant_keyword (merchant_id, status, relevance DESC);
-- ---------------------------------------------------------------- AEO 질문-답변
CREATE TABLE IF NOT EXISTS qa_pair (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
merchant_id uuid NOT NULL REFERENCES merchant(id) ON DELETE CASCADE,
question text NOT NULL,
answer text NOT NULL,
normalized_question text NOT NULL,
embedding vector(1536),
status merchant_keyword_status NOT NULL DEFAULT 'active',
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT qa_pair_merchant_question_uq UNIQUE (merchant_id, normalized_question)
);
CREATE INDEX IF NOT EXISTS qa_pair_merchant_idx ON qa_pair (merchant_id, status);
-- ---------------------------------------------------------------- 생성 감사 로그
CREATE TABLE IF NOT EXISTS generation_run (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
merchant_id uuid REFERENCES merchant(id) ON DELETE CASCADE,
provider text NOT NULL,
model text NOT NULL,
prompt_version text NOT NULL,
trigger text NOT NULL, -- published | scheduled | manual
status text NOT NULL DEFAULT 'running', -- running | succeeded | failed
input jsonb NOT NULL DEFAULT '{}'::jsonb,
output jsonb NOT NULL DEFAULT '{}'::jsonb,
stats jsonb NOT NULL DEFAULT '{}'::jsonb,
error text,
started_at timestamptz NOT NULL DEFAULT now(),
finished_at timestamptz
);
CREATE INDEX IF NOT EXISTS generation_run_merchant_idx
ON generation_run (merchant_id, started_at DESC);

8
nest-cli.json Normal file
View File

@ -0,0 +1,8 @@
{
"$schema": "https://json.schemastore.org/nest-cli",
"collection": "@nestjs/schematics",
"sourceRoot": "src",
"compilerOptions": {
"deleteOutDir": true
}
}

5449
package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

41
package.json Normal file
View File

@ -0,0 +1,41 @@
{
"name": "o2o-site-ontology",
"version": "0.1.0",
"description": "SEO/AEO keyword ontology service for o2o-site-AEO",
"private": true,
"scripts": {
"build": "nest build",
"start": "nest start",
"start:dev": "nest start --watch",
"start:prod": "node dist/main.js",
"db:up": "docker compose up -d",
"db:down": "docker compose down",
"db:migrate": "tsx src/db/migrate.ts",
"db:seed": "tsx src/db/seed.ts",
"db:reset": "docker compose down -v && docker compose up -d --wait && npm run db:migrate && npm run db:seed",
"smoke": "tsx scripts/smoke.ts"
},
"dependencies": {
"@nestjs/bullmq": "^11.0.2",
"@nestjs/common": "^11.0.12",
"@nestjs/core": "^11.0.12",
"@nestjs/platform-express": "^11.0.12",
"@nestjs/schedule": "^5.0.1",
"bullmq": "^5.44.0",
"class-transformer": "^0.5.1",
"class-validator": "^0.14.1",
"dotenv": "^16.4.7",
"openai": "^4.89.0",
"postgres": "^3.4.5",
"reflect-metadata": "^0.2.2",
"rxjs": "^7.8.2"
},
"devDependencies": {
"@nestjs/cli": "^11.0.5",
"@nestjs/schematics": "^11.0.2",
"@types/express": "^5.0.1",
"@types/node": "^22.13.14",
"tsx": "^4.19.3",
"typescript": "^5.8.2"
}
}

103
scripts/smoke.ts Normal file
View File

@ -0,0 +1,103 @@
/**
* 로컬 엔드투엔드 점검 스크립트.
* npm run db:reset && npm start (다른 터미널)
* npm run smoke
*/
const BASE = process.env.BASE_URL ?? 'http://localhost:3100';
const j = async (method: string, path: string, body?: unknown) => {
const res = await fetch(`${BASE}${path}`, {
method,
headers: body ? { 'content-type': 'application/json' } : undefined,
body: body ? JSON.stringify(body) : undefined,
});
const text = await res.text();
if (!res.ok) throw new Error(`${method} ${path} → ${res.status} ${text}`);
return text ? JSON.parse(text) : null;
};
const h = (t: string) => console.log(`\n\x1b[1m${t}\x1b[0m`);
async function main() {
h('0. health');
console.log(' ', await j('GET', '/health'));
h('1. site-1001 생성 (첫 업체 — 전부 신규)');
const a = await j('POST', '/v1/merchants/site-1001/generate?sync=true');
printStats(a);
h('2. site-1002 생성 (같은 강남/미용실 — 중복제거 발동)');
const b = await j('POST', '/v1/merchants/site-1002/generate?sync=true');
printStats(b);
printDetails(b);
h('3. publish 웹훅 + 표기 변형 (trigram 단계)');
const c = await j('POST', '/v1/merchants/publish', {
externalId: 'site-1003',
name: '강남 뷰티랩',
industryId: 'beauty.hair',
regionId: 'kr.seoul.gangnam',
description: '강남 미용실. 염색 전문.',
profile: { services: ['뿌리염색약', '여성펌'], features: ['주차가능'] },
sync: true,
});
printStats(c.generation);
printDetails(c.generation);
h('4. SEO payload');
const seo = await j('GET', '/v1/sites/site-1001/seo?limit=8');
console.log(' title :', seo.title);
console.log(' description:', seo.description);
console.log(' keywords :', seo.keywords.join(', '));
h('5. AEO payload');
const aeo = await j('GET', '/v1/sites/site-1001/aeo?limit=3');
for (const f of aeo.faqs) console.log(` Q. ${f.question}\n A. ${f.answer}`);
h('6. 의미 기반 키워드 검색');
const found = await j('POST', '/v1/keywords/search', { query: '강남 미용실 예약하고 싶어요', limit: 5 });
for (const r of found) console.log(` ${r.score.toFixed(3)} ${r.canonical} (${r.intent}, ${r.usage_count}개 업체)`);
h('7. 성과 피드백 → 저성과 강등');
console.log(
' ',
await j('POST', '/v1/sites/site-1001/performance', {
items: [
{ keyword: '강남 미용실 후기', impressions: 500, clicks: 0 },
{ keyword: '강남 미용실', impressions: 300, clicks: 40 },
],
}),
);
h('8. 비동기 큐 (BullMQ)');
console.log(' ', await j('POST', '/v1/merchants/site-2001/generate'));
for (let i = 0; i < 30; i++) {
const s = await j('GET', '/v1/sites/site-2001/seo?limit=5');
if (s.keywords.length) {
console.log(' 워커 처리 완료 →', s.keywords.join(', '));
break;
}
await new Promise((r) => setTimeout(r, 500));
}
console.log('\n✅ smoke 완료');
}
function printStats(s: any) {
console.log(
` 후보 ${s.candidates} → 신규 ${s.created} / 중복(정확 ${s.matchedExact}, 표기 ${s.matchedTrigram}, 의미 ${s.matchedVector})` +
` / 차단 ${s.rejected} / 연결 ${s.linked} / QA ${s.qaCreated} (${s.durationMs}ms)`,
);
}
function printDetails(s: any) {
for (const d of s.details ?? []) {
const sim = d.similarity != null ? ` (sim=${d.similarity.toFixed(3)} → '${d.matchedTo}')` : '';
console.log(` ${d.action.padEnd(16)} ${d.candidate}${sim}`);
}
}
main().catch((e) => {
console.error('\n❌', e.message);
process.exit(1);
});

29
src/app.module.ts Normal file
View File

@ -0,0 +1,29 @@
import { BullModule } from '@nestjs/bullmq';
import { Controller, Get, Module } from '@nestjs/common';
import { ScheduleModule } from '@nestjs/schedule';
import { env } from './config/env';
import { DbModule } from './db/db.module';
import { GenerationModule } from './generation/generation.module';
import { MerchantsHttpModule } from './merchants/merchants.controller.module';
import { ServingModule } from './serving/serving.module';
@Controller()
class HealthController {
@Get('health')
health() {
return { status: 'ok', llmProvider: env.llm.provider, ts: new Date().toISOString() };
}
}
@Module({
imports: [
DbModule,
ScheduleModule.forRoot(),
BullModule.forRoot({ connection: { host: env.redis.host, port: env.redis.port } }),
GenerationModule,
MerchantsHttpModule,
ServingModule,
],
controllers: [HealthController],
})
export class AppModule {}

30
src/config/env.ts Normal file
View File

@ -0,0 +1,30 @@
import 'dotenv/config';
const num = (v: string | undefined, d: number) => (v === undefined || v === '' ? d : Number(v));
export const env = {
port: num(process.env.PORT, 3100),
databaseUrl: process.env.DATABASE_URL ?? 'postgres://ontology:ontology@localhost:55432/ontology',
redis: {
host: process.env.REDIS_HOST ?? 'localhost',
port: num(process.env.REDIS_PORT, 56379),
},
llm: {
provider: (process.env.LLM_PROVIDER ?? 'mock') as 'mock' | 'openai',
apiKey: process.env.OPENAI_API_KEY ?? '',
model: process.env.OPENAI_MODEL ?? 'gpt-4.1-mini',
embeddingModel: process.env.OPENAI_EMBEDDING_MODEL ?? 'text-embedding-3-small',
},
dedup: {
cosineThreshold: num(process.env.DEDUP_COSINE_THRESHOLD, 0.92),
trigramThreshold: num(process.env.DEDUP_TRIGRAM_THRESHOLD, 0.6),
candidateLimit: num(process.env.DEDUP_CANDIDATE_LIMIT, 20),
},
generation: {
targetKeywords: num(process.env.GENERATION_TARGET_KEYWORDS, 15),
refreshIntervalDays: num(process.env.REFRESH_INTERVAL_DAYS, 30),
},
} as const;
export const EMBEDDING_DIM = 1536;
export const PROMPT_VERSION = 'kw-v1';

16
src/db/db.module.ts Normal file
View File

@ -0,0 +1,16 @@
import { Global, Module, OnModuleDestroy } from '@nestjs/common';
import { createSql, Sql } from './db';
export const PG = Symbol('PG');
@Global()
@Module({
providers: [{ provide: PG, useFactory: () => createSql() }],
exports: [PG],
})
export class DbModule implements OnModuleDestroy {
constructor() {}
async onModuleDestroy() {}
}
export type { Sql };

17
src/db/db.ts Normal file
View File

@ -0,0 +1,17 @@
import postgres from 'postgres';
import { env } from '../config/env';
export type Sql = postgres.Sql<{}>;
export const createSql = (): Sql =>
postgres(env.databaseUrl, {
max: 10,
// pgvector 컬럼은 텍스트로 주고받는다 ('[0.1,0.2,...]')
transform: { undefined: null },
});
/** number[] -> pgvector 리터럴 */
export const toVector = (v: number[]): string => `[${v.join(',')}]`;
/** postgres.js 의 JSONValue 타입 제약 우회용 캐스트 */
export const asJson = (v: unknown) => v as Parameters<Sql['json']>[0];

24
src/db/migrate.ts Normal file
View File

@ -0,0 +1,24 @@
import { readFileSync, readdirSync } from 'node:fs';
import { join } from 'node:path';
import { createSql } from './db';
async function main() {
const sql = createSql();
const dir = join(process.cwd(), 'drizzle');
const files = readdirSync(dir).filter((f) => f.endsWith('.sql')).sort();
for (const file of files) {
const ddl = readFileSync(join(dir, file), 'utf8');
process.stdout.write(`▶ applying ${file} ... `);
await sql.unsafe(ddl);
process.stdout.write('done\n');
}
await sql.end();
console.log('✅ migration complete');
}
main().catch((e) => {
console.error('❌ migration failed:', e);
process.exit(1);
});

92
src/db/seed.ts Normal file
View File

@ -0,0 +1,92 @@
import { asJson, createSql } from './db';
const industries = [
['beauty', 'beauty', '뷰티'],
['beauty.hair', 'beauty.hair', '미용실'],
['beauty.nail', 'beauty.nail', '네일샵'],
['food', 'food', '음식점'],
['food.korean', 'food.korean', '한식당'],
['health', 'health', '의료'],
['health.dental', 'health.dental', '치과'],
];
const regions = [
['kr', 'kr', '대한민국'],
['kr.seoul', 'kr.seoul', '서울'],
['kr.seoul.gangnam', 'kr.seoul.gangnam', '강남'],
['kr.seoul.mapo', 'kr.seoul.mapo', '마포'],
['kr.busan', 'kr.busan', '부산'],
['kr.busan.haeundae', 'kr.busan.haeundae', '해운대'],
];
const merchants = [
{
externalId: 'site-1001',
name: '레브살롱',
industryId: 'beauty.hair',
regionId: 'kr.seoul.gangnam',
description: '강남역 3번 출구 앞 프라이빗 헤어살롱. 1:1 디자이너 전담 시스템.',
siteUrl: 'https://rev-salon.example.com',
profile: {
services: ['남자 커트', '여성 펌', '뿌리염색', '두피 클리닉'],
features: ['주차 가능', '심야 영업', '예약제'],
priceRange: '30,000~120,000원',
},
},
{
externalId: 'site-1002',
name: '헤어랩 강남점',
industryId: 'beauty.hair',
regionId: 'kr.seoul.gangnam',
description: '강남 대형 헤어샵. 염색과 클리닉 전문.',
siteUrl: 'https://hairlab.example.com',
profile: {
services: ['뿌리 염색', '여성 펌', '두피클리닉'],
features: ['주차가능', '단체 예약'],
priceRange: '25,000~150,000원',
},
},
{
externalId: 'site-2001',
name: '해운대 소담한상',
industryId: 'food.korean',
regionId: 'kr.busan.haeundae',
description: '해운대 해변 인근 한정식집. 제철 해산물 코스 제공.',
siteUrl: 'https://sodam.example.com',
profile: {
services: ['한정식 코스', '점심 특선', '단체 예약'],
features: ['오션뷰', '룸 완비', '발렛파킹'],
priceRange: '25,000~80,000원',
},
},
];
async function main() {
const sql = createSql();
for (const [id, path, name] of industries) {
await sql`INSERT INTO industry (id, path, name) VALUES (${id}, ${path}::ltree, ${name})
ON CONFLICT (id) DO UPDATE SET name = EXCLUDED.name`;
}
for (const [id, path, name] of regions) {
await sql`INSERT INTO region (id, path, name) VALUES (${id}, ${path}::ltree, ${name})
ON CONFLICT (id) DO UPDATE SET name = EXCLUDED.name`;
}
for (const m of merchants) {
await sql`
INSERT INTO merchant (external_id, name, industry_id, region_id, description, profile, site_url)
VALUES (${m.externalId}, ${m.name}, ${m.industryId}, ${m.regionId},
${m.description}, ${sql.json(asJson(m.profile))}, ${m.siteUrl})
ON CONFLICT (external_id) DO UPDATE SET
name = EXCLUDED.name, description = EXCLUDED.description,
profile = EXCLUDED.profile, updated_at = now()`;
}
await sql.end();
console.log(`✅ seed: industry=${industries.length} region=${regions.length} merchant=${merchants.length}`);
}
main().catch((e) => {
console.error('❌ seed failed:', e);
process.exit(1);
});

View File

@ -0,0 +1,20 @@
import { BullModule } from '@nestjs/bullmq';
import { Module } from '@nestjs/common';
import { KeywordsModule } from '../keywords/keywords.module';
import { LlmModule } from '../llm/llm.module';
import { MerchantsModule } from '../merchants/merchants.module';
import { GenerationProcessor } from './generation.processor';
import { GENERATION_QUEUE, GenerationQueue } from './generation.queue';
import { GenerationService } from './generation.service';
@Module({
imports: [
BullModule.registerQueue({ name: GENERATION_QUEUE }),
LlmModule,
KeywordsModule,
MerchantsModule,
],
providers: [GenerationService, GenerationQueue, GenerationProcessor],
exports: [GenerationService, GenerationQueue],
})
export class GenerationModule {}

View File

@ -0,0 +1,21 @@
import { Processor, WorkerHost } from '@nestjs/bullmq';
import { Logger } from '@nestjs/common';
import { Job } from 'bullmq';
import { GenerationService } from './generation.service';
import { GENERATION_QUEUE, GenerationJob } from './generation.queue';
@Processor(GENERATION_QUEUE, { concurrency: 2 })
export class GenerationProcessor extends WorkerHost {
private readonly logger = new Logger(GenerationProcessor.name);
constructor(private readonly generation: GenerationService) {
super();
}
async process(job: Job<GenerationJob>) {
const { merchantId, trigger } = job.data;
this.logger.log(`processing ${job.id} (${trigger})`);
const stats = await this.generation.runForMerchant(merchantId, trigger);
return { ...stats, details: undefined };
}
}

View File

@ -0,0 +1,50 @@
import { InjectQueue } from '@nestjs/bullmq';
import { Injectable, Logger } from '@nestjs/common';
import { Cron, CronExpression } from '@nestjs/schedule';
import { Queue } from 'bullmq';
import { env } from '../config/env';
import { MerchantsService } from '../merchants/merchants.service';
import { GenerationTrigger } from './generation.service';
export const GENERATION_QUEUE = 'keyword-generation';
export interface GenerationJob {
merchantId: string;
trigger: GenerationTrigger;
}
@Injectable()
export class GenerationQueue {
private readonly logger = new Logger(GenerationQueue.name);
constructor(
@InjectQueue(GENERATION_QUEUE) private readonly queue: Queue<GenerationJob>,
private readonly merchants: MerchantsService,
) {}
async enqueue(merchantId: string, trigger: GenerationTrigger): Promise<string> {
// 짧은 시간 내 같은 업체가 여러 번 발행돼도 한 번만 처리 (60초 dedupe 창)
const job = await this.queue.add(
'generate',
{ merchantId, trigger },
{
deduplication: { id: `${merchantId}-${trigger}`, ttl: 60_000 },
removeOnComplete: 100,
removeOnFail: 500,
attempts: 3,
backoff: { type: 'exponential', delay: 5_000 },
},
);
return String(job.id);
}
/** 주기 리프레시: 매일 03:00, N일 지난 업체를 큐에 적재 */
@Cron(CronExpression.EVERY_DAY_AT_3AM)
async scheduleRefresh() {
const stale = await this.merchants.findStale(env.generation.refreshIntervalDays, 200);
for (const m of stale) {
await this.enqueue(m.id, 'scheduled');
}
if (stale.length) this.logger.log(`scheduled refresh queued: ${stale.length} merchants`);
}
}

View File

@ -0,0 +1,213 @@
import { Inject, Injectable, Logger } from '@nestjs/common';
import { PG } from '../db/db.module';
import { asJson, Sql, toVector } from '../db/db';
import { env, PROMPT_VERSION } from '../config/env';
import { DedupAction, DedupService } from '../keywords/dedup.service';
import { canonicalizeKeyword, normalizeKeyword } from '../keywords/normalize';
import { LlmProvider, MerchantContext } from '../llm/types';
import { MerchantsService } from '../merchants/merchants.service';
export type GenerationTrigger = 'published' | 'scheduled' | 'manual';
export interface GenerationStats {
runId: string;
merchantId: string;
merchantName: string;
provider: string;
model: string;
candidates: number;
created: number;
matchedExact: number;
matchedTrigram: number;
matchedVector: number;
rejected: number;
linked: number;
qaCreated: number;
durationMs: number;
details: Array<{ candidate: string; action: DedupAction; matchedTo?: string; similarity?: number }>;
}
@Injectable()
export class GenerationService {
private readonly logger = new Logger(GenerationService.name);
constructor(
@Inject(PG) private readonly sql: Sql,
private readonly merchants: MerchantsService,
private readonly llm: LlmProvider,
private readonly dedup: DedupService,
) {}
async runForMerchant(
idOrExternalId: string,
trigger: GenerationTrigger = 'manual',
): Promise<GenerationStats> {
const startedAt = Date.now();
const merchant = await this.merchants.findWithTaxonomy(idOrExternalId);
const runRows = await this.sql<Array<{ id: string }>>`
INSERT INTO generation_run (merchant_id, provider, model, prompt_version, trigger, status, input)
VALUES (${merchant.id}, ${this.llm.name}, ${this.llm.model}, ${PROMPT_VERSION},
${trigger}, 'running', ${this.sql.json(asJson({ externalId: merchant.external_id }))})
RETURNING id`;
const runId = runRows[0].id;
try {
const existing = await this.existingKeywordsFor(merchant.id, merchant.industry_id);
const ctx: MerchantContext = {
externalId: merchant.external_id,
name: merchant.name,
description: merchant.description,
industryName: merchant.industry_name,
industryPath: merchant.industry_path,
regionName: merchant.region_name,
regionPath: merchant.region_path,
profile: merchant.profile ?? {},
existingKeywords: existing,
targetCount: env.generation.targetKeywords,
};
const output = await this.llm.generate(ctx);
// 임베딩은 한 번에 배치 호출 (후보 수만큼 왕복하지 않는다)
const texts = output.keywords.map((k) => canonicalizeKeyword(k.keyword));
const embeddings = texts.length ? await this.llm.embed(texts) : [];
const stats: GenerationStats = {
runId,
merchantId: merchant.id,
merchantName: merchant.name,
provider: this.llm.name,
model: output.model,
candidates: output.keywords.length,
created: 0,
matchedExact: 0,
matchedTrigram: 0,
matchedVector: 0,
rejected: 0,
linked: 0,
qaCreated: 0,
durationMs: 0,
details: [],
};
for (let i = 0; i < output.keywords.length; i++) {
const cand = output.keywords[i];
const result = await this.dedup.resolve({
raw: cand.keyword,
intent: cand.intent,
embedding: embeddings[i],
locale: 'ko-KR',
industryId: merchant.industry_id,
regionId: merchant.region_id,
});
stats.details.push({
candidate: canonicalizeKeyword(cand.keyword),
action: result.action,
matchedTo: result.matchedTo,
similarity: result.similarity,
});
switch (result.action) {
case 'created': stats.created++; break;
case 'matched_exact': stats.matchedExact++; break;
case 'matched_trigram': stats.matchedTrigram++; break;
case 'matched_vector': stats.matchedVector++; break;
case 'rejected_banned': stats.rejected++; break;
}
if (result.keywordId) {
const linked = await this.linkKeyword(merchant.id, result.keywordId, cand.relevance, cand.rationale);
if (linked) stats.linked++;
}
}
stats.qaCreated = await this.upsertQaPairs(merchant.id, output.qaPairs);
await this.merchants.markGenerated(merchant.id);
stats.durationMs = Date.now() - startedAt;
await this.sql`
UPDATE generation_run
SET status = 'succeeded',
output = ${this.sql.json(asJson(output))},
stats = ${this.sql.json(asJson({ ...stats, details: undefined }))},
finished_at = now()
WHERE id = ${runId}`;
this.logger.log(
`[${merchant.name}] cand=${stats.candidates} new=${stats.created} ` +
`dup(exact/trg/vec)=${stats.matchedExact}/${stats.matchedTrigram}/${stats.matchedVector} ` +
`rejected=${stats.rejected} qa=${stats.qaCreated} ${stats.durationMs}ms`,
);
return stats;
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
await this.sql`
UPDATE generation_run
SET status = 'failed', error = ${message}, finished_at = now()
WHERE id = ${runId}`;
throw err;
}
}
/** 프롬프트에 넣을 "이미 보유한 키워드": 자기 것 + 같은 업종에서 많이 쓰는 것 */
private async existingKeywordsFor(merchantId: string, industryId: string | null): Promise<string[]> {
const rows = await this.sql<Array<{ canonical: string }>>`
SELECT DISTINCT k.canonical
FROM keyword k
LEFT JOIN merchant_keyword mk ON mk.keyword_id = k.id AND mk.merchant_id = ${merchantId}
WHERE mk.merchant_id IS NOT NULL
OR (${industryId}::text IS NOT NULL AND k.industry_id = ${industryId} AND k.usage_count > 0)
ORDER BY k.canonical
LIMIT 100`;
return rows.map((r) => r.canonical);
}
private async linkKeyword(
merchantId: string,
keywordId: string,
relevance: number,
rationale: string,
): Promise<boolean> {
const status = relevance >= 0.5 ? 'active' : 'candidate';
const rows = await this.sql<Array<{ inserted: boolean }>>`
INSERT INTO merchant_keyword (merchant_id, keyword_id, relevance, source, status, rationale)
VALUES (${merchantId}, ${keywordId}, ${relevance}, 'llm', ${status}, ${rationale})
ON CONFLICT (merchant_id, keyword_id) DO UPDATE SET
relevance = GREATEST(merchant_keyword.relevance, EXCLUDED.relevance),
rationale = COALESCE(EXCLUDED.rationale, merchant_keyword.rationale),
updated_at = now()
RETURNING (xmax = 0) AS inserted`;
if (rows[0]?.inserted) {
await this.sql`UPDATE keyword SET usage_count = usage_count + 1 WHERE id = ${keywordId}`;
return true;
}
return false;
}
private async upsertQaPairs(
merchantId: string,
pairs: Array<{ question: string; answer: string }>,
): Promise<number> {
if (pairs.length === 0) return 0;
const embeddings = await this.llm.embed(pairs.map((p) => p.question));
let created = 0;
for (let i = 0; i < pairs.length; i++) {
const p = pairs[i];
const nq = normalizeKeyword(p.question);
if (!nq) continue;
const rows = await this.sql<Array<{ inserted: boolean }>>`
INSERT INTO qa_pair (merchant_id, question, answer, normalized_question, embedding)
VALUES (${merchantId}, ${canonicalizeKeyword(p.question)}, ${p.answer.trim()},
${nq}, ${toVector(embeddings[i])}::vector)
ON CONFLICT (merchant_id, normalized_question) DO UPDATE SET
answer = EXCLUDED.answer, updated_at = now()
RETURNING (xmax = 0) AS inserted`;
if (rows[0]?.inserted) created++;
}
return created;
}
}

View File

@ -0,0 +1,107 @@
import { Injectable, Logger } from '@nestjs/common';
import { env } from '../config/env';
import { KeywordIntent } from '../llm/types';
import { KeywordRepository } from './keyword.repository';
import { canonicalizeKeyword, isBanned, normalizeKeyword } from './normalize';
export type DedupAction =
| 'created' // 새 키워드
| 'matched_exact' // 1단계: 정규화 해시 일치
| 'matched_trigram' // 2단계: 표기 변형/오타
| 'matched_vector' // 3단계: 의미 중복 → alias 흡수
| 'rejected_banned'; // 금칙어
export interface DedupResult {
action: DedupAction;
keywordId: string | null;
canonical: string;
matchedTo?: string;
similarity?: number;
}
export interface ResolveInput {
raw: string;
intent: KeywordIntent;
embedding: number[];
locale: string;
industryId: string | null;
regionId: string | null;
}
/**
* 4단계 계단식 중복제거.
* 값비싼 벡터 비교는 마지막에, 후보 집합 안에서만 수행한다.
*/
@Injectable()
export class DedupService {
private readonly logger = new Logger(DedupService.name);
constructor(private readonly repo: KeywordRepository) {}
async resolve(input: ResolveInput): Promise<DedupResult> {
const canonical = canonicalizeKeyword(input.raw);
const normalized = normalizeKeyword(input.raw);
// 0단계 — 금칙어/과장광고 차단
if (!normalized || isBanned(canonical)) {
return { action: 'rejected_banned', keywordId: null, canonical };
}
// 1단계 — 정규화 완전 일치 (공백/구두점 차이 흡수)
const exact = await this.repo.findByNormalized(normalized, input.locale);
if (exact) {
await this.repo.absorbAlias(exact.id, canonical);
return {
action: 'matched_exact',
keywordId: exact.id,
canonical: exact.canonical,
matchedTo: exact.canonical,
similarity: 1,
};
}
// 2~3단계 — trigram 후보 + 벡터 ANN 후보를 모아 최고 유사도 판정
const candidates = await this.repo.findDedupCandidates(
input.embedding,
normalized,
input.locale,
env.dedup.candidateLimit,
);
const trigramHit = candidates.find((c) => c.trg >= env.dedup.trigramThreshold);
if (trigramHit) {
await this.repo.absorbAlias(trigramHit.id, canonical);
return {
action: 'matched_trigram',
keywordId: trigramHit.id,
canonical: trigramHit.canonical,
matchedTo: trigramHit.canonical,
similarity: trigramHit.trg,
};
}
const best = candidates[0];
if (best && best.cosine >= env.dedup.cosineThreshold) {
await this.repo.absorbAlias(best.id, canonical);
return {
action: 'matched_vector',
keywordId: best.id,
canonical: best.canonical,
matchedTo: best.canonical,
similarity: best.cosine,
};
}
// 4단계 — 신규 등록
const created = await this.repo.insert({
canonical,
normalized,
locale: input.locale,
intent: input.intent,
embedding: input.embedding,
industryId: input.industryId,
regionId: input.regionId,
});
return { action: 'created', keywordId: created.id, canonical: created.canonical };
}
}

View File

@ -0,0 +1,124 @@
import { Inject, Injectable } from '@nestjs/common';
import { PG } from '../db/db.module';
import { Sql, toVector } from '../db/db';
import { KeywordIntent } from '../llm/types';
export interface KeywordRow {
id: string;
canonical: string;
normalized: string;
aliases: string[];
intent: KeywordIntent;
usage_count: number;
}
export interface CandidateRow {
id: string;
canonical: string;
normalized: string;
cosine: number;
trg: number;
}
@Injectable()
export class KeywordRepository {
constructor(@Inject(PG) private readonly sql: Sql) {}
async findByNormalized(normalized: string, locale: string): Promise<KeywordRow | null> {
const rows = await this.sql<KeywordRow[]>`
SELECT id, canonical, normalized, aliases, intent, usage_count
FROM keyword
WHERE normalized = ${normalized} AND locale = ${locale}
LIMIT 1`;
return rows[0] ?? null;
}
/**
* 중복 후보 수집: trigram 인덱스 히트 + 벡터 ANN 상위 N 을 합집합으로 가져온다.
* 벡터 비교는 이 후보 집합 안에서만 하므로 전수 비교가 일어나지 않는다.
*/
async findDedupCandidates(
embedding: number[],
normalized: string,
locale: string,
limit: number,
): Promise<CandidateRow[]> {
const vec = toVector(embedding);
const rows = await this.sql<CandidateRow[]>`
(
SELECT id, canonical, normalized,
1 - (embedding <=> ${vec}::vector) AS cosine,
similarity(normalized, ${normalized}) AS trg
FROM keyword
WHERE locale = ${locale}
AND embedding IS NOT NULL
AND normalized % ${normalized}
ORDER BY trg DESC
LIMIT ${limit}
)
UNION ALL
(
SELECT id, canonical, normalized,
1 - (embedding <=> ${vec}::vector) AS cosine,
0::real AS trg
FROM keyword
WHERE locale = ${locale}
AND embedding IS NOT NULL
ORDER BY embedding <=> ${vec}::vector
LIMIT ${limit}
)`;
const best = new Map<string, CandidateRow>();
for (const r of rows) {
const prev = best.get(r.id);
if (!prev || r.trg > prev.trg) best.set(r.id, { ...r, cosine: Number(r.cosine), trg: Number(r.trg) });
}
return [...best.values()].sort((a, b) => b.cosine - a.cosine);
}
async insert(input: {
canonical: string;
normalized: string;
locale: string;
intent: KeywordIntent;
embedding: number[];
industryId: string | null;
regionId: string | null;
}): Promise<KeywordRow> {
const rows = await this.sql<KeywordRow[]>`
INSERT INTO keyword (canonical, normalized, locale, intent, embedding, industry_id, region_id, usage_count)
VALUES (${input.canonical}, ${input.normalized}, ${input.locale}, ${input.intent},
${toVector(input.embedding)}::vector, ${input.industryId}, ${input.regionId}, 0)
ON CONFLICT (normalized, locale) DO UPDATE SET updated_at = now()
RETURNING id, canonical, normalized, aliases, intent, usage_count`;
return rows[0];
}
/** 표기 변형을 기존 키워드에 흡수 (롱테일 검색어 보존) */
async absorbAlias(keywordId: string, alias: string): Promise<void> {
await this.sql`
UPDATE keyword
SET aliases = (
SELECT ARRAY(SELECT DISTINCT unnest(aliases || ARRAY[${alias}]::text[]))
),
updated_at = now()
WHERE id = ${keywordId}
AND NOT (${alias} = ANY(aliases))
AND canonical <> ${alias}`;
}
async bumpUsage(keywordId: string): Promise<void> {
await this.sql`
UPDATE keyword SET usage_count = usage_count + 1, updated_at = now() WHERE id = ${keywordId}`;
}
async searchByVector(embedding: number[], locale: string, limit: number) {
const vec = toVector(embedding);
return this.sql<Array<{ id: string; canonical: string; intent: string; usage_count: number; score: number }>>`
SELECT id, canonical, intent, usage_count, 1 - (embedding <=> ${vec}::vector) AS score
FROM keyword
WHERE locale = ${locale} AND embedding IS NOT NULL
ORDER BY embedding <=> ${vec}::vector
LIMIT ${limit}`;
}
}

View File

@ -0,0 +1,9 @@
import { Module } from '@nestjs/common';
import { DedupService } from './dedup.service';
import { KeywordRepository } from './keyword.repository';
@Module({
providers: [KeywordRepository, DedupService],
exports: [KeywordRepository, DedupService],
})
export class KeywordsModule {}

33
src/keywords/normalize.ts Normal file
View File

@ -0,0 +1,33 @@
/**
* 중복 판정용 정규화.
* NFKC → 소문자 → 제로폭 문자 제거 → 구두점 제거 → 공백 전부 제거.
* "강남 미용실" 과 "강남미용실" 을 같은 키로 취급하기 위해 공백을 없앤다.
*/
const ZERO_WIDTH = /[\u200B-\u200D\uFEFF]/g;
const PUNCT = /[!-\/:-@\[-`{-~·ㆍ、。「-』]/g;
export function normalizeKeyword(raw: string): string {
return raw
.normalize('NFKC')
.toLowerCase()
.replace(ZERO_WIDTH, '')
.replace(PUNCT, '')
.replace(/\s+/g, '');
}
/** 표시용 정리: 앞뒤/중복 공백만 정리하고 원문 표기는 보존 */
export function canonicalizeKeyword(raw: string): string {
return raw.normalize('NFKC').replace(ZERO_WIDTH, '').replace(/\s+/g, ' ').trim();
}
/** 과장광고·금칙 표현 필터 (광고심의 리스크 차단) */
const BANNED = [
'최고', '1위', '일등', '넘버원', 'no.1', '100%', '무조건', '완치', '부작용없',
'영구', '평생보장', '유일한', '최저가보장', '전국최대',
];
const BANNED_NORMALIZED = BANNED.map(normalizeKeyword);
export function isBanned(text: string): boolean {
const n = normalizeKeyword(text);
return BANNED_NORMALIZED.some((b) => b.length > 0 && n.includes(b));
}

16
src/llm/llm.module.ts Normal file
View File

@ -0,0 +1,16 @@
import { Module } from '@nestjs/common';
import { env } from '../config/env';
import { MockLlmProvider } from './mock.provider';
import { OpenAiLlmProvider } from './openai.provider';
import { LlmProvider } from './types';
@Module({
providers: [
{
provide: LlmProvider,
useClass: env.llm.provider === 'openai' ? OpenAiLlmProvider : MockLlmProvider,
},
],
exports: [LlmProvider],
})
export class LlmModule {}

115
src/llm/mock.provider.ts Normal file
View File

@ -0,0 +1,115 @@
import { Injectable } from '@nestjs/common';
import { EMBEDDING_DIM } from '../config/env';
import { normalizeKeyword } from '../keywords/normalize';
import {
GenerationOutput,
KeywordCandidate,
KeywordIntent,
LlmProvider,
MerchantContext,
QaCandidate,
} from './types';
/**
* API 키 없이 로컬에서 전체 파이프라인(생성 → 중복제거 → 서빙)을 돌리기 위한 대체 구현.
*
* embed(): 문자 bigram 해싱 + L2 정규화.
* 랜덤이 아니라 "비슷한 문자열이면 비슷한 벡터"가 나오므로
* 코사인 임계값 기반 중복제거 동작을 실제와 유사하게 검증할 수 있다.
*/
@Injectable()
export class MockLlmProvider extends LlmProvider {
readonly name = 'mock';
readonly model = 'mock-keyword-v1';
async embed(texts: string[]): Promise<number[][]> {
return texts.map((t) => hashEmbedding(t));
}
async generate(ctx: MerchantContext): Promise<GenerationOutput> {
const region = ctx.regionName ?? '';
const industry = ctx.industryName ?? '업체';
const services = toStringArray(ctx.profile['services']);
const features = toStringArray(ctx.profile['features']);
const modifiers = ['추천', '예약', '가격', '후기', '잘하는곳', '근처'];
const raw: Array<[string, KeywordIntent, number]> = [];
raw.push([`${region} ${industry}`.trim(), 'local', 0.95]);
raw.push([ctx.name, 'brand', 0.99]);
for (const m of modifiers) {
raw.push([`${region} ${industry} ${m}`.trim(), m === '예약' ? 'transactional' : 'local', 0.8]);
}
for (const s of services) {
raw.push([`${region} ${s}`.trim(), 'local', 0.85]);
raw.push([`${s} 가격`, 'transactional', 0.7]);
raw.push([`${s} 잘하는 곳`, 'informational', 0.65]);
}
for (const f of features) {
raw.push([`${industry} ${f}`.trim(), 'informational', 0.6]);
}
// 표기 변형을 일부러 섞는다 — 중복제거 단계가 실제로 흡수하는지 확인용
raw.push([`${region}${industry}추천`, 'local', 0.5]);
raw.push([`${region} ${industry} 추천`, 'local', 0.5]);
const seen = new Set<string>();
const keywords: KeywordCandidate[] = [];
for (const [keyword, intent, relevance] of raw) {
const k = keyword.replace(/\s+/g, ' ').trim();
if (!k) continue;
if (seen.has(k)) continue;
seen.add(k);
keywords.push({ keyword: k, intent, relevance, rationale: `mock: ${intent}` });
if (keywords.length >= ctx.targetCount + 4) break;
}
const qaPairs: QaCandidate[] = [
{
question: `${ctx.name}은(는) 어디에 있나요?`,
answer: `${ctx.name}은(는) ${region || '해당 지역'}에 위치한 ${industry}입니다.`,
},
{
question: `${ctx.name} 예약은 어떻게 하나요?`,
answer: `${ctx.name}은(는) 사이트 예약 페이지 또는 전화로 예약할 수 있습니다.`,
},
{
question: `${ctx.name}의 주요 서비스는 무엇인가요?`,
answer: services.length
? `주요 서비스는 ${services.join(', ')} 입니다.`
: `${industry} 관련 서비스를 제공합니다.`,
},
];
return { keywords, qaPairs, model: this.model, provider: this.name };
}
}
function toStringArray(v: unknown): string[] {
return Array.isArray(v) ? v.filter((x): x is string => typeof x === 'string') : [];
}
/** 문자 bigram 해싱 임베딩 (결정적, L2 정규화) */
export function hashEmbedding(text: string, dim = EMBEDDING_DIM): number[] {
const s = ` ${normalizeKeyword(text)} `;
const vec = new Float64Array(dim);
for (let i = 0; i < s.length - 1; i++) {
const gram = s.slice(i, i + 2);
const h = fnv1a(gram);
vec[h % dim] += 1;
// 부호 해싱으로 충돌 편향 완화
vec[(h >>> 8) % dim] += h & 1 ? 1 : -1;
}
let norm = 0;
for (let i = 0; i < dim; i++) norm += vec[i] * vec[i];
norm = Math.sqrt(norm) || 1;
return Array.from(vec, (x) => x / norm);
}
function fnv1a(str: string): number {
let h = 0x811c9dc5;
for (let i = 0; i < str.length; i++) {
h ^= str.charCodeAt(i);
h = Math.imul(h, 0x01000193) >>> 0;
}
return h >>> 0;
}

122
src/llm/openai.provider.ts Normal file
View File

@ -0,0 +1,122 @@
import { Injectable, Logger } from '@nestjs/common';
import OpenAI from 'openai';
import { env } from '../config/env';
import { GenerationOutput, LlmProvider, MerchantContext } from './types';
/** Structured Outputs 로 강제하는 응답 스키마 — 자유 텍스트 파싱 금지 */
const RESPONSE_SCHEMA = {
type: 'object',
additionalProperties: false,
required: ['keywords', 'qa_pairs'],
properties: {
keywords: {
type: 'array',
items: {
type: 'object',
additionalProperties: false,
required: ['keyword', 'intent', 'relevance', 'rationale'],
properties: {
keyword: { type: 'string' },
intent: {
type: 'string',
enum: ['informational', 'navigational', 'transactional', 'local', 'brand'],
},
relevance: { type: 'number' },
rationale: { type: 'string' },
},
},
},
qa_pairs: {
type: 'array',
items: {
type: 'object',
additionalProperties: false,
required: ['question', 'answer'],
properties: {
question: { type: 'string' },
answer: { type: 'string' },
},
},
},
},
} as const;
@Injectable()
export class OpenAiLlmProvider extends LlmProvider {
readonly name = 'openai';
readonly model = env.llm.model;
private readonly logger = new Logger(OpenAiLlmProvider.name);
private readonly client = new OpenAI({ apiKey: env.llm.apiKey });
async embed(texts: string[]): Promise<number[][]> {
if (texts.length === 0) return [];
const res = await this.client.embeddings.create({
model: env.llm.embeddingModel,
input: texts,
});
return res.data.map((d) => d.embedding as number[]);
}
async generate(ctx: MerchantContext): Promise<GenerationOutput> {
const res = await this.client.chat.completions.create({
model: this.model,
temperature: 0.7,
messages: [
{ role: 'system', content: SYSTEM_PROMPT },
{ role: 'user', content: buildUserPrompt(ctx) },
],
response_format: {
type: 'json_schema',
json_schema: { name: 'seo_aeo_keywords', strict: true, schema: RESPONSE_SCHEMA as any },
},
});
const content = res.choices[0]?.message?.content ?? '{}';
const parsed = JSON.parse(content) as {
keywords?: GenerationOutput['keywords'];
qa_pairs?: GenerationOutput['qaPairs'];
};
return {
keywords: parsed.keywords ?? [],
qaPairs: parsed.qa_pairs ?? [],
model: this.model,
provider: this.name,
usage: {
prompt_tokens: res.usage?.prompt_tokens ?? 0,
completion_tokens: res.usage?.completion_tokens ?? 0,
total_tokens: res.usage?.total_tokens ?? 0,
},
};
}
}
const SYSTEM_PROMPT = `당신은 한국 로컬 비즈니스 SEO/AEO 전문가입니다.
주어진 업체 정보를 바탕으로 검색 유입에 실제로 도움이 되는 키워드와,
답변엔진(AI 검색)이 인용하기 좋은 질문-답변 쌍을 생성합니다.
규칙:
- 실제 사용자가 검색창에 입력할 법한 자연스러운 한국어 표현만 사용합니다.
- 지역명 + 업종 + 의도어 조합을 적극 활용합니다.
- 과장광고 표현(최고, 1위, 100%, 무조건, 완치 등)은 절대 사용하지 않습니다.
- 제공된 "이미 보유한 키워드"와 의미가 겹치는 것은 생성하지 않습니다.
- relevance 는 해당 업체와의 관련도를 0.0~1.0 으로 매깁니다.
- 답변(answer)은 2~3문장, 업체 정보에 근거한 사실만 씁니다.`;
function buildUserPrompt(ctx: MerchantContext): string {
return [
`# 업체 정보`,
`- 상호: ${ctx.name}`,
`- 업종: ${ctx.industryName ?? '미상'} (${ctx.industryPath ?? '-'})`,
`- 지역: ${ctx.regionName ?? '미상'} (${ctx.regionPath ?? '-'})`,
`- 소개: ${ctx.description || '없음'}`,
`- 상세: ${JSON.stringify(ctx.profile, null, 2)}`,
``,
`# 이미 보유한 키워드 (이것들과 겹치지 않는 새 후보만 생성)`,
ctx.existingKeywords.length ? ctx.existingKeywords.map((k) => `- ${k}`).join('\n') : '- (없음)',
``,
`# 요청`,
`- 키워드 ${ctx.targetCount}개`,
`- 질문-답변 쌍 5개`,
].join('\n');
}

47
src/llm/types.ts Normal file
View File

@ -0,0 +1,47 @@
export type KeywordIntent =
| 'informational'
| 'navigational'
| 'transactional'
| 'local'
| 'brand';
export interface MerchantContext {
externalId: string;
name: string;
description: string;
industryName?: string | null;
industryPath?: string | null;
regionName?: string | null;
regionPath?: string | null;
profile: Record<string, unknown>;
/** 이미 보유한 키워드 — 프롬프트에 넣어 중복 후보 생성 자체를 줄인다 */
existingKeywords: string[];
targetCount: number;
}
export interface KeywordCandidate {
keyword: string;
intent: KeywordIntent;
relevance: number; // 0..1
rationale: string;
}
export interface QaCandidate {
question: string;
answer: string;
}
export interface GenerationOutput {
keywords: KeywordCandidate[];
qaPairs: QaCandidate[];
model: string;
provider: string;
usage?: Record<string, number>;
}
export abstract class LlmProvider {
abstract readonly name: string;
abstract readonly model: string;
abstract generate(ctx: MerchantContext): Promise<GenerationOutput>;
abstract embed(texts: string[]): Promise<number[][]>;
}

17
src/main.ts Normal file
View File

@ -0,0 +1,17 @@
import 'reflect-metadata';
import { Logger, ValidationPipe } from '@nestjs/common';
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { env } from './config/env';
async function bootstrap() {
const app = await NestFactory.create(AppModule);
app.useGlobalPipes(new ValidationPipe({ transform: true, whitelist: true }));
app.enableCors();
await app.listen(env.port);
new Logger('bootstrap').log(
`o2o-site-ontology listening on http://localhost:${env.port} (llm=${env.llm.provider})`,
);
}
bootstrap();

View File

@ -0,0 +1,10 @@
import { Module } from '@nestjs/common';
import { GenerationModule } from '../generation/generation.module';
import { MerchantsController } from './merchants.controller';
import { MerchantsModule } from './merchants.module';
@Module({
imports: [MerchantsModule, GenerationModule],
controllers: [MerchantsController],
})
export class MerchantsHttpModule {}

View File

@ -0,0 +1,48 @@
import { Body, Controller, Get, Param, Post, Query } from '@nestjs/common';
import { GenerationService } from '../generation/generation.service';
import { GenerationQueue } from '../generation/generation.queue';
import { MerchantsService, UpsertMerchantDto } from './merchants.service';
@Controller('v1/merchants')
export class MerchantsController {
constructor(
private readonly merchants: MerchantsService,
private readonly generation: GenerationService,
private readonly queue: GenerationQueue,
) {}
@Get()
list() {
return this.merchants.list();
}
@Get(':id')
get(@Param('id') id: string) {
return this.merchants.findWithTaxonomy(id);
}
/** o2o-site-AEO 사이트 발행 웹훅: 업체 등록 + 키워드 생성 예약 */
@Post('publish')
async publish(@Body() dto: UpsertMerchantDto & { generate?: boolean; sync?: boolean }) {
const merchant = await this.merchants.upsert(dto);
if (dto.generate === false) return { merchant, generation: 'skipped' };
if (dto.sync) {
const stats = await this.generation.runForMerchant(merchant.id, 'published');
return { merchant, generation: stats };
}
const jobId = await this.queue.enqueue(merchant.id, 'published');
return { merchant, generation: { queued: true, jobId } };
}
/** 수동 재생성 */
@Post(':id/generate')
async generate(@Param('id') id: string, @Query('sync') sync?: string) {
if (sync === 'true' || sync === '1') {
return this.generation.runForMerchant(id, 'manual');
}
const m = await this.merchants.findWithTaxonomy(id);
const jobId = await this.queue.enqueue(m.id, 'manual');
return { queued: true, jobId };
}
}

View File

@ -0,0 +1,8 @@
import { Module } from '@nestjs/common';
import { MerchantsService } from './merchants.service';
@Module({
providers: [MerchantsService],
exports: [MerchantsService],
})
export class MerchantsModule {}

View File

@ -0,0 +1,92 @@
import { Inject, Injectable, NotFoundException } from '@nestjs/common';
import { PG } from '../db/db.module';
import { asJson, Sql } from '../db/db';
export interface MerchantRow {
id: string;
external_id: string;
name: string;
industry_id: string | null;
region_id: string | null;
description: string;
profile: Record<string, unknown>;
site_url: string | null;
last_generated_at: Date | null;
}
export interface MerchantWithTaxonomy extends MerchantRow {
industry_name: string | null;
industry_path: string | null;
region_name: string | null;
region_path: string | null;
}
export interface UpsertMerchantDto {
externalId: string;
name: string;
industryId?: string | null;
regionId?: string | null;
description?: string;
profile?: Record<string, unknown>;
siteUrl?: string | null;
}
@Injectable()
export class MerchantsService {
constructor(@Inject(PG) private readonly sql: Sql) {}
/** o2o-site-AEO 가 사이트를 발행할 때 호출하는 진입점 */
async upsert(dto: UpsertMerchantDto): Promise<MerchantRow> {
const rows = await this.sql<MerchantRow[]>`
INSERT INTO merchant (external_id, name, industry_id, region_id, description, profile, site_url)
VALUES (${dto.externalId}, ${dto.name}, ${dto.industryId ?? null}, ${dto.regionId ?? null},
${dto.description ?? ''}, ${this.sql.json(asJson(dto.profile ?? {}))}, ${dto.siteUrl ?? null})
ON CONFLICT (external_id) DO UPDATE SET
name = EXCLUDED.name,
industry_id = EXCLUDED.industry_id,
region_id = EXCLUDED.region_id,
description = EXCLUDED.description,
profile = EXCLUDED.profile,
site_url = EXCLUDED.site_url,
updated_at = now()
RETURNING *`;
return rows[0];
}
async findWithTaxonomy(idOrExternalId: string): Promise<MerchantWithTaxonomy> {
const rows = await this.sql<MerchantWithTaxonomy[]>`
SELECT m.*,
i.name AS industry_name, i.path::text AS industry_path,
r.name AS region_name, r.path::text AS region_path
FROM merchant m
LEFT JOIN industry i ON i.id = m.industry_id
LEFT JOIN region r ON r.id = m.region_id
WHERE m.external_id = ${idOrExternalId}
OR (${isUuid(idOrExternalId)} AND m.id::text = ${idOrExternalId})
LIMIT 1`;
if (!rows[0]) throw new NotFoundException(`merchant not found: ${idOrExternalId}`);
return rows[0];
}
async list(limit = 50) {
return this.sql<MerchantRow[]>`
SELECT * FROM merchant ORDER BY created_at DESC LIMIT ${limit}`;
}
/** 주기 리프레시 대상: 한 번도 생성 안 됐거나 N일 지난 업체 */
async findStale(intervalDays: number, limit: number) {
return this.sql<Array<{ id: string; external_id: string }>>`
SELECT id, external_id FROM merchant
WHERE last_generated_at IS NULL
OR last_generated_at < now() - (${intervalDays} || ' days')::interval
ORDER BY last_generated_at NULLS FIRST
LIMIT ${limit}`;
}
async markGenerated(merchantId: string) {
await this.sql`UPDATE merchant SET last_generated_at = now() WHERE id = ${merchantId}`;
}
}
const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
const isUuid = (v: string) => UUID_RE.test(v);

View File

@ -0,0 +1,39 @@
import { Body, Controller, Get, Param, Post, Query } from '@nestjs/common';
import { ServingService } from './serving.service';
@Controller('v1')
export class ServingController {
constructor(private readonly serving: ServingService) {}
/** 발행된 사이트가 렌더링 시 호출 — SEO 메타 */
@Get('sites/:id/seo')
seo(@Param('id') id: string, @Query('limit') limit?: string) {
return this.serving.seo(id, clamp(limit, 20, 50));
}
/** 발행된 사이트가 렌더링 시 호출 — AEO(답변엔진) 소스 */
@Get('sites/:id/aeo')
aeo(@Param('id') id: string, @Query('limit') limit?: string) {
return this.serving.aeo(id, clamp(limit, 10, 30));
}
/** 어드민: 의미 기반 키워드 검색 */
@Post('keywords/search')
search(@Body() body: { query: string; limit?: number }) {
return this.serving.searchKeywords(body.query, Math.min(body.limit ?? 10, 50));
}
/** 성과 피드백 주입 (Search Console / 유입 로그) */
@Post('sites/:id/performance')
performance(
@Param('id') id: string,
@Body() body: { items: Array<{ keyword: string; impressions: number; clicks: number }> },
) {
return this.serving.applyPerformance(id, body.items ?? []);
}
}
function clamp(v: string | undefined, def: number, max: number): number {
const n = v ? Number(v) : def;
return Number.isFinite(n) ? Math.min(Math.max(1, n), max) : def;
}

View File

@ -0,0 +1,13 @@
import { Module } from '@nestjs/common';
import { KeywordsModule } from '../keywords/keywords.module';
import { LlmModule } from '../llm/llm.module';
import { MerchantsModule } from '../merchants/merchants.module';
import { ServingController } from './serving.controller';
import { ServingService } from './serving.service';
@Module({
imports: [MerchantsModule, KeywordsModule, LlmModule],
controllers: [ServingController],
providers: [ServingService],
})
export class ServingModule {}

View File

@ -0,0 +1,144 @@
import { Inject, Injectable } from '@nestjs/common';
import { PG } from '../db/db.module';
import { Sql } from '../db/db';
import { KeywordRepository } from '../keywords/keyword.repository';
import { LlmProvider } from '../llm/types';
import { MerchantsService } from '../merchants/merchants.service';
export interface SeoPayload {
merchant: { id: string; externalId: string; name: string; siteUrl: string | null };
title: string;
description: string;
keywords: string[];
tags: Array<{ keyword: string; intent: string; relevance: number; aliases: string[] }>;
generatedAt: string | null;
}
export interface AeoPayload {
merchant: { id: string; externalId: string; name: string };
topics: string[];
faqs: Array<{ question: string; answer: string }>;
/** JSON-LD 는 후속 단계에서 이 payload 를 그대로 매핑해 생성한다 */
structuredDataHints: {
type: 'LocalBusiness';
name: string;
description: string;
areaServed: string | null;
category: string | null;
};
}
@Injectable()
export class ServingService {
constructor(
@Inject(PG) private readonly sql: Sql,
private readonly merchants: MerchantsService,
private readonly keywords: KeywordRepository,
private readonly llm: LlmProvider,
) {}
async seo(idOrExternalId: string, limit: number): Promise<SeoPayload> {
const m = await this.merchants.findWithTaxonomy(idOrExternalId);
const rows = await this.sql<
Array<{ canonical: string; intent: string; relevance: number; aliases: string[] }>
>`
SELECT k.canonical, k.intent, mk.relevance, k.aliases
FROM merchant_keyword mk
JOIN keyword k ON k.id = mk.keyword_id
WHERE mk.merchant_id = ${m.id} AND mk.status = 'active'
ORDER BY mk.relevance DESC, k.usage_count ASC
LIMIT ${limit}`;
const kws = rows.map((r) => r.canonical);
const locality = [m.region_name, m.industry_name].filter(Boolean).join(' ');
return {
merchant: { id: m.id, externalId: m.external_id, name: m.name, siteUrl: m.site_url },
title: locality ? `${m.name} | ${locality}` : m.name,
description: buildDescription(m.name, m.description, kws),
keywords: kws,
tags: rows.map((r) => ({
keyword: r.canonical,
intent: r.intent,
relevance: Number(r.relevance),
aliases: r.aliases ?? [],
})),
generatedAt: m.last_generated_at ? new Date(m.last_generated_at).toISOString() : null,
};
}
async aeo(idOrExternalId: string, limit: number): Promise<AeoPayload> {
const m = await this.merchants.findWithTaxonomy(idOrExternalId);
const faqs = await this.sql<Array<{ question: string; answer: string }>>`
SELECT question, answer FROM qa_pair
WHERE merchant_id = ${m.id} AND status = 'active'
ORDER BY created_at ASC
LIMIT ${limit}`;
const topics = await this.sql<Array<{ canonical: string }>>`
SELECT k.canonical FROM merchant_keyword mk
JOIN keyword k ON k.id = mk.keyword_id
WHERE mk.merchant_id = ${m.id} AND mk.status = 'active'
AND k.intent IN ('informational', 'local')
ORDER BY mk.relevance DESC LIMIT ${limit}`;
return {
merchant: { id: m.id, externalId: m.external_id, name: m.name },
topics: topics.map((t) => t.canonical),
faqs,
structuredDataHints: {
type: 'LocalBusiness',
name: m.name,
description: m.description,
areaServed: m.region_name,
category: m.industry_name,
},
};
}
async searchKeywords(query: string, limit: number) {
const [embedding] = await this.llm.embed([query]);
return this.keywords.searchByVector(embedding, 'ko-KR', limit);
}
/** Search Console / 유입 로그 피드백 → 저성과 키워드 강등 */
async applyPerformance(
idOrExternalId: string,
items: Array<{ keyword: string; impressions: number; clicks: number }>,
) {
const m = await this.merchants.findWithTaxonomy(idOrExternalId);
let updated = 0;
for (const it of items) {
const rows = await this.sql<Array<{ keyword_id: string }>>`
UPDATE merchant_keyword mk
SET impressions = mk.impressions + ${it.impressions},
clicks = mk.clicks + ${it.clicks},
ctr = CASE WHEN (mk.impressions + ${it.impressions}) > 0
THEN (mk.clicks + ${it.clicks})::real / (mk.impressions + ${it.impressions})
ELSE 0 END,
updated_at = now()
FROM keyword k
WHERE k.id = mk.keyword_id
AND mk.merchant_id = ${m.id}
AND (k.canonical = ${it.keyword} OR ${it.keyword} = ANY(k.aliases))
RETURNING mk.keyword_id`;
updated += rows.length;
}
// 노출은 충분한데 클릭이 없는 키워드는 강등 → 다음 생성 사이클에서 대체
const demoted = await this.sql<Array<{ keyword_id: string }>>`
UPDATE merchant_keyword
SET status = 'demoted', updated_at = now()
WHERE merchant_id = ${m.id} AND status = 'active'
AND impressions >= 100 AND ctr < 0.002
RETURNING keyword_id`;
return { matched: updated, demoted: demoted.length };
}
}
function buildDescription(name: string, desc: string, keywords: string[]): string {
const base = desc?.trim() || `${name} 안내`;
const tail = keywords.slice(0, 3).join(', ');
const full = tail ? `${base} ${tail} 정보를 확인하세요.` : base;
return full.length > 155 ? `${full.slice(0, 152)}...` : full;
}

26
tsconfig.json Normal file
View File

@ -0,0 +1,26 @@
{
"compilerOptions": {
"module": "commonjs",
"target": "ES2022",
"lib": ["ES2022"],
"moduleResolution": "node",
"declaration": false,
"removeComments": true,
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
"allowSyntheticDefaultImports": true,
"esModuleInterop": true,
"sourceMap": true,
"outDir": "./dist",
"baseUrl": "./",
"incremental": true,
"skipLibCheck": true,
"strict": true,
"strictNullChecks": true,
"noImplicitAny": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true
},
"include": ["src/**/*", "scripts/**/*"],
"exclude": ["node_modules", "dist"]
}