feat(phase-1a): external data fetchers + refresh pipeline + scheduler

10종목 시드 + pykrx OHLCV / 외인·기관 거래대금, KIS read-only EOD, OpenDART
공시, 네이버 금융 뉴스 스크레이퍼, 구글 뉴스 RSS, yfinance 거시(KOSPI/KOSDAQ/
USDKRW/US10Y) fetcher 를 추가하고 refresh_one / daily_batch / backfill /
APScheduler(16:00 KST) 파이프라인으로 묶음.

- backend/app/seed: 10종목 시드 (대형/고변동/테마/플랫폼/방어)
- backend/app/fetch: pykrx, kis, dart, news, macro, symbols_seed
- backend/app/pipelines: refresh_one, daily_batch, backfill(CLI), scheduler
- backend/app/api/refresh.py: POST /api/refresh/{code}?lookback_days=N
- backend/app/main.py: lifespan 으로 스케줄러 기동/종료, /health/keys 추가
- README: .env 복사 안내 보강

스모크 테스트 (실제 키 사용) 결과:
  KIS token  : ok (token 346자 발급)
  KIS daily  : 005930 11rows
  DART list  : 005930 30일 10건
  Naver news : 005930 12건
  Google RSS : "삼성전자" 92건

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
tkrmagid
2026-05-20 15:43:18 +09:00
parent cacddf5adf
commit 56f73a1f12
15 changed files with 1203 additions and 7 deletions

View File

@@ -1,18 +1,37 @@
from __future__ import annotations
import logging
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from app.api.refresh import router as refresh_router
from app.config import settings
from app.db.connection import ping as db_ping
from app.fetch import dart as dart_mod
from app.fetch import kis as kis_mod
from app.pipelines.scheduler import shutdown_scheduler, start_scheduler
logging.basicConfig(level=settings.log_level)
logging.basicConfig(
level=settings.log_level,
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
logger = logging.getLogger(__name__)
app = FastAPI(title="stock_chart_site", version="0.0.1")
@asynccontextmanager
async def lifespan(_: FastAPI):
# 스케줄러는 옵션. CI/테스트에서 disable 하고 싶으면 SCHEDULER_DISABLED 같은 env 추가 가능.
try:
start_scheduler()
except Exception: # noqa: BLE001
logger.exception("scheduler start failed")
yield
shutdown_scheduler()
app = FastAPI(title="stock_chart_site", version="0.1.0", lifespan=lifespan)
app.add_middleware(
CORSMiddleware,
@@ -21,6 +40,8 @@ app.add_middleware(
allow_headers=["*"],
)
app.include_router(refresh_router)
def _resolved_device() -> str:
if settings.model_device != "auto":
@@ -34,13 +55,19 @@ def _resolved_device() -> str:
@app.get("/health")
def health() -> dict[str, object]:
return {
"ok": True,
"device": _resolved_device(),
"version": "0.0.1",
}
return {"ok": True, "device": _resolved_device(), "version": "0.1.0"}
@app.get("/health/db")
def health_db() -> dict[str, object]:
return {"ok": True, **db_ping()}
@app.get("/health/keys")
def health_keys() -> dict[str, object]:
"""등록된 외부 키들 ping (key 값은 노출하지 않음)."""
return {
"kis": kis_mod.ping(),
"dart": dart_mod.ping(),
# huggingface 는 모델 다운로드 시점에 확인 (별도 ping 호출 비용 회피)
}