코딩 수정

[정민섭]

D:\Sahel>python sahel_scraper_final_v3.py
2026-10-12 23:11:11,377 [INFO] 🚀 [시작] 사헬 주요 언론 뉴스 수집 파이프라인 가동 (분산 DB 라우팅 적용)
2026-10-12 23:11:31,395 [INFO] ✅ Niger 매체 원천 크롤링 완료: 10건 수집됨
2026-10-12 23:12:53,354 [INFO] ✅ Mali 매체 원천 크롤링 완료: 64건 수집됨
2026-10-12 23:13:00,198 [INFO] ✅ Burkina Faso 매체 원천 크롤링 완료: 62건 수집됨
2026-10-12 23:13:00,198 [INFO] 🧠 수집된 136건의 기사 심층 텍스트 분석 및 메타데이터 분류 시작...
2026-10-12 23:13:00,828 [INFO] 💾 [ActuNiger] -> 분산 DB 저장 완료 (niger/actuniger_pri.db): 신규 1건
2026-10-12 23:13:00,837 [INFO] 💾 [Mali Web] -> 분산 DB 저장 완료 (mali/maliweb_pri.db): 신규 3건
2026-10-12 23:13:00,846 [INFO] 💾 [Le Faso] -> 분산 DB 저장 완료 (burkina_faso/lefaso_pri.db): 신규 2건
2026-10-12 23:13:00,846 [INFO] 💾 [완료] 총 6건의 정세 뉴스가 각 매체별 분산 DB에 정상 적재되었습니다.
2026-10-12 23:13:00,848 [INFO] 📝 마크다운 리포트 생성 완료: sahel_media_report_20261012_231300.md
2026-10-12 23:13:00,848 [INFO] 🎉 파이프라인 가동 종료!

D:\Sahel>python sahel_analytics.py

======================================================================
📈 분산 DB 기반 사헬 뉴스 분석 및 논문 데이터 추출 모듈 실행
======================================================================

        ╔════════════════════════════════════════════════════╗
        ║    분산 DB 기반 사헬 뉴스 담론 및 모순 통계 (360일) ║
        ╠════════════════════════════════════════════════════╣
        ║ 총 기사 수:                6362                    ║
        ║ 국가 수:                      3                    ║
        ║ 매체 수:                      3                    ║
        ╚════════════════════════════════════════════════════╝


📊 국가 및 매체별 기사 현황 (360일):
  - [Burkina Faso] Le Faso      | 1877건 | 중요도: 2.0 | 신뢰도: 0.85
  - [Mali] Mali Web     | 3773건 | 중요도: 2.0 | 신뢰도: 0.85
  - [Niger] ActuNiger    |  712건 | 중요도: 2.0 | 신뢰도: 0.85

💾 논문 분석용 통합 CSV 파일 일괄 추출 시작...
✅ 일반 기사 통합 데이터 내보내기 완료: sahel_news_export.csv
✅ 담론 분석용 개별 기사 정량화 데이터 내보내기 완료: sahel_discourse_metrics.csv

D:\Sahel>python sahel_scraper_sidwaya.py
2026-10-13 11:18:50,354 [INFO] 🚀 [시작] Sidwaya (부르키나파소 국영) 기사 수집 및 검증 파이프라인 가동
2026-10-13 11:18:50,354 [INFO] ⏳ 크롤링 중: https://www.sidwaya.info/bfcategories/politique/
2026-10-13 11:21:02,427 [INFO]    └─ politique [페이지 1] 유효 기사 35건 확보
2026-10-13 11:21:02,427 [INFO] ⏳ 크롤링 중: https://www.sidwaya.info/bfcategories/politique/page/2/
2026-10-13 11:21:36,174 [INFO]    └─ politique [페이지 2] 유효 기사 9건 확보
2026-10-13 11:21:36,175 [INFO] ⏳ 크롤링 중: https://www.sidwaya.info/bfcategories/politique/page/3/
2026-10-13 11:22:06,423 [INFO]    └─ politique [페이지 3] 유효 기사 9건 확보

...
2026-10-13 11:42:22,752 [INFO] ⏳ 크롤링 중: https://www.sidwaya.info/bfcategories/nouvelle-du-front/page/5/
2026-10-13 11:42:56,984 [INFO]    └─ nouvelle-du-front [페이지 5] 유효 기사 0건 확보
2026-10-13 11:42:56,984 [INFO] 🧠 수집된 375건의 기사 심층 텍스트 분석 중...
2026-10-13 11:42:58,063 [INFO] 💾 [완료] 신규 0건의 기사가 'burkina_faso/sidwaya_gov.db'에 적재되었습니다.

D:\Sahel>python sahel_scraper_lessor.py
2026-10-13 13:58:45,297 [INFO] 🚀 [시작] L'Essor (말리 국영) 기사 수집 및 스키마 보정 적재 가동
2026-10-13 13:58:45,297 [INFO] ⏳ 크롤링 중 [POLITIQUE]: https://lessor.ml/categories/1
2026-10-13 14:00:15,123 [INFO]    └─ POLITIQUE [페이지 1] 유효 기사 31건 확보
2026-10-13 14:00:15,123 [INFO] ⏳ 크롤링 중 [POLITIQUE]: https://lessor.ml/categories/1/page/2
2026-10-13 14:00:16,862 [INFO] ⏳ 크롤링 중 [ECONOMIE]: https://lessor.ml/categories/2
2026-10-13 14:00:58,396 [INFO]    └─ ECONOMIE [페이지 1] 유효 기사 16건 확보
2026-10-13 14:00:58,396 [INFO] ⏳ 크롤링 중 [ECONOMIE]: https://lessor.ml/categories/2/page/2
2026-10-13 14:01:01,458 [INFO] ⏳ 크롤링 중 [SPORT]: https://lessor.ml/categories/3
2026-10-13 14:01:29,807 [INFO]    └─ SPORT [페이지 1] 유효 기사 11건 확보
2026-10-13 14:01:29,808 [INFO] ⏳ 크롤링 중 [SPORT]: https://lessor.ml/categories/3/page/2
2026-10-13 14:01:31,594 [INFO] ⏳ 크롤링 중 [SOCIETE]: https://lessor.ml/categories/8
2026-10-13 14:02:17,653 [INFO]    └─ SOCIETE [페이지 1] 유효 기사 16건 확보
2026-10-13 14:02:17,653 [INFO] ⏳ 크롤링 중 [SOCIETE]: https://lessor.ml/categories/8/page/2
2026-10-13 14:02:19,431 [INFO] ⏳ 크롤링 중 [ACTUALITES]: https://lessor.ml/categories/10
2026-10-13 14:02:50,724 [INFO]    └─ ACTUALITES [페이지 1] 유효 기사 11건 확보
2026-10-13 14:02:50,724 [INFO] ⏳ 크롤링 중 [ACTUALITES]: https://lessor.ml/categories/10/page/2
2026-10-13 14:02:52,555 [INFO] ⏳ 크롤링 중 [FAITS_DIVERS]: https://lessor.ml/categories/15
2026-10-13 14:03:34,968 [INFO]    └─ FAITS_DIVERS [페이지 1] 유효 기사 16건 확보
2026-10-13 14:03:34,968 [INFO] ⏳ 크롤링 중 [FAITS_DIVERS]: https://lessor.ml/categories/15/page/2
2026-10-13 14:03:36,724 [INFO] ⏳ 크롤링 중 [AES]: https://lessor.ml/categories/17
2026-10-13 14:04:20,126 [INFO]    └─ AES [페이지 1] 유효 기사 16건 확보
2026-10-13 14:04:20,127 [INFO] ⏳ 크롤링 중 [AES]: https://lessor.ml/categories/17/page/2
2026-10-13 14:04:21,833 [INFO] 🧠 수집된 117건의 기사 심층 텍스트 분석 중...
2026-10-13 14:04:22,729 [INFO] 💾 [완료] 신규 117건의 기사가 'mali/lessor_gov.db'에 적재되었습니다.
2026-10-13 14:04:22,877 [INFO] 📝 Word(.docx) 리포트 생성 완료: lessor_gov_report_20261013_140422.docx

D:\Sahel>python sahel_scraper_lesahel.py
2026-10-13 15:03:58,411 [INFO] 🚀 [시작] Le Sahel (니제르 국영) 기사 수집 및 아카이브 파이프라인 가동
2026-10-13 15:03:58,411 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/
2026-10-13 15:05:44,503 [INFO]    └─ 유효 기사 59건 확보
2026-10-13 15:05:44,504 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/page/2/
2026-10-13 15:05:50,279 [INFO]    └─ 유효 기사 2건 확보
2026-10-13 15:05:50,280 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/page/3/
2026-10-13 15:05:52,404 [INFO]    └─ 유효 기사 0건 확보
2026-10-13 15:05:52,404 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/actualites/
2026-10-13 15:05:55,601 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/politique/
2026-10-13 15:06:15,041 [INFO]    └─ 유효 기사 10건 확보
2026-10-13 15:06:15,041 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/politique/page/2/
2026-10-13 15:08:11,995 [INFO]    └─ 유효 기사 13건 확보
2026-10-13 15:08:11,995 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/politique/page/3/
2026-10-13 15:10:26,711 [INFO]    └─ 유효 기사 13건 확보
2026-10-13 15:10:26,711 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/economie/
2026-10-13 15:10:50,686 [INFO]    └─ 유효 기사 10건 확보
2026-10-13 15:10:50,686 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/economie/page/2/
2026-10-13 15:11:18,669 [INFO]    └─ 유효 기사 13건 확보
2026-10-13 15:11:18,669 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/economie/page/3/
2026-10-13 15:12:34,075 [INFO]    └─ 유효 기사 13건 확보
2026-10-13 15:12:34,075 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/societe/
2026-10-13 15:12:49,784 [INFO]    └─ 유효 기사 8건 확보
2026-10-13 15:12:49,784 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/societe/page/2/
2026-10-13 15:13:14,556 [INFO]    └─ 유효 기사 13건 확보
2026-10-13 15:13:14,556 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/societe/page/3/
2026-10-13 15:13:37,917 [INFO]    └─ 유효 기사 12건 확보
2026-10-13 15:13:37,917 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/international/
2026-10-13 15:13:56,547 [INFO]    └─ 유효 기사 10건 확보
2026-10-13 15:13:56,547 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/international/page/2/
2026-10-13 15:14:23,495 [INFO]    └─ 유효 기사 13건 확보
2026-10-13 15:14:23,495 [INFO] ⏳ 크롤링 중: https://www.lesahel.org/category/international/page/3/
2026-10-13 15:16:58,893 [INFO]    └─ 유효 기사 13건 확보
2026-10-13 15:16:58,894 [INFO] 🧠 수집된 202건의 기사 심층 텍스트 분석 중...
2026-10-13 15:16:59,543 [INFO] 💾 [완료] 신규 202건의 기사가 'niger/lesahel_gov.db'에 적재되었습니다.
2026-10-13 15:16:59,708 [INFO] 📝 Word(.docx) 리포트 생성 완료: lesahel_gov_report_20261013_151659.docx

D:\Sahel>python sahel_scraper_rfi_auto.py
🚀 [시작] Playwright 기반 RFI 태그 페이징 자동 수집 파이프라인 가동
⏳ 브라우저 스캔 중 [Mali]: https://www.rfi.fr/fr/tag/mali/
   └─ 발견된 유효 기사 링크: 48개
      📄 기사 파싱 중: https://www.rfi.fr/fr/afrique/
      📄 기사 파싱 중: https://www.rfi.fr/fr/afrique/20200921-mali-naissance-dun-pays-les-cendres-la-fédération

...

📄 기사 파싱 중: https://www.rfi.fr/fr/afrique/20250824-niger-la-dissolution-de-cinq-syndicats-et-la-radiation-de-deux-magistrats-suscitent-des-r%C3%A9actions
      📄 기사 파싱 중: https://www.rfi.fr/fr/afrique/20250823-niger-la-soci%C3%A9t%C3%A9-civile-d-agadez-interpelle-l-%C3%A9tat-sur-les-manquements-de-la-soci%C3%A9t%C3%A9-mini%C3%A8re-somida
🧠 수집된 642건의 기사 심층 텍스트 분석 중...
💾 [완료] 총 642건의 기사가 'international/rfi_occ.db'에 정상 적재되었습니다.
📝 Word(.docx) 리포트 생성 완료: rfi_occ_report_20261013_165758.docx

D:\Sahel>python sahel_scraper_rt.py
🚀 [시작] RT Afrique(Oriental) 본문 날짜 정밀 추출 파이프라인 가동
⏳ RT Afrique 스캔 및 무한 스크롤 수행 중 [Mali]...
   ⟳ 스크롤 확장 중... (1/30)

...

      ✅ [수집 확정] (Niger) Niger : des cours de russe pour les... [2026-08-25]
🧠 수집된 327건의 유효 기사 심층 텍스트 분석 중...
💾 [완료] 총 327건의 신규 기사가 'international/rt_ori.db'에 정상 적재되었습니다.
📝 Word(.docx) 리포트 생성 완료: rt_ori_report_20261014_195506.docx

rt의 파이썬 스크립트의 INSERT 문은 source_credibility, importance_level, sector 등 총 18개 컬럼

다른 db는 컬럼이 총 15개만 존재

==> fix_db_schema.py
🔄 기존 데이터를 안전하게 백업 및 테이블 구조 재정비 중...
✨ [완료] 데이터 손실 없이 15개 표준 컬럼 구조로 완벽하게 동기화되었습니다!