{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# ๐Ÿ›ก๏ธ DAT10: ๋ฐ์ดํ„ฐ ๋งˆ์ด๋‹ ๊ธฐ๋ฐ˜ ๋ฐฉ์‚ฐ ๊ณต๊ธ‰๋ง ์žฌ๋‚œ ์‹๋ณ„ SaaS & ์•ผ์ „ ๊ตฐ์ˆ˜ ์žฌ๊ณ  ๊ด€๋ฆฌ ํ”Œ๋žซํผ\n", "## ๐Ÿ‘จโ€๐Ÿ’ป [ํ•™์Šต์ž์šฉ ์‹ค์Šต ๋ฐ ๊ณผ์ œ ์ˆ˜ํ–‰ ๋…ธํŠธ๋ถ] Student Assignment Notebook\n", "\n", "๋ณธ ๋…ธํŠธ๋ถ์€ **์ „์ˆ  ํ†ต์‹  ๋‘์ ˆ ํ™˜๊ฒฝ์—์„œ์˜ ๋‹จ๋ง ์ž๋ฆฝํ˜• ์žฌ๊ณ  ๊ด€๋ฆฌ, CRDT ๋ฌด์ถฉ๋Œ ๋™๊ธฐํ™”, ์œ„๋ณ€์กฐ ๋ฐฉ์ง€ ํ•ด์‹œ์ฒด์ธ, DIB ๊ณต๊ธ‰๋ง ์žฌ๋‚œ ์ด์ƒ์น˜ ๋งˆ์ด๋‹, HITL ์Šน์ธ ์›Œํฌํ”Œ๋กœ์šฐ ๋ฐ Streamlit ์ „์ˆ  SaaS ๋Œ€์‹œ๋ณด๋“œ**๋ฅผ ๋‹จ๊ณ„๋ณ„๋กœ ๊ตฌํ˜„ํ•˜๋Š” ์‹ค์Šต ๊ณผ์ œ์ž…๋‹ˆ๋‹ค.\n", "\n", "### ๐Ÿ“Œ ์‹ค์Šต ์ง„ํ–‰ ์•ˆ๋‚ด\n", "1. ๊ฐ ๋ฏธ์…˜์˜ **[Mission ์„ค๋ช…]**๊ณผ **[์š”๊ตฌ์‚ฌํ•ญ]**์„ ๊ผผ๊ผผํžˆ ํ™•์ธํ•˜์„ธ์š”.\n", "2. ๊ฐ ์ฝ”๋“œ ์…€ ์ƒ๋‹จ์˜ `%%writefile [๊ฒฝ๋กœ]` ๋งค์ง ์ปค๋งจ๋“œ๋ฅผ ํ†ตํ•ด ์‚ฐ์ถœ๋ฌผ ํŒŒ์ผ์ด ์ƒ์„ฑ๋˜๋„๋ก ๊ตฌ์„ฑ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค.\n", "3. `# TODO:` ๋กœ ํ‘œ์‹œ๋œ ๋นˆ์นธ์„ ๊ตฌํ˜„ ์กฐ๊ฑด์— ๋งž์ถ”์–ด ์™„์„ฑํ•˜์„ธ์š”.\n", "4. ๊ฐ ๋ฏธ์…˜ ํ•˜๋‹จ์˜ **[๊ฒ€์ฆ ์…€]**์„ ์‹คํ–‰ํ•˜์—ฌ ์ •์ƒ ๋™์ž‘(`PASS`) ์—ฌ๋ถ€๋ฅผ ํ™•์ธํ•˜์„ธ์š”." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### โš™๏ธ 0. ์‹ค์Šต ํ™˜๊ฒฝ ์ค€๋น„ ๋ฐ ํ•„์ˆ˜ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์„ค์น˜\n", "- ํ•„์š”ํ•œ ์˜์กด์„ฑ ํŒจํ‚ค์ง€๋ฅผ ์„ค์น˜ํ•˜๊ณ  ํ”„๋กœ์ ํŠธ ๋””๋ ‰ํ„ฐ๋ฆฌ ๊ตฌ์กฐ๋ฅผ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# 1. ํ•„์ˆ˜ ํŒจํ‚ค์ง€ ์„ค์น˜\n", "!pip install -q jsonschema pytest pyyaml plotly streamlit pandas scikit-learn\n", "\n", "# 2. ํ”„๋กœ์ ํŠธ ๋””๋ ‰ํ„ฐ๋ฆฌ ์ƒ์„ฑ\n", "!mkdir -p data/schema data/samples docs src/preprocess src/models src/verification src/ui src/hitl tests configs\n", "print('โœ… ํ™˜๊ฒฝ ์„ค์น˜ ๋ฐ ๋””๋ ‰ํ„ฐ๋ฆฌ ์ƒ์„ฑ ์™„๋ฃŒ!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ๐Ÿ“‹ [Mission 01] ๋ฐ์ดํ„ฐ ์Šคํ‚ค๋งˆ ์ •์˜ ๋ฐ ์ƒ˜ํ”Œ ๋ฐ์ดํ„ฐ์…‹ ๊ตฌ์ถ•\n", "- **๋ชฉํ‘œ**: ์˜คํ”„๋ผ์ธ ํŠธ๋žœ์žญ์…˜์˜ JSON Schema๋ฅผ ์ •์˜ํ•˜๊ณ , ์ „์ˆ  ํ†ต์‹  ๋‘์ ˆ ์ƒํ™ฉ ๋ฐ ๋ฐฉ์‚ฐ ๊ณต๊ธ‰๋ง(DIB) ์ƒ˜ํ”Œ ๋ฐ์ดํ„ฐ๋ฅผ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.\n", "- **์‚ฐ์ถœ๋ฌผ**: `data/schema/dat10_input_schema.json`, `data/samples/raw_offline_inventory.csv`, `data/samples/dib_supply_chain_disruptions.csv`, `docs/dat10_data_spec.md`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile data/schema/dat10_input_schema.json\n", "{\n", " \"$schema\": \"http://json-schema.org/draft-07/schema#\",\n", " \"title\": \"FieldLogisticsTransaction\",\n", " \"type\": \"object\",\n", " \"required\": [\"tx_id\", \"unit_id\", \"item_id\", \"tx_type\", \"quantity\", \"timestamp\", \"status\"],\n", " \"properties\": {\n", " \"tx_id\": {\n", " \"type\": \"string\",\n", " \"pattern\": \"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-4[0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}$\"\n", " },\n", " \"unit_id\": {\"type\": \"string\"},\n", " \"item_id\": {\"type\": \"string\"},\n", " \"tx_type\": {\"type\": \"string\", \"enum\": [\"ISSUE\", \"RECEIPT\", \"RETURN\", \"TRANSFER\"]},\n", " \"quantity\": {\"type\": \"integer\", \"minimum\": 1},\n", " \"timestamp\": {\"type\": \"string\", \"format\": \"date-time\"},\n", " \"status\": {\"type\": \"string\", \"enum\": [\"LOCAL_COMMITTED\", \"SYNC_PENDING\", \"SYNCED\", \"CONFLICT\"]}\n", " }\n", "}\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile data/samples/raw_offline_inventory.csv\n", "transaction_id,terminal_id,unit_id,nsn_code,item_name,supply_class,tx_type,quantity,current_stock,unit_cost,comms_status,grid_coords,weather_condition,lamport_timestamp,timestamp_utc,prev_hash,record_hash\n", "TX-20261001-001,TERM-A101,BN-01,1005-01-584-1234,5.56mm K2 ๋ณดํ†ตํƒ„,Class_V,RECEIPT,10000,25000,0.45,OFFLINE_LOCAL,52SDJ12345678,CLEAR,1,2026-10-01T08:00:00Z,0000000000000000000000000000000000000000000000000000000000000000,52531766d0f0539725e3d8fd44c54cf792d0f531214c478a6e2af5323466aac1\n", "TX-20261001-002,TERM-A101,BN-01,1005-01-584-1234,5.56mm K2 ๋ณดํ†ตํƒ„,Class_V,ISSUE,2400,22600,0.45,OFFLINE_LOCAL,52SDJ12345678,CLEAR,2,2026-10-01T10:30:00Z,52531766d0f0539725e3d8fd44c54cf792d0f531214c478a6e2af5323466aac1,4a1336df6225a56130d713bffaeaa145045b6da6e9ea44784c78abda36ea8d00\n", "TX-20261002-003,TERM-A102,BN-02,2530-01-450-8899,K9 ์œ ์•• ์„œ๋ณด๋ฐธ๋ธŒ,Class_IX,ISSUE,2,8,3400.0,DEGRADED_EW,52SDJ87654321,HEAVY_RAIN,1,2026-10-02T14:15:00Z,4a1336df6225a56130d713bffaeaa145045b6da6e9ea44784c78abda36ea8d00,911e7e88a87598fd271b04f9b8d42d350f73975f76b0f4c667802c7a04aa53a5\n", "TX-20261003-004,TERM-A101,BN-01,8970-01-345-6789,์ „ํˆฌ์‹๋Ÿ‰ 1ํ˜•(์†Œ๊ณ ๊ธฐ๋ณถ์Œ๋ฐฅ),Class_I,ISSUE,350,1650,8.5,OFFLINE_LOCAL,52SDJ12345678,FOG,3,2026-10-03T09:00:00Z,911e7e88a87598fd271b04f9b8d42d350f73975f76b0f4c667802c7a04aa53a5,be6367eacc821c322ad7e33a2c46b53bbb10113c8cb8b8bb697388755392fc62\n", "TX-20261004-005,TERM-A103,BDE-03,9130-01-222-3333,JP-8 ํ•ญ๊ณต์œ  (๋“œ๋Ÿผ),Class_III,TRANSFER,50,450,180.0,OFFLINE_P2P,52SDJ55556666,CLEAR,1,2026-10-04T16:40:00Z,be6367eacc821c322ad7e33a2c46b53bbb10113c8cb8b8bb697388755392fc62,3b8125af645689c41da1c5a791e393dbcca9badeecd2530e58c43ee819fb3e6a\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile data/samples/dib_supply_chain_disruptions.csv\n", "nsn_code,item_name,supplier_tier1,supplier_tier2,lead_time_days,baseline_lead_time,single_source_flag,geopolitical_risk_score,raw_material_criticality,failure_rate_monthly,disruption_label\n", "1005-01-584-1234,5.56mm K2 ๋ณดํ†ตํƒ„,Poongsan Corp,Dong-Ah Raw Metals,18,15,0,0.15,0.20,0.01,NORMAL\n", "2530-01-450-8899,K9 ์œ ์•• ์„œ๋ณด๋ฐธ๋ธŒ,Hanwha Aerospace,Moog Precision,145,45,1,0.85,0.92,0.18,CRITICAL_BOTTLENECK\n", "8970-01-345-6789,์ „ํˆฌ์‹๋Ÿ‰ 1ํ˜•(์†Œ๊ณ ๊ธฐ๋ณถ์Œ๋ฐฅ),CJ CheilJedang,Local Farms,25,20,0,0.10,0.15,0.00,NORMAL\n", "9130-01-222-3333,JP-8 ํ•ญ๊ณต์œ  (๋“œ๋Ÿผ),GS Caltex,MiddleEast Crude,40,30,0,0.65,0.80,0.05,WARNING_VOLATILITY\n", "5998-01-777-8888,์ „์ˆ  ํ†ต์‹ ์šฉ FPGA ๋ฐ˜๋„์ฒด ๋ชจ๋“ˆ,LIG Nex1,Xilinx/TSMC,320,60,1,0.95,0.98,0.25,DISASTER_OBSOLESCENCE\n", "2920-01-111-2222,K2 ์ „์ฐจ ์ ํ™”ํ”Œ๋Ÿฌ๊ทธ,Hyundai Rotem,Denso Tech,75,30,1,0.70,0.60,0.12,WARNING_SUPPLY_DELAY\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile docs/dat10_data_spec.md\n", "# DAT10 ์ „์ˆ  ๊ตฐ์ˆ˜ ์žฌ๊ณ  ๋ฐ์ดํ„ฐ ๋ช…์„ธ์„œ\n", "\n", "## 1. ๊ฐœ์š”\n", "์ „์ˆ  ํ†ต์‹  ๋‘์ ˆ ํ™˜๊ฒฝ์—์„œ์˜ ๋‹จ๋ง ์ž๋ฆฝํ˜• ์žฌ๊ณ  ๊ด€๋ฆฌ ํ‘œ์ค€ ์Šคํ‚ค๋งˆ ๋ช…์„ธ.\n", "\n", "## 2. ์ฃผ์š” ํ•„๋“œ ๊ทœ๊ฒฉ\n", "- `tx_id`: ํŠธ๋žœ์žญ์…˜ ๊ณ ์œ  ์‹๋ณ„์ž (UUID v4 ๊ทœ๊ฒฉ)\n", "- `unit_id`: ๋ถ€๋Œ€ ์‹๋ณ„ ์ฝ”๋“œ (์˜ˆ: BN-01, BDE-03)\n", "- `item_id` / `nsn_code`: ๊ตฐ์ˆ˜ ํ’ˆ๋ชฉ ๊ตญ๊ฐ€์žฌ๊ณ ๋ฒˆํ˜ธ(NSN)\n", "- `tx_type`: ์ˆ˜๋ถˆ ์œ ํ˜• (ISSUE, RECEIPT, RETURN, TRANSFER)\n", "- `quantity`: ์ˆ˜๋Ÿ‰ (์ •์ˆ˜ํ˜•, 1 ์ด์ƒ)\n", "- `status`: ๋™๊ธฐํ™” ์ƒํƒœ (LOCAL_COMMITTED, SYNC_PENDING, SYNCED, CONFLICT)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [Mission 01 ๊ฒ€์ฆ] JSON Schema ์œ ํšจ์„ฑ ํ™•์ธ\n", "import json, jsonschema\n", "with open('data/schema/dat10_input_schema.json') as f:\n", " schema = json.load(f)\n", "print('โœ… Schema Title:', schema.get('title'))\n", "print('โœ… Required Fields:', schema.get('required'))\n", "assert 'tx_id' in schema.get('required')\n", "print('๐ŸŽ‰ Mission 01 ์‚ฐ์ถœ๋ฌผ ๊ฒ€์ฆ ์„ฑ๊ณต!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ๐Ÿ”„ [Mission 02] ์ „์ฒ˜๋ฆฌ ๋ฐ ๋กœ์ปฌ SQLite ํŠธ๋žœ์žญ์…˜ ๋ชจ๋“ˆ ๊ตฌํ˜„\n", "- **๋ชฉํ‘œ**: ์•ผ์ „ ์˜คํ”„๋ผ์ธ ํŠธ๋žœ์žญ์…˜์„ ๋กœ์ปฌ SQLite์— ์บ์‹ฑํ•˜๊ณ , ๋ฐ์ดํ„ฐ ๊ฒฐ์ธก์น˜/์Œ์ˆ˜ ์ˆ˜๋Ÿ‰์„ ๋ณด์ •ํ•˜๋Š” ์ „์ฒ˜๋ฆฌ๊ธฐ๋ฅผ ๊ตฌํ˜„ํ•ฉ๋‹ˆ๋‹ค.\n", "- **์š”๊ตฌ์‚ฌํ•ญ**: `process_batch(records)` ๋ฉ”์„œ๋“œ์—์„œ ์Œ์ˆ˜ ์ˆ˜๋Ÿ‰์„ ์ ˆ๋Œ€๊ฐ’(`abs`)์œผ๋กœ ๋ณด์ •ํ•˜๊ณ  ์œ ํšจ ๋ ˆ์ฝ”๋“œ ๋ชฉ๋ก์„ ๋ฐ˜ํ™˜ํ•˜๋„๋ก ์™„์„ฑํ•˜์„ธ์š”.\n", "- **์‚ฐ์ถœ๋ฌผ**: `src/preprocess/dat10_preprocessor.py`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile src/preprocess/dat10_preprocessor.py\n", "import sqlite3\n", "import csv\n", "import os\n", "from typing import List, Dict, Any\n", "\n", "class TacticalDataPreprocessor:\n", " def __init__(self, db_path=':memory:'):\n", " self.db_path = db_path\n", " self.conn = sqlite3.connect(self.db_path)\n", " self._init_db()\n", "\n", " def _init_db(self):\n", " with self.conn:\n", " self.conn.execute('''\n", " CREATE TABLE IF NOT EXISTS tx_log (\n", " transaction_id TEXT PRIMARY KEY,\n", " terminal_id TEXT,\n", " unit_id TEXT,\n", " nsn_code TEXT,\n", " item_name TEXT,\n", " supply_class TEXT,\n", " tx_type TEXT,\n", " quantity INTEGER,\n", " current_stock INTEGER,\n", " unit_cost REAL,\n", " comms_status TEXT,\n", " timestamp_utc TEXT\n", " )\n", " ''')\n", "\n", " def process_batch(self, raw_records: List[Dict[str, Any]]) -> Dict[str, Any]:\n", " \"\"\"\n", " ํŠธ๋žœ์žญ์…˜ ๋ฐฐ์น˜ ์ „์ฒ˜๋ฆฌ\n", " - ์Œ์ˆ˜ quantity๋Š” abs(quantity)๋กœ ๋ณด์ •\n", " - ํ•„์ˆ˜ ํ•„๋“œ ๋ˆ„๋ฝ ์—ฌ๋ถ€ ๊ฒ€์ฆ\n", " \"\"\"\n", " valid_records = []\n", " invalid_count = 0\n", " \n", " for r in raw_records:\n", " # TODO 1: quantity๊ฐ€ ์Œ์ˆ˜์ธ ๊ฒฝ์šฐ abs(quantity)๋กœ ๋ณ€ํ™˜ํ•˜์„ธ์š”.\n", " # TODO 2: transaction_id ๋ฐ nsn_code๊ฐ€ ์กด์žฌํ•˜๋Š”์ง€ ๊ฒ€์ฆํ•˜์„ธ์š”.\n", " # -----------------------------------------------------\n", " # [ํ•™์Šต์ž ์ž‘์„ฑ ์˜์—ญ]\n", " rec = dict(r)\n", " if 'quantity' in rec:\n", " rec['quantity'] = abs(int(rec['quantity']))\n", " \n", " if rec.get('transaction_id') and rec.get('nsn_code'):\n", " valid_records.append(rec)\n", " else:\n", " invalid_count += 1\n", " # -----------------------------------------------------\n", " \n", " return {\n", " 'total_processed': len(raw_records),\n", " 'valid_records': len(valid_records),\n", " 'invalid_records': invalid_count,\n", " 'records': valid_records\n", " }\n", "\n", " def load_from_csv(self, csv_path: str) -> Dict[str, Any]:\n", " if not os.path.exists(csv_path):\n", " return {'total_processed': 0, 'valid_records': 0, 'invalid_records': 0, 'records': []}\n", " records = []\n", " with open(csv_path, 'r', encoding='utf-8') as f:\n", " reader = csv.DictReader(f)\n", " for row in reader:\n", " records.append(row)\n", " return self.process_batch(records)\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [Mission 02 ๊ฒ€์ฆ] ์ „์ฒ˜๋ฆฌ๊ธฐ ๋™์ž‘ ํ…Œ์ŠคํŠธ\n", "from src.preprocess.dat10_preprocessor import TacticalDataPreprocessor\n", "\n", "prep = TacticalDataPreprocessor(':memory:')\n", "sample_input = [\n", " {'transaction_id': 'TX-001', 'nsn_code': '1005-01', 'quantity': -150},\n", " {'transaction_id': '', 'nsn_code': '1005-01', 'quantity': 10} # ์œ ํšจํ•˜์ง€ ์•Š์€ ID\n", "]\n", "res = prep.process_batch(sample_input)\n", "print('์ „์ฒ˜๋ฆฌ ๊ฒฐ๊ณผ:', res)\n", "assert res['records'][0]['quantity'] == 150, '์Œ์ˆ˜ ์ˆ˜๋Ÿ‰ ๋ณด์ • ์‹คํŒจ!'\n", "assert res['valid_records'] == 1, '์œ ํšจ์„ฑ ํ•„ํ„ฐ๋ง ์˜ค๋ฅ˜!'\n", "print('๐ŸŽ‰ Mission 02 ๋‹จ์œ„ ํ…Œ์ŠคํŠธ PASS!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ๐Ÿ“ก [Mission 03] P2P ์žฌ๊ณ  ๊ตํ™˜ Baseline ๊ตฌํ˜„\n", "- **๋ชฉํ‘œ**: ๋‹จ์ˆœ ๋ฎ์–ด์“ฐ๊ธฐ ๋ฐ ๋žจํฌํŠธ ํƒ€์ž„์Šคํƒฌํ”„(Lamport TS) ๊ธฐ๋ฐ˜์˜ 1์ฐจ P2P ๋™๊ธฐํ™” ๋ฒ ์ด์Šค๋ผ์ธ์„ ๊ตฌํ˜„ํ•ฉ๋‹ˆ๋‹ค.\n", "- **์š”๊ตฌ์‚ฌํ•ญ**: `merge_p2p_payload(remote_state)` ๋ฉ”์„œ๋“œ์—์„œ ์›๊ฒฉ ์ƒํƒœ์™€ ๋กœ์ปฌ ์ƒํƒœ์˜ ๋ถˆ์ผ์น˜(Conflict)๋ฅผ ๊ฐ์ง€ํ•˜๋„๋ก ๊ตฌํ˜„ํ•˜์„ธ์š”.\n", "- **์‚ฐ์ถœ๋ฌผ**: `src/models/dat10_baseline_model.py`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile src/models/dat10_baseline_model.py\n", "from typing import Dict, Any\n", "\n", "class P2PInventoryBaselineSync:\n", " def __init__(self, node_id: str):\n", " self.node_id = node_id\n", " self.local_state: Dict[str, Dict[str, Any]] = {}\n", "\n", " def update_local_inventory(self, nsn_code: str, quantity: int, lamport_ts: int):\n", " self.local_state[nsn_code] = {'quantity': quantity, 'lamport_ts': lamport_ts}\n", "\n", " def merge_p2p_payload(self, remote_state: Dict[str, Dict[str, Any]]) -> Dict[str, int]:\n", " \"\"\"\n", " ์›๊ฒฉ ํ”ผ์–ด์˜ ์žฌ๊ณ  ์ƒํƒœ ๋ณ‘ํ•ฉ ๋ฐ ์ถฉ๋Œ ๊ฐ์ง€\n", " - ๋™์ผ nsn_code์— ๋Œ€ํ•ด ์ˆ˜๋Ÿ‰์ด ์„œ๋กœ ๋‹ค๋ฅด๋ฉด ์ถฉ๋Œ(conflict)๋กœ ์ง‘๊ณ„\n", " \"\"\"\n", " conflicts = 0\n", " # TODO: remote_state๋ฅผ ์ˆœํšŒํ•˜๋ฉฐ local_state์™€ ๋น„๊ตํ•˜๊ณ  ์ถฉ๋Œ ๊ฑด์ˆ˜๋ฅผ ์นด์šดํŠธํ•˜์„ธ์š”.\n", " # -----------------------------------------------------\n", " # [ํ•™์Šต์ž ์ž‘์„ฑ ์˜์—ญ]\n", " for k, v in remote_state.items():\n", " if k in self.local_state and self.local_state[k]['quantity'] != v['quantity']:\n", " conflicts += 1\n", " else:\n", " self.local_state[k] = v\n", " # -----------------------------------------------------\n", " return {'conflicts_count': conflicts, 'merged_count': len(self.local_state)}\n", "\n", "class BaselineRuleDetector:\n", " def evaluate(self, record: Dict[str, Any]) -> Dict[str, bool]:\n", " return {'is_valid': bool(record.get('transaction_id'))}\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [Mission 03 ๊ฒ€์ฆ] P2P ๋™๊ธฐํ™” ๋ฐ ์ถฉ๋Œ ๊ฐ์ง€ ํ…Œ์ŠคํŠธ\n", "from src.models.dat10_baseline_model import P2PInventoryBaselineSync\n", "\n", "node_a = P2PInventoryBaselineSync('TERM-A')\n", "node_b = P2PInventoryBaselineSync('TERM-B')\n", "\n", "node_a.update_local_inventory('1005-01-584-1234', 22600, 2)\n", "node_b.update_local_inventory('1005-01-584-1234', 24000, 2) # ๋ถˆ์ผ์น˜ ๋ฐœ์ƒ\n", "\n", "res = node_a.merge_p2p_payload(node_b.local_state)\n", "print('P2P ๋ณ‘ํ•ฉ ๊ฒฐ๊ณผ:', res)\n", "assert res['conflicts_count'] == 1, '์ถฉ๋Œ ๊ฐ์ง€ ์‹คํŒจ!'\n", "print('๐ŸŽ‰ Mission 03 ๋‹จ์œ„ ํ…Œ์ŠคํŠธ PASS!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ๐Ÿงฌ [Mission 04] CRDT ๋ฌด์ถฉ๋Œ ๋™๊ธฐํ™” & ๊ณต๊ธ‰๋ง ์žฌ๋‚œ ์ด์ƒ์น˜ ํƒ์ง€๊ธฐ\n", "- **๋ชฉํ‘œ**: ์ˆ˜๋ถˆ(RECEIPT/ISSUE)์„ ์ˆ˜ํ•™์ ์œผ๋กœ ์•ˆ์ „ํ•˜๊ฒŒ ๋ณ‘ํ•ฉํ•˜๋Š” **CRDT PN-Counter**์™€ DIB ๊ณต๊ธ‰๋ง ๋ฆฌ์Šคํฌ(๋ฆฌ๋“œํƒ€์ž„, ์ง€์ •ํ•™ ๋ฆฌ์Šคํฌ)๋ฅผ ์ข…ํ•ฉ ํ‰๊ฐ€ํ•˜๋Š” ํƒ์ง€๊ธฐ๋ฅผ ๊ตฌ์ถ•ํ•ฉ๋‹ˆ๋‹ค.\n", "- **์‚ฐ์ถœ๋ฌผ**: `src/models/dat10_advanced_detector.py`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile src/models/dat10_advanced_detector.py\n", "import numpy as np\n", "import pandas as pd\n", "from typing import Dict, List, Any\n", "\n", "class AdvancedCRDTSyncEngine:\n", " \"\"\"CRDT PN-Counter (Positive-Negative Counter) ๊ธฐ๋ฐ˜ ์–‘๋ฐฉํ–ฅ ๋ฌด์ถฉ๋Œ ๋™๊ธฐํ™”\"\"\"\n", " def __init__(self, replica_id: str):\n", " self.replica_id = replica_id\n", " self.pn_counters: Dict[str, Dict[str, Dict[str, int]]] = {}\n", "\n", " def _ensure_item(self, nsn_code: str):\n", " if nsn_code not in self.pn_counters:\n", " self.pn_counters[nsn_code] = {'P': {}, 'N': {}}\n", "\n", " def increment(self, nsn_code: str, value: int):\n", " self._ensure_item(nsn_code)\n", " self.pn_counters[nsn_code]['P'][self.replica_id] = (\n", " self.pn_counters[nsn_code]['P'].get(self.replica_id, 0) + value\n", " )\n", "\n", " def decrement(self, nsn_code: str, value: int):\n", " self._ensure_item(nsn_code)\n", " self.pn_counters[nsn_code]['N'][self.replica_id] = (\n", " self.pn_counters[nsn_code]['N'].get(self.replica_id, 0) + value\n", " )\n", "\n", " def value(self, nsn_code: str) -> int:\n", " \"\"\"\n", " ์ตœ์ข… ์žฌ๊ณ  ๊ณ„์‚ฐ: sum(P) - sum(N)\n", " \"\"\"\n", " # TODO: P์˜ ์ดํ•ฉ์—์„œ N์˜ ์ดํ•ฉ์„ ์ฐจ๊ฐํ•œ ์žฌ๊ณ  ์ˆ˜๋Ÿ‰์„ ๋ฐ˜ํ™˜ํ•˜์„ธ์š”. (์ตœ์†Œ๊ฐ’ 0)\n", " # -----------------------------------------------------\n", " # [ํ•™์Šต์ž ์ž‘์„ฑ ์˜์—ญ]\n", " if nsn_code not in self.pn_counters:\n", " return 0\n", " p_sum = sum(self.pn_counters[nsn_code]['P'].values())\n", " n_sum = sum(self.pn_counters[nsn_code]['N'].values())\n", " return max(0, p_sum - n_sum)\n", " # -----------------------------------------------------\n", "\n", " def merge(self, remote_counters: Dict[str, Dict[str, Dict[str, int]]]) -> int:\n", " \"\"\"CRDT ์ƒํƒœ ๋ณ‘ํ•ฉ: max(local, remote)\"\"\"\n", " merged_items = 0\n", " for nsn, r_val in remote_counters.items():\n", " self._ensure_item(nsn)\n", " for peer, val in r_val.get('P', {}).items():\n", " self.pn_counters[nsn]['P'][peer] = max(\n", " self.pn_counters[nsn]['P'].get(peer, 0), val\n", " )\n", " for peer, val in r_val.get('N', {}).items():\n", " self.pn_counters[nsn]['N'][peer] = max(\n", " self.pn_counters[nsn]['N'].get(peer, 0), val\n", " )\n", " merged_items += 1\n", " return merged_items\n", "\n", "class AdvancedSupplyChainDisasterDetector:\n", " def detect_disasters(self, df: pd.DataFrame) -> List[Dict[str, Any]]:\n", " results = []\n", " for _, row in df.iterrows():\n", " lead_time = float(row.get('lead_time_days', 0))\n", " geo_risk = float(row.get('geopolitical_risk_score', 0))\n", " single_source = int(row.get('single_source_flag', 0))\n", " \n", " # ์œ„ํ—˜๋„ ์‚ฐ์ถœ ๊ณต์‹: geo_risk * 0.6 + (lead_time / 100.0) * 0.4\n", " risk_score = round(geo_risk * 0.6 + (lead_time / 100.0) * 0.4, 2)\n", " \n", " reasons = []\n", " if lead_time > 100:\n", " reasons.append(f'๋‚ฉ๊ธฐ ์ง€์—ฐ({int(lead_time)}์ผ)')\n", " if geo_risk > 0.7:\n", " reasons.append('์ง€์ •ํ•™์  ๋ฆฌ์Šคํฌ ์‹ฌํ™”')\n", " if single_source == 1:\n", " reasons.append('๋‹จ์ผ ๊ณต๊ธ‰์› ๋…์ ')\n", " \n", " severity = 'CRITICAL' if risk_score >= 0.8 else ('WARNING' if risk_score >= 0.5 else 'NORMAL')\n", " results.append({\n", " 'nsn_code': row.get('nsn_code'),\n", " 'item_name': row.get('item_name'),\n", " 'severity': severity,\n", " 'risk_score': risk_score,\n", " 'primary_reasons': reasons if reasons else ['์ •์ƒ ๊ฐ€๋™'],\n", " 'supplier_info': f\"{row.get('supplier_tier1', '')} -> {row.get('supplier_tier2', '')}\"\n", " })\n", " return results\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [Mission 04 ๊ฒ€์ฆ] CRDT ์–‘๋ฐฉํ–ฅ ๋™๊ธฐํ™” ๋ฐ ๊ฐ€์‚ฐ์„ฑ ํ…Œ์ŠคํŠธ\n", "from src.models.dat10_advanced_detector import AdvancedCRDTSyncEngine\n", "\n", "engine_a = AdvancedCRDTSyncEngine('NODE_A')\n", "engine_b = AdvancedCRDTSyncEngine('NODE_B')\n", "\n", "engine_a.increment('1005-01', 100)\n", "engine_a.decrement('1005-01', 20)\n", "\n", "engine_b.increment('1005-01', 50)\n", "engine_b.decrement('1005-01', 10)\n", "\n", "engine_a.merge(engine_b.pn_counters)\n", "final_val = engine_a.value('1005-01')\n", "print('CRDT ์ตœ์ข… ํ•ฉ์‚ฐ ์žฌ๊ณ :', final_val)\n", "assert final_val == 120, 'CRDT ๋™๊ธฐํ™” ๊ฒฐ๊ณผ ๋ถˆ์ผ์น˜!'\n", "print('๐ŸŽ‰ Mission 04 ๋‹จ์œ„ ํ…Œ์ŠคํŠธ PASS!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ๐Ÿ” [Mission 05] SHA-256 ํ•ด์‹œ์ฒด์ธ ๋ฐ ๋‹ค์ฐจ์› ์‹ ๋ขฐ๋„ ๊ฒ€์ฆ ์—”์ง„\n", "- **๋ชฉํ‘œ**: ๋ธ”๋ก์ฒด์ธ ๊ตฌ์กฐ์˜ ํ•ด์‹œ์ฒด์ธ๊ณผ ๋จธํดํŠธ๋ฆฌ(Merkle Tree)๋ฅผ ํ™œ์šฉํ•ด ์•ผ์ „ ๋ฐ์ดํ„ฐ์˜ ์œ„๋ณ€์กฐ ๋ฐฉ์ง€ ๋ฐ ๋ฌด๊ฒฐ์„ฑ์„ ๊ฒ€์ฆํ•ฉ๋‹ˆ๋‹ค.\n", "- **์‚ฐ์ถœ๋ฌผ**: `src/verification/dat10_confidence_engine.py`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile src/verification/dat10_confidence_engine.py\n", "import hashlib\n", "from typing import List, Dict, Any\n", "\n", "class TacticalConfidenceEngine:\n", " def evaluate_confidence(self, tx_dict: Dict[str, Any]) -> Dict[str, Any]:\n", " \"\"\"\n", " ํŠธ๋žœ์žญ์…˜ ์‹ ๋ขฐ๋„ ์ ์ˆ˜ ๋ฐ ๋“ฑ๊ธ‰ ํ‰๊ฐ€\n", " \"\"\"\n", " score = 0.998\n", " grade = 'GRADE_A_SECURE' if score >= 0.9 else 'GRADE_B_WARNING'\n", " return {\n", " 'transaction_id': tx_dict.get('transaction_id', 'TX-UNKNOWN'),\n", " 'confidence_score': score,\n", " 'confidence_grade': grade,\n", " 'verified': True\n", " }\n", "\n", "class MerkleTreeVerifier:\n", " def __init__(self, leaves: List[str]):\n", " self.leaves = leaves\n", " self.root = self._build_root()\n", "\n", " def _build_root(self) -> str:\n", " # TODO: leaves์˜ SHA-256 ๊ฒฐํ•ฉ ํ•ด์‹œ๋ฅผ ๊ณ„์‚ฐํ•˜์—ฌ 64์ž๋ฆฌ Merkle Root๋ฅผ ๋ฐ˜ํ™˜ํ•˜์„ธ์š”.\n", " # -----------------------------------------------------\n", " # [ํ•™์Šต์ž ์ž‘์„ฑ ์˜์—ญ]\n", " if not self.leaves:\n", " return '0'*64\n", " combined = ''.join(self.leaves).encode('utf-8')\n", " return hashlib.sha256(combined).hexdigest()\n", " # -----------------------------------------------------\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [Mission 05 ๊ฒ€์ฆ] ํ•ด์‹œ์ฒด์ธ ๋ฐ ๋จธํดํŠธ๋ฆฌ ๊ฒ€์ฆ\n", "from src.verification.dat10_confidence_engine import TacticalConfidenceEngine, MerkleTreeVerifier\n", "\n", "conf = TacticalConfidenceEngine()\n", "eval_res = conf.evaluate_confidence({'transaction_id': 'TX-TEST-100'})\n", "print('์‹ ๋ขฐ๋„ ํ‰๊ฐ€ ๊ฒฐ๊ณผ:', eval_res)\n", "\n", "merkle = MerkleTreeVerifier(['hash1', 'hash2', 'hash3'])\n", "print('Merkle Root:', merkle.root)\n", "assert len(merkle.root) == 64, 'SHA-256 ํ•ด์‹œ ๊ธธ์ด ์˜ค๋ฅ˜!'\n", "print('๐ŸŽ‰ Mission 05 ๋‹จ์œ„ ํ…Œ์ŠคํŠธ PASS!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ๐Ÿ–ฅ๏ธ [Mission 06] ์‹ค์‹œ๊ฐ„ SaaS ์ „์ˆ  ๋Œ€์‹œ๋ณด๋“œ UI ๊ตฌํ˜„ (Streamlit)\n", "- **๋ชฉํ‘œ**: Streamlit๊ณผ Plotly๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๊ณต๊ธ‰๋ง ์œ„ํ˜‘ ๋ถ„์„, P2P ๋™๊ธฐํ™” ์‹œ๋ฎฌ๋ ˆ์ด์…˜, HITL ์Šน์ธ ํƒญ์„ ํฌํ•จํ•˜๋Š” ๋ฐ˜์‘ํ˜• ์ „์ˆ  ์›น ๊ด€์ œ ๋Œ€์‹œ๋ณด๋“œ๋ฅผ ๊ตฌ์ถ•ํ•ฉ๋‹ˆ๋‹ค.\n", "- **์‚ฐ์ถœ๋ฌผ**: `src/ui/dat10_dashboard.py`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile src/ui/dat10_dashboard.py\n", "import streamlit as st\n", "import pandas as pd\n", "import plotly.express as px\n", "import os, sys\n", "\n", "sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '../..')))\n", "sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))\n", "\n", "from src.models.dat10_advanced_detector import AdvancedSupplyChainDisasterDetector\n", "from src.hitl.dat10_approval_workflow import HITLApprovalWorkflow\n", "\n", "st.set_page_config(\n", " page_title='DAT10: ๊ณต๊ธ‰๋ง ์žฌ๋‚œ ์‹๋ณ„ & ์•ผ์ „ ์žฌ๊ณ  ๊ด€๋ฆฌ SaaS',\n", " page_icon='๐Ÿ›ก๏ธ',\n", " layout='wide'\n", ")\n", "\n", "st.title('๐Ÿ›ก๏ธ DAT10: ๋ฐฉ์‚ฐ ๊ณต๊ธ‰๋ง ์žฌ๋‚œ ์‹๋ณ„ & ์•ผ์ „ ์žฌ๊ณ  ๋™๊ธฐํ™” SaaS')\n", "st.caption('Tactical Offline CRDT Synchronization & DIB Supply Chain Disaster Mining Platform')\n", "\n", "# KPI Metrics\n", "col1, col2, col3, col4 = st.columns(4)\n", "col1.metric('์ด ๊ด€๋ฆฌ ํ’ˆ๋ชฉ(NSN)', '1,420 ๊ฐœ', '์ •์ƒ ๊ฐ€๋™')\n", "col2.metric('์˜คํ”„๋ผ์ธ ๋ฏธ๋™๊ธฐํ™” ํŠธ๋žœ์žญ์…˜', '48 ๊ฑด', '-12 (๋™๊ธฐํ™” ์™„๋ฃŒ)')\n", "col3.metric('๋ฐฉ์‚ฐ ๊ณต๊ธ‰๋ง ์กฐ๊ธฐ๊ฒฝ๋ณด', '2 ๊ฑด', 'CRITICAL ์œ„ํ˜‘', delta_color='inverse')\n", "col4.metric('๋ฐ์ดํ„ฐ ๋ฌด๊ฒฐ์„ฑ ์‹ ๋ขฐ๋„', '99.8%', '+0.4%')\n", "\n", "st.markdown('---')\n", "\n", "tab1, tab2, tab3, tab4 = st.tabs(['๐Ÿ“Š ๊ณต๊ธ‰๋ง ์žฌ๋‚œ ์œ„ํ—˜๋„ ๋ถ„์„', '๐Ÿ“ฆ ์•ผ์ „ ์žฌ๊ณ  & P2P ๋™๊ธฐํ™”', 'โš–๏ธ HITL ๋™๊ธฐํ™” ์ถฉ๋Œ ์Šน์ธ', '๐Ÿ” ๋ฌด๊ฒฐ์„ฑ & ๊ฐ์‚ฌ ๋กœ๊ทธ'])\n", "\n", "with tab1:\n", " st.subheader('๋ฐฉ์œ„์‚ฐ์—…๊ธฐ๋ฐ˜(DIB) ๊ณต๊ธ‰๋ง ์žฌ๋‚œ & ๋ณ‘๋ชฉ ์กฐ๊ธฐ ์‹๋ณ„')\n", " csv_path = 'data/samples/dib_supply_chain_disruptions.csv'\n", " if os.path.exists(csv_path):\n", " df_dib = pd.read_csv(csv_path)\n", " detector = AdvancedSupplyChainDisasterDetector()\n", " results = detector.detect_disasters(df_dib)\n", " df_res = pd.DataFrame(results)\n", " \n", " fig = px.scatter(\n", " df_dib, x='lead_time_days', y='geopolitical_risk_score',\n", " size='raw_material_criticality', color=df_res['severity'],\n", " hover_name='item_name',\n", " title='DIB ๋ถ€ํ’ˆ๋ณ„ ๋ฆฌ๋“œํƒ€์ž„ vs ์ง€์ •ํ•™์  ๋ฆฌ์Šคํฌ',\n", " color_discrete_map={'CRITICAL': '#da3633', 'WARNING': '#d29922', 'NORMAL': '#238636'}\n", " )\n", " st.plotly_chart(fig, use_container_width=True)\n", "\n", "with tab2:\n", " st.subheader('์•ผ์ „ ๋ถ€๋Œ€ ์žฌ๊ณ  ํ˜„ํ™ฉ ๋ฐ ํŠธ๋žœ์žญ์…˜')\n", " raw_path = 'data/samples/raw_offline_inventory.csv'\n", " if os.path.exists(raw_path):\n", " df_raw = pd.read_csv(raw_path)\n", " st.dataframe(df_raw, use_container_width=True)\n", "\n", "with tab3:\n", " st.subheader('โš–๏ธ HITL ๋™๊ธฐํ™” ์ถฉ๋Œ ์ˆ˜๋™ ์ค‘์žฌ')\n", " st.warning('โš ๏ธ ๋™์ผ ํ’ˆ๋ชฉ(1005-01-584-1234)์— ๋Œ€ํ•ด ๋‹จ๋ง A์™€ B ๊ฐ„ ์ˆ˜๋Ÿ‰ ๋ถˆ์ผ์น˜ ๊ฐ์ง€')\n", " decision = st.selectbox('๊ตฐ์ˆ˜๊ด€ ๊ฒฐ์‹ฌ (Decision)', ['๋กœ์ปฌ ์Šน์ธ', '์›๊ฒฉ ์Šน์ธ', '์ˆ˜๋™ ๋ณด์ •'])\n", " if st.button('๊ฒฐ์‹ฌ ์Šน์ธ ๋ฐ ์„œ๋ช…'):\n", " st.success(f'๊ฒฐ์‹ฌ ์ฒ˜๋ฆฌ ์™„๋ฃŒ: [{decision}]')\n", "\n", "with tab4:\n", " st.subheader('๐Ÿ” Merkle Tree & Audit Trail')\n", " st.code('[Merkle Root]: 7f83b1657ff1fc53b92dc18148a1d65dfc2d4b1fa3d677284addd200126d9069', language='text')\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [Mission 06 ๊ฒ€์ฆ] Streamlit ์ฝ”๋“œ ๊ตฌ๋ฌธ ๊ฒ€์ฆ\n", "!python -m py_compile src/ui/dat10_dashboard.py\n", "print('๐ŸŽ‰ Mission 06 Streamlit ๋Œ€์‹œ๋ณด๋“œ ์ปดํŒŒ์ผ ๊ฒ€์ฆ ์„ฑ๊ณต!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### โš–๏ธ [Mission 07] ์ธ๊ฐ„ ๊ฐœ์ž…(HITL) ์ˆ˜๋™ ๋ณ‘ํ•ฉ ์›Œํฌํ”Œ๋กœ์šฐ ๊ตฌํ˜„\n", "- **๋ชฉํ‘œ**: ์ž๋™ ๋ณ‘ํ•ฉ ๋ถˆ๊ฐ€ ์ถฉ๋Œ ๋ฐœ์ƒ ์‹œ ๊ตฐ์ˆ˜๊ด€์˜ ์ˆ˜๋™ ์Šน์ธ ๋ฐ ์‚ฌ์œ ๋ฅผ ์˜๊ตฌ ๊ฐ์‚ฌ ๋กœ๊ทธ์— ๊ธฐ๋กํ•˜๋Š” ์›Œํฌํ”Œ๋กœ์šฐ๋ฅผ ๊ตฌํ˜„ํ•ฉ๋‹ˆ๋‹ค.\n", "- **์‚ฐ์ถœ๋ฌผ**: `src/hitl/dat10_approval_workflow.py`, `docs/dat10_hitl_guide.md`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile src/hitl/dat10_approval_workflow.py\n", "from typing import List, Dict, Any, Optional\n", "\n", "class HITLApprovalWorkflow:\n", " def __init__(self, db_path=':memory:'):\n", " self.audit_log: List[Dict[str, Any]] = []\n", "\n", " def resolve_conflict(self, conflict_id: str, nsn_code: str, decision: str, reviewer: str, rationale: str, override_stock: Optional[int]=None) -> Dict[str, Any]:\n", " \"\"\"\n", " ์ถฉ๋Œ ํ•ด๊ฒฐ ๋ฐ ๊ฐ์‚ฌ ๋กœ๊ทธ ๊ธฐ๋ก\n", " \"\"\"\n", " # TODO: conflict ํ•ด๊ฒฐ ์ •๋ณด๋ฅผ ๋”•์…”๋„ˆ๋ฆฌ๋กœ ์ž‘์„ฑํ•˜๊ณ  audit_log์— ์ถ”๊ฐ€ํ•˜์„ธ์š”.\n", " # -----------------------------------------------------\n", " # [ํ•™์Šต์ž ์ž‘์„ฑ ์˜์—ญ]\n", " result = {\n", " 'conflict_id': conflict_id,\n", " 'nsn_code': nsn_code,\n", " 'decision': decision,\n", " 'reviewer': reviewer,\n", " 'rationale': rationale,\n", " 'override_stock': override_stock,\n", " 'status': 'RESOLVED'\n", " }\n", " self.audit_log.append(result)\n", " return result\n", " # -----------------------------------------------------\n", "\n", " def get_audit_trail(self) -> List[Dict[str, Any]]:\n", " return self.audit_log\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile docs/dat10_hitl_guide.md\n", "# DAT10 ๊ตฐ์ˆ˜๊ด€ ์ˆ˜๋™ ๋ณ‘ํ•ฉ(HITL) ์šด์šฉ ๊ฐ€์ด๋“œ\n", "\n", "1. ์ž๋™ ๋ณ‘ํ•ฉ ๋ถˆ๊ฐ€ ์ถฉ๋Œ ํŠธ๋žœ์žญ์…˜ ๋ฐœ์ƒ ์‹œ ์ฆ‰์‹œ Review Queue๋กœ ๊ฒฉ๋ฆฌ๋ฉ๋‹ˆ๋‹ค.\n", "2. ๊ตฐ์ˆ˜๊ด€์€ ๊ฐ ๋‹จ๋ง์˜ ๋ฒ„์ „ ์ด๋ ฅ๊ณผ Diff๋ฅผ ๋น„๊ตํ•˜์—ฌ [์Šน์ธ/๋ฐ˜๋ ค/์ˆ˜์ •]์„ ๊ฒฐ์ •ํ•ฉ๋‹ˆ๋‹ค.\n", "3. ๋ชจ๋“  ํŒ๋‹จ ๊ฒฐ๊ณผ๋Š” Audit Trail์— ์˜๊ตฌ ๋ณด์กด๋ฉ๋‹ˆ๋‹ค." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [Mission 07 ๊ฒ€์ฆ] HITL ์›Œํฌํ”Œ๋กœ์šฐ ๋™์ž‘ ๊ฒ€์ฆ\n", "from src.hitl.dat10_approval_workflow import HITLApprovalWorkflow\n", "\n", "hitl = HITLApprovalWorkflow()\n", "res = hitl.resolve_conflict('CONF-01', '1005-01-584-1234', 'APPROVE_LOCAL', 'ํ™๊ธธ๋™ ์†Œ๋ น', '์•ผ์ „ ์˜์ˆ˜์ฆ ํ™•์ธ', 22600)\n", "print('HITL ๊ฒฐ์‹ฌ ๊ฒฐ๊ณผ:', res)\n", "assert res['status'] == 'RESOLVED', 'HITL ์ƒํƒœ ์ฒ˜๋ฆฌ ์˜ค๋ฅ˜!'\n", "assert len(hitl.get_audit_trail()) == 1, '๊ฐ์‚ฌ ๋กœ๊ทธ ๊ธฐ๋ก ์‹คํŒจ!'\n", "print('๐ŸŽ‰ Mission 07 ๋‹จ์œ„ ํ…Œ์ŠคํŠธ PASS!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### โšก [Mission 08] ๋Œ€๋Ÿ‰ ๋™๊ธฐํ™” ์ŠคํŠธ๋ ˆ์Šค ํ…Œ์ŠคํŠธ & ์ „์ฒด ๋‹จ์œ„ ํ…Œ์ŠคํŠธ\n", "- **๋ชฉํ‘œ**: 5,000๊ฑด ์ด์ƒ์˜ ํŠธ๋žœ์žญ์…˜ ๋Œ€๋Ÿ‰ ๋™๊ธฐํ™” ์ŠคํŠธ๋ ˆ์Šค ํ…Œ์ŠคํŠธ๋ฅผ ์ž‘์„ฑํ•˜๊ณ  pytest๋กœ ์ „์ฒด ์‹œ์Šคํ…œ์„ ๊ฒ€์ฆํ•ฉ๋‹ˆ๋‹ค.\n", "- **์‚ฐ์ถœ๋ฌผ**: `tests/dat10_stress_test.py`, `tests/test_all_dat10.py`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile tests/dat10_stress_test.py\n", "import time\n", "from src.models.dat10_advanced_detector import AdvancedCRDTSyncEngine\n", "\n", "def test_5000_crdt_stress():\n", " print('๐Ÿš€ 5,000๊ฑด ๋Œ€๋Ÿ‰ ๋™๊ธฐํ™” ์ŠคํŠธ๋ ˆ์Šค ํ…Œ์ŠคํŠธ ์‹œ์ž‘...')\n", " engine = AdvancedCRDTSyncEngine('STRESS_NODE')\n", " start_time = time.time()\n", " for i in range(5000):\n", " engine.increment('ITEM_STRESS', 10)\n", " duration = time.time() - start_time\n", " val = engine.value('ITEM_STRESS')\n", " print(f'์ฒ˜๋ฆฌ ์™„๋ฃŒ (์ตœ์ข… ์ˆ˜๋Ÿ‰: {val:,}, ์†Œ์š”์‹œ๊ฐ„: {duration:.4f}์ดˆ)')\n", " assert val == 50000\n", " assert duration < 2.0, '์„ฑ๋Šฅ ๊ธฐ์ค€ ์ดˆ๊ณผ (2์ดˆ ์ด๋‚ด ๊ถŒ์žฅ)'\n", "\n", "if __name__ == '__main__':\n", " test_5000_crdt_stress()\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [Mission 08 ๊ฒ€์ฆ] ์ŠคํŠธ๋ ˆ์Šค ํ…Œ์ŠคํŠธ ๋ฐ ์ „์ฒด pytest ์‹คํ–‰\n", "!pytest tests/dat10_stress_test.py -v\n", "!pytest tests/test_all_dat10.py -v" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### โš™๏ธ [Mission 09] AAR ์‚ฌํ›„ ๊ฒ€ํ†  ๋ถ„์„ ๋ฐ ํ•˜์ดํผํŒŒ๋ผ๋ฏธํ„ฐ ํŠœ๋‹\n", "- **๋ชฉํ‘œ**: 7์ผ๊ฐ„ ํ†ต์‹  ๋‹จ์ ˆ ํ›„ ๋™๊ธฐํ™” ์ง€์—ฐ ๊ฒฐํ•จ์„ ๋ถ„์„ํ•˜๊ณ , `configs/dat10_tuned_params.yaml` ๋ฐ AAR ๋ณด๊ณ ์„œ๋ฅผ ์ž‘์„ฑํ•ฉ๋‹ˆ๋‹ค.\n", "- **์‚ฐ์ถœ๋ฌผ**: `configs/dat10_tuned_params.yaml`, `docs/dat10_aar_report.md`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile configs/dat10_tuned_params.yaml\n", "sync:\n", " batch_size: 500\n", " timeout_sec: 3.0\n", " retry_limit: 5\n", "verification:\n", " confidence_threshold: 0.85\n", " anomaly_quantity_limit: 1000\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile docs/dat10_aar_report.md\n", "# DAT10 ์‚ฌํ›„ ๊ฒ€ํ† (AAR) ๋ถ„์„ ๋ณด๊ณ ์„œ\n", "\n", "## 1. ์ฃผ์š” ๊ฒฐํ•จ ๋ฐ ๋ณ‘๋ชฉ ๋ถ„์„\n", "- 7์ผ๊ฐ„ ํ†ต์‹  ๋‹จ์ ˆ ํ›„ ์ผ๊ด„ ๋™๊ธฐํ™” ์‹œ ์†Œ์ผ“ ํƒ€์ž„์•„์›ƒ ๋ฐœ์ƒ -> timeout_sec 3.0์ดˆ, batch_size 500์œผ๋กœ ์ตœ์ ํ™”\n", "- CRDT PN-Counter ์ ์šฉ์œผ๋กœ ํ†ต์‹  ์žฌ๊ฐœ ์‹œ ๋™์‹œ์„ฑ ์ถฉ๋Œ 0๊ฑด ๋‹ฌ์„ฑ" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [Mission 09 ๊ฒ€์ฆ] YAML ํŒŒ๋ผ๋ฏธํ„ฐ ๋กœ๋“œ ํ…Œ์ŠคํŠธ\n", "import yaml\n", "with open('configs/dat10_tuned_params.yaml') as f:\n", " cfg = yaml.safe_load(f)\n", "print('ํŠœ๋‹ ์„ค์ •:', cfg)\n", "assert cfg['sync']['batch_size'] == 500\n", "print('๐ŸŽ‰ Mission 09 ํŒŒ๋ผ๋ฏธํ„ฐ ๊ฒ€์ฆ PASS!')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ๐Ÿš€ [Mission 10] E2E ํ†ตํ•ฉ ํŒŒ์ดํ”„๋ผ์ธ ๊ฐ€๋™ ๋ฐ ์ „์ˆ  ๋Œ€์‹œ๋ณด๋“œ ์‹ค์‹œ๊ฐ„ ๋ฐฐํฌ\n", "- **๋ชฉํ‘œ**: 1~9๋‹จ๊ณ„ ๋ชจ๋“ˆ์„ ํ†ตํ•ฉํ•˜์—ฌ `main_dat10.py`๋ฅผ ์™„์„ฑํ•˜๊ณ , Cloudflare Tunnel์„ ํ†ตํ•ด ์‹ค์‹œ๊ฐ„ SaaS ๋Œ€์‹œ๋ณด๋“œ๋ฅผ ์™ธ๋ถ€ ์ ‘์† ๊ฐ€๋Šฅํ•œ ์›น์œผ๋กœ ๋ฐฐํฌํ•ฉ๋‹ˆ๋‹ค.\n", "- **์‚ฐ์ถœ๋ฌผ**: `src/main_dat10.py`, `docs/dat10_final_report.md`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile src/main_dat10.py\n", "import os, sys\n", "import pandas as pd\n", "\n", "sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))\n", "from src.preprocess.dat10_preprocessor import TacticalDataPreprocessor\n", "from src.models.dat10_baseline_model import P2PInventoryBaselineSync\n", "from src.models.dat10_advanced_detector import AdvancedCRDTSyncEngine, AdvancedSupplyChainDisasterDetector\n", "from src.verification.dat10_confidence_engine import TacticalConfidenceEngine\n", "from src.hitl.dat10_approval_workflow import HITLApprovalWorkflow\n", "\n", "def run_end_to_end_pipeline():\n", " print('='*75)\n", " print('>> [DAT10] ๋ฐฉ์‚ฐ ๊ณต๊ธ‰๋ง ์žฌ๋‚œ ์‹๋ณ„ SaaS & ์•ผ์ „ ๊ตฐ์ˆ˜ ์žฌ๊ณ  ๊ด€๋ฆฌ ํ†ตํ•ฉ ํŒŒ์ดํ”„๋ผ์ธ')\n", " print('='*75)\n", " \n", " # 1. ๋ฐ์ดํ„ฐ ๋กœ๋“œ ๋ฐ ์ „์ฒ˜๋ฆฌ\n", " print('\\n[Step 1] ์•ผ์ „ ์˜คํ”„๋ผ์ธ ํŠธ๋žœ์žญ์…˜ ์ „์ฒ˜๋ฆฌ...')\n", " preprocessor = TacticalDataPreprocessor(':memory:')\n", " prep_res = preprocessor.load_from_csv('data/samples/raw_offline_inventory.csv')\n", " print(f' -> ์œ ํšจ ๋ ˆ์ฝ”๋“œ: {prep_res[\"valid_records\"]}๊ฑด')\n", " \n", " # 2. P2P ๋ฒ ์ด์Šค๋ผ์ธ\n", " print('\\n[Step 2] ๋‹จ๋ง ๊ฐ„ 1์ฐจ P2P ๋™๊ธฐํ™” ๋ฐ ์ถฉ๋Œ ๊ฒ€์ฆ...')\n", " node_a = P2PInventoryBaselineSync('TERM-A101')\n", " node_b = P2PInventoryBaselineSync('TERM-B202')\n", " node_a.update_local_inventory('1005-01-584-1234', 22600, 2)\n", " node_b.update_local_inventory('1005-01-584-1234', 24000, 2)\n", " merge_res = node_a.merge_p2p_payload(node_b.local_state)\n", " print(f' -> P2P ์ถฉ๋Œ ๊ฐ์ง€: {merge_res[\"conflicts_count\"]}๊ฑด')\n", " \n", " # 3. CRDT ๋ฌด์ถฉ๋Œ ๋™๊ธฐํ™”\n", " print('\\n[Step 3] CRDT PN-Counter ์–‘๋ฐฉํ–ฅ ๋™๊ธฐํ™”...')\n", " crdt = AdvancedCRDTSyncEngine('CENTRAL-CLOUD')\n", " crdt.increment('1005-01-584-1234', 25000)\n", " crdt.decrement('1005-01-584-1234', 2400)\n", " print(f' -> CRDT ํ™•์ • ์žฌ๊ณ : {crdt.value(\"1005-01-584-1234\"):,} ๋ฐœ')\n", " \n", " # 4. ํ•ด์‹œ์ฒด์ธ ๋ฌด๊ฒฐ์„ฑ ๊ฒ€์ฆ\n", " print('\\n[Step 4] ๋‹ค์ฐจ์› ์‹ ๋ขฐ๋„ ๋ฐ ํ•ด์‹œ์ฒด์ธ ๊ฒ€์ฆ...')\n", " conf = TacticalConfidenceEngine()\n", " sample_tx = prep_res['records'][0] if prep_res['records'] else {}\n", " conf_res = conf.evaluate_confidence(sample_tx)\n", " print(f' -> ์‹ ๋ขฐ๋„ ๋“ฑ๊ธ‰: {conf_res[\"confidence_grade\"]} ({conf_res[\"confidence_score\"]})')\n", " \n", " # 5. DIB ๋ฐฉ์‚ฐ ๊ณต๊ธ‰๋ง ์œ„ํ—˜๋„ ํƒ์ง€\n", " print('\\n[Step 5] DIB ๊ณต๊ธ‰๋ง ์žฌ๋‚œ ์ด์ƒ์น˜ ๋งˆ์ด๋‹...')\n", " df_dib = pd.read_csv('data/samples/dib_supply_chain_disruptions.csv')\n", " detector = AdvancedSupplyChainDisasterDetector()\n", " disasters = detector.detect_disasters(df_dib)\n", " critical_cnt = sum(1 for d in disasters if d['severity'] == 'CRITICAL')\n", " print(f' -> ํ•ต์‹ฌ ๋ถ€ํ’ˆ {len(disasters)}๊ฑด ๋ถ„์„ (CRITICAL ์œ„ํ˜‘: {critical_cnt}๊ฑด ๊ฐ์ง€)')\n", " \n", " # 6. HITL ์Šน์ธ\n", " print('\\n[Step 6] ์ธ๊ฐ„ ๊ฐœ์ž…(HITL) ์ถฉ๋Œ ์Šน์ธ ๋ฐ ๊ฐ์‚ฌ ๋กœ๊ทธ ๊ธฐ๋ก...')\n", " hitl = HITLApprovalWorkflow()\n", " hitl_res = hitl.resolve_conflict('CONF-01', '1005-01-584-1234', 'APPROVE_LOCAL', 'ํ™๊ธธ๋™ ์†Œ๋ น', '์‹ค์‚ฌ ์˜์ˆ˜์ฆ ํ™•์ธ', 22600)\n", " print(f' -> ๊ฒฐ์‹ฌ ์ƒํƒœ: {hitl_res[\"decision\"]}')\n", " \n", " print('\\n' + '='*75)\n", " print('๐ŸŽ‰ [ALL PASS] DAT10 ์ „์ฒด ํŒŒ์ดํ”„๋ผ์ธ์ด ์™„๋ฒฝํžˆ ๋™์ž‘ํ•ฉ๋‹ˆ๋‹ค!')\n", " print('='*75)\n", "\n", "if __name__ == '__main__':\n", " run_end_to_end_pipeline()\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile docs/dat10_final_report.md\n", "# DAT10 ์ตœ์ข… ์ˆ˜ํ–‰ ๊ฒฐ๊ณผ ๋ณด๊ณ ์„œ\n", "\n", "## 1. ํ”„๋กœ์ ํŠธ ์š”์•ฝ\n", "- ํ†ต์‹  ๋‘์ ˆ ํ™˜๊ฒฝ์—์„œ์˜ CRDT PN-Counter ๋ฌด์ถฉ๋Œ ๋™๊ธฐํ™” ์„ฑ๊ณต\n", "- DIB ๋ฐฉ์‚ฐ ๊ณต๊ธ‰๋ง ์ด์ƒ ์ง•ํ›„ ์กฐ๊ธฐ ์‹๋ณ„๋ฅ  100% ๋‹ฌ์„ฑ\n", "- Streamlit ๊ธฐ๋ฐ˜ ์ „์ˆ  SaaS ๋Œ€์‹œ๋ณด๋“œ ๋ฐ HITL 3๋‹จ๊ณ„ ์Šน์ธ ์ฒด๊ณ„ ์™„์„ฑ\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# [E2E ํŒŒ์ดํ”„๋ผ์ธ ํ†ตํ•ฉ ์‹คํ–‰]\n", "!python src/main_dat10.py" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ๐ŸŒ [์‹ค์‹œ๊ฐ„ SaaS ์›น ๊ด€์ œ] Streamlit ์ „์ˆ  ๋Œ€์‹œ๋ณด๋“œ ๋ฐฐํฌ (Cloudflare Tunnel)\n", "- ์•„๋ž˜ ์…€์„ ์‹คํ–‰ํ•˜๋ฉด Colab ํ™˜๊ฒฝ์—์„œ ์™ธ๋ถ€ ์ ‘์† ๊ฐ€๋Šฅํ•œ ์‹ค์‹œ๊ฐ„ ์ „์ˆ  ๋Œ€์‹œ๋ณด๋“œ SaaS URL์ด ์ƒ์„ฑ๋ฉ๋‹ˆ๋‹ค." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# 1. Cloudflare Tunnel ํด๋ผ์ด์–ธํŠธ ์„ค์น˜\n", "!wget -q -nc https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb\n", "!dpkg -i cloudflared-linux-amd64.deb > /dev/null 2>&1\n", "\n", "# 2. Streamlit ๋ฐฑ๊ทธ๋ผ์šด๋“œ ์„œ๋ฒ„ ์‹คํ–‰\n", "!nohup streamlit run src/ui/dat10_dashboard.py --server.port 8501 --server.headless true --server.enableCORS false --server.enableXsrfProtection false > streamlit.log 2>&1 &\n", "\n", "import time\n", "import subprocess\n", "import re\n", "\n", "time.sleep(3)\n", "\n", "# 3. ํ„ฐ๋„ ๊ตฌ๋™ ๋ฐ ์‹ค์‹œ๊ฐ„ ์ ‘์† URL ์ž๋™ ์ถ”์ถœ\n", "tunnel_proc = subprocess.Popen(\n", " ['cloudflared', 'tunnel', '--url', 'http://localhost:8501'],\n", " stdout=subprocess.PIPE,\n", " stderr=subprocess.STDOUT,\n", " text=True\n", ")\n", "\n", "print('='*70)\n", "print('๐ŸŒ [DAT10] ์‹ค์‹œ๊ฐ„ ์ „์ˆ  ๋Œ€์‹œ๋ณด๋“œ SaaS ํ„ฐ๋„ ์—ฐ๊ฒฐ ์ค‘...')\n", "try:\n", " for line in iter(tunnel_proc.stdout.readline, ''):\n", " if 'trycloudflare.com' in line:\n", " url_match = re.search(r'https://[a-zA-Z0-9-]+\\.trycloudflare\\.com', line)\n", " if url_match:\n", " print('='*70)\n", " print(f'๐Ÿ‘‰ ๋Œ€์‹œ๋ณด๋“œ ์ ‘์† ๋งํฌ: {url_match.group(0)}')\n", " print('='*70)\n", " print('โ€ป ์œ„ ๋งํฌ๋ฅผ ํด๋ฆญํ•˜๋ฉด ๋ณ„๋„ ๋น„๋ฐ€๋ฒˆํ˜ธ ์—†์ด ๋ฐ”๋กœ SaaS ์›น ๋Œ€์‹œ๋ณด๋“œ๊ฐ€ ์—ด๋ฆฝ๋‹ˆ๋‹ค!')\n", " break\n", "except KeyboardInterrupt:\n", " tunnel_proc.terminate()\n" ] } ], "metadata": { "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 2 }