パラメータを増やさずに、MoEのエキスパートの組み合わせを約18倍に増やせるとしたら？ タイトル: MoRE: Mixture of Reused Experts URL: ❓ MoREとは何ですか？ 隣り合う層でエキスパートのプールを共有するMoEです。再利用係数R=2なら、2層で2N個のプールを共有し、物理パラメータは標準MoEと同じです。 💡 何が嬉しいのですか？ N=32、K=4のとき、選べる組み合わせは35,960通りから635,376通りへ約18倍に増えます。連続する層で同じエキスパートを使うことも可能になります。 ❓ 実装は難しいですか？ 重み共有と、層ごとの深さ埋め込み1つだけです。ルーターは層ごとに別で、独自の正規化や安定化の仕組みも不要と報告されています。 ❓ 効果はありますか？ 1.15B規模の検証perplexityは、DeepSeek 11.88に対しMoRE 11.75。6ベンチマーク平均の精度は47.05から48.12に上がりました。SonicMoEではスループットも約96%を保ちます。 💡 注意点は？ 検証は1.15Bまで、学習は約65.5億トークンです。100B超での挙動は未検証で、再利用率もランダムの約2倍にとどまります。 #MoE# #LLM#

モデルを切り替えるたびに会話の文脈を再計算し直すの、実はもったいないかもしれません。 タイトル: Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse URL: ❓ そもそも何をする研究？ 💡 NVIDIAの研究チームが、あるサイズのLLMが計算したKVキャッシュを、別サイズの同系統モデルにそのまま「翻訳」して使い回す手法を提案しました。学習不要、閉形式のリッジ回帰だけで変換できます。 ❓ なぜモデルサイズをまたいで使い回したいの？ 💡 実運用ではコストと品質のバランスを取るために、小さいモデルから大きいモデルへエスカレーションしたり、会話途中でモデルを切り替えたりします。そのたびにゼロから文脈を再計算（re-prefill）するのは無駄が多いためです。 ❓ 精度は落ちないの？ 💡 良い組み合わせでは元モデル単体の73〜98%の精度を維持します。相性の悪いペアでは大きく劣化しますが、小さなMLPを追加するだけでHellaSwagが最大36.8ポイント回復しました。 ❓ どれくらい速くなるの？ 💡 再prefillと比べて2.7〜25倍高速。32Kトークンの長い文脈では最大25倍の差がつき、10ターン程度の会話でも精度の劣化はごくわずかでした。 #LLM# #KVキャッシュ#

TL;DR: 言語モデルに自分のコンテキストを「編集可能なファイル」として自由に管理させる新しいパラダイム「Context Language Models」が、多くのベンチマークで精度向上とFLOPs削減を同時に達成しました。 タイトル: Context Language Models URL: ポイント 📝 コンテキストをファイルとしてミラーリングし、モデル自身がBashコマンドで自由に編集・追記できる設計。要約や検索の閾値を人間が決める従来のハーネスから脱却 🧠 明示的な指示なしに「インコンテキスト・スコアボード」や再利用可能な関数定義など創発的な戦略を獲得 🔍 BrowseComp-Plusで精度59.4%・FLOPs21.5%削減、EdgeBench-10では要約ベース比スコアを上回りつつFLOPsを半分以下に 🎯 強化学習ではQwen3.5-9Bの精度を28.8%→42.5%に向上させつつFLOPsも削減（成功した軌跡の中でのみ効率に報酬を与える設計でハッキングを防止） ⚡ 新技術Suffix Cache Reuseが、コンテキスト編集後も残存部分のKVキャッシュを再利用し、サーバー側計算量を標準比35%削減 🧩 ContextBenchの診断タスクでは、要約ベースが失敗する高コンテキスト圧力下でもCLMは完全な精度を維持 コンテキスト管理そのものをモデルの能力として扱うという発想転換に、大きな可能性を感じます。 #コンテキスト管理# #LLM効率化#

# Learning Palantir Foundry 🚀 The first move that turns a dataset into a business object. How you design Object Types largely decides downstream app performance and UX. 📌 Title and Feature URL Title: オブジェクトタイプ URL: 📝 Overview An Object Type defines the schema for a real-world entity or event. A single occurrence is an object instance (e.g., employee "Melissa Chang"), while a group is an object set (e.g., all tenured employees). This mirrors how datasets handle rows and filtered row collections. 🔧 How It Works - Primary keys and identity: objects need a primary key to uniquely identify instances. Mapping a data source to the object type lets you create and display objects in applications. - Properties: define an object's characteristics, with options such as edit-only properties, required properties, and shared properties reused across multiple object types. - Property types: support time series data, geospatial information, and struct types (nested, complex properties). - Display and search: title/display settings and search indexing improve discoverability inside apps. - Value Types: custom value types with versions, permissions, and constraints standardize representation across the ontology. 🛠 Practical Usage - Connect an employee directory or enterprise data to an Employee object type, converting raw datasets into actionable ontology instances. - Nail down primary key design first and index for search to secure downstream app performance and UX. - Use struct properties to auto-map hierarchical data, combined with shared properties for reuse. 🎯 Use Cases - Turn a customer master into a Customer object so the whole company shares one identity. - Model sensor-equipped assets with time series properties to retain operating history. - Model sites and stores with geospatial properties for map-based search and aggregation. ⚠️ Caveats - Primary key design, property types, and search indexing largely determine later app performance and UX, so treat them as your most important modeling decisions. - You must correctly map a data source to the object type before objects can be created or displayed. #PalantirFoundry# #Ontology#

# Learning Palantir Foundry 🚀 Put business logic right on the ontology. Functions cure the "numbers don't match across departments" problem by centralizing logic in one place. 📌 Title and Feature URL Title: ファンクション URL: 📝 Overview Functions let you write server-side logic that executes in isolated environments, powering operational apps like dashboards and decision-support tools. They are designed to work with Foundry ontologies, so they can read object properties, traverse links, and perform flexible ontology edits. 🔧 How It Works - Supported languages: TypeScript (full feature support) and Python (beta, with growing support especially for serverless and deployed execution). - Serverless execution: spins up on demand when invoked and bills only during execution, with a 60-second total wall-clock timeout (30s CPU plus a 30s network buffer). Multiple versions can run simultaneously, making upgrades safer. - Deployed execution: reserves dedicated resources for cases serverless cannot meet, runs a single version at a time, and bills continuously while deployed. - Capability differences: ontology read/write, Workshop integration, and external API calls work in both languages. Pipeline Builder is Python, while model embedding and semantic search are TypeScript. 🛠 Practical Usage - Derived properties: display function-computed values as table columns. - Function-backed Actions: implement complex edits spanning multiple objects. - Workshop integration: run functions to compute or display variables. - API gateway: invoke query functions programmatically to reuse the same logic everywhere. 🎯 Use Cases - Implement derived-KPI logic once and return identical results to Workshop, OSDK, and the API. - Query external systems to enrich ontology objects. - Build complex validation or bulk updates as function-backed Actions. ⚠️ Caveats - The 60-second timeout applies uniformly across execution modes, so optimize for efficiency. - Available capabilities depend on the invocation context (for example, model embedding and semantic search are TypeScript only), so decide on language early. #PalantirFoundry# #DataEngineering#

# Learning Palantir Foundry 🚀 Action Types are what decisively separate Foundry from read-only BI. Approvals and assignments become safe, validated, structured writes. 📌 Title and Feature URL Title: アクションタイプ URL: 📝 Overview An Action Type defines a set of changes a user can apply to ontology objects, properties, and links in a single transaction. It encapsulates both the data modifications and any side effects triggered on submission, letting users think in terms of overall goals rather than individual property edits. 🔧 How It Works - Write-back to the ontology: when an action runs, all changes are committed to the ontology and reflected across every app. The latest object data, including user edits, is captured in the object type's write-back dataset. - Parameters and defaults: parameters standardize input, supporting default values, filtered dropdown results, and overrides. - Rules: define when and how an action executes, including object relationships and property constraints. - Submission criteria and validation: validation rules control execution eligibility and error handling before changes persist. - Action logs: a full audit trail of every executed action supports accountability and compliance. 🛠 Practical Usage - Run an "Assign Employee" style action that changes a role property, auto-creates a manager-employee link, and notifies stakeholders in one transaction. - Embed submission criteria like "only a director may submit amounts over 1M yen" as validation, replacing Excel-plus-email approvals with structured operations. - Reuse the same validation logic and workflow consistently across every user-facing app. 🎯 Use Cases - Standardize status changes, approvals, and assignments as permissioned, criteria-bound operations. - Let non-technical users safely execute multi-step changes spanning several objects. - Use the action log of every operation as an audit trail for internal controls. ⚠️ Caveats - Actions execute only after passing their validation rules, so submission-criteria design drives the quality of your controls. - Changes propagate immediately across the ontology and all apps, so do not leave rule and parameter design ambiguous. #PalantirFoundry# #Ontology#