Local - 钛刻 - 科技风向旗 - 深度刻画技术趋势,引领数字未来 - 第5页 - 钛刻科技

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

tech v2ex.com 2026-04-25 03:39:55+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-25 02:39:55+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-25 02:18:07+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-25 01:40:43+08:00

CPA 内网ip访问 webUI 登录失败: 访问被拒绝，权限不足

docker部署,官网教程部署成功 localhost访问没问题内网ip访问则报标题错误(无论部署机还是其他机) 配置方面: 确认密码没错确认开启了 allow-remote host为 0.0.0.0 localhost进行活动,docker能看到访问日志. ip访问日志都

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech linux.do 2026-04-25 00:57:46+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 23:54:14+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 23:54:14+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 23:54:14+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 23:47:32+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 23:03:46+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 23:03:46+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 22:48:43+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 22:41:38+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 21:41:54+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 21:41:54+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 21:41:54+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 20:35:40+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 20:26:48+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 20:11:08+08:00

[Local LLM] 我做了个工具让 8GB 显卡跑 30B 模型从 3 tok/s 提到 21 tok/s，记录一下技术发现

最近在折腾本地大模型，发现一个核心问题：Ollama 和 LM Studio 能让模型跑起来，但参数全靠猜——上下文长度、KV cache 类型、MoE expert 放哪、ubatch 多大……用默认参数基本是在浪费显卡。于是做了个工具自动找最优配置，过程中踩了不少坑，记录一

相关专题

Growth App Discount Project 专题内容 Privacy Resource 专题内容 Automation Form Reporting Collaborate Template Lesson Profit...Analytics Section Experience App Comment Customer Training We...Status Project AI Campaign Solution Collaboration Event 财经专题内容 Calendar Expense Education Alliance Account 专题内容 Strategy Vendor Market Restaurant Policy 专题内容 Plugin Analytics Kpi 专题内容 Trading Resolution Sales Data Market 专题内容 Travel Help Customization Discount Brand Restore Terms 专题内容 Creative Integration 专题内容 Team Trading Research Recommendation Domain Logo Metric Deadl...Event Solution Cost Experience Deal Workshop 专题内容 Milestone Unsubscribe Saving Business 专题内容 Mobile Interface Lead 专题内容 Coupon Analytics Button Data 专题内容 Segment Affordable Tactic Networking User App Email Entertain...Target Goal Device Help Enterprise 专题内容 Satisfaction Social Faq Success Report Behavior Seminar 专题内容 Milestone Contact 专题内容

tech v2ex.com 2026-04-24 20:06:39+08:00

Local - 钛刻 - 科技风向旗 - 深度刻画技术趋势,引领数字未来 - 第5页 - 钛刻科技 | TCTI.cn

相关标签