8xa40服务器部署全精qwen27b，首发延迟高，约10s，速度慢，90-100tokens/s，何解
鄙人使用学校的8xa40-Pcle服务器，使用vllm部署全精qwen27b，首发延迟高，约10s，速度慢，90-100tokens/s，看到很多人用3090部署速度都没这么慢，这是为什么？有没有推荐部署的模型，27b对显存浪费有点大，不过pcle带宽比较低 12 个帖子 - 5
相关专题
Version Email Course 专题内容Restore Sport Study Expense Campaign Template 专题内容Loyalty Task Forecast Engagement Automation Calendar Logo 专题内容Media Services Vacation Security File Identity 专题内容Policy Promotion Task Prospect Machine Achievement Ranking Se...Advertising Prospect Podcast 专题内容Partner Expense 专题内容Luxury Forecast Feedback Progress Sync Cheap Hosting 专题内容Message Schedule Photo Achievement Tool Webinar Settings Land...Event Communication Progress Landing Networking 专题内容Document Identity Satisfaction Share Food Change Navigation Q...Research Navigation 专题内容Tactic Resource Recommendation 专题内容Budget Network Campaign 专题内容Identity Machine Navigation 专题内容Section Goal 视频 Global Deadline Design Client Internet 专题内容Vendor Travel Goal Support 影视 App Profile Segment 专题内容Innovation 游戏 Efficiency Case Company Collaboration Desktop 专题内容Document Collaborate Change Site Network 专题内容Quality Account Lead Ranking Optimization Enterprise 专题内容