Performance Modeling Engineer, AI Inference Deployments (Senior/Principal)
Salary
$220,000–$270,000
Experience
5+ years
Jobgether is hiring a Performance Modeling Engineer, AI Inference Deployments (Senior/Principal), fully remote. This role is tied to United States; check the apply page for its specific location or time-zone requirements.
The role typically suits candidates with around 5+ years of experience in engineering.
Key skills
About this role at Jobgether
This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Performance Modeling Engineer, AI Inference Deployments (Senior / Principal) based in the United States.
In this role, you will develop advanced performance models and deployment-planning tools that help customers understand how large-scale AI inference workloads will perform on next-generation hardware. You will work at the intersection of silicon architecture, systems engineering, and customer solutions to translate complex deployment requirements into accurate, actionable performance estimates. Your work will cover everything from chip-level simulation to large-scale deployment configurations, including throughput, latency, scalability, and hardware requirements. You will collaborate directly with customers, product teams, and chip modeling engineers to evaluate deployment strategies and identify performance optimization opportunities. A key part of your mission will be developing analytical models and customer-facing tools that enable organizations to independently estimate and plan their AI infrastructure needs. You will also validate predictions against real hardware measurements, ensuring that models remain accurate as hardware, software, and workloads evolve. This is an opportunity to influence major AI infrastructure decisions and help shape how frontier-scale inference deployments are designed, evaluated, and scaled.
Accountabilities:
- Partner with customers and product teams to translate deployment requirements into structured performance studies, defining assumptions, scenarios, metrics, and actionable recommendations.
- Conduct objective, apples-to-apples performance comparisons between competing hardware platforms, including GPUs and other AI accelerators, identifying the factors driving differences in performance.
- Design, develop, and deliver a standalone performance-estimation tool that customers can use to evaluate their own AI inference workloads and deployment configurations.
- Build and maintain deployment-level performance models for large-scale inference systems, accounting for tensor, pipeline, expert, and data parallelism strategies.
- Evaluate the impact of batch sizes, context lengths, workload characteristics, and mixed hardware configurations involving different accelerator platforms.
- Model network and system-level performance, including interconnect latency, bandwidth, topology, and redundancy, to understand how these factors influence throughput and scalability as the number of chips increases.
- Develop and extend analytical and simulation-based deployment models that leverage existing chip performance simulators, ensuring that all deployment estimates remain consistent with a shared hardware performance model.
- Analyze system configurations and identify performance bottlenecks, scalability limitations, and opportunities to optimize deployment efficiency.
- Validate performance projections against actual hardware measurements and benchmark results, continuously calibrating models as hardware architectures, software environments, and workloads evolve.
- Collaborate closely with chip modeling and engineering teams to incorporate new hardware capabilities and improve the accuracy of deployment predictions.
- Communicate technical findings clearly to customers and internal stakeholders, explaining performance estimates, key assumptions, influencing factors, and confidence levels.
- Depending on experience and seniority, take ownership of either the overall modeling architecture or individual model deployment studies and the development of customer-facing performance tools.
Requirements:
- At least 5 years of professional experience in performance modeling, capacity planning, systems architecture, or a related field involving large-scale AI inference, high-performance computing (HPC), or data-center infrastructure.
- Demonstrated experience developing analytical or queueing-based performance models and calibrating predictions against real-world system measurements.
- Deep understanding of large language model (LLM) inference at the systems level, including prefill and decode phases, KV cache management, parallelism strategies, mixture-of-experts (MoE) routing, batching, time-to-first-token (TTFT), and tokens-per-second performance metrics.
- Strong knowledge of large-scale deployment architectures and the factors affecting inference throughput, latency, hardware utilization, and scalability.
- Solid understanding of data-center networking concepts, including interconnect latency, bandwidth, network topology, and their impact on distributed system performance.
- Good knowledge of GPU architecture and performance characteristics, with the ability to evaluate and compare different accelerator platforms.
- Experience building analytical models, conducting performance experiments, interpreting benchmark results, and translating…
Description provided by the employer’s job posting.
Apply directly through the employer
This listing links straight to Jobgether’s own application page.
More remote jobs in United States
View all →Veteran Affairs Medical Sales Representative
150501- Electronic Wave Form Lab Inc
IX Mid-Market Account Executive, Foundations
Gong.io
Member of Technical Staff
Flox