Member of Technical Staff - Inference Platform

About the role

Avra is an AI lab building relational foundation models from the ground up, used by leading Latin American enterprises in credit, growth and fraud. As a Member of Technical Staff on the Platform team, you will own where the models execute: running governed model releases reliably and efficiently in the cloud and on customer-hosted Kubernetes, making inference fast, predictable and cheap enough for enterprise and mid-market customers.

Key responsibilities

• Evolve Sophos, the online and batch inference runtime built on Kubernetes.
• Run large batch inference on ephemeral jobs with multi-dimensional admission control (CPU, memory, GPU).
• Build and extend the controller and its Kubernetes custom resources.
• Optimize each model's inference engine and feature processing, and serve graphs and data efficiently.
• Own execution of training, post-training and fine-tuning jobs in the cloud and in customer dataplanes or on-premise cloud.
• Drive autoscaling, GPU serving, performance and cost optimization, with telemetry for every model.

Required profile

• Experience running model serving or large-scale batch compute on Kubernetes.
• Experience building Kubernetes controllers or operators.
• Clear sense of cost, treating compute efficiency as a product feature.
• Production-quality code and reviews, and willingness to operate what you build.

Required skills

• Profiling and optimizing data-heavy Python pipelines.
• Nice to have: Ray, Ray Serve or KubeRay in production; admission-control systems; GPU serving and performance optimization; Arrow, Parquet, Lance or other columnar formats; shipping software to customer-hosted Kubernetes; GCP/AWS and GKE/EKS, and experience in financial services or regulated environments.