Senior Software Engineer, Research Data Infrastructure
(Frontier AI Research Systems & Trusted Data Infrastructure)
Compensation: $850K-$1M total compensation
Location: New York, NY. Four days per week, onsite
The Company
An established investment technology organization is building a dedicated AI research lab focused on transforming how machine intelligence supports complex market research. The team is working on systems that reason across large-scale, noisy, human-generated, and causally complex information.
This is a rare environment combining frontier AI research, production-grade infrastructure, and high-stakes decision-making.
The Opportunity
This is a Senior/Staff-level software engineering role focused on building the core data infrastructure behind advanced AI and investment research workflows. The work is not traditional reporting or analytics data engineering; it is deep software engineering across distributed systems, trusted data platforms, orchestration, and research infrastructure.
You will help shape foundational systems that make complex research data reliable, traceable, observable, and usable at scale.
The Role
You will design and scale the infrastructure that powers critical research and production workflows across an AI-native research lab. You will work closely with Research Scientists to turn novel ideas into robust platforms, with a strong emphasis on correctness, provenance, operational trust, and long-term maintainability.
What You’ll Do
- Build distributed systems and trusted data infrastructure for frontier AI and research workflows.
- Design resilient processing and orchestration systems with strong guarantees around reliability, correctness, and data integrity.
- Partner with Research Scientists to productionize workflows using messy, human-generated, and unstructured datasets.
- Develop systems for observability, lineage, monitoring, data quality, and infrastructure health.
- Improve platform reliability, resiliency, automation, and operational maturity across critical research infrastructure.
- Modernize existing systems while architecting new foundational platform capabilities.
- Use AI-native engineering workflows, coding agents, and automated development systems to accelerate delivery and improve quality.
What You’ll Bring
- 8+ years of experience building scalable distributed systems, data platforms, or production research infrastructure.
- Strong Python engineering skills and excellent software design fundamentals.
- Deep experience with orchestration frameworks, distributed processing, and production-grade infrastructure.
- Strong understanding of data modeling, schema design, lineage, observability, monitoring, and data quality.
- Experience supporting large-scale research, ML, scientific experimentation, or knowledge-intensive workflows.
- Ability to work with ambiguous, noisy, human-generated, or unstructured data while preserving context and traceability.
- Strong communication skills and comfort partnering closely with research, engineering, and technical stakeholders.
What This Role Requires
- 8+ years building scalable distributed systems and data platforms.
- Strong Python engineering ability and software design fundamentals.
- Experience building infrastructure with strong guarantees around correctness, traceability, reliability, and data integrity.
- Hands-on experience with distributed systems, orchestration frameworks, and production data infrastructure.
- Practical fluency using AI-native development workflows, coding agents, and automated engineering systems.
Tech Stack
- Python
- Flink
- Ray
- Spark
- Kafka
- Airflow
- Databricks
- Snowflake
- Distributed systems
- Data infrastructure
- Orchestration frameworks
- Observability, lineage, monitoring, and data quality tooling
Why Join
- Build foundational infrastructure for one of the most technically ambitious applications of AI in investment research.
- Own high-impact systems where correctness, trust, provenance, and scale are critical.
- Join a well-resourced, AI-native research environment with strong compensation and significant technical upside.
About People in AI
People in AI is a specialist recruitment partner connecting exceptional AI, machine learning, data, and engineering talent with some of the most ambitious technology companies in the world. We work closely with founders, technical leaders, and hiring teams to represent opportunities accurately and help candidates assess genuine fit.
