Senior Software Engineer, Research Data Infrastructure
New York City, 4 days per week onsite
Compensation: Not disclosed
AI-driven institutional investment platform
We are working with a highly regarded institutional investment firm building a specialist AI research and engineering group focused on applying advanced machine learning to live investment decision-making.
This is not a typical data engineering role and it is not an internal analytics function. The team builds AI systems that operate in production, support real investment workflows, and are held accountable to measurable outcomes. You would join a small, high-caliber group of research scientists, machine learning engineers, platform engineers, and investment specialists working at the intersection of AI, markets, distributed systems, and large-scale experimentation.
The role is best suited to a strong software engineer who has built production-grade distributed systems, data platforms, or ML infrastructure, and wants to work directly with research teams on hard, ambiguous problems.
What you’ll work on
You will build the core research data infrastructure that powers AI models, experimentation systems, and production investment workflows.
This includes designing and scaling systems for high-throughput data movement, processing, storage, orchestration, observability, and reproducibility. You will partner closely with research scientists to turn early-stage research needs into reliable infrastructure, while also modernizing existing systems and building new platforms where the current tooling does not yet exist.
The work will span pipelines, orchestration layers, monitoring, introspection, data quality checks, developer tooling, researcher tooling, and infrastructure that helps research move from idea to production more quickly and safely.
What we’re looking for
We are looking for someone with strong software engineering fundamentals and real production ownership.
You should have experience building distributed systems, data platforms, orchestration frameworks, or infrastructure that supports machine learning, scientific research, or complex experimentation. Strong Python and systems design ability are important, as is evidence that you have owned architecture, scalability, reliability, observability, and operational maturity.
This is a hands-on role. You should be comfortable designing the system, writing the code, debugging production issues, and improving the platform over time. You should also be comfortable working with research scientists where the requirements may be ambiguous and the best solution needs to be shaped collaboratively.
Tech stack and environment
The environment includes technologies such as Python, Spark, Kafka, Flink, Ray, Airflow, Databricks, Snowflake, Kubernetes, and similar distributed data systems.
You do not need to have used every tool listed, but you should have strong depth across distributed systems, high-throughput data infrastructure, workflow orchestration, batch or streaming systems, monitoring, schema design, data quality, and failure recovery.
The team also uses AI-native development workflows, coding agents, and automated engineering systems as part of how software is built.
Why this role stands out
This is a rare opportunity to work on infrastructure that sits directly behind advanced AI research and live investment systems. The team is applying modern machine learning and reasoning models to financial markets, with a strong focus on causality, diagnostics, reliability, and explainability.
The culture is fast-moving, direct, and highly collaborative. Engineers and scientists work closely together, with real ownership from paper to production. This will suit someone who wants to be close to the research, close to the systems, and close to the business impact.
Location
This is an onsite role based in New York City, with an expectation of being in the office four days per week.
People In AI partners with leading AI, machine learning, and data-driven companies to help candidates understand the real scope, team context, and technical expectations behind each opportunity. We aim to make the process clear, direct, and useful, so you can make an informed decision about whether the role is genuinely aligned with your goals.
