Machine Learning Engineer — AI Architecture Research

About the Role

We’re looking for a

Machine Learning Engineer focused on AI architecture research to help design, prototype, and validate next-generation model architectures. You’ll work at the intersection of research and production — turning new ideas into scalable, real-world systems.

This role is ideal for someone who enjoys questioning architectural assumptions

, experimenting with novel model designs, and pushing beyond standard Transformer-style approaches.

What You’ll Work On

Research and develop new neural network architectures (e.g. alternatives or extensions to Transformers, recurrent / hybrid models, long-context systems)

Design and run architecture-level experiments (scaling laws, memory mechanisms, compute trade-offs)

Prototype models end-to-end — from research code to training-ready implementations

Collaborate with inference and systems engineers to ensure architectures are deployable and efficient

Analyze model behavior, failure modes, and inductive biases

Read, reproduce, and extend cutting-edge research papers

Contribute to internal research notes, benchmarks, and open-source efforts (where applicable)

What We’re Looking For

Strong background in machine learning fundamentals and deep learning

Hands-on experience implementing model architectures from scratch

Solid understanding of:

Attention mechanisms, RNNs, state-space models, or hybrid architectures

Training dynamics, scaling behavior, and optimization

Memory, latency, and compute constraints at the model level

Comfortable working in

PyTorch or JAX

Ability to move fluidly between theory, experimentation, and engineering

Clear communicator who can explain architectural trade-offs

Nice to Have

Experience with

non-Transformer architectures (RNN variants, SSMs, long-context models)

Background in research-driven startups or open-source ML projects

Experience with large-scale training or custom training loops

Publications, preprints, or notable research contributions

Familiarity with inference optimization and deployment constraints

Why Join

Work on core model architecture

, not just fine-tuning

Direct influence on the technical direction of a Series-A company

Small, high-caliber team with fast feedback loops

Opportunity to ship research into production

Competitive compensation + meaningful equity

Originally posted on

Himalayas