S Souranil Sen / Portfolio

Souranil Sen

Senior ML Engineer · Meta Reality Labs · Burlingame, California

Building foundational generative AI, computer vision, and 3D systems for platform-scale consumer products — facial perception, synthetic data infrastructure, and text-to-3D animation.

Generative AI Computer Vision 3D / Graphics Synthetic Data Production ML Billion-user systems Meta Reality Labs
01

Professional experience

2022 — Now

Meta/Senior Machine Learning Engineer · Reality Labs

Burlingame, California · Generative AI · Computer Vision · 3D
Generative AI Synthetic Data Sim-to-Real 3D Reconstruction Diffusion Models DECA PyTorch3D Unity Production ML

Engineering lead on foundational generative AI and computer vision systems deployed at consumer scale — including a production multi-frame facial perception model, a Unity-based synthetic data generation infrastructure, and a diffusion-based model for generating 3D character animations from natural language. Systems deployed to 1.6 billion+ users across Facebook, Instagram, WhatsApp, and Horizon.

1.6B+
Facebook · Instagram · WhatsApp · Horizon
+33%
L1 on multi-pose set
−4%
Pipeline rejections
12×
Real-time inference speedup
  • Word-to-Animation (2026, current) — tech lead on diffusion-based model generating 3D character animations from natural language; owned eval framework, training infra, and end-to-end pipeline.
  • Autogen (2023 — 2025) — multi-phase generative facial perception initiative for Parametric Avatar: Multi-Task Model → 3D reconstruction for parametric faces → Multi-frame Autogen. Outperformed prior production system on all 92 blendshapes — +33% L1, +1.3% global completion, 1.5M+ additional successful generations/yr.
  • 12× inference speedup on production facial expression model serving 1.5B+ users — 10.0s → 0.8s for a 300-frame sequence.
  • Synthetic data pipeline — Unity-based procedural face generation; 15M+ labeled images. Foundational training infrastructure that powered every phase of Autogen and downstream generative models.
  • Primary oncall at billion-user scale across 2023–2025; led incident review for outage affecting 23,000 user requests.
Platform · Delivery
1.6B+Consumer users
4 appsFacebook · Instagram · WhatsApp · Horizon
Synthetic data infrastructure
DECA feature extraction
Multi-frame 3D reconstruction
Production release + telemetry
Model changes
+33%Accuracy lift@ Multi-pose L1
+1.3%Global completion+1.5M generations/yr · at 1.6B scale
−4%Failure reduction@ Pipeline rejections
12×Inference speedup10.0s → 0.8s
  • Word-to-Animation (2026, current) — Tech lead for modeling on a diffusion-based product generating 3D character animations from natural language. Established the automated-evaluation framework, training infrastructure, and end-to-end multi-model pipeline.
  • Autogen (2023 — 2025) — multi-phase generative facial perception initiative for Parametric Avatar. Took it through three phases: Multi-Task Model3D reconstruction for parametric facesMulti-frame Autogen for parametric avatar. Finetuned the DECA encoder as feature extractor and designed a novel per-frame aggregator over shape/pose parameters; outperformed the prior production system on all 92 blendshapes. Drove +33% L1 on the multi-pose benchmark and +1.3% global completion — delivering 1.5M+ additional successful generations per year and a −4% reduction in pipeline rejections, ~7× the previous model's lift.
  • Synthetic data pipeline — foundation for all Autogen work. Designed and built the Unity-based synthetic data generation pipeline for training large-scale facial perception models — 15M+ labeled images via procedural face generation with controlled variation across lighting, pose, and expression. Every phase of Autogen depended on this infrastructure, and it was reused across downstream generative models.
  • FACS-based synthetic expression generation pipeline with learned domain transfer across a diverse identity space — adopted as training infrastructure for downstream generative animation models.
  • 12× inference speedup on a production facial expression model serving 1.5B+ users10.0s → 0.8s for a 300-frame sequence, removing ~92% of critical-path latency.
  • Shipped end-to-end model lifecycle for parametric facial reconstruction — owned training, evaluation, internal testing, A/B experimentation, release, fine-tuning, and post-launch telemetry.
  • Production ownership at billion-user scale — primary oncall across 2023–2025; led incident review for a high-priority production incident affecting 23,000 user requests.
  • Cross-functional collaboration at Meta scale — partnered with Technical Artists, Product Engineers, Production Engineers, and other MLEs across Reality Labs and adjacent organisations to take foundational models from prototype to consumer-product deployment.
  • Lead engineer on multiple high-visibility product releases and executive-level technical demonstrations.
2018 — 2022

Unity Technologies/Machine Learning Engineer · AI Computer Vision

San Francisco, CA · ~4 years · Synthetic data · sim-to-real · robotics
Synthetic Data Sim-to-Real Unity Detectron2 Robotics SDK

Led synthetic-data research in AI computer vision — designing the data and tooling that bridged Unity simulation environments with ML training pipelines for robotics and autonomous-vehicle customers. Established the sim-to-real transfer effectiveness result for object detection that validated the synthetic-data thesis at production scale, and was lead author on the team's peer-reviewed and industry-blog output.

IROS
2021 · peer-reviewed
Summit
Unity AI · keynote 2021
SDK
unity-vision · shipped
  • Designed & shipped unity-vision SDK bridging Unity-generated synthetic datasets with ML training pipelines — used internally and by robotics / AV customers.
  • Established sim-to-real transfer result for object detection — validated the synthetic-data thesis at production scale; became Unity's external anchor for data-centric AI positioning.
  • IROS 2021 co-author (drone pose estimation); delivered invited keynote at Unity AI Summit 2021; featured author on Unity engineering blog.
  • Built cloud simulation services for running Unity scenes at scale with hyperparameter tuning; collaborated with a major game studio on automated playtesting.
Research output
IROSPeer-reviewed
SummitUnity AI keynote
SDKShipped tooling
BlogFeatured author
Sim-to-real stack
Unity simulation scenes
Synthetic labels + variation
Detectron2 training pipeline
Benchmark transfer results
Applied domains
RoboticsPose estimation
AVAutonomous vehicles
GamingPlaytesting
CloudSimulation scale
  • unity-vision SDK. Designed and shipped the SDK bridging Unity-generated synthetic datasets and ML training pipelines, used internally and by Unity's robotics / AV customers.
  • Sim-to-real transfer for object detection. Validated the synthetic-data approach end-to-end: showed Unity-rendered training data delivers benchmark-beating object-detection performance with transfer learning — the result Unity used externally to anchor its data-centric-AI positioning.
  • IEEE/RSJ IROS 2021 — Drone Pose Estimation Using Synthetic Data in Unity. Co-authored, presented as poster at the International Conference on Intelligent Robots and Systems.
  • Unity AI Summit 2021 — invited keynote / workshop. Delivered the synthetic-data-for-computer-vision workshop with measurable end-to-end benchmarks.
  • Unity Technologies blog — featured author. Published "Data-Centric AI with Unity Computer Vision Datasets" on the official corporate engineering blog.
  • Cloud simulation services. Designed core cloud services for running Unity simulations at scale with hyperparameter tuning; collaborated with a major game studio on automated playtesting optimization.
  • Open-source — Indoor Pet Detection. Authored the canonical Unity-Perception synthetic-data sample for large-scale indoor dog detection with Faster R-CNN + transfer learning.
2021

MIT Fluid Interfaces Lab/Technical Mentor — "AI Generated Media"

MIT Media Lab · Remote · Sep — Nov 2021 · ~20 graduate students
MentorshipGenAI

Technical mentor for the AI Generated Media graduate course at the MIT Media Lab — provided guidance to a cohort of ~20 graduate students on deep-fakes, generative AI, and content authenticity, including project-level review of student work on generative-media systems.

  • Technical mentor for ~20 graduate students on deep-fakes, generative AI, and content authenticity.
  • Project-level review of student work on generative-media systems.
2017 — 2018

Stony Brook University/Graduate Research Assistant · M.S. Computer Science

Stony Brook, NY · GPA 3.73 · Human Interaction Lab
Computer Vision VR React Native

Graduate research under Prof. Roy Shilkrot on dark-pattern detection in user interfaces using computer vision; parallel work under Prof. Rong Zhao on a VR application and ML-assisted clustering framework for schizophrenia patient research. Designed and deployed the multi-tenant NYSTAR conference app (React Native) to Google Play and the App Store with full CI/CD.

  • CV research on detecting dark patterns in user-facing interfaces (Prof. Roy Shilkrot).
  • VR + ML clustering for schizophrenia clinical research — VR app paired with unsupervised clustering framework (Prof. Rong Zhao).
  • Shipped NYSTAR conference app (React Native) to Google Play and App Store with full CI/CD; multi-tenant architecture.
3.73
M.S. CS GPA
CV
Dark-pattern research
VR
Clinical research app
App
NYSTAR shipped
  • Computer-vision research on detecting dark patterns in user-facing UI surfaces (Prof. Roy Shilkrot).
  • VR + ML clustering for schizophrenia research — VR application paired with a clustering framework supporting clinical analysis (Prof. Rong Zhao).
  • NYSTAR conference application — multi-tenant React Native app deployed to Google Play and App Store with CI/CD automation.
  • Hack@CEWIT 2017 — Most Original. Workout-detection armband with 3D-printed enclosure, gyroscope, and DTW-based motion-quality classification.
2016 — 2017

Quintype Inc./Full Stack Engineer

Bangalore, India · Mar 2016 — Jan 2017 · Publishing platform · real-time data
ClojureMicroservicesNode.jsRxJSWebSocketsRSS ImportsOncall

Full-stack platform engineer on Quintype's publishing stack — helped redesign a monolith into Clojure microservices, built RSS import workflows, and developed a real-time stock-data streaming service handling 1M+ QPS using RxJS, WebSockets, and Node.js.

  • Microservices migration — helped decompose the publishing platform monolith into Clojure microservices.
  • RSS import workflows — built ingestion pipelines for importing feed content into the publishing platform.
  • Real-time stock streaming at 1M+ QPS — built a high-throughput streaming service using RxJS, WebSockets, and Node.js.
1M+
QPS · stock streaming
Clojure
Monolith → microservices
RSS
Content ingestion pipelines
  • Product architecture redesign — worked on decomposing the publishing platform from a monolith into Clojure microservices.
  • Data-team backend development — built ingestion workflows for importing RSS feeds into the publishing platform.
  • Real-time stock streaming at 1M+ QPS — built a high-throughput streaming service using RxJS, WebSockets, Node.js, and the observer pattern.
  • Production leadership — received appreciation for effectively mitigating production issues on call and leading a team through operational incidents.
2014 — 2015

ThoughtWorks/Software Consultant

Bangalore, India · Aug 2014 — Jun 2015 · Enterprise data services · DevOps
RubyPostgreSQLState MachinesD3.jsAWS OpsWorksDashboards

Software consultant on RedE, an enterprise data-driven application — built backend services, refactored transition logic with state machines, developed live D3.js dashboards, and migrated deployment automation to AWS OpsWorks.

  • Backend services for RedE — data-driven enterprise application using Ruby and PostgreSQL.
  • State-machine refactor — reworked core workflow transition logic for modularity.
  • Live D3.js dashboards for operational data visibility; migrated deployments to AWS OpsWorks.
RedE
Enterprise app
D3.js
Live dashboards
AWS
OpsWorks deploys
  • Backend services for RedE — developed data-driven enterprise services using Ruby and PostgreSQL.
  • State-machine refactor — reworked core transition logic to make workflow behavior more modular and easier to change.
  • Live visualization dashboards — built D3.js dashboards for operational data visibility.
  • Deployment automation — migrated and automated application deployments using AWS OpsWorks.
2012 — 2014

TCS Innovations Lab, Tata Research/Solutions Developer

Chennai, India · Jan 2013 — Aug 2014 · Re-imagination of Workplaces initiative
JavaRubyPostgreSQLElasticSearchEnterprise Social

Solutions developer in TCS Innovations Lab on an enterprise social-networking platform for the Re-imagination of Workplaces initiative, spanning Java/Ruby services, PostgreSQL persistence, and ElasticSearch-backed discovery.

  • Enterprise social networking platform — Java/Ruby services with PostgreSQL persistence for the Re-imagination of Workplaces initiative.
  • ElasticSearch-backed discovery — built search and content-surfacing features for workplace collaboration at scale.
ESN
Enterprise social platform
Search
ElasticSearch-backed
R&D
Innovations Lab
  • Enterprise social networking platform — built Java and Ruby services with PostgreSQL persistence for the Re-imagination of Workplaces initiative.
  • ElasticSearch-backed discovery — implemented search and content-surfacing features for workplace collaboration at enterprise scale.
02

Recognition

2025Senior Member@IEEETop ~10% of IEEE membership · approval by admissions committee
2025Judge@Meta Horizon Creator CompetitionMobile Genre Showdown · Invited Judge
2025Mentorship@MIT Reality Hack 2025Invited Mentor · XR/VR/AR · Boston
2025Speaker@Mohawk College — AI & Gaming PanelSchool of Creative Industries · Invited Panelist
2021Speaker@Unity AI Summit 2021 ↗Invited keynote · synthetic data & sim-to-real for CV
2021Mentorship@MIT Fluid Interfaces LabTechnical Mentor · ~20 graduate students · AI Generated Media
2017Award@Hack@CEWIT — Most OriginalWorkout-detection armband · IoT & Microservices Hackathon
03

Open source contributions

Active on github.com/sladebot. Most current work is orb — a multi-tenant multi-agent coding runtime with 4 agent topologies, live TUI, browser dashboard, and Python SDK; published on PyPI as orb-agents.

2024+orbPythonPyPIMulti-agentMulti-agent coding runtime — 4 topologies, per-node model allocation, TUI + browser dashboard
2024+turboquant-mlxPythonMLXOn-deviceQuantization & inference experiments on Apple's MLX framework for on-device ML
2023+swe2mlJupyterMLEducationReference implementations and curriculum bridging software engineering and applied ML
2018deepvoPyTorchTensorFlowCVOpen-source DeepVO — RCNN for monocular ego-motion estimation from optical flow, tested on KITTI
+ 45 more·github.com/sladebot49 public repos across CV, ML, robotics, and infra · Arctic Code Vault contributor
04

Publications & press

2021
Poster

S. Erfanian Ebadi, S. Sen, J. Leban, and P. Wani, “Drone Pose Estimation with Unity Simulation,” presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021. [Poster session].

Citations · Scholar
2018
Paper

S. Sen and A. Balasubramanian, “A highly resilient and scalable broker architecture for IoT applications,” 2018 10th International Conference on Communication Systems & Networks (COMSNETS), Bengaluru, India, 2018, pp. 336–341, doi: 10.1109/COMSNETS.2018.8328216.

~33 Citations · Scholar

Press & Featured Writing

2024+ Substack — souranil.substack.com ↗ Practitioner-audience series on the SWE → ML engineering transition Authored
2021 Data-Centric AI with Unity Computer Vision Datasets Unity Technologies engineering blog · featured author Industry
2021 Unity News Feature — synthetic-data research Corporate news coverage of synthetic-data research Press

Peer Review & Service

2026 Peer Reviewer — Precision Conference (PCS) CHI 2026 Papers — BlocksDeFix · debugging hints in block-based programming CHI 2026 Posters — Social-metadata in video previews CHI 2026 Posters — ALLEF · agency assessment in AI textbooks CUI 2026 Short Papers — Conversational pedagogy · in AI-supported art viewing Service
05

Skills

The distinctive combination is the bridge: classical ML and deep learning, applied computer vision and 3D, on top of a real graphics + simulation engineering foundation. That bridge is what enabled the Style 2 / Autogen synthetic-data pipeline at Meta.

Domain 01
Machine Learning & Deep Learning
PyTorchTensorFlowScikit-learnDetectron2 DECAMulti-task LearningTransfer Learning Neural RenderingDiffusion ModelsHyperNet Bayesian HP OptimizationContinuous Evaluation
Domain 02
Computer Vision & Spatial AI
3D ReconstructionFacial Feature Extraction Pose EstimationVisual Odometry Blendshape MappingSim-to-Real Transfer Domain Randomization
Domain 03
3D Graphics & Synthetic Data
UnityPyTorch3D Procedural Generation Avatar SystemsFACS-based Pipelines Unity Perception3D MoCap Datasets
Foundations
Languages & Infrastructure
PythonC++ScalaJava JavaScriptSQLClojure GCPAWSKubernetesDocker GitLab CI/CDJenkins
06

Education

2018M.S. Computer Science@Stony Brook UniversityGPA 3.73 · Graduate Research Assistant · Human Interaction Lab
2012B.Tech. Information Technology@Techno India, KolkataWest Bengal University of Technology
Certifications & Coursework
AI Generated Media · MIT Media Lab Robotics: Perception · UPenn / Coursera MITx 6.86x · ML with Python