Shuvo Barman — Data & Automation Engineer

85%

Faster Data Processing

4K+

Fintech Transactions / Day

Years Experience

M.Sc.

in Computer Science

About Me

Building data infrastructure
that drives decisions.

I'm a data and software professional with an M.Sc. in Computer Science, passionate about data engineering and analytics. My career has taken me from building Android healthcare applications in Bangladesh, to processing financial transactions at scale in fintech — and now to designing and automating data workflows for Canada's federal government.

At Environment and Climate Change Canada, I build end-to-end data pipelines in Azure DevOps, develop Power BI dashboards for policy and analysis teams, and write Python and Julia tools that replace slow manual processes with reliable, automated systems. My official title is Junior Economist — but the work is engineering through and through.

I'm actively seeking opportunities in Data Engineering where I can design scalable pipelines, build clean data infrastructure, and help teams spend less time moving data — and more time using it.

            85%
            reduction in data processing time after architecting Azure DevOps pipelines with YAML-based build and release definitions at ECCC
          

            99.99%
            accuracy on an automated Python system processing 4,000+ daily cashback transactions at Upay, replacing an entirely manual process
          

            40%
            improvement in stakeholder response time after delivering real-time Power BI dashboards to decision-makers at ECCC
          

Currently

Junior Economist — Data & Automation · Environment and Climate Change Canada · Gatineau, QC

Technical Skills

Data Pipelines & Orchestration

ETL / ELT Pipelines Apache Airflow Azure DevOps CI/CD Batch Processing & Ingestion

Data Processing & Transformation

Python SQL PySpark Julia Pandas NumPy Data Wrangling Data Modeling

Cloud, Storage & Databases

Microsoft Azure Amazon Web Services (AWS) Azure Blob Storage Data Lake Data Warehouse PostgreSQL Vector Databases NoSQL File Systems Parquet

Business Intelligence

Power BI DAX Tableau Data Visualization Matplotlib

Data Quality & Governance

Schema Validation Data Quality Testing Great Expectations Ensuring Data Integrity Data Management/Governance

Machine Learning

Scikit-learn Feature Engineering Predictive Modeling Customer Segmentation Statistical Analysis

Selected Work

Projects

Python Scikit-Surprise SVD KNNWithMeans Pandas

Amazon Beauty Product Recommendation System

Implemented and compared five recommendation algorithms on Amazon's Beauty product dataset to tackle cold-start and data sparsity challenges in real-world recommender systems.

Built demographic (weighted average), rank-based, item-based (KNNWithMeans), and matrix factorisation models (SVD, SVD++) evaluated with 5-fold cross-validation
SVD achieved best cold-start performance (RMSE: 0.72, MAE: 0.007); user-based SVD performed best on sparse data (RMSE: 1.25, MAE: 0.97)
Addressed the sparsity problem by reducing the dataset to users with 5+ ratings, bringing interaction density to 0.28% — a realistic production scenario

View on GitHub

Python EDA Feature Engineering KNN Regression Linear Regression

New York Airbnb — EDA & Price Prediction

Analysed the September 2022 NYC Airbnb listings dataset to identify price-driving features and build a listing price prediction model using supervised regression.

Used ML-based imputation to handle outliers and missing values — KNN with Robust Scaler for minimum nights, Linear Regression with MaxAbs Scaler for review scores
Applied Mutual Information scoring across features to select the 14 most price-relevant attributes; compared 5 scaling techniques to optimise model input
Identified host listing count, review scores, and room type as the strongest predictors of listing price

View on GitHub

Python Logistic Regression KNN K-Fold CV EDA

Heart Attack Analysis & Prediction

Built a binary classifier to predict heart attack risk from clinical patient data, comparing three algorithms across five scaling techniques to find the best-performing combination.

Compared KNN, Logistic Regression with K-Fold CV, and Linear Classification — K-Fold Logistic Regression achieved the highest training accuracy (~90%) and was selected for test prediction
Applied 5 scaling methods (MinMax, Z-score, MaxAbs, Robust, Quantile) to evaluate their effect on each model's F-score before choosing the final configuration
Achieved 86% accuracy on test data using K-Fold cross-validated Logistic Regression with the liblinear solver

View on GitHub

Career

Experience

Sep 2023 – Present

Environment & Climate Change Canada

↗ canada.ca/eccc

Junior Economist · May 2024 – Present

Scaled the Azure DevOps CI/CD pipeline to process 15 scenarios end-to-end in under 40 minutes — down from hours of manual work using Access database files that could not be shared across the team
Engineered the pipeline to extract 58 target .dta files from 460+ files per scenario folder, producing 58 unique Parquet files, 63 master files, and 5 delta comparison files tracking changes across Greenhouse Gas (GHG) Emissions, Air Pollution, Electricity Generation, and Generation Capacity — covering economic sectors including Oil & Gas, Electricity, Transportation, Heavy Industry, Agriculture, Buildings, and Waste & Others
Automated VM cleanup and Azure Blob Storage upload as part of the same pipeline run, ensuring no manual handoff steps between processing and reporting
Scaled Power BI reporting to support up to 20 scenarios across 18 stakeholder-facing tabs — replacing an R/Access workflow that could not handle multi-scenario comparison at this scale, with each tab tailored to a specific visual need
Built a post-pipeline email alert system in Python that monitors key economic variables and z-driver values against sector-specific thresholds — proactively notifying Economic Sector Leads of significant cross-sector changes after every pipeline run
Cut province-specific stakeholder report distribution from 3–4 hours to under 2 minutes using Python automation — covering 13 provinces, 10–30 tabs per workbook, with consistent formatting, bilingual data validation, and dynamic schema handling across all outputs

Junior Economist · Jan 2024 – Apr 2024

Built automated Azure ETL pipelines to replace fragmented manual workflows, reducing manual data handling by 70% with zero data integrity issues
Developed Python scripts to convert legacy .dta files to Parquet format, enabling direct Power BI integration and eliminating intermediate processing steps
Saved Economic Sector Leads 10 hours of manual reporting work per week by delivering Power BI dashboards that automated previously manual weekly scenario runs

Co-op Student · Sep 2023 – Dec 2023

Eliminated manual scientific data conversion by building a Julia package that automated transformation of raw datasets into NeXus/HDF5 format, with full technical documentation for team knowledge transfer

Aug 2021 – Jul 2022

Upay — UCB Fintech Company Limited

Sr. Executive, Revenue Assurance & Business Intelligence

↗ upaybd.com

Engineered ETL pipelines using PostgreSQL and Python to process customer transaction data daily, enabling real-time analytics across customer care, marketing, sales, and tech operations teams
Replaced a fully manual cashback disbursement process by building an automated Python system handling 4,000+ daily transactions with 99.99% accuracy — reducing processing time from 24 hours to 30 minutes and cutting weekly customer complaints by 25%
Built Tableau dashboards tracking revenue performance and operational metrics, giving business teams a single source of truth for day-to-day decision making
Developed proof-of-concept ML models for customer churn prediction, next-transaction prediction, and transaction pattern analysis — identifying at-risk customers with 85% accuracy
Built customer segmentation models that improved campaign targeting accuracy by 30% and increased weekly transaction volume by 15%
Automated financial reporting and stakeholder email alerts using Python, replacing manual report preparation across the revenue assurance team
Evaluated campaign plans submitted by product and sales teams against historical data and target benchmarks before go-live — ensuring every campaign launched was grounded in evidence, not assumption

Dec 2018 – Aug 2021

USAID MaMoni Maternal & Newborn Care Project

Java Programmer — Mobile Application · ~3 Years

↗ MaMoni Project — Save the Children Resource Centre

Retained via: Save the Children in Bangladesh (Mar 2020 – Aug 2021) Enroute International Limited (Oct 2019 – Feb 2020) Dnet — a social enterprise (Dec 2018 – Sep 2019)

Digitised maternal and child health records for the Government of Bangladesh by converting paper-based registers into a structured mobile application — capturing routine checkups, clinical visits, and full health histories for pregnant women and newborns
Improved application performance and maintainability by replacing third-party UI libraries with native Android design components, reducing external dependencies and ensuring a consistent user experience across devices
Delivered new modules end-to-end — from requirements gathered with system analysts and DGFP stakeholders through to interface design, database schema, and production code
Supported nationwide adoption by conducting hands-on training across 10 districts, training 300+ government health workers on using the system in the field
Maintained delivery schedules across a multi-organisation project structure by collaborating with project managers, system analysts, and government counterparts on implementation planning

Data Engineer
& Analyst

Building data infrastructure
that drives decisions.

Projects

Amazon Beauty Product Recommendation System

New York Airbnb — EDA & Price Prediction

Heart Attack Analysis & Prediction

Experience

Education

Certifications

Extracurricular Activities

Let's Connect

Data Engineer& Analyst

Building data infrastructurethat drives decisions.

Projects

Amazon Beauty Product Recommendation System

New York Airbnb — EDA & Price Prediction

Heart Attack Analysis & Prediction

Experience

Education

Certifications

Extracurricular Activities

Let's Connect

Data Engineer
& Analyst

Building data infrastructure
that drives decisions.