Pipelines & Engineering
Production-grade cloud systems for collecting, enriching, and analyzing large-scale job market data — built on GCP, Apache Beam, and Vertex AI.
Job Postings Intelligence Pipeline
A fully GCP-native pipeline collecting job postings across CS, MBA, Data Analytics, and AI domains from USAJobs and Jooble, scoring relevance, and mapping every role to O*NET 24.2 competency frameworks. Migrated from a Google Colab prototype to Cloud Run, BigQuery, and Cloud Scheduler, with a nine-phase interactive analytics dashboard and GitHub Actions auto-deploy.
System Architecture
Religious Job Listings Dataset (RJLD)
A multi-source pipeline integrating IRS, Census, USDA, and SerpAPI data to track digital labor market participation among chaplains, pastors, clergy, and related roles. Features a fine-tuned RoBERTa multi-label classifier (four thematic labels: community_service, campus_ministry, spiritual_care, digital_facilitation) with an inter-annotator agreement gate (Cohen's κ ≥ 0.6) before model training.
System Architecture
Job Pipeline — ROAR HPC Variant
The GCP pipeline adapted for Penn State's ROAR supercomputer cluster. Replaces every managed cloud service with a local equivalent: DuckDB for BigQuery, POSIX file queue for Pub/Sub, SLURM sbatch for Cloud Scheduler, scikit-learn RandomForest for Vertex AI AutoML, and a Streamlit dashboard for Cloud Run.