Architecture & methodology

How I build data systems

Two real end-to-end systems: a cloud-native data pipeline with RAG, and a reproducible machine-learning workflow on HPC.

Production RAG pipeline · GCP

RAG · GCP

Findspo — from scientific literature to clinical answers

PubMed

Scientific literature as source

Ingestion

Extraction and parsing in Python

Cloud Storage

Raw data landing zone

BigQuery

Structured, queryable warehouse

Embeddings + Vector search

Semantic retrieval (Vertex AI)

LLM Reranking

Relevance re-scoring

Cloud Run API

Clinical-answer chatbot

All infrastructure is provisioned with Terraform (Infrastructure as Code).

ML · HPC

CSIC-IBMB — biological age prediction from scRNA-seq

Raw scRNA-seq

Single-cell expression data

QC & preprocessing

Quality control with scanpy

Feature selection

Variable genes · age correlation

HPC training

Models trained via SLURM

Evaluation

R² · classification metrics

Interpretation

Biological feature importance

All compute runs on the IBMB HPC cluster through SLURM.

Infrastructure as code

Reproducible, versioned environments with Terraform and Docker.

Data quality by design

Validation and monitoring built into every pipeline stage.

Reproducibility

Deterministic workflows: same data and code, same results.

Scale

From local prototype to distributed cloud and HPC workloads.