Data Engineer
Data Engineer builds the batch and incremental ingestion pipelines (ETL, document parsing, vector index syncs) that keep MacroCoderz's AI assistant fed with clean, up-to-date data. Core stack: Python, SQL, Postgres/warehouses, and orchestrators like Airflow, Dagster or Prefect. Fully remote, open worldwide.
- Compensation
- Not disclosed
- Location
- Khanpur, Pakistan
- Work setup
- Remote
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 2+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 2+ years of data engineering
- Strong Python and SQL
- Experience with an orchestration tool such as Airflow, Dagster or Prefect
- Understands incremental loads, idempotency and schema changes
Nice to have
- Has built ingestion for a RAG or search system
- dbt, Spark or a cloud data warehouse
- Document parsing tools such as Unstructured or Docling
Full description
Remote Data Engineer at MacroCoderz: build the pipelines that feed AI systems - document ingestion, ETL, data syncs and vector indexing at scale.
An AI assistant that answers from stale or broken data is worse than no assistant. As a Data Engineer you build the pipelines that keep our systems fed: ingesting documents and records from client systems, cleaning and transforming them, and keeping vector indexes in sync as the source data changes.
What you will do
- Build batch and incremental ingestion pipelines from databases, APIs and file stores.
- Parse and clean documents - PDFs, HTML, spreadsheets - for retrieval.
- Keep vector indexes and metadata in sync with source systems.
- Model data in Postgres and warehouses for analytics and retrieval.
- Monitor pipeline health and data quality.
What we are looking for
- 2+ years of data engineering.
- Strong Python and SQL.
- Experience with an orchestration tool such as Airflow, Dagster or Prefect.
- Understands incremental loads, idempotency and schema changes.
Nice to have
- Has built ingestion for a RAG or search system.
- dbt, Spark or a cloud data warehouse.
- Document parsing tools such as Unstructured or Docling.
Location: fully remote, open to applicants worldwide.