Lead Data Engineer Job at WorkHQ, Los Angeles, CA

TlZKUE9qalVGcXhNcW84eXY0bWIwNGdlNXc9PQ==
  • WorkHQ
  • Los Angeles, CA

Job Description

Company Context

Series A, well-funded US startup in HRTech developing WorkHQ.com and an AI Recruiter product.

This is a US-only, Remote role (Mainland).

Role Overview

Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.

Hire data engineers to aid you in that journey.

Core Responsibilities

  • Design scalable data pipelines processing massive record volumes

  • Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)

  • Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch

  • Integrate new data sources into the main pipeline

  • Implement advanced data matching using Splink

Technical Requirements

  • 5-8 years professional data engineering experience

  • Good proficiency in:

    • PySpark and distributed computing

    • AWS data services (EMR, Glue, Athena)

    • Docker

    • Pandas and DataFrame manipulation

    • Complex data format handling (JSONL, Parquet)

  • Strong background in:

    • Big data processing architectures

    • Data warehouse design

    • Performance optimization

  • Advanced Python, SQL skills

Nice to Have

  • Probabilistic record linking expertise

  • OpenSearch/elasticsearch technologies

  • Machine learning data pipeline design

  • Recruitment tech ecosystem knowledge

Technical Stack

  • Big Data: PySpark, EMR

  • Databases: Postgres, OpenSearch

  • Cloud: AWS

  • Containerization: Docker

  • Data Formats: JSONL, Parquet

  • Analytics: Metabase, Athena, Glue

  • Data Processing: Pandas, Splink

Other Considerations

While this role has specific requirements - if you lack a few technical skills, but motivated to learn and lead the platform, please apply for consideration.

If you are coming from Director/Head of/VP levels that is relevant to this job, you can apply as well.

You will need to apply directly on our platform.

Thank you for your time.

Job Tags

Permanent employment, Remote job, Shift work,

Similar Jobs

Archer Travel

Remote Travel Agent - Work From Anywhere! Job at Archer Travel

 ...STATES, UNITED KINGDOM, OR MEXICO) Do you have a passion for travel and love helping others plan unforgettable vacations? We're looking...  ...for enthusiastic individuals to join our team as Remote Travel Agents! No prior experience is necessary - we provide comprehensive... 

KB Search Team

Service Department Manager Job at KB Search Team

 ...Drive Service Excellence: Lead High-Performing Team! Due to an upcoming retirement, we are looking for a Service Department Leader to...  ...practices to enhance efficiency and consistency across the team. Manage department operations effectively within budget constraints.... 

The Custom Group of Companies

Workday People Analytics Analyst Job at The Custom Group of Companies

Workday People Analytics Analyst 25-00263Position is REMOTE in EST12+ MONTHSHourly W2 - DOENo Corp to Corp or 3rd party agenciesHealthcareWe are looking for a Workday People Analytic Analyst to join our team of professionals!The Workday People Analytics Analyst... 

Signify Health

Physician Assistant, In-Home Visits Job at Signify Health

 ...transforming health care. As the nations leading health solutions company, we reach millions of Americans through our local presence,...  ...screenings, tobacco cessation and weight management programs, confidential counseling and financial coaching. Benefit solutions that... 

Monk Construction LLC

Experienced Reliable Subcontractors Needed Job at Monk Construction LLC

 ...in Northern Wisconsin is still very high and we are seeking subcontractors of all types to help us complete the many projects that we are...  ...and 2026. We need sub-contractors who do concrete, framing, roofing, plumbing, electrical, HVAC, drywall, insulation, siding, masonry...