Lead Data Engineer Job at Altimetrik, Cary, NC

  • Altimetrik
  • Cary, NC

Job Description

Programming & Data Engineering

Expert - level proficiency in Python, Scala, and PySpark, with a strong track record of designing and delivering production-ready, modular, and well-tested solutions; developing and troubleshooting Spark workloads; and optimizing large-scale batch and streaming data pipelines using Delta Lake and Spark technologies.

Strong SQL and data modelling — dimensional and normalised; schema design and data contract definition.

Databricks expertise — Delta Lake, Unity Catalog, Jobs & Workflows, cluster and pool management, performance tuning, Model Serving.

Azure data stack — ADLS Gen2 (zone design, ACLs, lifecycle), Azure Data Factory (parameterized / metadata-driven frameworks, error handling), Azure Event Hubs.

AI & Machine Learning

3+ years designing and shipping LLM-based systems in production: RAG pipelines, agentic / tool-calling workflows, structured output, chunking and embedding strategy, vector and hybrid retrieval, and prompt engineering.

Evaluation discipline — golden datasets, regression suites, accuracy and hallucination tracking, human-in-the-loop feedback loops; you measure AI quality, not assert it.

Hands-on experience with LangChain, LlamaIndex, or LangGraph, plus at least one provider stack (Azure OpenAI, OpenAI, or Databricks Model Serving).

Metadata-driven thinking — schema inference, data profiling, lineage, catalogs, and configuration-driven frameworks that onboard the next source without new code.

Job Tags

Full time, Contract work

Similar Jobs

Salt Lake Valley Gmc Truck

General Sales Manager Job at Salt Lake Valley Gmc Truck

 ...Sales Manager Automotive DealershipSalt Lake Valley Auto Group is seeking a driven, experienced Sales Manager to lead the sales team. Were committed to delivering an exceptional car-buying experience while building a team culture thats high-performing, collaborative... 

North American Rail Solutions

Sales Director Job at North American Rail Solutions

 ...sales operations with teams of five (5) or more direct reports. \n Knowledge of rail maintenance services, railroad operations, commercial construction, or related industries preferred. \n Familiarity with track inspection processes (FRA or non-FRA) and the ability... 

Priority Dispatch, Inc.

Medical Delivery Driver (Tuesday - Saturday) Job at Priority Dispatch, Inc.

 ...for Extra Holiday Income? We've Got a Route for You. Medication Delivery Driver | Indianapolis, IN. The holidays are busy, and so are we. Priority Dispatch is seeking reliable Independent Contractors to help deliver critical medications to nursing facilities... 

Allied Universal

Security Officer Oil Refinery Job at Allied Universal

 ...customers we serve. Job Description As a Security Officer Oil Refinery in Detroit MI you will serve and safeguard clients in a...  ...Safety Academy (military or civilian) or verified work experience in the law enforcement adult corrections or firefighter field... 

Mercy

LPN - Mercy Hospital Lebanon - Wound Care - Part Time Job at Mercy

 ...calling at Mercy! Utilizes the nursing process to provide patient care under the supervision of person(s) licensed by a State...  ...through innovation, technology, and compassion. As we grow, you'll grow with us. keyword(s): wound care, LPN, Lebanon, Part time...