Data Pipeline Engineer (ADF And ADLS) job in Diamond Bar,...

Data Pipeline Engineer - (ADF and ADLS)
Niagara BottlingDiamond Bar, CA2 days ago
At Niagara, we’re looking for Team Members who want to be part of achieving our mission to provide our customers the highest quality most affordable bottled water.

Consider applying here, if you want to:

  • Work in an entrepreneurial and dynamic environment with a chance to make an impact.

  • Develop lasting relationships with great people.

  • Have the opportunity to build a satisfying career.



We offer competitive compensation and benefits packages for our Team Members.

Data Pipeline Engineer - (ADF and ADLS)

As an integral member of our team, the Data Pipeline Engineer is responsible for the design, development, implementation and support of mission-critical enterprise projects. Working with cross-functional teams, this person will be responsible for preparation of detailed specifications, technical design, development, documentation, testing, implementation and support of large-scale projects in the cloud, with a firm grasp of emerging technologies, platforms, and applications, and ability to customize them to help our business become more secure and efficient. Extensive knowledge of cloud maintenance and advanced industry knowledge is key to keeping us on the cutting edge. From day one, this role will have an immediate impact on the day-to-day efficiency of our IT operations, and an ongoing impact on our overall growth.

Detailed Description

  • Participate in creation of a well-informed cloud strategy and administer the adaption process

  • Regularly evaluate cloud applications, hardware, software and propose best practices to optimize cloud presence, performance, security, and cost

  • Build and Manage Azure DevOps pipelines for CI/CD and Release Management workflows

  • Manage Azure DevOps repos and branch policies

  • Create and maintain optimal data pipeline architecture

  • Design, develop and implement large scale, high volume, high performance data models and pipelines for Data Lake and Data Warehouse

  • Develop and implement data quality checks, conduct QA and implement monitoring routines

  • Proficient with cloud Technology on Azure or AWS or GCP.

  • Carrying out deployment, maintenance, monitoring, and management tasks within this implemented cloud structure

  • Working with business to understand their needs and to implement the cloud strategies designed to meet those needs

  • Translating project-specific needs into a cloud structure that will suit those needs and considering the project needs in terms of resource usage and scalability

  • Work closely with IT security to monitor the cloud privacy

  • Employ exceptional problem-solving skills, with the ability to see and solve issues before they affect business productivity

  • Respond to technical issues in a professional and timely manner

  • Identify the top cloud architecture solutions to successfully meet the strategic needs

  • Ensure proper configuration management and change controls are implemented

  • Work closely with offshore team(s) and participate in scrum calls

  • Provide technical assistance and cross training to other team members

  • Provide training, user documentation and assistance to end users as required



  • Design and implement technology best practices, guidelines and repeatable processes



  • Understanding Data Sources and Data Modeling concepts to develop and enhance the existing Data Warehouse

  • Knowledge of logical and physical database modeling concepts and techniques

  • Understand normalized and star-schema physical models and logical modeling concepts

  • Follow and implement Business Intelligence best practices and architectural standards

  • Participate and handle full Software Development Life Cycle (SDLC)

  • Please note that this job description is not designed to contain a comprehensive list of activities, duties or responsibilities that are required of the employee for this job. Duties, responsibilities and activities may change at any time with or without prior notice



Work Experience/KSA’s

  • Required:

    • 5+ years – Experience in AI/ML projects, DevOps, CI/CD pipelines/ML Studio

    • 5+ years – Experience in Python/R, Spark/Kafka

    • 5+ years – Experience with Azure Databricks/Data Factory

    • 5+ years – Experience with Data Modeling, Data Mart design, ETL Architecture

    • 3+ years – Experience with PL/SQL to support ETL processes

    • 3+ years – Experience in OBIEE/Power BI/OAC/Tableau

    • 3+ years – Experience in performance tuning of ETL process, reports/dashboards

    • 3+ years – Experience with Java programming





  • Preferred:

    • 7+ years – Experience in AI/ML projects, DevOps, CI/CD pipelines/ML Studio

    • 7+ years – Experience in Python/R, Spark/Kafka

    • 7+ years – Experience with Azure Databricks/Data Factory/ODI/Informatica

    • 7+ years – Experience with Data Modeling, Data Mart design, ETL Architecture

    • 7+ years – Experience with PL/SQL to support ETL processes

    • 7+ years – Experience in OBIEE/Power BI/OAC/Tableau

    • 7+ years – Experience in performance tuning of ETL process, reports/dashboards

    • 5+ years – Experience with Snowflake/Synapse/ADW

    • 5+ years – Experience with Java programming




Competencies

  • Listens and responds constructively to other team member’s ideas

  • Proven leadership qualities

  • Pays close attention to detail, accuracy and completeness

  • Promptly notifies his/her manager about any problems that affect his/her ability to accomplish planned goals

  • Expresses pride in the group and encourages people to feel good about their accomplishments



Education

  • Minimum Required:

    • Bachelor's Degree in Computer Science or Engineering




  • Preferred:

    • Master's Degree in Computer Science or Engineering




Certification/License:

  • Required: Cloud Architect certification.

  • Preferred: Oracle Business Intelligence Foundation Suite 11g/12c



Foreign Language

  • Required: None Required

  • Preferred: None



Benefits
https://careers.niagarawater.com/us/en/benefits

This job is intended to be performed entirely outside of Colorado



Any employment agency, person or entity that submits a résumé into this career site or to a hiring manager does so with the understanding that the applicant's résumé will become the property of Niagara Bottling, LLC. Niagara Bottling, LLC will have the right to hire that applicant at its discretion without any fee owed to the submitting employment agency, person or entity.

Employment agencies that have fee agreements with Niagara Bottling, LLC and have been engaged on a search shall submit résumé to the designated Niagara Bottling, LLC recruiter or, upon authorization, submit résumé into this career site to be eligible for placement fees.
Loss Mitigation Specialist, Team Lead
Carrington Mortgage Holdings, LLC

Anaheim, CA

2 days ago
Ability to compute, analyze and interpret complex statistical data and/or...
DevOps Engineering II
Leidos

San Dimas, CA

2 days ago
Experience integrating DevOps pipeline with test automation tools. Two, b...
.Net Software Engineer III
Leidos

San Dimas, CA

2 days ago
Two, bring in new technology to make existing processes better (e.g., more effic...
Loss Mitigation Specialist, Team Lead
Vylla

Anaheim, CA

2 days ago
Ability to compute, analyze and interpret complex statistical data and/or...
.Net UI Software Engineer III
Leidos

San Dimas, CA

2 days ago
Two, bring in new technology to make existing processes better (e.g., more effic...
Java Software Engineer III
Leidos

San Dimas, CA

2 days ago
Two, bring in new technology to make existing processes better (e.g., more effic...
Java UI Software Engineer III
Leidos

San Dimas, CA

2 days ago
Two, bring in new technology to make existing processes better (e.g., more effic...
Volunteer Services Manager LEAP Program
OneOC

Santa Ana, CA

2 days ago
Able to project likely future occurrences based on current or historic data...
Senior Business Intelligence Developer - SBID22-03314111000
Evergreen Technologies, LLC.

Citrus, CA

2 days ago
Use the Microsoft BI tool set while also having an understanding of other relati...
Senior DevOps Engineer - 22-03314222000
Evergreen Technologies, LLC.

Citrus, CA

2 days ago
Experience maintaining continuous integration/deployment (CI/CD) pipelines
REMOTE - Data Engineer (Machine Learning)
City of Hope

Irwindale, CA

3 days ago
Maintain and support data pipelines in production. Design and buil...
Deputy Strategic Talent Management
City of Hope

Irwindale, CA

3 days ago
Develops an integrated tracking system to display performance metrics that regul...
Sr. Recruiter - Corporate Functions (Hybrid/Remote)
City of Hope

Irwindale, CA

3 days ago
Ensures accurate data input into the applicant tracking system and People...
Sr Manager, Talent Acquisition (Remote/Hybrid)
City of Hope

Irwindale, CA

3 days ago
Strong understanding of social media and demonstrated strategies implementing so...